FailedProxy/Varaverk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Storage-mode awareness pass + doc update for System_Essentials through Partnership

Browse Source 
All state/data file paths in scripts and PHP now resolve via STATE_DIR / DATA_DIR /
PERSISTENT_CONF_CACHE instead of hardcoded /boot/config/ or /tmp/ paths, so the
ecosystem works in both internal and appdata storage modes.

PHP layer (watchdog.php, partnership.php, fallback.php, monitor.php, snapshot.php,
config.php): all state reads switched to STATE_DIR constant; remote state reads use
the new vv_remote_state_cmd() helper which resolves the remote's SCRIPTS_DIR via
their varaverk.cfg before building the path.

conf_sync.sh: fixed SCRIPTS_ROOT → SCRIPTS_DIR bug on MY_CONF path; added
_remote_scripts_dir() to resolve partner's SCRIPTS_DIR before SCP pull.

fallback.php page: added controls card (PARTNERSHIP_ENABLED, FALLBACK_ENABLED,
FALLBACK_RSYNC_ENABLED toggles), status grid, and settings card.

README and Manual updated for System_Essentials, Watchdogs, Fallback, Rsync,
Media, Monitors, Orchestrators, Partnership: added new scripts (conf_sync,
conf_cache_save/restore, conf_cache_watchdog, play_state_sync, start_webhook_listener,
upgrade_webhook_handler), corrected all stale /boot/config/ state file paths to
$STATE_DIR/$DATA_DIR, noted webgui/php_fpm/mover/user_scripts scripts moved to
Plugin/unraid/System_Essentials, fixed start_webhook_listener.sh header (Node.js,
not PHP -S).
This commit is contained in:
Gmer4Lfe
2026-06-19 19:32:39 -04:00
parent 0564580605
commit bf3e7cc2c4
35 changed files with 835 additions and 489 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Media/Manual-Media.md
+45 -4
View File
@@ -132,8 +132,8 @@ LIDARR_IMPORT_SCAN_TIMEOUT=600 # seconds to wait for pre-flight import scan
LIDARR_VERSION_MAJOR=3 # expected Lidarr major version (API safety check)
LIDARR_EXTENSIONS=("flac" "mp3" "m4a" "wav" "aac" "ogg" "opus" "wma")
LIDARR_PROTECTED_PATTERNS=("*.jpg" "*.jpeg" "*.png" "*.nfo" "*.lrc")
LIDARR_TRACKED_COUNT_FILE=/boot/config/lidarr_tracked_count # persistent baseline
ARR_CLEANUP_STATS=/boot/config/arr_cleanup_stats.db # read by coffee report
LIDARR_TRACKED_COUNT_FILE="$DATA_DIR/lidarr_tracked.count" # persistent baseline
ARR_CLEANUP_STATS="$DATA_DIR/arr_cleanup_stats.db" # read by coffee report
```
---
@@ -171,7 +171,7 @@ RADARR_PROTECTED_PATTERNS=("*.jpg" "*.jpeg" "*.png" "*.nfo" "*.srt" "*.sub" "*.a
```bash
ARR_SYNC_ENABLED=true
ARR_SYNC_BLOCKLIST=/boot/config/arr_sync_blocklist.tsv # tombstone file
ARR_SYNC_BLOCKLIST="$DATA_DIR/arr_sync_blocklist.tsv" # tombstone file
ARR_SYNC_CONNECT_TIMEOUT=10 # SSH connect timeout in seconds
ARR_SYNC_API_TIMEOUT=60 # curl API call timeout in seconds
DOCKER_APPDATA_BASE=/mnt/user/appdata
@@ -189,7 +189,7 @@ ARR_IMPORT_RECOVERY_AGE=6 # hours — items newer than this are skipped
SONARR_VERSION_MAJOR=4
RADARR_VERSION_MAJOR=6
LIDARR_VERSION_MAJOR=3
ARR_RECOVERY_STATS=/boot/config/arr_recovery_stats.db # read by coffee report
ARR_RECOVERY_STATS="$DATA_DIR/arr_recovery_stats.db" # read by coffee report
```
---
@@ -274,6 +274,47 @@ Requires `HOST*_TMDB_API_KEY` in `host*.conf`.
---
### Play State Sync
```bash
PLAY_STATE_SYNC_ENABLED=true # toggle entire sync
PLAY_STATE_SYNC_LOOKBACK_DAYS=30 # history window for played items
```
Emby and Jellyfin servers configured per-host:
```bash
# host*.conf
HOST1_EMBY_URL="http://192.168.50.2:8096"
HOST1_EMBY_API_KEY="..."
HOST1_JELLYFIN_URL="" # empty = skip Jellyfin on this host
HOST1_JELLYFIN_API_KEY=""
```
---
### Upgrade Webhook
```bash
# master.conf
WEBHOOK_PORT=9456 # 0 = disable listener
WEBHOOK_SECRET="" # auto-generated on first start if empty
WEBHOOK_UPGRADE_RSYNC_ENABLED=true # push upgraded file to remote nodes
```
```bash
# host*.conf
HOST1_WEBHOOK_LAN_IP="192.168.50.2" # IP arrs call for webhook delivery
```
When `WEBHOOK_PORT=0`, `start_webhook_listener.sh` exits cleanly and no listener starts.
When `WEBHOOK_SECRET` is empty, a 32-byte hex secret is generated on first start and
written back to `master.conf`. Run `Tools/webhook_setup.sh` to register the URL in arrs.
Webhook log: `/var/log/varaverk/upgrade_webhook.log`
---
### Orchestrator Job Order
```bash
Media/README-Media.md
+48 -2
View File
@@ -4,8 +4,9 @@ Library health, consistency, sync, and behavior-driven discovery for a multi-ser
stack. Correct permissions so arrs can manage files. Junk removal so orphan detection
isn't confused by scene debris. Library sync so every node tracks the same content.
Orphan cleanup against live arr APIs so deleted content actually leaves disk. Emby
notified automatically after every deletion. Weekly discovery adds new music, movies,
and TV shows based on what you actually play — no manual browsing required.
notified automatically after every deletion. Watch state synced across Emby/Jellyfin
every 30 minutes. Quality upgrades propagate to all nodes immediately via webhook.
Weekly discovery adds new music, movies, and TV shows based on what you actually play.
> **These scripts permanently delete files.** The arr cleanup scripts are protected by
> multiple safety layers that must all pass before anything is touched — but dry runs and
@@ -91,6 +92,19 @@ free overnight recovery.
`lidarr_missing_art.sh` — fetch missing album and artist artwork from fanart.tv and
fallback sources. Never overwrites existing files.
**Play State Sync**
`play_state_sync.sh` — syncs watched/played state and resume positions across all
configured Emby and Jellyfin servers. Newest timestamp wins. Runs every 30 minutes
via critical_sync_maintenance.sh.
**Upgrade Propagation**
`start_webhook_listener.sh` — Node.js HTTP server that receives Sonarr/Radarr/Lidarr
OnUpgrade webhooks. Continuous; started at array start. Writes to
`/var/log/varaverk/upgrade_webhook.log`.
`upgrade_webhook_handler.sh` — triggered by the webhook listener. Pushes the upgraded
item folder to every remote node immediately, then triggers an arr library rescan on
each remote so the upgraded file is accepted without triggering a redundant quality search.
**Discovery**
`playback_aware_lidarr_discovery.sh` — behavior-driven music discovery. Scores your
Emby play history, runs Last.fm getSimilar on top artists, adds the best matches to
@@ -138,6 +152,24 @@ playback_aware_radarr_discovery.sh — score watch history → TMDB recommenda
playback_aware_sonarr_discovery.sh — score episode history → TMDB TV recommendations → add to Sonarr
```
**Every 30 min via `critical_sync_maintenance.sh` (CRITICAL_MAINTENANCE_SCRIPTS):**
```
play_state_sync.sh — sync watched/resume state across Emby + Jellyfin
```
**Continuous (started by `array_started.sh`):**
```
start_webhook_listener.sh — Node.js webhook server; dispatch upgrade_webhook_handler.sh
```
**On every arr upgrade (triggered by webhook):**
```
upgrade_webhook_handler.sh — push upgraded folder to all remote nodes + trigger arr rescan
```
**Ad-hoc or separate schedule:**
```
@@ -189,6 +221,9 @@ in `HOST*_MEDIA_PERMISSION_SHARES` and `HOST*_MEDIA_CLEAN_FOLDERS` in host*.conf
| `lidarr_missing_art.sh` | Fetch missing album and artist artwork | Ad-hoc or separate schedule |
| `radarr_tmdb_removed.sh` | Remove movies dropped from TMDb | Ad-hoc or weekly |
| `sonarr_tvdb_removed.sh` | Remove series dropped from TVDB | Ad-hoc or weekly |
| `play_state_sync.sh` | Sync watched/played state + resume positions across Emby + Jellyfin | Every 30 min via `critical_sync_maintenance.sh` |
| `start_webhook_listener.sh` | Node.js webhook server — receive arr OnUpgrade and dispatch handler | Continuous (started by `array_started.sh`) |
| `upgrade_webhook_handler.sh` | Push upgraded item folder to remote nodes + trigger arr rescan | On each arr upgrade (via webhook) |
| `playback_aware_lidarr_discovery.sh` | Behavior-driven music discovery — Emby plays → Last.fm similar → Lidarr | Weekly via `weekly_sync_maintenance.sh` |
| `playback_aware_radarr_discovery.sh` | Behavior-driven movie discovery — Emby watches → TMDB recommendations → Radarr | Weekly via `weekly_sync_maintenance.sh` |
| `playback_aware_sonarr_discovery.sh` | Behavior-driven TV discovery — Emby episodes → TMDB TV recommendations → Sonarr | Weekly via `weekly_sync_maintenance.sh` |
@@ -228,6 +263,17 @@ Weekly discovery (WEEKLY_MAINTENANCE_SCRIPTS):
│
└── each discovery script fires arr search immediately after successful add
Every 30 min (critical_sync_maintenance.sh CRITICAL_MAINTENANCE_SCRIPTS):
play_state_sync.sh ─── newest timestamp wins → watched/resume state synced
across all configured Emby + Jellyfin servers
Continuous (started by array_started.sh):
start_webhook_listener.sh ── Node.js HTTP server listens on WEBHOOK_PORT
│ arr OnUpgrade fires webhook → POST to http://HOST_LAN_IP:WEBHOOK_PORT/webhook?key=SECRET
└── upgrade_webhook_handler.sh
├── rsync upgraded folder → all remote nodes immediately
└── trigger arr library rescan on each remote (accept new file, no quality search)
Ad-hoc enrichment:
lidarr_missing_art.sh ─────── discovers missing artwork → fetches from fanart.tv
radarr_tmdb_removed.sh ────── status="deleted" → remove from Radarr + add exclusion
Media/start_webhook_listener.sh
+5 -2
View File
@@ -5,7 +5,7 @@
#
# PURPOSE
# ─────────────────────────────────────────────────────────────────────────────
# Starts a standalone PHP HTTP server that receives Sonarr/Radarr/Lidarr
# Starts a standalone Node.js HTTP server that receives Sonarr/Radarr/Lidarr
# OnUpgrade webhooks and dispatches upgrade_webhook_handler.sh.
#
# Runs outside Unraid nginx — no session auth required. The shared secret in
@@ -13,9 +13,12 @@
#
# http://<HOST_LAN_IP>:<WEBHOOK_PORT>/webhook?key=<WEBHOOK_SECRET>
#
# Runs as a continuous script started by array_started.sh. Execs php -S which
# Runs as a continuous script started by array_started.sh. Execs node which
# replaces this process — the PID stays the same for array_started.sh's check.
#
# Uses Node.js instead of php -S: php -S on Unraid PHP 8.4 silently drops
# POST request bodies, making webhook payloads arrive empty.
#
# If WEBHOOK_SECRET is empty in master.conf: generates and saves one, then starts.
# If WEBHOOK_PORT is 0: exits cleanly (disables the listener).
#