Comments asserting a partner is down, being rebuilt or not yet installed are true on the day they are written and wrong afterwards, and nothing prompts anyone to revisit them. Behaviour is described generically; current state belongs in operator memory, not in tracked files.
Chunker splits on the header sections the audit standardised, then sub-splits
named-paragraph safeguards — without that a specific question about one of
rsync.sh's fourteen safeguards scored below unrelated chunks, because the other
thirteen dominated the vector. Index is SQLite with raw float32 blobs and is
incremental on mtime; a no-op re-index takes 66ms.
The stale defaults were found by asking the system a question and checking its
answer: it correctly reported what mover_stop.sh's header claimed, and the
header was wrong.
The fallback docs used an older COVERS naming in the wrong conf file, and rsync's merge-run
mode — the only path that carries --delete — was undocumented entirely.
Codebase-wide audit pass: fixed real bugs (SSH hangs missing BatchMode,
local-outside-function no-ops, variable name collisions, a truncated
ratio calc, wrong state-dir path, DARK vs NO_INTERNET drift, and more),
then pulled logic that was duplicated across multiple scripts — arr
cleanup safety gates, docker restart ordering, container maintenance
stop/restart, watchdog state-file helpers, partnership role resolution,
cert expiry checks, remote node discovery, and TMDB discovery scoring —
into common.sh so each now has a single implementation.
Retries up to PLAY_SYNC_HANDBACK_RETRIES times (default 5, 60s apart)
before giving up — one successful run catches all state regardless of
outage length, so users land on current watch state after DNS flips.
Emby now runs continuously on both hosts — play_state_sync via API
handles watch state reconciliation, making the 30-min dirty rsync
redundant. Handback runs play_state_sync --wait before DNS cutover
so users land on current state after a failover event.
- Emby removed from FALLBACK_HOST1_TIER1 (always running)
- Emby dirty sync removed from HOST1_CRITICAL_SYNC_SHARES
- Emby writeback removed from FALLBACK_HOST1_WRITEBACK_TIER1
- play_state_sync gains --wait flag (uses lock wait mode for handback)
- Tier 1 emby-fallback special case removed from handback writeback loop
- play_state_sync --wait added as Step 7 in handback, before DNS cutover
All state/data file paths in scripts and PHP now resolve via STATE_DIR / DATA_DIR /
PERSISTENT_CONF_CACHE instead of hardcoded /boot/config/ or /tmp/ paths, so the
ecosystem works in both internal and appdata storage modes.
PHP layer (watchdog.php, partnership.php, fallback.php, monitor.php, snapshot.php,
config.php): all state reads switched to STATE_DIR constant; remote state reads use
the new vv_remote_state_cmd() helper which resolves the remote's SCRIPTS_DIR via
their varaverk.cfg before building the path.
conf_sync.sh: fixed SCRIPTS_ROOT → SCRIPTS_DIR bug on MY_CONF path; added
_remote_scripts_dir() to resolve partner's SCRIPTS_DIR before SCP pull.
fallback.php page: added controls card (PARTNERSHIP_ENABLED, FALLBACK_ENABLED,
FALLBACK_RSYNC_ENABLED toggles), status grid, and settings card.
README and Manual updated for System_Essentials, Watchdogs, Fallback, Rsync,
Media, Monitors, Orchestrators, Partnership: added new scripts (conf_sync,
conf_cache_save/restore, conf_cache_watchdog, play_state_sync, start_webhook_listener,
upgrade_webhook_handler), corrected all stale /boot/config/ state file paths to
$STATE_DIR/$DATA_DIR, noted webgui/php_fpm/mover/user_scripts scripts moved to
Plugin/unraid/System_Essentials, fixed start_webhook_listener.sh header (Node.js,
not PHP -S).
Adds require_partnership() to common.sh — exits cleanly when PARTNERSHIP_ENABLED=false.
Removes FALLBACK_PARTNERSHIP_REQUIRED toggle — partnership is now always required,
not optional. Cross-server scripts (rsync, conf sync, fallback, arr sync, play state,
backup verify) all call require_partnership after detect_hosts.
All FALLBACK_${MY_ID}_COVERS_${REMOTE_ID}_TIER* references updated to
FALLBACK_${REMOTE_ID}_TIER* across fallback_test.sh, partnership_manager.sh,
docker_update.sh, mesh_monitor.sh, and monitor.php. mesh_monitor.sh drops
the inner covering-host loop — tier data now lives in the covered host's own
conf so no cross-host scan is needed. monitor.php reads from the covered
host's conf file rather than the local host's.
Previously HOST1 defined what it would run for HOST2 (FALLBACK_HOST1_COVERS_HOST2_TIER*).
Now each host defines what it wants run when IT goes down (FALLBACK_HOST1_TIER*), and the
covering server reads the down host's conf via the RAM/persistent cache.
get_tier_containers() reads FALLBACK_${REMOTE_ID}_TIER* instead of
FALLBACK_${MY_ID}_COVERS_${REMOTE_ID}_TIER*. Tier data migrated to the correct host confs.
Writeback paths and delays were already REMOTE_ID-based — no change needed there.
stability_watchdog.sh mover check now uses platform_is_mover_running()
instead of raw pgrep, matching server_reboot.sh.
check_unraid_version_parity() renamed to check_os_version_parity() —
the function internals were already platform-neutral after the adapter
refactor; the name was the last Unraid artifact.
OS version reads now go through platform_get_os_version() and
platform_os_version_probe_cmd() instead of grepping /etc/unraid-version directly.
STATE_DIR fallbacks to /boot/config removed — STATE_DIR is always set by
load_config.sh and the fallback encoded a platform-specific path.
Setup DB path references use platform_setup_db_path() instead of the
VARAVERK_SETUP_FILE/-/boot/config compound fallback.
DOCKER_APPDATA_BASE default removed from arr_sync.sh — the adapter sets it.
All bash scripts are now platform-neutral. Unraid-specific paths, commands,
and service checks moved to Plugin/unraid/adapter.sh. Core scripts call
platform_*() functions exclusively — no direct OS paths in runtime logic.
New adapter functions: platform_storage_path, platform_webui_install_path,
platform_scripts_dir_probe_cmd, platform_setup_db_path, platform_storage_healthy,
platform_is_service_enabled, platform_get_temp_thresholds, platform_disk_states_path,
platform_rebuild_container, platform_push_conf, platform_push_setup_state,
platform_get_templates_dir, platform_send_os_notification.
Partnership services stack (Emby/Jellyfin/Seerr/SeerrFin) added as third
onboarding stack alongside auth and arr stacks.
- Move SCRIPTS_DIR/DATA_DIR/STATE_DIR from appdata to /boot/config/plugins/varaverk
- All state files now in STATE_DIR (no more /tmp or /boot/config root writes)
- Bootstrap: Gitea-first clone with GitHub fallback, no array dependency
- varaverk.cfg seeded with Gitea connection settings
- .gitignore: add State_Files/, varaverk.cfg, varaverk-*.txz
- Partnership/transcode/fallback scripts use STATE_DIR variables
- PHP config.php: DATA_DIR/STATE_DIR constants, VV_SETUP_STATE_FILE dynamic
- deploy.sh PROD_ROOT updated to plugin flash dir
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
appdata-Failover → appdata-Fallback:
All .md files updated — scripts and confs already used appdata-Fallback.
Share was renamed; docs were the only stale references.
critical_sync_maintenance.sh: */15 → */30, 96/day → 48/day:
critical_sync_maintenance.sh header, master.conf, host1/2.conf comments,
Orchestrators README, User Script Plug-in README, Partnership README/manual/script,
Docker_Essentials README/downloaders_reset.sh, Fallback README,
weekly_sync_maintenance.sh, user_script_plug-in.sh all updated.
rsync_emby_failover User Script superseded:
Emby dirty sync was already in HOST1_CRITICAL_SYNC_SHARES — it now runs inside
critical_sync_maintenance.sh every 30min alongside auth sync and partnership check.
Section in User Script Plug-in README marked for deletion from plugin.
transcode_management.sh: */3 → */7:
transcode_management.sh, transcode_manager.sh, transcode_cleanup.sh headers,
Transcodes README/manual, Orchestrators README, User Script Plug-in README,
user_script_plug-in.sh all updated.
array_start.sh → array_started.sh (bulk rename — actual file has always been
array_started.sh; docs consistently used the wrong name across 9 files)
failover.sh → fallback.sh (bulk rename — script is Fallback/fallback.sh;
docs used the old name throughout)
FAILOVER_HOST2_RUNS_FOR_HOST1_TIER1 → FALLBACK_HOST2_COVERS_HOST1_TIER1
(README.md — wrong variable name, actual vars use FALLBACK_ prefix and
COVERS_ not RUNS_FOR_)
docker_watchdog.sh attribution: "started by array_started.sh" → "called every
minute by watchdog_orchestrator.sh" (docker_watchdog is not in ARRAY_START_SCRIPTS)
server_reboot.sh shutdown sequence: step 7 "rc.docker stop" → calls
array_stopping.sh (user_scripts, fallback, rsync, mover, docker_container_stop)
Rsync manual ARRAY_START_SCRIPTS list: removed stability_watchdog and
docker_watchdog (now in watchdog_orchestrator); failover → fallback
arr_sync.sh bidirectional union replaced the old split-ownership model.
Old: HOST1 owns Movies/TV/Music, HOST2 owns Anime — each arr manages
different shares to avoid conflicts.
New: arr_sync.sh keeps all arr databases in union. Either server can
download to any share at any time. The union is the source of truth.
Auth stack references left intact — HOST1 still owns auth config
(one-way sync to HOST2). Only media library ownership language updated.
Files changed:
README.md — server block + share annotations
Fallback/README-Fallback.md — role line + Split Source of Truth section
Orchestrators/README-Orchestrators — daily sync share comments
Rsync/Manual-Rsync.md — DEFAULT_RSYNC_OPTS comment block
Media/README-Media.md — HOST AWARENESS section
22 scripts used docker without verifying it's present. Consistent with the
pattern already established in Docker_Essentials scripts. Added after acquire_lock
and before detect_hosts in each script's setup section.
- downloaders_reset: connection check block before slskd API sections;
triggers PUT /api/v0/server reconnect if disconnected, polls 60s,
gates Stuck Searches and Dead Transfer Records on SLSKD_CONNECTED
- Sync all modified/new/deleted files from v2 refactor across Docker_Essentials,
Media, Monitors, Partnership, Rsync, Tools, Transcodes, unRAID_Essentials,
common.sh, master confs, and new Manual/README docs
Fallback runs as a bare subprocess (cmdline: bash .../Fallback/fallback.sh),
not via /tmp/user.scripts, so user_scripts_stop.sh cannot find it.
Adding --stop mode closes that gap in the array shutdown sequence.
fallback.sh --stop: reads lock file (/tmp/unraid_locks/fallback.lock) to
get the running PID, sends SIGTERM (hits the existing trap that sets
FALLBACK_RUNNING=false), waits up to 10s, SIGKILLs if still alive.
master.conf: insert Fallback/fallback.sh --stop as step 2 in
ARRAY_STOP_SCRIPTS (after user_scripts_stop, before rsync_stop).
user_script_plug-in.sh: update ARRAY STOPPING block to show 5-step order
and add v2.5 changelog entry.
common.sh:
- Add resolve_tailscale_ip() helper — tries `tailscale ip -4` first, falls back to
parsing `tailscale status` output; handles hosts where MagicDNS short-name resolution
is not active
- Add PARTNERSHIP_OWN_CONTAINERS alias in detect_hosts()
- Add aliasing for 4 Emby provisioning vars (PARTNERSHIP_PROVISION_EMBY_ADMIN,
PARTNERSHIP_EMBY_ADMIN_USER, PARTNERSHIP_EMBY_ADMIN_PASS, PARTNERSHIP_EMBY_PORT)
Partnership/partnership_manager.sh:
- Replace 9 bare `tailscale ip -4` calls with resolve_tailscale_ip()
- Add read_remote_conf_var() and read_remote_conf_array() — SSH to mirror, source its
own load_config.sh + detect_hosts(), return aliased variable; solves sparse-checkout
problem where HOST1 cannot read master_host2.conf directly
- Add derive_short_name() — strips unraid- prefix, capitalises first char
- Add cleanup_partner_containers() — removes partner containers via FolderView3 folder
if enabled, else falls back to FALLBACK_*_COVERS_*_TIER* arrays
- Add cleanup_owner_containers_on_mirror() — SSH to mirror, stops and removes containers
matching *-${OWNER_SHORT} naming convention
- Add start_own_stack() and start_mirror_own_stack() — restart own containers locally
or on mirror via SSH using PARTNERSHIP_OWN_CONTAINERS
- Add provision_emby_admin() — reads mirror credentials via read_remote_conf_var, checks
for username collision, creates user + sets password + grants admin policy via Emby API
- Add revoke_emby_admin() — looks up mirror username on local Emby, deletes via REST API
- Wire offboard paths (both mirror-initiated and owner-initiated) to call container
cleanup and stack restart; update --check finalisation paths accordingly
- Fix write_state_file in --onboard not gated on DRY_RUN (was writing ACTIVE state on
dry runs)
master_host1.conf:
- Add HOST1_PARTNERSHIP_OWN_CONTAINERS array
- Add partnership Emby provisioning config (toggle + port + per-host credentials)
master_host2.conf:
- Add HOST2_PARTNERSHIP_OWN_CONTAINERS array
- Add HOST2_PARTNERSHIP_EMBY_ADMIN_USER and HOST2_PARTNERSHIP_EMBY_ADMIN_PASS
Tailscale fix applied to:
- Initial_run/ssh_setup.sh (2 callsites)
- unRAID_Essentials/rsync_stop.sh (1 callsite)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Arr sync (new):
- Media/arr_sync.sh — full mesh bidirectional sync across all HOST* nodes
- Lidarr (MusicBrainz), Sonarr (TVDB), Radarr (TMDB) all handled in one script
- Remote API keys read live from config.xml via SSH — never stored in conf files
- Shared blocklist (DATA_DIR/arr_sync_blocklist.tsv) merged from all nodes at runtime
- Graceful skip if arr not configured locally or not reachable on a remote node
- --blocklist-add / --blocklist-remove / --blocklist-list management flags
- daily_sync_maintenance.sh — arr sync runs as explicit phase before rsync
- partnership_onboard.sh — Step 3 bootstraps merged library on both sides at onboard
- master.conf — ARR_SYNC_* config block, DOCKER_APPDATA_BASE
Rsync / cleanup:
- DEFAULT_RSYNC_OPTS — removed --delete; arr_cleanup.sh owns orphan enforcement
- lidarr_cleanup.sh — removed HOST1-only guard; runs on any node with Lidarr configured
Config architecture:
- HOST1/HOST2 hostnames moved from master_host*.conf → master.conf (not credentials)
- Sparse checkout now works correctly: each server only needs its own host conf
- detect_hosts() still resolves MY_ID + REMOTE_ID via master.conf hostname values
Bug fix:
- common.sh line 493 — watchdog toggle eval had broken quoting; all SYS_WATCHDOG_CHECK_*
globals were silently set to empty instead of their configured values
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Gmer4Lfe