Writes partner confs from RAM cache to /boot/config/.cache/vv/d/ while
remote is offline, and removes the backup when remote comes back. Called
each minute via SYSTEM_WATCHDOG_SCRIPTS so crashes and power loss are
covered — not just graceful shutdowns.
All FALLBACK_${MY_ID}_COVERS_${REMOTE_ID}_TIER* references updated to
FALLBACK_${REMOTE_ID}_TIER* across fallback_test.sh, partnership_manager.sh,
docker_update.sh, mesh_monitor.sh, and monitor.php. mesh_monitor.sh drops
the inner covering-host loop — tier data now lives in the covered host's own
conf so no cross-host scan is needed. monitor.php reads from the covered
host's conf file rather than the local host's.
Previously HOST1 defined what it would run for HOST2 (FALLBACK_HOST1_COVERS_HOST2_TIER*).
Now each host defines what it wants run when IT goes down (FALLBACK_HOST1_TIER*), and the
covering server reads the down host's conf via the RAM/persistent cache.
get_tier_containers() reads FALLBACK_${REMOTE_ID}_TIER* instead of
FALLBACK_${MY_ID}_COVERS_${REMOTE_ID}_TIER*. Tier data migrated to the correct host confs.
Writeback paths and delays were already REMOTE_ID-based — no change needed there.
conf_cache_save.sh runs first on array stop — snapshots partner confs from RAM
cache to /boot/config/varaverk/conf_bak/ before anything else shuts down.
conf_cache_restore.sh runs after conf_sync.sh on array start — if partner was
unreachable and RAM cache is incomplete, loads the backup into RAM then removes
it. Normal reboots: backup written, fresh pull succeeds, backup deleted unused.
Edge case (partner down at boot): backup fills the gap so fallback.sh has the
partner vars it needs to operate correctly.
Web files now served via symlink to the git repo so git pull changes survive
reboots without rebuilding the txz. Also includes: docker pull/rebuild/restart
with live log streaming, arr_profile_enforcer for Sonarr/Radarr quality
profiles, monitor page cache fix (background writer now in cron), and
ARR_KIDS/SONARR/RADARR profile name vars in master.conf.
Adds SSH key (hostname convention), arr path maps (docker volume mounts),
Authelia container + config path, boot device storage mode detection,
and master.conf HOST identity + Gitea container. Also fixes RADARR_MOVIE_ROOT
→ RADARR_MOVIES_ROOT to match the variable name used by all other scripts.
Live TV and Direct Stream sessions never appear in the transcoding
session list but do write segments to the ramdisk. Card now shows
the active segment count so the ramdisk usage is explained.
HLS segments are written atomically so lsof never sees them as open.
Cleanup now shows "Active" count (too-young files) which correctly
reflects Live TV / Direct Stream segments. Manager session count now
counts flat file prefixes in addition to subdirs.
Weekly sync window was pulling images but using docker start after rsync,
which ignores the new digest. Containers in the emby/critical-data profiles
(Emby, Mariadb, Redis) never actually landed on updated images.
docker_update_remaining.sh merged into docker_update.sh --remainder, which
already had better exclusion logic. Added WEEKLY_REMAINING_UPDATES toggle
and WEEKLY_RESTART_CONTAINERS exclusion to remainder mode.
Onboard now runs docker_network_connect.sh on mirror before deploying stacks.
Fix discrepancies found by auditing docs against headers:
- stability_watchdog.sh: wrong variable name (STRIKES → STRIKE_LIMIT) and wrong default (2hr → 12hr) in header
- Watchdogs manual: REBOOT_WINDOW_HRS example value was 2, should be 12
- Transcodes README: transcode_cleanup/manager table said "Every 3 min", should be "Every 7 minutes"
- Tools manual: fallback_state_reset.sh still marked "not yet built" — rewrite section to reflect current script
- Tools manual: claude_startup --setup flag doesn't exist; modes were inverted — fix and add --launch
- Tools README + manual: add docker_prune_images.sh (existed but undocumented)
- Tools manual: add play_state_sync.sh --full to flag reference
- Orchestrators README: arrs_failed_stalled_recovery runs via intermediate_sync (every 4hr), not standalone every 6hr
- Plugin README: add build.sh to scripts table
- Partnership README + manual: add partnership_transfer.sh and onboard_cancel.sh to script tables and flag reference
PHP reads master.conf values like STATE_DIR="${SCRIPTS_DIR}/State_Files"
literally — the ${SCRIPTS_DIR} token is never expanded, so any PHP code
using the returned STATE_DIR value got an invalid path. Two-pass resolution:
SCRIPTS_DIR first (from PHP constant), then remaining ${VAR} tokens using
the now-resolved var set (covers DATA_DIR-based paths like ARR_SYNC_BLOCKLIST).
Behavior fixes:
- docker_daily_restart.sh: implement RESTART_VERIFY_WAIT sleep before
verify_running() — variable was logged as config but never applied
- cert_monitor.sh: --log now shows expiry date and days remaining for
healthy domains in the per-domain summary; problems always show details
Header corrections:
- downloaders_reset.sh: fix frequency contradiction (15 min → 30 min)
- lidarr_cleanup.sh: Emby scan triggers on actual deletions, not every run
- ssh_setup.sh: document --local-only flag in USAGE section
- zfs_memory_snapshot.sh: clarify dry-run skips log write in PURPOSE
- ramdisk_stop.sh / ramdisk_setup.sh: document TRANSCODE_STATE_FILE var
- smart_long_test.sh: remove SMART_TEMP_WARN/CRIT — not used in code
stability_watchdog.sh mover check now uses platform_is_mover_running()
instead of raw pgrep, matching server_reboot.sh.
check_unraid_version_parity() renamed to check_os_version_parity() —
the function internals were already platform-neutral after the adapter
refactor; the name was the last Unraid artifact.
OS version reads now go through platform_get_os_version() and
platform_os_version_probe_cmd() instead of grepping /etc/unraid-version directly.
STATE_DIR fallbacks to /boot/config removed — STATE_DIR is always set by
load_config.sh and the fallback encoded a platform-specific path.
Setup DB path references use platform_setup_db_path() instead of the
VARAVERK_SETUP_FILE/-/boot/config compound fallback.
DOCKER_APPDATA_BASE default removed from arr_sync.sh — the adapter sets it.
detect_hosts() now sets REMOTE_STORAGE_PATH from HOST*_STORAGE_PATH in the
remote's conf instead of calling platform_storage_path() locally. SSH commands
that check the remote's storage (check_remote_array, check_remote_disks,
check_both_healthy) use $REMOTE_STORAGE_PATH so they work correctly when the
remote is a different platform with a different storage root.
All bash scripts are now platform-neutral. Unraid-specific paths, commands,
and service checks moved to Plugin/unraid/adapter.sh. Core scripts call
platform_*() functions exclusively — no direct OS paths in runtime logic.
New adapter functions: platform_storage_path, platform_webui_install_path,
platform_scripts_dir_probe_cmd, platform_setup_db_path, platform_storage_healthy,
platform_is_service_enabled, platform_get_temp_thresholds, platform_disk_states_path,
platform_rebuild_container, platform_push_conf, platform_push_setup_state,
platform_get_templates_dir, platform_send_os_notification.
Partnership services stack (Emby/Jellyfin/Seerr/SeerrFin) added as third
onboarding stack alongside auth and arr stacks.
host.conf.template: add AUTH STACK section (NPM/lldap/Authelia) so
conf_upgrade.sh stops stripping those keys from host*.conf at 1am.
setup wizard: storage mode selector auto-detects USB vs NVMe at first
run; user can override; triggers storage_migrate.sh when mode differs
from current SCRIPTS_DIR location.
claude_startup.sh: reads HOST*_STORAGE_MODE_INTERNAL to pick internal
or appdata path rather than always requiring array to be mounted.
- REMOTE_ID_TIER*_DELAY renamed to HOSTN_TIER*_DELAY — these are own-host outage timers
- HOSTN_RADARR_MOVIE_ROOT → HOSTN_RADARR_MOVIES_ROOT (matches common.sh)
- HOSTN_SYS_WATCHDOG_CHECK_DOCKER → CHECK_DOCKER_DAEMON, CHECK_TEMP → CHECK_CPU_TEMP
- Add REMOTE_ID substitution to the sed in git_pull_execute.sh so COVERS_REMOTE_ID
resolves to the actual remote host (e.g. FALLBACK_HOST1_COVERS_HOST2_TIER1)
- Add HOSTN_PARTNERSHIP_REPLACE_CONTAINERS, HOSTN_PARTNERSHIP_ARR_REPLACE_CONTAINERS,
and HOSTN_WATCHDOG_CONTAINER_API_CHECKS to template so they aren't removed each pull
Section order now matches master.conf: IDENTITY → PARTNERSHIP → RSYNC →
FALLBACK → DOCKER → MEDIA → ARR STACK → TRANSCODES → MONITORS →
RESOURCE MANAGER → SYSTEM WATCHDOG. Extracted PARTNERSHIP from IDENTITY
into its own top-level section. PERSONAL SHARES moved to sit directly
after DAILY SYNC SHARES. NIC var moved to top of SYSTEM WATCHDOG block.
STATE_DIR introduced, DATA_DIR path updated to flash, all state files
migrated from /tmp and /boot/config root to $STATE_DIR. TRANSCODE_STATE_FILE
and DOCKER_WATCHDOG_FAILED_FILE added; SYS_WATCHDOG_FAILED_FILE renamed to
DOCKER_WATCHDOG_FAILED_FILE to reflect what it actually tracks.
Deployment/conf_templates/ had two stale files: host.conf.template was a copy of host1.conf
(not a HOSTN template), and master.conf had hardcoded hostnames and an old appdata DATA_DIR path.
The correct templates already exist in Configurations/ (master.conf.template, host.conf.template).
Updated varaverk.plg and git_pull_execute.sh to read from Configurations/ directly.
Removed TMPL_DIR variable from git_pull_execute.sh — CONF_DIR covers both.
OLD_ID vs NEW_ID only caught updates landed during this run. If an image
was pulled externally (Unraid UI), the container stayed on the old image
while :latest moved on — this run saw nothing to do. Now also compare
the container's current image ID against :latest and rebuild if behind.
exec 1>&- closed stdout but left stderr holding the tee pipe open,
so tee never got EOF and wait blocked indefinitely — keeping the
process alive and the locks held until manually killed.
- Replace python3/PyYAML Authelia ACL parser with pure PHP (no deps available on Unraid)
- Cert tab now pulls live from NPM API instead of cert_monitor.sh — auto-discovers all managed certs sorted by urgency
- Watchdog page: add missing GB constant and _fmtBytes/_relTime functions that were causing silent render failure
- Rsync settings card: pin to far-right 3 columns (grid-column:6/-1), toggle grid narrowed to 2 columns
- Add CLAUDE.md project context file on /boot for session persistence across reboots
- claude_startup.sh: symlink CLAUDE.md into /root on array start
- Auth stack: fold cert monitor into Auth Stack page as fourth tab (Certs);
remove standalone cert page and top-level tab
- cert_monitor.sh: write JSON status cache to State_Files/cert_status.json
after each run; expose per-domain days/expiry via _CERT_DAYS/_CERT_EXPIRY globals
- api/cert.php: new — serves cached cert status; falls back to configured
domains as UNKN when no cache exists; POST action=run triggers live check
- arrs db fallbacks: vv_arr_cleanup_stats/discovery_stats/recovery_stats now
read from data/*.db files when log JSON files don't yet exist
- config.php vv_conf_vars(): unescape bash \$ → $ so passwords with dollar
signs read correctly from conf files
- host1.conf: fill in HOST1_NPM_USER/PASS and HOST1_LLDAP_USER/PASS
- Partnership adapter pattern: Unraid-specific container logic extracted to
Plugin/unraid/Partnership/; platform-agnostic structure stays in Partnership/
- First-run wizard: uniform multi-step flow for all hosts; HOST2 pull moved
to checklist; auto SSH keygen and API key creation on save
- api/checklist.php: live setup checklist with pull_master action
- Fullscreen toggle: hide Unraid header/menu; state persists via localStorage
Tests all drives sequentially (one at a time to avoid I/O saturation).
Polls every 60s until complete. Notifies on failure. Respects SMART_IGNORE_DRIVES.
Enable by uncommenting in MONTHLY_MAINTENANCE_SCRIPTS in master.conf.
zfs_pool_scrub.sh was already fully implemented — also uncommented locally.
Partner conf cache (/tmp/.vv/) was only refreshed at array start. If a partner
updated their conf mid-day, this host's RAM cache went stale until next reboot.
Intermediate orchestrator now pulls partner confs every 4 hours as step 1.
conf_sync.sh --pull-only: pulls partner confs into local cache, skips push
(push is already handled by conf_populate.sh --push-only on conf save).
vv_conf_vars() reads conf as text; TRANSCODE_STATE_FILE=$STATE_DIR/... is
returned as a literal string, not expanded. Build path from STATE_DIR directly.
vv_transcode_sessions() was hardcoded to /tmp/transcode_state.db but master.conf
sets TRANSCODE_STATE_FILE=$STATE_DIR/transcode_state.db which resolves to
/boot/config/plugins/varaverk/State_Files/transcode_state.db. Page always showed
"ramdisk_setup.sh not yet run" despite state file being present and populated.
Default behavior is now setup-only (symlinks + binary, no interactive launch)
so array_started.sh can call it directly without arguments. Use --launch to
start an interactive Claude session.
Remote functions (_remote_arr_up, _remote_library, _remote_defaults,
_remote_add, _delete_remote_item) now check ${node_id}_<ARR>_API_KEY
from the environment first. When found (conf cache populated by
conf_sync.sh + load_config.sh sourcing /tmp/.vv/), they call the arr
API directly over Tailscale — no SSH connection needed.
SSH fallback (grep config.xml + localhost API) remains for the first
boot before conf_sync has run or when the partner is offline.
config_xml is no longer passed between functions — computed internally
in the SSH path only. Call sites in _sync_arr and blocklist-add updated
accordingly.
conf_sync.sh has always populated /tmp/.vv/config/cached/.confs/ with
partner host*.conf files pulled via SCP, but load_config.sh never read
from it — leaving HOST2_* vars undefined on HOST1 and vice versa.
After the disk host*.conf loop, source any cached conf whose basename
wasn't already loaded from disk. Disk copy always wins (authoritative);
cache supplies partner vars that sparse checkout intentionally withholds.
Cache is tmpfs, cleared on reboot, repopulated by conf_sync.sh on array
start.
Adds a "Wrap" button to the editor status bar that toggles word wrap
on/off, persisted in localStorage. Default is nowrap with horizontal
scroll (standard code editor behaviour). In wrap mode the line-number
gutter and current-line highlight are hidden — both rely on fixed
line-height calculations that break when lines wrap visually.
CSS: textarea/overlay default to white-space:pre; .vv-ed-wrap class
(on #vv-editor-wrap) overrides to pre-wrap, hides gutter and cur-line.
JS: vvWordWrap state, vvApplyWordWrap(), vvToggleWordWrap(); guards in
vvUpdateLineNums() and vvUpdateCurLine(); restored via
vvRestoreEditorPrefs() on both script and raw-conf editor entry points.
common.sh gains:
- docker_cmd() + verify_running() + retry_docker() — removed from all 3 Docker_Essentials
scripts where they were byte-for-byte duplicates
- emby_api(endpoint, [timeout=30]) — removed from 6 Media/Tools scripts that each defined
their own _emby_api() with the same curl/parse/error pattern; call sites renamed emby_api
- format_duration() extended with days/hours branch (was capped at minutes+seconds)
- notify() comment: scripts do not need to preflight the notify script via validate_unraid_cmd
Tailscale deduplication:
- arr_sync.sh: _resolve_node_ip() and inline block in _delete_remote_item() both replaced
with resolve_tailscale_ip() from common.sh
- git_pull_execute.sh: inline tailscale ip -4 replaced with resolve_tailscale_ip() (adds
the tailscale status fallback that was missing)
- ARRAY_START_SCRIPTS: add api_cache_writer.sh so monitor/arrs cache is populated
immediately on array start (/tmp is tmpfs — cleared on reboot, so first-boot
load was hitting live API for up to 60 seconds until the cron fired)
- api/monitor.php + arrs.php: raise cache TTL from 90s to 300s — stale-but-instant
beats a 6-second live API wait if the writer is momentarily behind
- include/unraid_api.php: replace last inline uptime formatter with vv_format_uptime()
Consolidations (config.php gains 5 shared utilities):
- vv_format_uptime() replaces 4 inline uptime-formatting blocks
- vv_parse_conf_scalar() replaces vv_arr_scalar/vv_wd_scalar/vv_fb_scalar/vv_media_conf_scalar
- vv_known_hosts() replaces vv_arr_known_hosts/vv_fb_known_hosts + inline parser in watchdog
- vv_parse_kv_db() replaces inline key=value parsing in snapshot and monitor
- vv_local_ip() replaces duplicate in docker_folders.php and inline in docker.php
All module-level function names kept as thin aliases so call sites unchanged.
Critical bug fixes:
- api/system.php: added require_once config.php and POST-only guard (no auth on shutdown)
- api/movescript.php + reorderarray.php: use vv_write_conf_raw (atomic) + vv_push_master_conf
- api/snapshot.php: share /tmp/vv_cpu_stat.json with vv_cpu_per_core() instead of own state file
Correctness:
- vv_cpu_per_core() and vv_network_stats(): atomic tmp+rename for state files (concurrent poll safety)
- ext_ip curl cache moved from /tmp/vv_ext_ip.cache to vv_cache_read/write (canonical cache dir)
- monitor_remote.php + board.php + snapshot.php: all use vv_cache_read/write instead of ad-hoc /tmp files
HTTP method guards added to write-only APIs that were missing them:
- api/scheduler.php, conf_toggle.php, flag_toggle.php
Split slow cached monitor endpoint from the live stats. monitor_fast.php
reads /proc/stat, /proc/meminfo, ZFS arcstats, and /proc/net/dev directly
— no cache wrapper, 87ms response. Docker/vm/swap pulled from last full
cache so mem card stays complete. Full monitor poll stays at 2s for
everything else (GPU, containers, storage, etc).
unRAID API rejects underscores and hyphens in key names — only letters, numbers,
and spaces allowed. Varaverk_HOST1 / Varaverk_Gmer4Lfe both fail; now uses
"Varaverk <hostname>" (space). Also adds monitor_remote.php, conf_populate.sh,
and conf_sync.sh from dev branch that were missing from production.
Gmer4Lfe