Commit Graph
621 Commits
Author SHA1 Message Date
Gmer4Lfe bfc3cbe204 Include WEEKLY_RESTART_CONTAINERS in remainder updates
docker_weekly_restart.sh restarts them but never pulls images — remainder
is the only update path they have.
2026-06-14 11:03:26 -04:00
Gmer4Lfe a5292c4bb7 Document which orchestrator calls each docker_update.sh mode 2026-06-14 10:59:48 -04:00
Gmer4Lfe fdef61cc25 Fix container updates: rebuild on new image, consolidate update scripts
Weekly sync window was pulling images but using docker start after rsync,
which ignores the new digest. Containers in the emby/critical-data profiles
(Emby, Mariadb, Redis) never actually landed on updated images.

docker_update_remaining.sh merged into docker_update.sh --remainder, which
already had better exclusion logic. Added WEEKLY_REMAINING_UPDATES toggle
and WEEKLY_RESTART_CONTAINERS exclusion to remainder mode.

Onboard now runs docker_network_connect.sh on mirror before deploying stacks.
2026-06-14 10:58:19 -04:00
Gmer4Lfe ba3eed39e3 Sync READMEs and manuals to current script headers
Fix discrepancies found by auditing docs against headers:
- stability_watchdog.sh: wrong variable name (STRIKES → STRIKE_LIMIT) and wrong default (2hr → 12hr) in header
- Watchdogs manual: REBOOT_WINDOW_HRS example value was 2, should be 12
- Transcodes README: transcode_cleanup/manager table said "Every 3 min", should be "Every 7 minutes"
- Tools manual: fallback_state_reset.sh still marked "not yet built" — rewrite section to reflect current script
- Tools manual: claude_startup --setup flag doesn't exist; modes were inverted — fix and add --launch
- Tools README + manual: add docker_prune_images.sh (existed but undocumented)
- Tools manual: add play_state_sync.sh --full to flag reference
- Orchestrators README: arrs_failed_stalled_recovery runs via intermediate_sync (every 4hr), not standalone every 6hr
- Plugin README: add build.sh to scripts table
- Partnership README + manual: add partnership_transfer.sh and onboard_cancel.sh to script tables and flag reference
2026-06-14 02:01:27 -04:00
Gmer4Lfe 420e13a746 Resolve bash variable references in vv_conf_vars()
PHP reads master.conf values like STATE_DIR="${SCRIPTS_DIR}/State_Files"
literally — the ${SCRIPTS_DIR} token is never expanded, so any PHP code
using the returned STATE_DIR value got an invalid path. Two-pass resolution:
SCRIPTS_DIR first (from PHP constant), then remaining ${VAR} tokens using
the now-resolved var set (covers DATA_DIR-based paths like ARR_SYNC_BLOCKLIST).
2026-06-14 01:46:51 -04:00
Gmer4Lfe 181114aed5 Header/code audit fixes across all core scripts
Behavior fixes:
- docker_daily_restart.sh: implement RESTART_VERIFY_WAIT sleep before
  verify_running() — variable was logged as config but never applied
- cert_monitor.sh: --log now shows expiry date and days remaining for
  healthy domains in the per-domain summary; problems always show details

Header corrections:
- downloaders_reset.sh: fix frequency contradiction (15 min → 30 min)
- lidarr_cleanup.sh: Emby scan triggers on actual deletions, not every run
- ssh_setup.sh: document --local-only flag in USAGE section
- zfs_memory_snapshot.sh: clarify dry-run skips log write in PURPOSE
- ramdisk_stop.sh / ramdisk_setup.sh: document TRANSCODE_STATE_FILE var
- smart_long_test.sh: remove SMART_TEMP_WARN/CRIT — not used in code
2026-06-14 01:40:35 -04:00
Gmer4Lfe b4d6866f40 Polish before first GitHub push: consistent adapter usage and naming
stability_watchdog.sh mover check now uses platform_is_mover_running()
instead of raw pgrep, matching server_reboot.sh.

check_unraid_version_parity() renamed to check_os_version_parity() —
the function internals were already platform-neutral after the adapter
refactor; the name was the last Unraid artifact.
2026-06-14 01:30:16 -04:00
Gmer4Lfe ee5a07be1c Remove remaining OS-specific hardcodes from core scripts
OS version reads now go through platform_get_os_version() and
platform_os_version_probe_cmd() instead of grepping /etc/unraid-version directly.

STATE_DIR fallbacks to /boot/config removed — STATE_DIR is always set by
load_config.sh and the fallback encoded a platform-specific path.

Setup DB path references use platform_setup_db_path() instead of the
VARAVERK_SETUP_FILE/-/boot/config compound fallback.

DOCKER_APPDATA_BASE default removed from arr_sync.sh — the adapter sets it.
2026-06-14 01:22:02 -04:00
Gmer4Lfe 3c900ae5c1 Resolve remote storage path from host*.conf for true cross-platform SSH support
detect_hosts() now sets REMOTE_STORAGE_PATH from HOST*_STORAGE_PATH in the
remote's conf instead of calling platform_storage_path() locally. SSH commands
that check the remote's storage (check_remote_array, check_remote_disks,
check_both_healthy) use $REMOTE_STORAGE_PATH so they work correctly when the
remote is a different platform with a different storage root.
2026-06-14 01:09:53 -04:00
Gmer4Lfe 27bfc21cb0 Platform-agnostic refactor: eliminate OS-specific hardcodes from core scripts
All bash scripts are now platform-neutral. Unraid-specific paths, commands,
and service checks moved to Plugin/unraid/adapter.sh. Core scripts call
platform_*() functions exclusively — no direct OS paths in runtime logic.

New adapter functions: platform_storage_path, platform_webui_install_path,
platform_scripts_dir_probe_cmd, platform_setup_db_path, platform_storage_healthy,
platform_is_service_enabled, platform_get_temp_thresholds, platform_disk_states_path,
platform_rebuild_container, platform_push_conf, platform_push_setup_state,
platform_get_templates_dir, platform_send_os_notification.

Partnership services stack (Emby/Jellyfin/Seerr/SeerrFin) added as third
onboarding stack alongside auth and arr stacks.
2026-06-14 00:59:19 -04:00
Gmer4Lfe fd1c4c7e3a Add CONF_SYNC_ENABLED to master.conf.template so nightly upgrade doesn't strip it 2026-06-13 13:30:40 -04:00
Gmer4Lfe ba1259c05c Add auth stack conf vars and storage mode selection to setup wizard
host.conf.template: add AUTH STACK section (NPM/lldap/Authelia) so
conf_upgrade.sh stops stripping those keys from host*.conf at 1am.

setup wizard: storage mode selector auto-detects USB vs NVMe at first
run; user can override; triggers storage_migrate.sh when mode differs
from current SCRIPTS_DIR location.

claude_startup.sh: reads HOST*_STORAGE_MODE_INTERNAL to pick internal
or appdata path rather than always requiring array to be mounted.
2026-06-13 13:05:46 -04:00
Gmer4Lfe 60ff658181 Fix conf_upgrade template naming bugs that caused false ADDED/REMOVED on every nightly pull
- REMOTE_ID_TIER*_DELAY renamed to HOSTN_TIER*_DELAY — these are own-host outage timers
- HOSTN_RADARR_MOVIE_ROOT → HOSTN_RADARR_MOVIES_ROOT (matches common.sh)
- HOSTN_SYS_WATCHDOG_CHECK_DOCKER → CHECK_DOCKER_DAEMON, CHECK_TEMP → CHECK_CPU_TEMP
- Add REMOTE_ID substitution to the sed in git_pull_execute.sh so COVERS_REMOTE_ID
  resolves to the actual remote host (e.g. FALLBACK_HOST1_COVERS_HOST2_TIER1)
- Add HOSTN_PARTNERSHIP_REPLACE_CONTAINERS, HOSTN_PARTNERSHIP_ARR_REPLACE_CONTAINERS,
  and HOSTN_WATCHDOG_CONTAINER_API_CHECKS to template so they aren't removed each pull
2026-06-13 10:22:02 -04:00
Gmer4Lfe 1bc76f8490 Align host template to unified section structure
Section order now matches master.conf: IDENTITY → PARTNERSHIP → RSYNC →
FALLBACK → DOCKER → MEDIA → ARR STACK → TRANSCODES → MONITORS →
RESOURCE MANAGER → SYSTEM WATCHDOG. Extracted PARTNERSHIP from IDENTITY
into its own top-level section. PERSONAL SHARES moved to sit directly
after DAILY SYNC SHARES. NIC var moved to top of SYSTEM WATCHDOG block.
2026-06-12 22:09:54 -04:00
Gmer4Lfe f036fbfe4a Sync master.conf.template schema to match live master.conf
STATE_DIR introduced, DATA_DIR path updated to flash, all state files
migrated from /tmp and /boot/config root to $STATE_DIR. TRANSCODE_STATE_FILE
and DOCKER_WATCHDOG_FAILED_FILE added; SYS_WATCHDOG_FAILED_FILE renamed to
DOCKER_WATCHDOG_FAILED_FILE to reflect what it actually tracks.
2026-06-12 21:55:15 -04:00
Gmer4Lfe 1f766fef31 Sync host.conf.template schema to match host1.conf
Added HOSTN_PERSONAL_SHARES, HOSTN_TMDB_API_KEY, and 8 missing system
watchdog check toggles (KERNEL_OOPS, BOOT, OOM, ARC, CONTAINERS, MDSTAT,
NETWORK, SSHD). Reordered watchdog checks to reflect tier structure from
host1.conf. Remaining template/live diffs are intentional renames
(CPU_TEMP→TEMP, DOCKER_DAEMON→DOCKER, MOVIES_ROOT→MOVIE_ROOT, TIER*_DELAY→REMOTE_ID).
2026-06-12 21:54:18 -04:00
Gmer4Lfe f10f09eeb1 Bug fixes, arr cleanup API-driven scan, shared JS formatters
Bug fixes:
- fallback.sh: escape sed metacharacters (\ & |) in state_set values
- common.sh: parse PID from lock file content correctly (handles pid:metadata format)
- unraid_api_key_renew.sh: fix path depth (../../../) and sync registry key to conf when stale
- stop.php: only clear pid/status if process is actually dead — D-state survives SIGKILL
- array_started.sh: check ARRAY_START_SCRIPTS empty before printing launch header

Arr cleanup:
- radarr_cleanup.sh / sonarr_cleanup.sh: fetch root folders from arr API instead of
  reverse-looking up the path map — handles multi-root-folder setups correctly

UI:
- varaverk.js: extract shared formatters (_relTime, _fmtBytes, _sz, _uptime, _gb, _tb, _n)
- arrs.php / partnership.php: use shared formatters, remove duplicates
- arrs.php / fallback.php: show error message on fetch failure instead of silent empty
- docker.php: disable rename input during request, restore original value on failure
- setup.php: abort controller timeout on detect fetch
- partnership.php: remove Re-run Phase 2 button opacity dimming
2026-06-12 21:41:11 -04:00
Gmer4Lfe 3937d5e33b Consolidate conf templates into Configurations/ — remove stale Deployment/conf_templates/
Deployment/conf_templates/ had two stale files: host.conf.template was a copy of host1.conf
(not a HOSTN template), and master.conf had hardcoded hostnames and an old appdata DATA_DIR path.
The correct templates already exist in Configurations/ (master.conf.template, host.conf.template).

Updated varaverk.plg and git_pull_execute.sh to read from Configurations/ directly.
Removed TMPL_DIR variable from git_pull_execute.sh — CONF_DIR covers both.
2026-06-12 21:35:58 -04:00
Gmer4Lfe 129e8ae14a Fix Emby HTTP 500 on PlayedItems — convert DatePlayed to yyyyMMddHHmmss
Emby's API spec requires DatePlayed as yyyyMMddHHmmss; sending ISO 8601 caused 500 on every mark-played call.
2026-06-11 19:22:51 -04:00
Gmer4Lfe ca0d17ec2a All rsync flags re-enabled — HOST2 rebuild complete 2026-06-07 10:12:20 -04:00
Gmer4Lfe 30e7c2f6b0 Detect containers running behind already-pulled images
OLD_ID vs NEW_ID only caught updates landed during this run. If an image
was pulled externally (Unraid UI), the container stayed on the old image
while :latest moved on — this run saw nothing to do. Now also compare
the container's current image ID against :latest and rebuild if behind.
2026-06-07 09:57:00 -04:00
Gmer4Lfe 819f5e6450 Remove old images by ID after container updates, not just dangling prune
Tagged images are never caught by docker image prune -f. Collect OLD_IDs
during the pull loop and explicitly rmi them after rebuilds complete.
2026-06-07 09:54:42 -04:00
Gmer4Lfe 1e54752478 Fix digest falsely counting = style state entries as system watchdog strikes
grep -v ':0$' was picking up kernel_oops_count=00, mdstat_errors=0, and
watchdog_cycle as active strikes. Restrict to : separator lines only.
2026-06-07 09:39:16 -04:00
Gmer4Lfe 80b1ee4f0d Fix rsync wrapper hanging forever after sync completes
exec 1>&- closed stdout but left stderr holding the tee pipe open,
so tee never got EOF and wait blocked indefinitely — keeping the
process alive and the locks held until manually killed.
2026-06-07 09:27:38 -04:00
Gmer4Lfe af313352c0 arr_sync: enforce monitored on every run — bulk re-monitor unmonitored items before sync 2026-06-06 18:35:41 -04:00
Gmer4Lfe aa92511ebc arr_sync: always add with monitored:true — don't propagate partner's unmonitored state 2026-06-06 18:31:36 -04:00
Gmer4Lfe cac9d8e82f Fix Sonarr episode count — episodeFileCount is nested under statistics in v4 2026-06-06 18:24:31 -04:00
Gmer4Lfe a24f9db2ff Remove orphaned API key code — correct key name format, collapse duplicate buttons 2026-06-06 18:17:30 -04:00
Gmer4Lfe dab8d62e95 Fix API key detection — name is 'Varaverk <hostname>', not 'Varaverk' 2026-06-06 18:09:37 -04:00
Gmer4Lfe a535319f1d API key button delegates to unraid_api_key_renew.sh — checks registry before creating 2026-06-06 18:02:37 -04:00
Gmer4Lfe bd73e87bf9 Add CONF_SYNC_ENABLED gate — skip partner sync when HOST2 is offline 2026-06-06 16:37:05 -04:00
Gmer4Lfe 273a2973e0 Ignore vanished-file chmod errors in Temp_Storage — race condition with Emby transcodes 2026-06-06 16:26:26 -04:00
Gmer4Lfe 57773bcaf5 Rsync sync log: stream live output from tee'd log file, preserve last run 2026-06-06 16:06:42 -04:00
Gmer4Lfe d05c5ac198 Sync activity card: fill full height instead of capping at 200px 2026-06-06 15:57:13 -04:00
Gmer4Lfe 04cd15a22f Fix rsync page: add missing _esc helper that broke all card rendering 2026-06-06 15:55:31 -04:00
Gmer4Lfe 4bb6bbf009 Rsync page: add live sync activity log card left of settings 2026-06-06 15:48:55 -04:00
Gmer4Lfe bac0f7c535 Auth page: pure-PHP Authelia ACL parser, NPM-sourced certs tab, watchdog JS fixes, rsync settings layout
- Replace python3/PyYAML Authelia ACL parser with pure PHP (no deps available on Unraid)
- Cert tab now pulls live from NPM API instead of cert_monitor.sh — auto-discovers all managed certs sorted by urgency
- Watchdog page: add missing GB constant and _fmtBytes/_relTime functions that were causing silent render failure
- Rsync settings card: pin to far-right 3 columns (grid-column:6/-1), toggle grid narrowed to 2 columns
- Add CLAUDE.md project context file on /boot for session persistence across reboots
- claude_startup.sh: symlink CLAUDE.md into /root on array start
2026-06-06 15:39:34 -04:00
Gmer4Lfe 9c3ace95a7 Auth stack certs tab, arrs db fallbacks, cert monitor cache, conf parser fix
- Auth stack: fold cert monitor into Auth Stack page as fourth tab (Certs);
  remove standalone cert page and top-level tab
- cert_monitor.sh: write JSON status cache to State_Files/cert_status.json
  after each run; expose per-domain days/expiry via _CERT_DAYS/_CERT_EXPIRY globals
- api/cert.php: new — serves cached cert status; falls back to configured
  domains as UNKN when no cache exists; POST action=run triggers live check
- arrs db fallbacks: vv_arr_cleanup_stats/discovery_stats/recovery_stats now
  read from data/*.db files when log JSON files don't yet exist
- config.php vv_conf_vars(): unescape bash \$ → $ so passwords with dollar
  signs read correctly from conf files
- host1.conf: fill in HOST1_NPM_USER/PASS and HOST1_LLDAP_USER/PASS
- Partnership adapter pattern: Unraid-specific container logic extracted to
  Plugin/unraid/Partnership/; platform-agnostic structure stays in Partnership/
- First-run wizard: uniform multi-step flow for all hosts; HOST2 pull moved
  to checklist; auto SSH keygen and API key creation on save
- api/checklist.php: live setup checklist with pull_master action
- Fullscreen toggle: hide Unraid header/menu; state persists via localStorage
2026-06-05 23:17:30 -04:00
Gmer4Lfe 17012bcd8c refactor: move unraid-specific tools into Plugin/unraid/tools/
api_cache_writer, remote_arr_cache_writer, conf_populate, storage_migrate are
all Unraid-adapter-specific — they read Docker appdata paths, Unraid service
configs, and drive the Unraid web plugin cache. They don't belong in Tools/.

- Tools/{4 scripts + php} → Plugin/unraid/tools/
- Fix source depth: ../ → ../../../ (now 3 levels from repo root)
- Fix conf_populate SCRIPTS_ROOT bug: was never set, would expand to empty
- api/arrs.php, api/storage.php: SCRIPTS_DIR/Tools/ → dirname(__DIR__)/tools/
- schedule.json: key IDs updated to Plugin/unraid/tools/...
- Cron rebuilt — live entries updated immediately
2026-06-05 20:04:51 -04:00
Gmer4Lfe bdcdfea175 feat: build smart_long_test.sh — SMART extended self-test for monthly maintenance
Tests all drives sequentially (one at a time to avoid I/O saturation).
Polls every 60s until complete. Notifies on failure. Respects SMART_IGNORE_DRIVES.
Enable by uncommenting in MONTHLY_MAINTENANCE_SCRIPTS in master.conf.

zfs_pool_scrub.sh was already fully implemented — also uncommented locally.
2026-06-05 19:53:15 -04:00
Gmer4Lfe c441ac443a feat: add --pull-only to conf_sync, wire into intermediate orchestrator
Partner conf cache (/tmp/.vv/) was only refreshed at array start. If a partner
updated their conf mid-day, this host's RAM cache went stale until next reboot.
Intermediate orchestrator now pulls partner confs every 4 hours as step 1.

conf_sync.sh --pull-only: pulls partner confs into local cache, skips push
(push is already handled by conf_populate.sh --push-only on conf save).
2026-06-05 19:44:10 -04:00
Gmer4Lfe f9fe13bdc1 fix: use STATE_DIR directly — TRANSCODE_STATE_FILE contains unexpanded bash var
vv_conf_vars() reads conf as text; TRANSCODE_STATE_FILE=$STATE_DIR/... is
returned as a literal string, not expanded. Build path from STATE_DIR directly.
2026-06-05 19:32:48 -04:00
Gmer4Lfe 415c260175 fix: transcode monitor reads STATE_DIR from conf instead of hardcoded /tmp
vv_transcode_sessions() was hardcoded to /tmp/transcode_state.db but master.conf
sets TRANSCODE_STATE_FILE=$STATE_DIR/transcode_state.db which resolves to
/boot/config/plugins/varaverk/State_Files/transcode_state.db. Page always showed
"ramdisk_setup.sh not yet run" despite state file being present and populated.
2026-06-05 19:29:31 -04:00
Gmer4Lfe 437c525fbd git pull: replace reset --hard + clean -fd with --ff-only 2026-06-05 19:24:48 -04:00
Gmer4Lfe a5936331c9 auth stack page, monitor cache fix, symlink install 2026-06-05 19:20:28 -04:00
Gmer4Lfe d9ed8c7704 claude_startup: flip default to setup-only, add --launch flag
Default behavior is now setup-only (symlinks + binary, no interactive launch)
so array_started.sh can call it directly without arguments. Use --launch to
start an interactive Claude session.
2026-06-05 16:04:53 -04:00
Gmer4Lfe 0ded0c87a1 arr_sync: cache-first remote API calls, SSH fallback
Remote functions (_remote_arr_up, _remote_library, _remote_defaults,
_remote_add, _delete_remote_item) now check ${node_id}_<ARR>_API_KEY
from the environment first. When found (conf cache populated by
conf_sync.sh + load_config.sh sourcing /tmp/.vv/), they call the arr
API directly over Tailscale — no SSH connection needed.

SSH fallback (grep config.xml + localhost API) remains for the first
boot before conf_sync has run or when the partner is offline.

config_xml is no longer passed between functions — computed internally
in the SSH path only. Call sites in _sync_arr and blocklist-add updated
accordingly.
2026-06-04 22:20:38 -04:00
Gmer4Lfe d70c682be2 load_config: source partner confs from /tmp cache
conf_sync.sh has always populated /tmp/.vv/config/cached/.confs/ with
partner host*.conf files pulled via SCP, but load_config.sh never read
from it — leaving HOST2_* vars undefined on HOST1 and vice versa.

After the disk host*.conf loop, source any cached conf whose basename
wasn't already loaded from disk. Disk copy always wins (authoritative);
cache supplies partner vars that sparse checkout intentionally withholds.
Cache is tmpfs, cleared on reboot, repopulated by conf_sync.sh on array
start.
2026-06-04 22:09:20 -04:00
Gmer4Lfe 0b3d7d542f editor: add word wrap toggle to status bar
Adds a "Wrap" button to the editor status bar that toggles word wrap
on/off, persisted in localStorage. Default is nowrap with horizontal
scroll (standard code editor behaviour). In wrap mode the line-number
gutter and current-line highlight are hidden — both rely on fixed
line-height calculations that break when lines wrap visually.

CSS: textarea/overlay default to white-space:pre; .vv-ed-wrap class
(on #vv-editor-wrap) overrides to pre-wrap, hides gutter and cur-line.
JS: vvWordWrap state, vvApplyWordWrap(), vvToggleWordWrap(); guards in
vvUpdateLineNums() and vvUpdateCurLine(); restored via
vvRestoreEditorPrefs() on both script and raw-conf editor entry points.
2026-06-04 21:56:22 -04:00
Gmer4Lfe c1919febc2 Monitor: fix memory bar ZFS field name zfs_kb → arc_kb 2026-06-04 21:40:07 -04:00