Commit Graph
824 Commits
Author SHA1 Message Date
Gmer4Lfe df18657183 Fix transcode session detection for Live TV and Direct Stream
HLS segments are written atomically so lsof never sees them as open.
Cleanup now shows "Active" count (too-young files) which correctly
reflects Live TV / Direct Stream segments. Manager session count now
counts flat file prefixes in addition to subdirs.
2026-06-14 11:24:21 -04:00
Gmer4Lfe a2bcd58f78 Add --weekly mode to docker_update.sh for WEEKLY_RESTART_CONTAINERS
Weekly now has the same update→restart pattern as daily. Remainder
excludes weekly containers since they're now owned by this new mode.
2026-06-14 11:13:23 -04:00
Gmer4Lfe 55f8842dbd Move remainder container updates from weekly to monthly
Daily updates daily containers, weekly updates its sync-window containers,
monthly now handles everything else via MONTHLY_MAINTENANCE_SCRIPTS.
2026-06-14 11:07:18 -04:00
Gmer4Lfe bfc3cbe204 Include WEEKLY_RESTART_CONTAINERS in remainder updates
docker_weekly_restart.sh restarts them but never pulls images — remainder
is the only update path they have.
2026-06-14 11:03:26 -04:00
Gmer4Lfe a5292c4bb7 Document which orchestrator calls each docker_update.sh mode 2026-06-14 10:59:48 -04:00
Gmer4Lfe fdef61cc25 Fix container updates: rebuild on new image, consolidate update scripts
Weekly sync window was pulling images but using docker start after rsync,
which ignores the new digest. Containers in the emby/critical-data profiles
(Emby, Mariadb, Redis) never actually landed on updated images.

docker_update_remaining.sh merged into docker_update.sh --remainder, which
already had better exclusion logic. Added WEEKLY_REMAINING_UPDATES toggle
and WEEKLY_RESTART_CONTAINERS exclusion to remainder mode.

Onboard now runs docker_network_connect.sh on mirror before deploying stacks.
2026-06-14 10:58:19 -04:00
Gmer4Lfe ba3eed39e3 Sync READMEs and manuals to current script headers
Fix discrepancies found by auditing docs against headers:
- stability_watchdog.sh: wrong variable name (STRIKES → STRIKE_LIMIT) and wrong default (2hr → 12hr) in header
- Watchdogs manual: REBOOT_WINDOW_HRS example value was 2, should be 12
- Transcodes README: transcode_cleanup/manager table said "Every 3 min", should be "Every 7 minutes"
- Tools manual: fallback_state_reset.sh still marked "not yet built" — rewrite section to reflect current script
- Tools manual: claude_startup --setup flag doesn't exist; modes were inverted — fix and add --launch
- Tools README + manual: add docker_prune_images.sh (existed but undocumented)
- Tools manual: add play_state_sync.sh --full to flag reference
- Orchestrators README: arrs_failed_stalled_recovery runs via intermediate_sync (every 4hr), not standalone every 6hr
- Plugin README: add build.sh to scripts table
- Partnership README + manual: add partnership_transfer.sh and onboard_cancel.sh to script tables and flag reference
2026-06-14 02:01:27 -04:00
Gmer4Lfe 420e13a746 Resolve bash variable references in vv_conf_vars()
PHP reads master.conf values like STATE_DIR="${SCRIPTS_DIR}/State_Files"
literally — the ${SCRIPTS_DIR} token is never expanded, so any PHP code
using the returned STATE_DIR value got an invalid path. Two-pass resolution:
SCRIPTS_DIR first (from PHP constant), then remaining ${VAR} tokens using
the now-resolved var set (covers DATA_DIR-based paths like ARR_SYNC_BLOCKLIST).
2026-06-14 01:46:51 -04:00
Gmer4Lfe 181114aed5 Header/code audit fixes across all core scripts
Behavior fixes:
- docker_daily_restart.sh: implement RESTART_VERIFY_WAIT sleep before
  verify_running() — variable was logged as config but never applied
- cert_monitor.sh: --log now shows expiry date and days remaining for
  healthy domains in the per-domain summary; problems always show details

Header corrections:
- downloaders_reset.sh: fix frequency contradiction (15 min → 30 min)
- lidarr_cleanup.sh: Emby scan triggers on actual deletions, not every run
- ssh_setup.sh: document --local-only flag in USAGE section
- zfs_memory_snapshot.sh: clarify dry-run skips log write in PURPOSE
- ramdisk_stop.sh / ramdisk_setup.sh: document TRANSCODE_STATE_FILE var
- smart_long_test.sh: remove SMART_TEMP_WARN/CRIT — not used in code
2026-06-14 01:40:35 -04:00
Gmer4Lfe b4d6866f40 Polish before first GitHub push: consistent adapter usage and naming
stability_watchdog.sh mover check now uses platform_is_mover_running()
instead of raw pgrep, matching server_reboot.sh.

check_unraid_version_parity() renamed to check_os_version_parity() —
the function internals were already platform-neutral after the adapter
refactor; the name was the last Unraid artifact.
2026-06-14 01:30:16 -04:00
Gmer4Lfe ee5a07be1c Remove remaining OS-specific hardcodes from core scripts
OS version reads now go through platform_get_os_version() and
platform_os_version_probe_cmd() instead of grepping /etc/unraid-version directly.

STATE_DIR fallbacks to /boot/config removed — STATE_DIR is always set by
load_config.sh and the fallback encoded a platform-specific path.

Setup DB path references use platform_setup_db_path() instead of the
VARAVERK_SETUP_FILE/-/boot/config compound fallback.

DOCKER_APPDATA_BASE default removed from arr_sync.sh — the adapter sets it.
2026-06-14 01:22:02 -04:00
Gmer4Lfe 3c900ae5c1 Resolve remote storage path from host*.conf for true cross-platform SSH support
detect_hosts() now sets REMOTE_STORAGE_PATH from HOST*_STORAGE_PATH in the
remote's conf instead of calling platform_storage_path() locally. SSH commands
that check the remote's storage (check_remote_array, check_remote_disks,
check_both_healthy) use $REMOTE_STORAGE_PATH so they work correctly when the
remote is a different platform with a different storage root.
2026-06-14 01:09:53 -04:00
Gmer4Lfe 27bfc21cb0 Platform-agnostic refactor: eliminate OS-specific hardcodes from core scripts
All bash scripts are now platform-neutral. Unraid-specific paths, commands,
and service checks moved to Plugin/unraid/adapter.sh. Core scripts call
platform_*() functions exclusively — no direct OS paths in runtime logic.

New adapter functions: platform_storage_path, platform_webui_install_path,
platform_scripts_dir_probe_cmd, platform_setup_db_path, platform_storage_healthy,
platform_is_service_enabled, platform_get_temp_thresholds, platform_disk_states_path,
platform_rebuild_container, platform_push_conf, platform_push_setup_state,
platform_get_templates_dir, platform_send_os_notification.

Partnership services stack (Emby/Jellyfin/Seerr/SeerrFin) added as third
onboarding stack alongside auth and arr stacks.
2026-06-14 00:59:19 -04:00
Gmer4Lfe fd1c4c7e3a Add CONF_SYNC_ENABLED to master.conf.template so nightly upgrade doesn't strip it 2026-06-13 13:30:40 -04:00
Gmer4Lfe ba1259c05c Add auth stack conf vars and storage mode selection to setup wizard
host.conf.template: add AUTH STACK section (NPM/lldap/Authelia) so
conf_upgrade.sh stops stripping those keys from host*.conf at 1am.

setup wizard: storage mode selector auto-detects USB vs NVMe at first
run; user can override; triggers storage_migrate.sh when mode differs
from current SCRIPTS_DIR location.

claude_startup.sh: reads HOST*_STORAGE_MODE_INTERNAL to pick internal
or appdata path rather than always requiring array to be mounted.
2026-06-13 13:05:46 -04:00
Gmer4Lfe 60ff658181 Fix conf_upgrade template naming bugs that caused false ADDED/REMOVED on every nightly pull
- REMOTE_ID_TIER*_DELAY renamed to HOSTN_TIER*_DELAY — these are own-host outage timers
- HOSTN_RADARR_MOVIE_ROOT → HOSTN_RADARR_MOVIES_ROOT (matches common.sh)
- HOSTN_SYS_WATCHDOG_CHECK_DOCKER → CHECK_DOCKER_DAEMON, CHECK_TEMP → CHECK_CPU_TEMP
- Add REMOTE_ID substitution to the sed in git_pull_execute.sh so COVERS_REMOTE_ID
  resolves to the actual remote host (e.g. FALLBACK_HOST1_COVERS_HOST2_TIER1)
- Add HOSTN_PARTNERSHIP_REPLACE_CONTAINERS, HOSTN_PARTNERSHIP_ARR_REPLACE_CONTAINERS,
  and HOSTN_WATCHDOG_CONTAINER_API_CHECKS to template so they aren't removed each pull
2026-06-13 10:22:02 -04:00
Gmer4Lfe 1bc76f8490 Align host template to unified section structure
Section order now matches master.conf: IDENTITY → PARTNERSHIP → RSYNC →
FALLBACK → DOCKER → MEDIA → ARR STACK → TRANSCODES → MONITORS →
RESOURCE MANAGER → SYSTEM WATCHDOG. Extracted PARTNERSHIP from IDENTITY
into its own top-level section. PERSONAL SHARES moved to sit directly
after DAILY SYNC SHARES. NIC var moved to top of SYSTEM WATCHDOG block.
2026-06-12 22:09:54 -04:00
Gmer4Lfe f036fbfe4a Sync master.conf.template schema to match live master.conf
STATE_DIR introduced, DATA_DIR path updated to flash, all state files
migrated from /tmp and /boot/config root to $STATE_DIR. TRANSCODE_STATE_FILE
and DOCKER_WATCHDOG_FAILED_FILE added; SYS_WATCHDOG_FAILED_FILE renamed to
DOCKER_WATCHDOG_FAILED_FILE to reflect what it actually tracks.
2026-06-12 21:55:15 -04:00
Gmer4Lfe 1f766fef31 Sync host.conf.template schema to match host1.conf
Added HOSTN_PERSONAL_SHARES, HOSTN_TMDB_API_KEY, and 8 missing system
watchdog check toggles (KERNEL_OOPS, BOOT, OOM, ARC, CONTAINERS, MDSTAT,
NETWORK, SSHD). Reordered watchdog checks to reflect tier structure from
host1.conf. Remaining template/live diffs are intentional renames
(CPU_TEMP→TEMP, DOCKER_DAEMON→DOCKER, MOVIES_ROOT→MOVIE_ROOT, TIER*_DELAY→REMOTE_ID).
2026-06-12 21:54:18 -04:00
Gmer4Lfe f10f09eeb1 Bug fixes, arr cleanup API-driven scan, shared JS formatters
Bug fixes:
- fallback.sh: escape sed metacharacters (\ & |) in state_set values
- common.sh: parse PID from lock file content correctly (handles pid:metadata format)
- unraid_api_key_renew.sh: fix path depth (../../../) and sync registry key to conf when stale
- stop.php: only clear pid/status if process is actually dead — D-state survives SIGKILL
- array_started.sh: check ARRAY_START_SCRIPTS empty before printing launch header

Arr cleanup:
- radarr_cleanup.sh / sonarr_cleanup.sh: fetch root folders from arr API instead of
  reverse-looking up the path map — handles multi-root-folder setups correctly

UI:
- varaverk.js: extract shared formatters (_relTime, _fmtBytes, _sz, _uptime, _gb, _tb, _n)
- arrs.php / partnership.php: use shared formatters, remove duplicates
- arrs.php / fallback.php: show error message on fetch failure instead of silent empty
- docker.php: disable rename input during request, restore original value on failure
- setup.php: abort controller timeout on detect fetch
- partnership.php: remove Re-run Phase 2 button opacity dimming
2026-06-12 21:41:11 -04:00
Gmer4Lfe 3937d5e33b Consolidate conf templates into Configurations/ — remove stale Deployment/conf_templates/
Deployment/conf_templates/ had two stale files: host.conf.template was a copy of host1.conf
(not a HOSTN template), and master.conf had hardcoded hostnames and an old appdata DATA_DIR path.
The correct templates already exist in Configurations/ (master.conf.template, host.conf.template).

Updated varaverk.plg and git_pull_execute.sh to read from Configurations/ directly.
Removed TMPL_DIR variable from git_pull_execute.sh — CONF_DIR covers both.
2026-06-12 21:35:58 -04:00
Gmer4Lfe 129e8ae14a Fix Emby HTTP 500 on PlayedItems — convert DatePlayed to yyyyMMddHHmmss
Emby's API spec requires DatePlayed as yyyyMMddHHmmss; sending ISO 8601 caused 500 on every mark-played call.
2026-06-11 19:22:51 -04:00
Gmer4Lfe ca0d17ec2a All rsync flags re-enabled — HOST2 rebuild complete 2026-06-07 10:12:20 -04:00
Gmer4Lfe 30e7c2f6b0 Detect containers running behind already-pulled images
OLD_ID vs NEW_ID only caught updates landed during this run. If an image
was pulled externally (Unraid UI), the container stayed on the old image
while :latest moved on — this run saw nothing to do. Now also compare
the container's current image ID against :latest and rebuild if behind.
2026-06-07 09:57:00 -04:00
Gmer4Lfe 819f5e6450 Remove old images by ID after container updates, not just dangling prune
Tagged images are never caught by docker image prune -f. Collect OLD_IDs
during the pull loop and explicitly rmi them after rebuilds complete.
2026-06-07 09:54:42 -04:00
Gmer4Lfe 1e54752478 Fix digest falsely counting = style state entries as system watchdog strikes
grep -v ':0$' was picking up kernel_oops_count=00, mdstat_errors=0, and
watchdog_cycle as active strikes. Restrict to : separator lines only.
2026-06-07 09:39:16 -04:00
Gmer4Lfe 80b1ee4f0d Fix rsync wrapper hanging forever after sync completes
exec 1>&- closed stdout but left stderr holding the tee pipe open,
so tee never got EOF and wait blocked indefinitely — keeping the
process alive and the locks held until manually killed.
2026-06-07 09:27:38 -04:00
Gmer4Lfe af313352c0 arr_sync: enforce monitored on every run — bulk re-monitor unmonitored items before sync 2026-06-06 18:35:41 -04:00
Gmer4Lfe aa92511ebc arr_sync: always add with monitored:true — don't propagate partner's unmonitored state 2026-06-06 18:31:36 -04:00
Gmer4Lfe cac9d8e82f Fix Sonarr episode count — episodeFileCount is nested under statistics in v4 2026-06-06 18:24:31 -04:00
Gmer4Lfe a24f9db2ff Remove orphaned API key code — correct key name format, collapse duplicate buttons 2026-06-06 18:17:30 -04:00
Gmer4Lfe dab8d62e95 Fix API key detection — name is 'Varaverk <hostname>', not 'Varaverk' 2026-06-06 18:09:37 -04:00
Gmer4Lfe a535319f1d API key button delegates to unraid_api_key_renew.sh — checks registry before creating 2026-06-06 18:02:37 -04:00
Gmer4Lfe bd73e87bf9 Add CONF_SYNC_ENABLED gate — skip partner sync when HOST2 is offline 2026-06-06 16:37:05 -04:00
Gmer4Lfe 273a2973e0 Ignore vanished-file chmod errors in Temp_Storage — race condition with Emby transcodes 2026-06-06 16:26:26 -04:00
Gmer4Lfe 57773bcaf5 Rsync sync log: stream live output from tee'd log file, preserve last run 2026-06-06 16:06:42 -04:00
Gmer4Lfe d05c5ac198 Sync activity card: fill full height instead of capping at 200px 2026-06-06 15:57:13 -04:00
Gmer4Lfe 04cd15a22f Fix rsync page: add missing _esc helper that broke all card rendering 2026-06-06 15:55:31 -04:00
Gmer4Lfe 4bb6bbf009 Rsync page: add live sync activity log card left of settings 2026-06-06 15:48:55 -04:00
Gmer4Lfe bac0f7c535 Auth page: pure-PHP Authelia ACL parser, NPM-sourced certs tab, watchdog JS fixes, rsync settings layout
- Replace python3/PyYAML Authelia ACL parser with pure PHP (no deps available on Unraid)
- Cert tab now pulls live from NPM API instead of cert_monitor.sh — auto-discovers all managed certs sorted by urgency
- Watchdog page: add missing GB constant and _fmtBytes/_relTime functions that were causing silent render failure
- Rsync settings card: pin to far-right 3 columns (grid-column:6/-1), toggle grid narrowed to 2 columns
- Add CLAUDE.md project context file on /boot for session persistence across reboots
- claude_startup.sh: symlink CLAUDE.md into /root on array start
2026-06-06 15:39:34 -04:00
Gmer4Lfe 9c3ace95a7 Auth stack certs tab, arrs db fallbacks, cert monitor cache, conf parser fix
- Auth stack: fold cert monitor into Auth Stack page as fourth tab (Certs);
  remove standalone cert page and top-level tab
- cert_monitor.sh: write JSON status cache to State_Files/cert_status.json
  after each run; expose per-domain days/expiry via _CERT_DAYS/_CERT_EXPIRY globals
- api/cert.php: new — serves cached cert status; falls back to configured
  domains as UNKN when no cache exists; POST action=run triggers live check
- arrs db fallbacks: vv_arr_cleanup_stats/discovery_stats/recovery_stats now
  read from data/*.db files when log JSON files don't yet exist
- config.php vv_conf_vars(): unescape bash \$ → $ so passwords with dollar
  signs read correctly from conf files
- host1.conf: fill in HOST1_NPM_USER/PASS and HOST1_LLDAP_USER/PASS
- Partnership adapter pattern: Unraid-specific container logic extracted to
  Plugin/unraid/Partnership/; platform-agnostic structure stays in Partnership/
- First-run wizard: uniform multi-step flow for all hosts; HOST2 pull moved
  to checklist; auto SSH keygen and API key creation on save
- api/checklist.php: live setup checklist with pull_master action
- Fullscreen toggle: hide Unraid header/menu; state persists via localStorage
2026-06-05 23:17:30 -04:00
Gmer4Lfe 17012bcd8c refactor: move unraid-specific tools into Plugin/unraid/tools/
api_cache_writer, remote_arr_cache_writer, conf_populate, storage_migrate are
all Unraid-adapter-specific — they read Docker appdata paths, Unraid service
configs, and drive the Unraid web plugin cache. They don't belong in Tools/.

- Tools/{4 scripts + php} → Plugin/unraid/tools/
- Fix source depth: ../ → ../../../ (now 3 levels from repo root)
- Fix conf_populate SCRIPTS_ROOT bug: was never set, would expand to empty
- api/arrs.php, api/storage.php: SCRIPTS_DIR/Tools/ → dirname(__DIR__)/tools/
- schedule.json: key IDs updated to Plugin/unraid/tools/...
- Cron rebuilt — live entries updated immediately
2026-06-05 20:04:51 -04:00
Gmer4Lfe bdcdfea175 feat: build smart_long_test.sh — SMART extended self-test for monthly maintenance
Tests all drives sequentially (one at a time to avoid I/O saturation).
Polls every 60s until complete. Notifies on failure. Respects SMART_IGNORE_DRIVES.
Enable by uncommenting in MONTHLY_MAINTENANCE_SCRIPTS in master.conf.

zfs_pool_scrub.sh was already fully implemented — also uncommented locally.
2026-06-05 19:53:15 -04:00
Gmer4Lfe c441ac443a feat: add --pull-only to conf_sync, wire into intermediate orchestrator
Partner conf cache (/tmp/.vv/) was only refreshed at array start. If a partner
updated their conf mid-day, this host's RAM cache went stale until next reboot.
Intermediate orchestrator now pulls partner confs every 4 hours as step 1.

conf_sync.sh --pull-only: pulls partner confs into local cache, skips push
(push is already handled by conf_populate.sh --push-only on conf save).
2026-06-05 19:44:10 -04:00
Gmer4Lfe f9fe13bdc1 fix: use STATE_DIR directly — TRANSCODE_STATE_FILE contains unexpanded bash var
vv_conf_vars() reads conf as text; TRANSCODE_STATE_FILE=$STATE_DIR/... is
returned as a literal string, not expanded. Build path from STATE_DIR directly.
2026-06-05 19:32:48 -04:00
Gmer4Lfe 415c260175 fix: transcode monitor reads STATE_DIR from conf instead of hardcoded /tmp
vv_transcode_sessions() was hardcoded to /tmp/transcode_state.db but master.conf
sets TRANSCODE_STATE_FILE=$STATE_DIR/transcode_state.db which resolves to
/boot/config/plugins/varaverk/State_Files/transcode_state.db. Page always showed
"ramdisk_setup.sh not yet run" despite state file being present and populated.
2026-06-05 19:29:31 -04:00
Gmer4Lfe 437c525fbd git pull: replace reset --hard + clean -fd with --ff-only 2026-06-05 19:24:48 -04:00
Gmer4Lfe a5936331c9 auth stack page, monitor cache fix, symlink install 2026-06-05 19:20:28 -04:00
Gmer4Lfe d9ed8c7704 claude_startup: flip default to setup-only, add --launch flag
Default behavior is now setup-only (symlinks + binary, no interactive launch)
so array_started.sh can call it directly without arguments. Use --launch to
start an interactive Claude session.
2026-06-05 16:04:53 -04:00
Gmer4Lfe 0ded0c87a1 arr_sync: cache-first remote API calls, SSH fallback
Remote functions (_remote_arr_up, _remote_library, _remote_defaults,
_remote_add, _delete_remote_item) now check ${node_id}_<ARR>_API_KEY
from the environment first. When found (conf cache populated by
conf_sync.sh + load_config.sh sourcing /tmp/.vv/), they call the arr
API directly over Tailscale — no SSH connection needed.

SSH fallback (grep config.xml + localhost API) remains for the first
boot before conf_sync has run or when the partner is offline.

config_xml is no longer passed between functions — computed internally
in the SSH path only. Call sites in _sync_arr and blocklist-add updated
accordingly.
2026-06-04 22:20:38 -04:00