docker_watchdog now writes daemon_confirmed_down=true to its state file
when restart is attempted and daemon is still unresponsive. Clears the
flag on recovery.
stability_watchdog removes the duplicate CRITICAL daemon check (which
bypassed all strikes, all abort conditions, and had its own rc.docker
restart). Replaced with a standard strike check reading the
daemon_confirmed_down flag from docker_watchdog's state file.
Result: docker_watchdog owns all daemon restart logic. stability_watchdog
escalates to reboot only after N consecutive strike cycles confirming the
daemon is truly unrecoverable — same pattern as all other standard checks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
stability_watchdog: container health is owned by docker_watchdog — rebooting
when docker_watchdog skip-lists a container creates a reboot loop. Removed
SYS_WATCHDOG_CHECK_CONTAINERS check entirely. docker_watchdog handles
strikes, restart attempts, skip-listing, and notifications.
varaverk.plg: warn on USB/removable boot device at install time. Plugin is
designed for internal NVMe/SSD boot. Warning is non-blocking but clear.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Exits cleanly before startup grace check if /mnt/user is not mounted
as shfs. Prevents watchdogs running against Docker/storage that isn't
available, and blocks false-positive reboots when array is stopped.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
runc v1.3.5 (Docker 29.x / Unraid 7.3+) crashes on bind-propagation=shared
with any container — not GPU-specific. Ramdisk tmpfs is already MS_SHARED
at kernel level; propagation is automatic and the option is unnecessary.
- Remove bind-propagation=shared from all example Extra Parameters
- Update "Why shared Is Required" → "Mount Propagation — Unraid 7.3+"
- Update GPU troubleshooting section to cover the general runc crash
- Verify commands updated (no longer checking for Propagation: shared)
- Historical notes preserved explaining why it existed before
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
runc v1.3.5 (Docker 29.x / Unraid 7.3+) rejects bind-propagation=shared
on symlink sources with: open /proc/self/mountinfo: no such file or directory.
The ramdisk tmpfs is already MS_SHARED at the kernel level — propagation is
inherited automatically and does not need to be specified in Docker.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
228 path references updated from /mnt/user/appdata/Varaverk/ to
/boot/config/plugins/varaverk/. PATHS header block updated to match.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Plugin folder /boot/config/plugins/varaverk is the single home for
scripts, data, and state. Available at boot before array mounts.
USB boot is not a supported configuration.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Bootstrap varaverk.cfg default and conf template default back to
/mnt/user/appdata/Varaverk — safe for USB boot users. Flash paths
(/boot/config/plugins/varaverk) are set in the user's gitignored
personal configs and don't affect public installs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move SCRIPTS_DIR/DATA_DIR/STATE_DIR from appdata to /boot/config/plugins/varaverk
- All state files now in STATE_DIR (no more /tmp or /boot/config root writes)
- Bootstrap: Gitea-first clone with GitHub fallback, no array dependency
- varaverk.cfg seeded with Gitea connection settings
- .gitignore: add State_Files/, varaverk.cfg, varaverk-*.txz
- Partnership/transcode/fallback scripts use STATE_DIR variables
- PHP config.php: DATA_DIR/STATE_DIR constants, VV_SETUP_STATE_FILE dynamic
- deploy.sh PROD_ROOT updated to plugin flash dir
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Was looking for host1.conf at SCRIPTS_ROOT/host1.conf instead of
SCRIPTS_ROOT/Configurations/host1.conf — causing the 'host1.conf not found'
warning and leaving HOST1_SSH_KEY unwritten after key generation.
flag_key was declared with 'local' at script top level, which is only
valid inside bash functions. This left flag_key empty, writing '=true'
(malformed) to setup.db instead of 'HOST1_LOCAL_DONE=true'.
Changed to a plain variable assignment.
- Single '▶ Onboard' button per unprovisioned remote → expands that card
to show Step 1 (terminal link + click-to-copy command) and Step 2 (Push Conf)
- Onboard state persists across 10s poll re-renders via top-level _vvOnboarding map
and _vvPtReload hook set by the IIFE
- Phase 1 complete: amber 'SSH ready' pill + Tailscale IP + Phase 2 manual button
- Phase 2 complete: green 'Active' pill + date + IP + uptime + unRAID version + state
- Compact compact '○ Not provisioned' + Reset button when not in onboard mode
- Removed vvPtPhase1 (replaced by vvPtStartOnboard toggle)
- vvPtStartOnboard / vvPtEndOnboard are top-level functions
At phase 0, show below the buttons:
- 'Open Terminal' link using HOST1's Tailscale IP (/webterminal/ttyd/)
- Clickable command code block (click to copy) with the Phase 1 script path
- Note that tab auto-updates on next poll when the script completes
Tailscale IP comes from selfNode.ts_ip already in the API response.
partnership_onboard.sh (--phase1-only):
- Test if SSH already works first (BatchMode) — skip ssh-copy-id if yes
- If ssh-copy-id fails (no TTY in background): soft-fail, generate key locally,
print pubkey for manual install, write HOST2_KEY_READY=true, run local setup
- Only skip conf push if SSH truly unavailable; otherwise proceeds normally
- Exit 0 with clear manual-step instructions instead of hard failure
include/partnership.php:
- Read HOST2_KEY_READY from setup.db; expose as key_ready on remote nodes
pages/partnership.php:
- Node card: amber "Key generated · install on HOST2 then Push Conf" badge
when key_ready=true and phase=0
- "Push Conf (key installed)" button appears when key_ready=true — runs
--phase1-only --skip-ssh (assumes user manually did ssh-copy-id)
- vvPtPushConf() top-level function
vv_partner_state() now reads setup.db and adds onboard_phase (0/1/2/null)
to each host entry.
Partner card JS renders a badge under the hostname for remote hosts that
haven't completed Phase 2: amber "Awaiting onboard" (phase 1) or grey
"Not provisioned" (phase 0). Phase 2 complete shows nothing extra.
When HOST1 completes its local setup (--onboard --local-only), enable the
partnership in master.conf so sync scripts start functioning immediately.
Pushes the updated master.conf to all listed hosts via vv_push_master_conf().
partnership_manager.sh:
- --local-only flag for --onboard: skips remote pre-flight, WebUI reconfig,
state push; runs FolderView3 folder creation + writes HOST1_LOCAL_DONE=true
to setup.db + pushes it; exits without writing ACTIVE (partnership not yet
established until Phase 2)
partnership_onboard.sh:
- Phase 1 exit now runs partnership_manager.sh --onboard --local-only after
SSH + conf push, so HOST1 is fully configured on its own side before Phase 2
include/partnership.php:
- vv_pt_nodes() reads HOST1_LOCAL_DONE from setup.db; exposes as local_done
on the self node (null for remote nodes)
pages/partnership.php:
- Self node card shows green "Local setup complete · waiting for partner"
badge when local_done=true
- Actions: "Complete HOST1 Setup" button appears when HOST1 hasn't run local
setup yet but HOST2 is already at phase 1 — lets user trigger it manually
- vvPtLocalSetup() top-level function calls partnership_manager.sh via run.php
If run.php returns an empty body (transient nginx/PHP-FPM issue), the script
still runs via nohup. Treat empty response as ok instead of throwing a
JSON parse error that misleads the user into thinking the action failed.
onboard_cancel.sh:
- SSHes HOST2 while key still valid: removes HOST1 pubkey from authorized_keys
and clears HOST2_PHASE* from setup.db on the remote side
- Deletes local private + public key pair
- Clears HOST2_PHASE* from local setup.db
- Graceful if HOST2 unreachable (local side still cleaned up)
partnership.php:
- Cancel button (warn style, slightly dimmed) at phase 0 and phase 1
- vvPtCancel() top-level function, calls onboard_cancel.sh via run.php
window.fn = inside an IIFE is not reliable in all script-loading contexts.
Move vvPtPhase1/2, vvPtOnboard, vvPtOffboard and their _vvPtRun helper
above the IIFE as plain top-level function declarations — unconditionally
global and reachable from onclick= attributes.
Tab pages use full page reloads (not AJAX) — function declarations inside
the IIFE are not in global scope, so onclick= attributes fail silently.
Fix: assign all onclick-callable functions to window.*
Replace vvRunById() calls with direct fetch (vvRunById is also IIFE-scoped
in scheduler.php, unavailable in the partnership tab context).
Add .catch() handlers so network/parse errors surface as alerts.
include/monitor.php: vv_fallback_state() now returns:
enabled, check_interval, handback_strikes, handback_strikes_required,
partnership_suspended, partner_lost_at, partnership_required,
partnership_suspend_after — all read from state file + conf vars.
pages/monitor.php: fallback card render rebuilt:
Disabled: clear message with how to enable
Suspended: partnership inactive with elapsed time
FAILOVER: outage duration + T1/T2/T3/T4 tier badges (active=green,
pending=dark) + handback strike progress dots (●●○) when remote
comes back and strikes are counting
NO_INTERNET/DARK: short contextual line
NORMAL: quiet "monitoring · 30s · partnership gated" meta line
Accent border: disabled/suspended → warn, NORMAL → ok, FAILOVER → err
End users leave GITEA_REPO_PATH="" in master.conf (template default).
Before hitting any Gitea detection logic, the script now checks for this
and does git pull --ff-only origin main directly — origin is already
GitHub since the .plg cloned from there.
Sparse checkout skipped on this path: conf files are gitignored so
nothing sensitive is in the repo to exclude.
Developer path (GITEA_REPO_PATH configured) is completely unchanged.
unraid_api.php: track key_missing separately from general fallbacks.
vv_api_data() sets key_missing=true when HOST*_UNRAID_API_KEY is empty.
vv_api_get_status() includes key_missing in the status payload.
monitor.php: two distinct banner states instead of one orange warning.
key_missing → subtle dark/grey note: "API key not configured — add
HOST1_UNRAID_API_KEY in Scheduler → host conf to enable enhanced monitoring"
key present but unreachable → existing orange ⚠ with fallback list.
No banner at all when API is working correctly.
Plugin/varaverk.plg:
- Add support="https://github.com/FailedProxy/Varaverk/issues"
- Install completion log now says "open Varaverk to complete setup"
with navigation hint — user lands on wizard automatically
Plugin/unraid/VaraverkSettings.page:
- Fix stale SCRIPTS_DIR default (unraid_scripts → Varaverk)
- Add GitHub / Issues / Changelog links at bottom of settings page
Plugin/unraid/Varaverk.page:
- Small "⎋ GitHub" link right-aligned in the tab bar
links to github.com/FailedProxy/Varaverk in a new tab
After Step 8 (arr bootstrap), owner calls vv_push_master_conf() and
vv_push_setup_state() via php -r. SSH is established at this point so
the probe passes and all listed hosts get the authoritative master.conf
immediately — no manual save required.
Skipped if onboard failed, dry-run mode, or php unavailable (with
fallback instruction). Result shown in onboard summary as Step 9.
Fix 1 — Monitor partner card (common.php + monitor.php):
vv_remote_hosts_stats() no longer skips hosts with no API key.
Returns no_api_key:true entry instead. Monitor JS renders
"API key not configured — complete Onboard to enable" instead
of a blank space.
Fix 2 — rawconf push not alarming pre-onboard (config.php + scheduler.php):
Probe failures (Tailscale not found, plugin not installed) now carry
ready:false. JS treats ready:false results silently — button shows
"✓ Saved" not "push failed: HOST2" before SSH is set up.
Only genuine post-onboard failures (ready not false) show as errors.
Fix 3 — Partnership Onboard button (partnership.php):
Disabled with tooltip when no partner hostname is in master.conf.
When partner configured but not onboarded, adds explanation note:
"Onboard will generate your SSH key, exchange it with the partner..."
No partner configured: "Edit master.conf and set HOST2."
scheduler.php: after host1.conf save in setup mode, redirect to
?tab=partnership&vv_onboard=1 instead of Monitor.
HOST2+ still redirects to Monitor (they don't manage the partnership).
partnership.php: detects vv_onboard=1 URL param after actions render.
Adds pulse animation + box-shadow to the Onboard button and scrolls
it into view. One-shot — gone when user navigates away.
Varaverk.page: also shows wizard when local host.conf is missing
(handles master.conf pushed by HOST1 before HOST2 installs plugin).
pages/setup.php: three wizard flows
- Standard: blank master.conf, fill hostnames, redirect to scheduler
- Host2/state file: state file detected, pull master.conf from HOST1 via SSH
- Conf-only: master.conf already filled (was pushed), just create local host.conf
api/setup.php:
- save action: writes master.conf + host.conf, creates varaverk_setup.db state file,
redirects to ?tab=scheduler&vv_setup=master.conf
- pull action: resolves HOST1 Tailscale IP, queries HOST1 SCRIPTS_DIR, SCPs
master.conf, creates local host.conf, redirects to ?tab=scheduler&vv_setup=hostN.conf
include/config.php: vv_setup_state_read/write/push helpers.
vv_push_setup_state() pushes varaverk_setup.db to /boot/config/ on all known
remotes — no plugin-readiness probe needed (flash is always accessible).
api/rawconf.php: calls vv_push_setup_state() alongside master.conf push.
pages/scheduler.php: setup mode via ?vv_setup=<conf> URL param.
Auto-opens the specified conf file on page load (DOMContentLoaded).
vvSaveRawConf: in setup mode, skips confirm dialog and forces sequence:
master.conf save → auto-open hostN.conf
hostN.conf save → redirect to Monitor (setup complete)
Plugin/varaverk.plg: URL updated to github.com/FailedProxy/Varaverk
Configurations/host.conf.template:
Full host conf structure with HOSTN/hostn placeholders.
All personal values blank, all sections documented.
Covers identity, rsync, docker, fallback, media, monitors,
transcodes, arrs, system watchdog, resource manager.
Varaverk.page: checks if HOST1 is blank before rendering tabs.
If blank → shows setup wizard, returns early (tabs never render).
pages/setup.php: first-run wizard UI.
Auto-populates hostname from hostname -s.
Role selection: primary (HOST1) or partner (HOST2+).
Partner slot selector for HOST3+.
api/setup.php: handles wizard form POST.
Writes HOST1/HOST2 (and HOST3+) into master.conf preserving all
other content. Creates host*.conf from template with HOSTN/hostn
replaced and SSH key path pre-filled from hostname convention.
Never overwrites an existing host*.conf.
Plugin/varaverk.plg:
- Method="install update": git clone (fresh) or pull (update)
Validates Unraid, checks array is started via shfs mount, clones
to SCRIPTS_DIR (read from varaverk.cfg, default /mnt/user/appdata/Varaverk),
creates Plugin/unraid symlink, inits varaverk.cfg, bootstraps master.conf
from template on first install.
- Every boot: lightweight boot.sh recreates symlink if missing (no git ops).
- Method="remove": removes web symlink, leaves appdata intact.
Configurations/master.conf.template:
- Full master.conf with personal values blanked: HOST1/HOST2 empty,
GITEA_REPO_PATH empty, personal container names in profiles cleared,
FALLBACK_ENABLED=false, CRITICAL/DAILY rsync defaults corrected to true.
New users fill in HOST1, HOST2, then proceed to partnership setup.
Single SSH probe per remote checks all three conditions in one call:
- plugin cfg exists (varaverk.cfg present with SCRIPTS_DIR)
- Configurations/ dir exists at that path
- master.conf already present on remote
Any failure skips that host entirely rather than pushing blind.
Uses the discovered SCRIPTS_DIR as the SCP destination, not ours.
Instead of assuming the remote uses the same appdata path as the local
host, SSH-grep their varaverk.cfg first and SCP to their actual
Configurations/ dir. Falls back to /mnt/user/appdata/Varaverk if the
plugin isn't installed or the file can't be read.
vv_push_master_conf() in config.php: finds all remote HOST* entries,
resolves Tailscale IPs, SCPs master.conf to each.
rawconf.php calls it after a successful write when file=master.conf.
Scheduler save button shows sync status: "✓ Saved · synced to HOST2"
or "✓ Saved · push failed: HOST2" on error.
Also fix path bug in partnership_transfer.sh — was SCPing to
appdata root instead of Configurations/.
VaraverkSettings.page (Settings → Other Settings) now owns SCRIPTS_DIR.
The inline Advanced-mode field and vvSaveSettings() in the Scheduler tab
were a second path to the same write — removed.
Discover all HOST*_TRANSCODE_SERVERS from every configured host.
For remote hosts, rewrite localhost/127.0.0.1 in the URL to their
Tailscale IP (via resolve_tailscale_ip). Unreachable hosts are
warned and skipped — partial sync continues for reachable servers.
Adding a new host needs no script changes — just HOST*_TRANSCODE_SERVERS
in the new host conf.
Removed from DAILY_MAINTENANCE_SCRIPTS — critical orch already covers it
at a much tighter cadence. Play state stays in sync within 30 minutes of
any watched/resume activity on either server.
play_state_sync.sh:
- Add detect_hosts() call so TRANSCODE_SERVERS alias is populated
- Switch MEDIA_SERVERS → TRANSCODE_SERVERS (correct array name from host*.conf)
- Fix malformed jq fallback in combine step (was {"} now handled with if/else)
- Skip placeholder API keys in _add_server
load_config.sh:
- Replace 'for x in $(ls glob)' with 'while read < <(printf glob | sort)'
so paths with spaces (e.g. dev workspace) don't get word-split
Syncs played/resume state bidirectionally across all configured Emby
and Jellyfin servers. Newest LastPlayedDate wins; users matched by
name (case-insensitive); unmatched users skipped cleanly.
Matching: IMDb/TMDB for movies, TVDB+s/e for episodes, MusicBrainz
track ID for audio. Also syncs resume positions for partial plays.
Config: PLAY_SYNC_ENABLED, PLAY_SYNC_DAYS (default 90), PLAY_SYNC_TYPES.
Modes: --dry-run, --status, --full (ignore day window), --log.
Wired into DAILY_MAINTENANCE_SCRIPTS after arr cleanup.
- Tools and Custom: args input always visible (translucent at rest, full on hover)
- Orch children: args input hidden when orch is on, shown when orch is off
(same toggle as the cron field — orch off = standalone mode)
- vvRunJob/vvDryRun: pick up .vv-script-args value, POST as extra_args
- run.php / dryrun.php: accept extra_args, validate against shell metacharacters,
append to run_job.sh invocation (flows through SCRIPT_ARGS to the script)
- CSS: .vv-script-args — 30% opacity at rest, 100% on hover/focus
Default still removes only dangling (untagged) images.
--all removes stopped containers first, then all images not used
by any running container — clears tagged orphan images left behind
by removed or stopped apps.
PHP architecture:
- Extract common.php from monitor.php — shared system functions (vv_system_info,
vv_memory_breakdown, vv_remote_hosts_stats, disk/GPU/UPS/network/docker/parity, etc.)
now live in one place; monitor.php and watchdog.php both require common.php
- Add unraid_api.php as explicit include (was implicit via config.php chain)
- confform.php: add missing require_once config.php (implicit dep made explicit)
- Delete orphaned pages/docs.php and pages/config.php (absorbed into scheduler)
Watchdog page:
- Add Storage watchdog card (growth + log strikes, baseline age, suppress ceilings)
- Add Network watchdog card (NPM strikes, DDNS domain/container, NPM URL)
- One host per row layout — all 5 watchdog cards equally spaced via inner grid
- Watchdog now uses Unraid API for local system stats; remote nodes with API key
but no SSH get system info from vv_remote_hosts_stats() with api_only flag
- SSH bundle: /proc/meminfo passed as raw section instead of awk-parsed header
fields — fixes RAM showing 0 on remote hosts where awk quoting was unreliable
- vv_wd_local_system() rewritten as thin wrapper over vv_system_info() + vv_memory_breakdown()
Monitor page:
- Watchdog card: add stability strikes, storage watchdog strikes, network NPM status,
live system stats (rootfs/log/tmp %, RAM free, load, CPU temp, zombies, NIC, sshd)
- Row height: switch from max-height on cards to minmax(0, calc(...)) on grid track —
all cards in a row now fill to the tallest card's height correctly (fixes Pools card
being shorter than neighbours)
Scheduler page:
- Add Tools section above Custom Scripts — lists Tools/*.sh with run/dry-run/cron/log
- vv_tools_scripts() function in scheduler.php include
Tools:
- Add docker_prune_images.sh — removes dangling Docker images; --dry-run and --status modes
- Replace pill view with per-container rows showing full detail
- docker inspect batch call: networks+IPs, port mappings, bind mounts
- Container icon from template XML, name links to WebUI if configured
- Network IP badges (blue), port mapping badges (yellow)
- Bind mount paths shown src→dst, collapsible expand for >3 paths
- Large folders (>7 containers) auto-span full width
- Status badge: RUNNING/STOPPED/PAUSED/EXITED with colour
- Stopped containers dimmed, icons load from template XMLs
- Primary store moved to /boot/config/plugins/varaverk/docker_folders.json
- No dependency on folder.view3 plugin being installed
- Auto-imports folder.view3 JSON on first run if it exists (one-time bootstrap)
- Mirrors writes to folder.view3 JSON only if that plugin's directory is present
- Toolbar shows '⇄ folder.view3' badge when sync is active
- New Docker tab (between Scheduler and Watchdog)
- Reads /boot/config/plugins/folder.view3/docker.json — fully compatible with folder.view3 plugin
- Writes changes to both docker.json AND HOST*_DOCKER_FOLDER_MAP in host*.conf simultaneously
- Edit mode: rename folders (inline input), delete folders, move containers via popover picker, create new folders
- Drift banner: highlights containers where conf desired state doesn't match json actual state
- Sync conf→JSON: apply conf desired state to json (fixes drift after onboard)
- Sync JSON→conf: capture manual json edits back into conf
- Ungrouped section shows all containers not assigned to any folder
- Onboard scripts can read HOST*_DOCKER_FOLDER_MAP to auto-place new containers
Scheduler UI:
- Arrange mode: drag scripts between orchs and reorder within arrays; right
panel shows unassigned script pool; Save Arrangement commits to master.conf
- + Folder: named collapsible subfolders for Custom Scripts stored in schedule.json
- Rsync children: hide Run/Dry Run/Log/location when orch is ON; show standalone
location + cron controls when orch is OFF; cron only fires when both filled
- Non-conf-managed children (transcode): toggles now show enabled when orch is on
- Right panel height sync: fix ResizeObserver feedback loop via align-self:flex-start
on left panel and left.offsetHeight in vvFitRight
- How do I use this: updated to cover arrange, folders, rsync standalone, transcode
New API endpoints:
- board.php, clearlock.php, movescript.php, rawconf.php, readscript.php
- reorderarray.php, rsync_standalone.php, savefolders.php
run.php / dryrun.php: accept optional --location= arg for standalone rsync calls
Daily restarts create orphaned images that were never cleaned up.
Added docker image prune -f section matching docker_update_remaining.sh pattern.
Prune runs each cycle so orphans from the previous day's restart are removed daily
rather than waiting for the weekly docker_update_remaining.sh sweep.
Previously ran daily at 3am with internal self-gating. Now scheduled directly
on the 15th — uptime and interval gates still apply; a skip means next month's 15th.
- vv_script_children() now calls vv_script_suggested_cron() per child,
adding suggested_cron and suggested_label to each child entry
- Orch rows (non-event) and regular script child rows render a faint
.vv-cron-hint span to the left of the cron input showing the suggested
cron from the script's # Schedule: header
- Clicking the hint fills in the cron input (data-cron + onclick)
- Tooltip shows full "suggested_cron — label" string on hover
- .vv-cron-hint: color #444, brightens to #888 on hover
- Right panel title: "Suggested Schedules" → "Scheduler Information"
- Removed top hint paragraph; info now lives in the panel itself
- Added three collapsible info sections (open by default) at top of panel:
Controls — toggle, cron, run, dry run, log, stop, verbose, config
Orchestrators — enabled/disabled behavior, child cron rules, event triggers
Children & Advanced — toggle semantics, independent cron, rsync flag badge
- Bullet lists use CSS columns:2 with column-fill:balance for even distribution
- "Suggested Schedules" divider separates info from parsed schedule blocks
- Added vvToggleSug() for DOM-traversal-based accordion (no ID dependency)
- Rsync/rsync.sh children in Advanced now show as conf_flag type when the
parent orch controls a *_RSYNC_ENABLED tier flag; toggle writes true/false
to master.conf instead of comment/uncommenting a SCRIPTS array entry
- Added vv_conf_flag_value() and vv_conf_flag_set() helpers in scheduler.php
- Added api/flag_toggle.php endpoint (validates *_RSYNC_ENABLED pattern)
- Added api/stop.php: kills process group, sweeps stuck locks, updates stat
- vv-flag-badge CSS (amber, monospace) to distinguish from event/script rows
- vvSaveChild() routes conf_flag children to flag_toggle.php unconditionally
- vvApplyOrchState() keeps conf_flag toggle at real flag value; disables when orch off
- vvSaveAll() skips conf_flag children (immediate-save only)
When orch is ON (god mode):
- Children's toggle state is read from master.conf comment status
- Toggling a child comments/uncomments its line in the *_SCRIPTS array
- Child crons are suppressed in vv_cron_rebuild — orch is the sole trigger
- Children not found in any *_SCRIPTS array are shown disabled (read-only)
When orch is OFF:
- All children flip to off; schedule.json updated immediately
- A child with a cron value + enabled toggle gets its own independent cron entry
- A child with no cron does nothing when enabled
New: vv_conf_script_map() — cached per-request scan of master.conf arrays
New: vv_parse_conf_array_full() — includes commented entries (disabled scripts)
New: vv_conf_toggle_script() — comments/uncomments a script line in master.conf
New: api/conf_toggle.php — endpoint for child toggle → master.conf write
Two hardcoded event-triggered entries appear at the top of the scheduler
(array_started.sh / array_stopping.sh). They show an ⚡ Array Start /
⚡ Array Stop badge instead of a cron field and are enabled/disabled via
the normal toggle.
Static event scripts fire them via Unraid's event system:
- event/disks_mounted/array_start_jobs → runs in background (non-blocking)
- event/disks_unmounting/array_stop_jobs → runs foreground (blocks until done)
vv_cron_rebuild() skips @array_* entries so they never land in the cron file.
Scripts executed on each event are managed in master.conf via
ARRAY_START_SCRIPTS and ARRAY_STOP_SCRIPTS arrays.
Auto Scroll now pauses automatically when user scrolls up and resumes
when they reach the bottom — no separate Scroll Lock needed.
Lidarr discovery no longer hard-skips single-seed candidates; they
score low on breadth but can still reach threshold on merit.
Before update_cron was wired in, the plugin wrote directly to
/etc/cron.d/varaverk. That file persisted after the migration and
caused duplicate job firing (two watchdog instances at :00/:15/:30/:45).
The second instance hit the lock, exited with warn, and overwrote the
stat file — killing the running indicator while the first was still live.
vv_cron_rebuild() now unlinks the legacy file after every update_cron
call. Since rebuild_cron (boot event) calls vv_cron_rebuild(), the
cleanup runs on every reboot and every schedule save.
run_job.sh was overwriting the log on every cron fire, making it
impossible to see run history. Switch to append mode, print a
── YYYY-MM-DD HH:MM:SS ── separator before each run, and trim to
the last 1000 lines after each run to keep logs bounded.
Cron runs with PATH=/usr/bin:/bin, which omits /usr/local/sbin where
tailscale (and other Unraid tools) live. resolve_remote_ip silently got
empty results from both tailscale calls, failing all 3 retry attempts
every scheduled run. Export the full path at source time so all scripts
find tailscale regardless of how they're invoked.
Scripts calling acquire_lock followed by acquire_rsync_lock (rsync.sh)
or a custom EXIT trap (resource_watchdog.sh) would overwrite the lock
release trap, leaving the first lock file behind on exit.
Replace per-acquire trap with a _LOCK_FILES registry. _register_lock()
appends each lock file to the array and sets a single _release_all_locks
trap. All acquired locks release together on exit regardless of how many
traps are subsequently set or overwritten.
resource_watchdog.sh: chain _release_all_locks into both its custom trap
and the state-persisted disarm path so the lock releases in all exit paths.
The first cron run of critical_sync_maintenance failed because Tailscale
had a transient moment at startup. resolve_remote_ip() now retries 3 times
with a 5s delay before giving up, making it resilient to brief daemon
hiccups without masking real peer-offline conditions.
Add api/status.php — reads all scheduled jobs' stat files and returns a
{id: status} map. The scheduler page polls it every 3s and lights up the
dot on any job whose stat shows status=running, clearing it when done.
Replace single vvRunningId with vvRunningSet (Set) so multiple jobs can
show as running simultaneously. vvSetDot/vvClearDot handle per-job dots
and the header dot for the active job. Remove stale-count detection from
vvFetchRight — status poll is now the single source of truth.
Create varaverk.plg on flash (/boot/config/plugins/varaverk/varaverk.plg).
On boot, rebuild_cron copies it to /var/log/plugins/ so update_cron natively
picks up varaverk.cron and merges it into /etc/cron.d/root.
With the .plg in place the dev-mode direct-inject fallback in vv_cron_rebuild()
is no longer needed and has been removed.
dcron uses the filename in /etc/cron.d/ as the username, so a file named
'varaverk' ran jobs as a non-existent user and was silently ignored.
Switch to writing /boot/config/plugins/varaverk/varaverk.cron and calling
update_cron (which merges all plugin .cron files into /etc/cron.d/root).
In dev mode (no varaverk.plg registered), fall back to injecting our entries
directly into the live root crontab via crontab -c /etc/cron.d -.
run_job.sh: new wrapper called by scheduler cron for every job.
Writes /var/log/varaverk/<id>.json (status/start/end/exit/pid)
and /var/log/varaverk/<id>.log (latest run output, overwritten).
Maps exit 0→ok, 1→warn, 2+→error so watchdog "took action" runs
show warn not error.
scheduler: cron now calls bash run_job.sh instead of inline bash -c.
Added vv_job_stat_path(). LOG_DIR moved to config.php (shared).
monitor: vv_scripts_status() now reads /var/log/varaverk/*.json and
*/*.json — no User Scripts tmpScripts dependency at all.
Stale running detection via /proc/<pid> check.
Returns error_count in addition to running/ok/warn counts.
pages/monitor.php: added error pill (red ✗), error icon/color in
list, duration shown next to age for each script entry.
log() is gated on ENABLE_LOGGING — silent on normal runs. Healthy
confirmations (All healthy, Network healthy, Storage healthy, WebGUI
healthy) were invisible, making banners appear with nothing after them.
Switched all four to plain echo so they show every cycle.
All watchdog scripts now open with a consistent echo banner:
━━━ <icon> <Name> Watchdog — <timestamp> ━━━
Previously network and storage used a quiet log() line (no separator),
and stability_watchdog used warn() which made every run show ⚠️.
Also: network_watchdog now logs "Network healthy ✅" on clean exit,
and stability_watchdog's reboot-trigger block now says "Stability"
instead of the old "System Watchdog" label.
network_watchdog: Tailscale JSON uses "BackendState": "Running" (with space)
but grep pattern matched "BackendState":"Running" (no space) — always failed.
Switched to grep -qE with \s* to handle both forms.
system_watchdog: add echo banner at run start so its output is visually
separated from docker_watchdog in orchestrator logs. Previously appeared
as a continuation of the docker section since docker_watchdog has a banner
but system_watchdog did not.
Layout:
- Switch from three flex rows to single CSS grid (repeat(8,1fr)) — eliminates
gap-count discrepancy between rows; span 1/2/4/8 cards are now pixel-identical
across all rows regardless of how many cards share the row
New cards:
- Scripts card: last-run status (ok/warn/running/unknown), clickable filter pills,
7-row scroll, flex:1 matching Power/Parity
- Streams card: live per-second progress counters, color-coded play buttons
(blue=live, orange=transcoding, green=direct, yellow=paused), server badges
with counts, left-fill column layout (3 per col)
- Transcode card: NVMe indicator, active session count + scrollable list,
session rows with server icon + title + type + method
Dynamic thresholds:
- vv_disk_thresholds() reads /boot/config/plugins/dynamix/dynamix.cfg
- Disk utilization bars: green/orange/red based on warning/critical settings
- Temperature colors: hot/max for HDD, hotssd/maxssd for SSD/NVMe
- Thresholds update every poll cycle; changing Unraid settings takes effect automatically
Other:
- SSD path detection uses findmnt FSTYPE to exclude tmpfs/ramfs (fixes RAM
being reported as SSD transcode target)
- varaverk.css h3: add white-space:normal + overflow:hidden to neutralise
any Unraid global h3 styles that could force card width
- network_watchdog: remove ISSUES++ for Tailscale — it's notify-only by
design, no restart possible; cascading to system watchdog exit 1 was noise
- docker_watchdog: move banner to top of cycle so body errors appear under
the correct section header, not visually inside Resource Manager output
