Curated state copied to every node is state that can disagree, so the index,
the model and the shared memory stay on the owner and each node reaches them
over the SSH trust onboarding already builds. Chats stay on the node that had
them; memory and bug reports stay the owner's to write.
Phase 1 exists to hand the partner its identity before the install, but the push refused any
host without varaverk.cfg — so the conf could only ever reach a node that no longer needed it
to be told who it was. It now resolves the remote's conf directory across both install layouts
and creates the internal one when neither exists.
The value was written as typed, bash expanded it when the read-back sourced the file, and
the guard rolled the whole write back with nothing on screen but "save failed" — which is
also what an empty value, a trailing space, and a stale API-key check had been doing.
vv_is_ai_host() was `=== 'host1'`, which made a physical fact — that is where the
GPU is — look like a rule. AI_OWNER_HOST declares it, so the card can move to a
rebuilt host3 or a friend's spare.
The gate was also doing two jobs. Assistant docks and findings strips now ask
whether a model is reachable, so a node without a GPU gets them by borrowing;
the AI tab asks whether this is the owner, because that page carries the bug
reports, the index and the model configuration — the surface where the
vocabulary assumes you built the mesh.
Resolution is local, then owner, then anyone else declaring a model, pinned once
it answers. Pinned rather than re-derived per call: a mesh that re-decides every
request eventually decides differently mid-conversation, and a chat whose second
turn lands on another machine has no history there. Cleared only on a transport
failure, and only when there is somewhere else to go — a single-node mesh whose
model is down should say so, not report AI as unconfigured.
PHP defaults to UTC on Unraid while every bash script stamps local time, and the
two write into the same files — conf_changes.log was four hours out from every
log you would correlate it against. The parsing half was worse: vv_ai_syslog_ts()
reads local-time syslog lines through strtotime() under UTC, landing every event
four hours early, and the repair sweep bounds its scan to "since the last pass"
— so a fault that had just happened could read as four hours old and fall
outside the window. Also fixes date-string comparisons against bandwidth and
cleanup dbs, which bash writes with local dates.
Both the reader and the writer ran to the first closing paren, which is only the
array's own close if nothing inside contains one. HOST1_WATCHDOG_SCAN_IGNORE has
carried "(exit 127 — bad image)" for weeks: writes spliced into the middle of it
and were refused by bash -n with a bare false, and reads returned the entries
above it — the Rsync tab showed no intermediate scripts at all and 10 of 17
daily. Arrays now close on a ) that starts its own line, as confform already did.
The tab collected over SSH on every 30s poll — 8.3s a load with HOST2 down.
vv_parse_conf_scalar() captured to end of line, so a commented toggle parsed as
"true # HOST2 back online": every threshold read right because (int) stops at
the first non-digit, and 38 booleans read wrong. The Fallback tab has been
showing failover disabled while it was on.
vv_conf_vars() read and regex-parsed ~1,900 lines per call and reached a shell fork through
vv_detect_host(); the repair sweep asks it for a key per log line. Keyed on a hash of the
contents rather than mtime and size, which missed a same-second rewrite to the same length.
These files are gitignored, so a bad write had nothing to go back to; the escaping order
also stored any value containing a quote truncated but still parseable, which only a
read-back can catch.
The response was discarded, so a refused stop looked like a completed one, and the payload the
card redraws from is cached for 300s with no invalidation — the container carried on showing as
running until the once-a-minute writer caught up. Stop now confirms; start still does not.
The Tools card adopted the AI scripts on the host check alone, and api/ai.php only
tested AI_ENABLED in front of ask, so a disabled subsystem still had rows to run and
an endpoint that answered.
Writing down what each endpoint actually guarantees made the places it
didn't obvious — shell arguments reaching a crontab or a bash -c
unescaped, master.conf written without tmp+rename, and conf edits that
could be saved without ever being parsed.
All 16 include/ files now carry PURPOSE / DESIGN PRINCIPLES / OPERATIONAL
SAFEGUARDS / EXPORTS / CONFIGURATION, keeping the first three section names
identical to the bash headers so retrieval can route across both languages.
monitor.php read six watchdog state files from /tmp while the watchdogs write
to STATE_DIR, so every strike set came back empty and the summary reported
healthy unconditionally. docs.php gained path containment before it is wired
to a page.
Unraid truncates the Server Name to 15 chars. Depending on which form
ends up in master.conf's HOST* value (the truncated OS hostname, or a
longer name matching what Tailscale independently registered for that
peer), either host-identity detection or Tailscale IP resolution could
fail — hit live on HOST2 in both directions this session.
- common.sh detect_hosts(): was case-sensitive exact match only, with no
fallback and an exit 1 on failure — meaning every orchestrator/watchdog/
rsync script would hard-fail on a truncated-hostname host, not just the
web UI. Now case-insensitive, with a prefix-match fallback when the
local hostname is exactly 15 chars.
- common.sh resolve_tailscale_ip(): already did a bare regex prefix match
with zero ambiguity handling (pre-existing risk: e.g. server1/server10
could collide). Replaced with an explicit unambiguous-only check.
- config.php vv_detect_host() / vv_resolve_tailscale_ip(): same treatment,
kept as close a mirror of the bash logic as PHP allows.
All fallbacks require an EXACT prefix match (never fuzzy/percentage
similarity — considered and rejected, since names like server1/server2/
server3 would be dangerously similar under any generic similarity metric)
and require exactly one candidate to qualify; ambiguous matches are
treated as no match rather than guessed. Verified live against HOST1/
HOST2 in both master.conf configurations (short and long HOST2 value).
Unraid truncates the Server Name to 15 chars (NetBIOS limit). HOST2's
real hostname is "unRAID-Jayred36" but master.conf's HOST2 (matching
what Tailscale shows for this peer, since resolve_tailscale_ip() keys
off the same value) is the untruncated "unRAID-Jayred365" — confirmed
live, Tailscale's own Self.HostName on that machine is truncated too.
vv_detect_host() did a strict case-insensitive match against the bare
`hostname -s` output with no tolerance for this, so it always returned
'unknown' on HOST2. That silently broke the first-run wizard (Varaverk.page
explicitly excludes 'unknown' from the "needs setup" check) even though
host2.conf never existed, plus vv_partner_state() and vv_fallback_active()
in monitor.php which independently reimplemented the same hostname
comparison instead of calling vv_detect_host().
Fix: vv_detect_host() falls back to a prefix match when the local hostname
is exactly 15 chars; vv_partner_state()/vv_fallback_active() now call
vv_detect_host() instead of duplicating the comparison. Verified live on
HOST2 — vv_detect_host() now returns 'host2', partner state correctly
flags HOST2 as is_me, and the wizard-trigger condition now evaluates true.
The readiness probe wrapped the remote command in raw double quotes with
manually backslash-escaped inner quotes. shell_exec() runs its command
through an extra local `sh -c` layer beyond the ssh invocation itself, and
because the remote command was double-quoted (not single-quoted/opaque),
that extra local layer expanded the $(...)/${...} substitutions using
HOST1's own environment before ssh ever sent anything to the remote host.
Confirmed live: the exact same command run directly (one shell layer)
returned the correct remote SCRIPTS_DIR; run through an extra sh -c layer
(matching shell_exec's real behavior) it silently evaluated everything
against HOST1's local varaverk.cfg instead, producing an empty probe result
every time — so every push silently reported "plugin not installed" even
though HOST2 was fully installed and reachable.
Fix: build the remote command as a plain string and escapeshellarg() it as
a whole, same pattern vv_pt_ssh() already used safely elsewhere. Verified
live — probe now returns HOST2's real SCRIPTS_DIR and the master.conf push
lands with a matching checksum on both hosts.
Custom Scripts (the Scheduler page's inline editor) used to save into the
git-tracked Custom/ folder, so anything saved there would end up on GitHub.
They now live in /boot/config/plugins/user.scripts/Varaverk/Scripts, same
folder family as Unraid's own User Scripts plugin. Import Script lets you
browse the whole server and move an existing script in instead of only
creating new ones inline — always a move, never a copy, so no stray
duplicate is left where it came from.
All state/data file paths in scripts and PHP now resolve via STATE_DIR / DATA_DIR /
PERSISTENT_CONF_CACHE instead of hardcoded /boot/config/ or /tmp/ paths, so the
ecosystem works in both internal and appdata storage modes.
PHP layer (watchdog.php, partnership.php, fallback.php, monitor.php, snapshot.php,
config.php): all state reads switched to STATE_DIR constant; remote state reads use
the new vv_remote_state_cmd() helper which resolves the remote's SCRIPTS_DIR via
their varaverk.cfg before building the path.
conf_sync.sh: fixed SCRIPTS_ROOT → SCRIPTS_DIR bug on MY_CONF path; added
_remote_scripts_dir() to resolve partner's SCRIPTS_DIR before SCP pull.
fallback.php page: added controls card (PARTNERSHIP_ENABLED, FALLBACK_ENABLED,
FALLBACK_RSYNC_ENABLED toggles), status grid, and settings card.
README and Manual updated for System_Essentials, Watchdogs, Fallback, Rsync,
Media, Monitors, Orchestrators, Partnership: added new scripts (conf_sync,
conf_cache_save/restore, conf_cache_watchdog, play_state_sync, start_webhook_listener,
upgrade_webhook_handler), corrected all stale /boot/config/ state file paths to
$STATE_DIR/$DATA_DIR, noted webgui/php_fpm/mover/user_scripts scripts moved to
Plugin/unraid/System_Essentials, fixed start_webhook_listener.sh header (Node.js,
not PHP -S).
PHP reads master.conf values like STATE_DIR="${SCRIPTS_DIR}/State_Files"
literally — the ${SCRIPTS_DIR} token is never expanded, so any PHP code
using the returned STATE_DIR value got an invalid path. Two-pass resolution:
SCRIPTS_DIR first (from PHP constant), then remaining ${VAR} tokens using
the now-resolved var set (covers DATA_DIR-based paths like ARR_SYNC_BLOCKLIST).
- Auth stack: fold cert monitor into Auth Stack page as fourth tab (Certs);
remove standalone cert page and top-level tab
- cert_monitor.sh: write JSON status cache to State_Files/cert_status.json
after each run; expose per-domain days/expiry via _CERT_DAYS/_CERT_EXPIRY globals
- api/cert.php: new — serves cached cert status; falls back to configured
domains as UNKN when no cache exists; POST action=run triggers live check
- arrs db fallbacks: vv_arr_cleanup_stats/discovery_stats/recovery_stats now
read from data/*.db files when log JSON files don't yet exist
- config.php vv_conf_vars(): unescape bash \$ → $ so passwords with dollar
signs read correctly from conf files
- host1.conf: fill in HOST1_NPM_USER/PASS and HOST1_LLDAP_USER/PASS
- Partnership adapter pattern: Unraid-specific container logic extracted to
Plugin/unraid/Partnership/; platform-agnostic structure stays in Partnership/
- First-run wizard: uniform multi-step flow for all hosts; HOST2 pull moved
to checklist; auto SSH keygen and API key creation on save
- api/checklist.php: live setup checklist with pull_master action
- Fullscreen toggle: hide Unraid header/menu; state persists via localStorage
Consolidations (config.php gains 5 shared utilities):
- vv_format_uptime() replaces 4 inline uptime-formatting blocks
- vv_parse_conf_scalar() replaces vv_arr_scalar/vv_wd_scalar/vv_fb_scalar/vv_media_conf_scalar
- vv_known_hosts() replaces vv_arr_known_hosts/vv_fb_known_hosts + inline parser in watchdog
- vv_parse_kv_db() replaces inline key=value parsing in snapshot and monitor
- vv_local_ip() replaces duplicate in docker_folders.php and inline in docker.php
All module-level function names kept as thin aliases so call sites unchanged.
Critical bug fixes:
- api/system.php: added require_once config.php and POST-only guard (no auth on shutdown)
- api/movescript.php + reorderarray.php: use vv_write_conf_raw (atomic) + vv_push_master_conf
- api/snapshot.php: share /tmp/vv_cpu_stat.json with vv_cpu_per_core() instead of own state file
Correctness:
- vv_cpu_per_core() and vv_network_stats(): atomic tmp+rename for state files (concurrent poll safety)
- ext_ip curl cache moved from /tmp/vv_ext_ip.cache to vv_cache_read/write (canonical cache dir)
- monitor_remote.php + board.php + snapshot.php: all use vv_cache_read/write instead of ad-hoc /tmp files
HTTP method guards added to write-only APIs that were missing them:
- api/scheduler.php, conf_toggle.php, flag_toggle.php
- Move SCRIPTS_DIR/DATA_DIR/STATE_DIR from appdata to /boot/config/plugins/varaverk
- All state files now in STATE_DIR (no more /tmp or /boot/config root writes)
- Bootstrap: Gitea-first clone with GitHub fallback, no array dependency
- varaverk.cfg seeded with Gitea connection settings
- .gitignore: add State_Files/, varaverk.cfg, varaverk-*.txz
- Partnership/transcode/fallback scripts use STATE_DIR variables
- PHP config.php: DATA_DIR/STATE_DIR constants, VV_SETUP_STATE_FILE dynamic
- deploy.sh PROD_ROOT updated to plugin flash dir
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix 1 — Monitor partner card (common.php + monitor.php):
vv_remote_hosts_stats() no longer skips hosts with no API key.
Returns no_api_key:true entry instead. Monitor JS renders
"API key not configured — complete Onboard to enable" instead
of a blank space.
Fix 2 — rawconf push not alarming pre-onboard (config.php + scheduler.php):
Probe failures (Tailscale not found, plugin not installed) now carry
ready:false. JS treats ready:false results silently — button shows
"✓ Saved" not "push failed: HOST2" before SSH is set up.
Only genuine post-onboard failures (ready not false) show as errors.
Fix 3 — Partnership Onboard button (partnership.php):
Disabled with tooltip when no partner hostname is in master.conf.
When partner configured but not onboarded, adds explanation note:
"Onboard will generate your SSH key, exchange it with the partner..."
No partner configured: "Edit master.conf and set HOST2."
Varaverk.page: also shows wizard when local host.conf is missing
(handles master.conf pushed by HOST1 before HOST2 installs plugin).
pages/setup.php: three wizard flows
- Standard: blank master.conf, fill hostnames, redirect to scheduler
- Host2/state file: state file detected, pull master.conf from HOST1 via SSH
- Conf-only: master.conf already filled (was pushed), just create local host.conf
api/setup.php:
- save action: writes master.conf + host.conf, creates varaverk_setup.db state file,
redirects to ?tab=scheduler&vv_setup=master.conf
- pull action: resolves HOST1 Tailscale IP, queries HOST1 SCRIPTS_DIR, SCPs
master.conf, creates local host.conf, redirects to ?tab=scheduler&vv_setup=hostN.conf
include/config.php: vv_setup_state_read/write/push helpers.
vv_push_setup_state() pushes varaverk_setup.db to /boot/config/ on all known
remotes — no plugin-readiness probe needed (flash is always accessible).
api/rawconf.php: calls vv_push_setup_state() alongside master.conf push.
pages/scheduler.php: setup mode via ?vv_setup=<conf> URL param.
Auto-opens the specified conf file on page load (DOMContentLoaded).
vvSaveRawConf: in setup mode, skips confirm dialog and forces sequence:
master.conf save → auto-open hostN.conf
hostN.conf save → redirect to Monitor (setup complete)
Single SSH probe per remote checks all three conditions in one call:
- plugin cfg exists (varaverk.cfg present with SCRIPTS_DIR)
- Configurations/ dir exists at that path
- master.conf already present on remote
Any failure skips that host entirely rather than pushing blind.
Uses the discovered SCRIPTS_DIR as the SCP destination, not ours.
Instead of assuming the remote uses the same appdata path as the local
host, SSH-grep their varaverk.cfg first and SCP to their actual
Configurations/ dir. Falls back to /mnt/user/appdata/Varaverk if the
plugin isn't installed or the file can't be read.
vv_push_master_conf() in config.php: finds all remote HOST* entries,
resolves Tailscale IPs, SCPs master.conf to each.
rawconf.php calls it after a successful write when file=master.conf.
Scheduler save button shows sync status: "✓ Saved · synced to HOST2"
or "✓ Saved · push failed: HOST2" on error.
Also fix path bug in partnership_transfer.sh — was SCPing to
appdata root instead of Configurations/.
Gmer4Lfe