Commit Graph
226 Commits
Author SHA1 Message Date
Gmer4Lfe 0447fa6f86 Remove point-in-time HOST2 status from the codebase
Comments asserting a partner is down, being rebuilt or not yet installed are true on the day they are written and wrong afterwards, and nothing prompts anyone to revisit them. Behaviour is described generically; current state belongs in operator memory, not in tracked files.
2026-08-17 15:20:25 -04:00
Gmer4Lfe b108a6fe62 Report the partner's API key and whether it is actually answering 2026-08-17 14:45:38 -04:00
Gmer4Lfe f58ff68ab3 Mesh partnership leg reads this host's own state file, not one named after the remote 2026-08-17 14:43:03 -04:00
Gmer4Lfe e7e9d9fecf Green mesh banner on each remote card, naming the failing leg when it is not green 2026-08-17 14:42:15 -04:00
Gmer4Lfe d417bd5360 Match the tailnet device name to the conf hostname by unambiguous prefix
master.conf says unRAID-Jayred36 and the tailnet device is unraid-jayred365, so the exact-key lookup found nothing and every remote card rendered with no IP, no online state and no containers.
2026-08-17 14:41:35 -04:00
Gmer4Lfe bce3bf61e9 Partnership page: one settings card instead of three, and a Data Transferred card
Windows with runs but no bytes say 'no data moved' rather than 0.00 GB, and rows before the byte-parser fix are excluded rather than summed into a false total.
2026-08-17 14:36:21 -04:00
Gmer4Lfe 1231cd69a8 A host with the VM service off is not a host with a broken API key
The API answered and simply had no vms node, but that was recorded as an API fallback, and the monitor reads any fallback with a key present as 'Unraid API unreachable — check API key in host conf.'
2026-08-17 13:47:19 -04:00
Gmer4Lfe d8a77c700b Stop counting a gated-off critical rsync as an unreachable partner, and report the counter in the unit it is stored in
The guard tested Tier 1 only, so onboard Step 1d's posture — Tier 1 open, every Tier 2 closed — made a healthy partner climb toward auto-offboard every 30 minutes. The counter holds intervals, not days, and was rendered raw.
2026-08-17 11:39:19 -04:00
Gmer4Lfe 5fc53948c6 Treat an ACTIVE partnership as phase 2 regardless of the flags
HOST<n>_PHASE*_DONE only ever describe the mirror, so on the mirror the owner's card computed phase 0 and offered to onboard the host that had just onboarded it.
2026-08-17 11:36:00 -04:00
Gmer4Lfe ff94ccb343 Make the setup-state push report whether it worked
It was documented 'Always returns 0', so the offboard's new 'Phase flags pushed' check was testing a constant and ticking regardless.
2026-08-17 10:57:56 -04:00
Gmer4Lfe 42711ff9ac Bring the seed docs in line with Phase 3 2026-08-17 10:24:41 -04:00
Gmer4Lfe 7c957d96be Make seeding Phase 3 — its own operator-triggered phase, not the tail of an onboard
Onboard now ends at Phase 2: connected, running, and saying so. Phase 3 sets the seeding gate posture and dispatches the seed. With MEDIA_SEED_ENABLED false there is no Phase 3 and the model is two phases.
2026-08-17 10:23:03 -04:00
Gmer4Lfe 1a47f864f9 Put the media seed behind MEDIA_SEED_ENABLED so a partner filled by other means never starts a multi-week transfer
Tier 2 beside the per-orchestrator gates. Unset reads as on — the toggle postdates the seed.
2026-08-17 07:46:51 -04:00
Gmer4Lfe ad623353bc Stop the media seed from holding the onboard, and therefore the partnership, open for weeks
A first seed is ~28 TB behind a 12.5 MB/s bwlimit, and it ran inline as Step 9d, so the
phase-2 flag every status reader depends on was written only after it finished.
2026-08-17 07:40:00 -04:00
Gmer4Lfe 3676526daf Remove the Docker tab and stop overwriting folder.view3's file, which destroyed folders made in its own UI 2026-08-17 05:11:00 -04:00
Gmer4Lfe a46317de18 Make tailscale removal opt-in, since it depends on a key that expires and no-ops silently once it has 2026-08-17 04:49:32 -04:00
Gmer4Lfe cce1e25c2b Let a non-blocking checklist item be dismissed with a recorded decision, and run discovery on the mirror once there is something to discover 2026-08-16 21:56:00 -04:00
Gmer4Lfe 3ae6298656 Let a partner adopt the owner's custom networks from the conf phase 1 already cached, instead of asking for a value it cannot know 2026-08-16 21:20:59 -04:00
Gmer4Lfe 93cdbaac57 Give the mirror the terminal step its half of phase 1 actually needs, instead of a button that cannot answer a password prompt 2026-08-16 20:16:17 -04:00
Gmer4Lfe 35e59d2510 Let the plugin read partner vars from the same RAM cache bash reads, so PHP and bash cannot disagree about a partner 2026-08-16 19:40:49 -04:00
Gmer4Lfe 4f95cc6d13 Deliver master.conf to a node that has no Varaverk on it yet, which is the node that needs it
Phase 1 exists to hand the partner its identity before the install, but the push refused any
host without varaverk.cfg — so the conf could only ever reach a node that no longer needed it
to be told who it was. It now resolves the remote's conf directory across both install layouts
and creates the internal one when neither exists.
2026-08-16 16:14:23 -04:00
Gmer4Lfe 0fe3435033 Let the conf push seed a node that has no master.conf yet, which is the node it exists to seed 2026-08-16 13:05:04 -04:00
Gmer4Lfe f2ad41e8e2 Show how each domain has behaved over a day, a week, a month and a year, not just now 2026-08-16 11:55:25 -04:00
Gmer4Lfe 01601d210b Watch the auth stack on a schedule, so nobody has to open the tab
Both checks already answered their question on demand and both needed
somebody to press a button on the right row. One host here has returned
nothing but 5xx for months.

Filed as findings, which is the existing answer to a condition that
persists while nobody is looking. Grouped by cause rather than by
hostname: a default policy of bypass produced twenty-two findings that
were one sentence repeated, and they have one fix between them.
2026-08-16 01:59:11 -04:00
Gmer4Lfe 99b58c0c4f Make the Auth tab explain a number instead of only showing it
A low uptime figure, a refused login and a certificate that stopped
renewing all looked the same from the row: a number, with the reason
split across NPM, an Authelia config and the directory.

The why-check goes and looks — TCP to the forward target, HTTP through
the proxy, a second handshake with verification off to tell a broken
certificate from a broken service. Forward hosts are docker names that
only resolve on NPM's network, so an unresolvable one is redirected to
the container address and the substitution is reported; a check that
could not be made must never read as a check that failed.

The access simulator walks the rules the way Authelia does and shows the
ones it stepped over, reading whichever instance the chosen host points
at rather than the one conf names — there are two here.

Cert triage counts runs rather than log lines and orders by rotation
suffix rather than mtime, both of which change the answer.
2026-08-16 01:59:01 -04:00
Gmer4Lfe 1001c25487 Probe every domain every minute, and keep it out of its own traffic numbers
NPM's access log only describes hosts somebody visited; the host most likely to be quietly broken
is the one nobody does. Probes carry a User-Agent npm_access_stats.sh drops — unmarked, this
monitor would be fifty thousand requests a day in the very logs it reports on.
2026-08-15 20:28:30 -04:00
Gmer4Lfe 13de1dab82 Stop a scalar NPM response killing the request, and add certificate deletion
NPM answers a successful DELETE with the bare literal true, which vv_npm_raw() is typed to
return an array from — so deleting or toggling a proxy host has always thrown a TypeError
rather than doing the thing.
2026-08-15 15:51:29 -04:00
Gmer4Lfe 8157673291 Add AUTH_STACK so the Auth tab follows the stack in force
Authentik is the likely destination and the page had Authelia and lldap wired in at every
level, so the seam goes in now: the panels and every endpoint action route off one conf value,
and a stack that cannot be driven yet says so rather than drawing controls with nothing behind.
2026-08-15 15:27:50 -04:00
Gmer4Lfe 81aeb0619b Reach the rest of what lldap exposes — photos, real names, group rename
lldap lets you edit five things about a user and one about a group; the page reached two of
them, so correcting a surname or a group's name still meant opening the container's own WebUI.
2026-08-15 11:45:23 -04:00
Gmer4Lfe 693ac693a4 Make access-control domains an editable list, and stop saves eating the rule labels
The block is rebuilt from the parsed model on every save and the parser discarded comments,
so one press of Save deleted the five ## lines that are the only thing in the file saying
what each rule is for — which inline editing was about to make far easier to press.
2026-08-15 11:30:36 -04:00
Gmer4Lfe 0101aef51a Let a password containing a $ actually save, and say why when a save is refused
The value was written as typed, bash expanded it when the read-back sourced the file, and
the guard rolled the whole write back with nothing on screen but "save failed" — which is
also what an empty value, a trailing space, and a stale API-key check had been doing.
2026-08-14 23:39:12 -04:00
Gmer4Lfe 9eef5b50e6 Tell the three auth failures apart, and give the Auth tab a settings card
Not set, rejected and unreachable all reached the page as one message about checking
credentials, which sends you to a password when the field is simply empty — as both of
HOST1's were, with no card on the page to fill them in from.
2026-08-14 23:14:33 -04:00
Gmer4Lfe 2bda4cdaf8 Declare the AI owner, and let every node borrow its model over the mesh
vv_is_ai_host() was `=== 'host1'`, which made a physical fact — that is where the
GPU is — look like a rule. AI_OWNER_HOST declares it, so the card can move to a
rebuilt host3 or a friend's spare.

The gate was also doing two jobs. Assistant docks and findings strips now ask
whether a model is reachable, so a node without a GPU gets them by borrowing;
the AI tab asks whether this is the owner, because that page carries the bug
reports, the index and the model configuration — the surface where the
vocabulary assumes you built the mesh.

Resolution is local, then owner, then anyone else declaring a model, pinned once
it answers. Pinned rather than re-derived per call: a mesh that re-decides every
request eventually decides differently mid-conversation, and a chat whose second
turn lands on another machine has no history there. Cleared only on a transport
failure, and only when there is somewhere else to go — a single-node mesh whose
model is down should say so, not report AI as unconfigured.
2026-08-14 22:54:26 -04:00
Gmer4Lfe b708a90548 Attribute findings to the host they are about and the host that answered
Identity now includes the host: "Bazarr is on the skip list" is a different fact
on each machine that says it, and without the host in the hash the second node
to report a subject would land on the first node's record — one machine's
dismissal silencing another's live fault.

Nine dismissed findings existed when this changed, so the write path migrates a
legacy id once, at the only moment it matters. Verified: a dismissal survives the
rename rather than resurfacing as new.

acted_by is stamped in apply_action, not set_state — the sweep calls set_state to
move a candidate to needs_operator, and crediting a machine for classifying is
not the same as recording who decided.
2026-08-14 22:39:38 -04:00
Gmer4Lfe 90774f1442 Put the AI toggles and findings on the pages they act on
The five AI_ASSIST_ switches and the findings they produce were reachable only
from the AI tab, which is a long way from the page a finding is about.

Nothing was synchronised because nothing needed to be: findings are one file per
finding, and every surface is a view over that store with actions going to the
same endpoint. Acting on the Watchdog tab shows on the AI tab because they are
not two copies. The strip reloads rather than editing its own row, which is the
only way they could have drifted apart.

Each strip shows one page's kinds. Actions are whatever the server offers for
that row, so Move appears on media findings without this card knowing what a
move is.
2026-08-14 22:32:55 -04:00
Gmer4Lfe a705aa36b7 List each AI enhancement against the script it enhances
Declared in PHP rather than in each bash header, against scriptinfo's usual rule
that the header next to the code is authoritative: it is authoritative about what
the script does, and an enhancement is something else reading its output, gated
by a flag the script has never heard of. A description of PHP inside a file that
cannot enforce it would drift the first time either changed.

Shown with its switch, so "there is an enhancement" and "it is running" are never
the same claim. Discovery is listed even though nothing acts on its output — it
takes the first accessible root folder with no regard for content, and the
classification scan is what notices that night. That relationship explains where
misfiled series come from and was written down nowhere.
2026-08-14 22:25:24 -04:00
Gmer4Lfe 60971f1af4 Let a misfiled series be moved from the finding that reported it
The triage could say Narcos is filed under anime and offer nothing to do about
it. Findings that recorded a destination now carry a Move action, which the AI
tab renders as a button on its own — actions are read from the server, so no UI
knew this existed. Armed like Fix and Dismiss, because it is the only action in
that card that relocates files rather than settings and the undo is another move.
Offered only where the triage said misfiled, never for uncertain: three
narrowings before a file is touched. Re-reads the series from Sonarr first, so a
finding gone stale cannot undo a move made by hand in the meantime.
2026-08-14 22:13:25 -04:00
Gmer4Lfe 00dee37e22 Triage the classification leaks the metadata rule says it cannot judge
The scan's own header calls reverse-anime leaks "genuine judgment calls" and
reports them as a count. Seventeen of them concealed two live-action crime
dramas filed under anime for as long as nobody read the list. The scan now
persists that bucket as its own verdict, and the sweep asks the model to sort
only those eleven — not the eleven hundred it already classifies correctly.
Files findings for misfiled and uncertain only; donghua and anime-adjacent are
answers, not problems, and filing them would rebuild the undifferentiated list
this exists to break up. Moves nothing — --move stays a flag a human types.
2026-08-14 22:03:55 -04:00
Gmer4Lfe 5ad5c0df7e Rename the Arrs tab to Media Stack, now that it is one
It covers the pipeline rather than only acquisition: the arrs, the jobs that run
over the same files, and the servers that play them. The label moves; the tab id,
file and URL do not, so bookmarks and the cache key keep working.

The settings card widens with it — Play State Sync, Media Permissions, Media
Cleaner, Emby and Jellyfin were reachable only through the Settings catch-all,
which is a long way to go for a setting named after the page you are on. Match
is whole-word, so "Intermediate Sync" is not caught by "media"; section count is
unchanged at 126 with the same 5 gaps, so nothing was taken from another tab.
2026-08-14 21:47:33 -04:00
Gmer4Lfe 00328cbd0a Report network throughput in bits, which is what everything downstream says
/proc/net/dev counts octets; the field is named rx_bps, vvFmtBps renders Kb/s
and Mb/s, and the Monitor card prints it directly under the NIC link speed from
/sys/class/net/*/speed, which genuinely is megabits. So the one figure you would
read against the link understated traffic eightfold — 811 KB/s shown as "0.8
Mb/s" beside a 10 Gb/s link when it was 6.5. Converted at the source so the
field name stops lying; both consumers are bit-labelled or relative.
2026-08-14 21:36:30 -04:00
Gmer4Lfe 7f22bb0612 Run the PHP layer on the host's clock, not UTC
PHP defaults to UTC on Unraid while every bash script stamps local time, and the
two write into the same files — conf_changes.log was four hours out from every
log you would correlate it against. The parsing half was worse: vv_ai_syslog_ts()
reads local-time syslog lines through strtotime() under UTC, landing every event
four hours early, and the repair sweep bounds its scan to "since the last pass"
— so a fault that had just happened could read as four hours old and fall
outside the window. Also fixes date-string comparisons against bandwidth and
cleanup dbs, which bash writes with local dates.
2026-08-14 19:49:48 -04:00
Gmer4Lfe 7ebe8c0891 Judge load on the Watchdog page against the same count the watchdog uses
The page multiplies its core count by the RW_LOAD_* multipliers to colour the
load bar, and resource_watchdog.sh computes its real thresholds from nproc. The
page preferred the API's physical core count — 16 against 32 threads — so it
turned amber at load 32 and red at 48 while the watchdog did not reach soft
pressure until 64 or medium until 96. A page whose whole job is reporting what
the watchdogs think was reporting a crisis about a machine they considered idle.
The remote path already sent the partner's own nproc, so both nodes now agree.
2026-08-14 19:42:14 -04:00
Gmer4Lfe dbc0c8c6d2 Report container CPU as a share of the machine, matching Unraid's Docker page
docker stats sums CPU across every logical processor, so Jellyfin's "227%" is
2.3 threads busy — 7% of a 32-thread host, which is what Unraid shows and what
the operator was reading. The card was printing the raw figure and colouring it
against thresholds meant for a percentage, so ordinary background work looked
like an emergency; I twice flagged Jellyfin as pegged on that basis and was
wrong both times. Divides by nproc, not the watchdog's `cores` — that field is
the physical 16 because it compares load averages, and borrowing it here would
have looked reasonable and doubled every reading.
2026-08-14 19:39:14 -04:00
Gmer4Lfe 04a0dcb39a Let the media server card show the whole mesh, not just this host
A partner's Emby URL is http://localhost:8096 — true there, meaningless here —
so nobody queries a partner's media server directly. Each host reports its own
over the SSH call remote_arr_cache_writer.sh already makes, which keeps its API
keys on it and costs no extra connection. Local is the default: opening the tab
to check your own server should not first make you read past a partner's.
A partner with no cache is stated as not collected rather than omitted, because
an empty mesh view and a partner that is down look identical otherwise.
2026-08-14 19:30:23 -04:00
Gmer4Lfe 24710af156 Add media server cards, and stop re-counting whole libraries every minute
The counts need the entire library downloaded to compute six numbers — 16.0 MB
from Radarr and 5.5 MB from Sonarr, measured — and they were on the same
one-minute clock as queue depth and health, which pulled ~30 GB a day out of the
arrs to re-count records that had not changed. They now cache for fifteen
minutes against how often the numbers actually move: 7.6s to 3.0s per refresh.

Emby and Jellyfin get a card each: version, CPU, memory, uptime, streams, users
and transcodes, with update/restart flags. Two halves of one question — a server
answering happily at 145% CPU is a different situation from one at 8%, and
neither the app nor the container says so alone.
2026-08-14 19:23:10 -04:00
Gmer4Lfe 1381a526ab Surface the three media jobs that appeared in no tab at all
play_state_sync, media_shares_permissions and media_cleaner work on the same
files the arrs manage and were visible only by opening the Scheduler and reading
an orchestrator's log. Readable at all because run_orch_child() now writes a run
record and a per-script log for its children — this could not have been written
yesterday. Each card shows the sentence the script itself ended on rather than a
count re-derived here, since the three word their outcome differently and the
wording is the part worth reading.
2026-08-14 18:51:54 -04:00
Gmer4Lfe 04aba81ab3 Show orphan and junk sizes, and drop a cleanup field nothing could fill
The arr cleanups are entries in DAILY_MAINTENANCE_SCRIPTS, which the orchestrator
invokes with plain bash, so they never get the run record the primary parse needs
— every payload has come from the daily aggregate db. That path read every column
except the byte counts, leaving orphans_sz at its "0B" default, which is
invisible at zero orphans and would have read "12 orphans (0B)" the first time
there were any. 'total' had no source there and no reader anywhere, so it is gone.
Arr Sync now reads "disabled" rather than "never run" when its switch is off.
2026-08-14 18:15:56 -04:00
Gmer4Lfe 2cd1384786 Add timed mutes so a temporary problem stops needing a permanent exemption
Every existing exemption lasts until someone remembers to undo it, and nobody
does — Healarr has sat in a pressure list since it was uninstalled and seven
ignore entries name containers that are gone. A mute states when it ends and
then ends, capped by WATCHDOG_MUTE_MAX_HOURS so temporary is enforced rather
than intended. Applied where IGNORE_MAP is built, so all five check sites
inherit it, and shown with its countdown because an invisible suppression is
the thing being fixed.
2026-08-14 16:28:59 -04:00
Gmer4Lfe 9f32644c32 Show when a watchdog cycle last completed
Without it the page renders zeros, all-clear pills and green dots when the
orchestrator has stopped entirely — its healthiest appearance describing a
machine with no watchdogs, against a header promising absence is never drawn as
an all-clear. The heartbeat already existed; overdue is measured against the
cron schedule rather than an assumed fifteen minutes.
2026-08-14 16:18:14 -04:00
Gmer4Lfe a7387c27fb Mark watchdog list entries that name a container which no longer exists
Half the hard-pressure stop list is containers that are not installed, so at
level 3 it sheds half the memory the card implies. Badged on the lists where a
dead entry costs something and deliberately not on Scan ignore, where ignoring
something already gone is the point — a card of dashed pills by design would
teach the eye to skip the badge where it has to be believed.
2026-08-14 16:06:02 -04:00