Commit Graph
104 Commits
Author SHA1 Message Date
Gmer4Lfe 2ccc93bb14 Send a bug report to one place, on purpose, after reading it
The page built its own markdown and nothing could send it; local and upstream
are different people, so neither falls back to the other.
2026-08-13 16:32:36 -04:00
Gmer4Lfe a04368e9fc Notice when the question is about their own machine
The other detector recognises named components, so its complement is unbounded;
this one reads grammar instead, which is not.
2026-08-12 18:08:50 -04:00
Gmer4Lfe 0eccd7b36e File the repair log settings under the section that documents them
They sat under AI Web Search, which only mattered once the settings card
started grouping by the header above a key.
2026-08-11 18:36:37 -04:00
Gmer4Lfe 29e0daf80d Let the assistant propose what to remember, and the operator decide what is kept 2026-08-10 22:00:42 -04:00
Gmer4Lfe 57f3bf2807 Split memory into assisted and learned, and deny learned the precedence assisted has 2026-08-10 21:50:35 -04:00
Gmer4Lfe 6054f68754 Stop conf_upgrade losing every key that follows a single-line array 2026-08-10 18:42:25 -04:00
Gmer4Lfe 74b579e281 Remove dead PCIe hardware at array start so correctable AER errors stop burying the syslog 2026-08-10 18:22:09 -04:00
Gmer4Lfe 0c16c6db63 Add degoog as the search provider, and make it the default
The operator already runs it, so this is the one provider that costs nothing to adopt: no key,
no card, no third party, and no new container. It aggregates several engines and returns them
merged, and its /api/search reply is already the shape everything downstream expects — title,
url, snippet, with content as a fallback for the roughly one result in six that has no snippet.

It is also now the only provider whose response has been read from a live service rather than
from documentation, which is why it is the default.
2026-08-09 22:50:47 -04:00
Gmer4Lfe 613634473a Let General Chat search the web, and only General Chat
Search is the one capability that fits the profile holding none: every other capability either
reads this installation or changes it, and chat's whole contract is that it can do neither. The
assistant deliberately does not get it — its contract is that answers come from this install's
own documents, and a web result there is an answer that looks sourced and is not. A chat question
about this machine hands off to the assistant before the search would run, so it never reaches
the internet even with the box ticked.

Off by default, and not because it is dangerous: searching sends the operator's question outside
the house, which is theirs to decide. Asked for per turn as well as enabled in conf.

Provider-agnostic, as asked — searxng, brave, tavily. Only whichever is configured here can be
verified; all three read every field by name, so a shape that changes underneath yields no
results rather than wrong ones.

The explain fixtures asserted chat holds no capabilities at all, which is exactly the guarantee
worth keeping. caps=only: now states the set rather than its emptiness, so the check still fails
the day something else is granted there.
2026-08-09 22:42:43 -04:00
Gmer4Lfe 426ca2e5c7 Read the system log and the containers, not only Varaverk's own logs
Triage could see what Varaverk wrote about itself and what the arrs said about themselves, and
nothing else — so a disk throwing I/O errors, a filesystem going read-only or a PCIe link
retraining every two minutes was invisible to the thing whose job is noticing. Container state
was already watched; no line any container actually wrote ever was.

Container patterns match the environment rather than the application. Fifty containers are fifty
programs with no shared vocabulary for malfunctioning, but an exact shared one for a full disk or
a corrupt SQLite file, because those strings come from libc and SQLite rather than from the app.

Both halves are checked by Tools/ai_log_check.sh, which is two tests because the failure modes
are opposite: fixtures for recall on faults this host has never had, and a replay of its real
logs for precision — 74,519 syslog lines and 79,193 container lines, matching only the PCIe
errors it genuinely has.

Severity was being graded from a hand-picked three fields, so every one of these would have been
filed as a warning however bad it was, and notified as one.
2026-08-09 22:16:04 -04:00
Gmer4Lfe 91357a2d03 Tell the operator when a finding needs them
A finding nobody is told about is a finding nobody has, and the card added earlier only shows
them to someone who opens the tab. Only needs_operator is announced — an open finding may still
be repaired by the next pass — one notification for all of them, and each is announced once and
stays quiet until the fault changes or gets worse.

vv_notify() hands the message to common.sh's notify() rather than reimplementing the channels,
and calls detect_hosts() explicitly because load_config.sh deliberately does not: without it the
Unraid notification arrives and Discord silently never does. It also reports false when no
channel is switched on at all, since notify() exits 0 either way and a caller believing that
would mark a finding as told and never mention it again.

Notification text is folded to ASCII. Unraid's notifier dropped an em dash outright and left the
double space behind, which was found by sending one and reading what arrived.
2026-08-09 21:59:50 -04:00
Gmer4Lfe 3fac5751e4 Point the master template at scripts that exist
Nine entries named paths nothing lives at — a directory that was renamed to System_Essentials,
and docker_update_remaining.sh which has never existed — and the API key renewal was absent
from both arrays it runs from on this host. A fresh install would have skipped all of it.
2026-08-09 20:00:58 -04:00
Gmer4Lfe e91a74d31f Sweep finished runs for misconfigurations instead of waiting to be told
Nothing in Varaverk fires when a job ends, so this picks up run records that completed
since the last pass rather than adding a hook to forty scripts. Ahead of stability in the
cycle on purpose: a wrong port is not fixed by rebooting the machine.
2026-08-09 19:38:02 -04:00
Gmer4Lfe a8654280a7 Prove a repair before writing it, and gate writing separately from looking
A model can be confident a port should be 8686; a probe can report that 8686 answered, and
only the second is a fact. Detecting and repairing are also separate things to trust, so
the feature runs read-only until the write switch is turned on under it.
2026-08-09 19:23:57 -04:00
Gmer4Lfe 083b5ec5ad Record what is misconfigured on this host, apart from what is broken in Varaverk
A bug stays open until the code changes and nothing here can close it; a finding closes
itself when the probe that proved the fault starts passing, so the two cannot share a
store. Acknowledging one is scoped to the value it was acknowledged at.
2026-08-09 19:07:45 -04:00
Gmer4Lfe d9f917ecef Give conf writes a backup, a lock and a read-back before they count
These files are gitignored, so a bad write had nothing to go back to; the escaping order
also stored any value containing a quote truncated but still parseable, which only a
read-back can catch.
2026-08-09 19:07:22 -04:00
Gmer4Lfe 3a45c18555 Document today's restructures, and give data/ a README that ships with it 2026-08-09 00:01:05 -04:00
Gmer4Lfe 38cd3edf99 Scope the migration's busy check to the installation being migrated 2026-08-08 23:53:13 -04:00
Gmer4Lfe e406bd445d Move the data directories too, not only the files inside them 2026-08-08 23:43:57 -04:00
Gmer4Lfe f1603349cc Put everything Varaverk persists under one root, state included 2026-08-08 23:26:46 -04:00
Gmer4Lfe d5c36db531 Put every tmpfs path in one place both layers can read 2026-08-08 22:57:29 -04:00
Gmer4Lfe 1e324636e0 Keep conversations, so a useful answer outlives the tab it was asked in 2026-08-08 22:00:58 -04:00
Gmer4Lfe e0cf83c944 Let one host template upgrade any host's conf
The pull substituted only HOSTN_, so bare HOSTN in comments kept tripping
conf_upgrade's own guard and no host conf had upgraded since the guard landed.
2026-08-08 13:08:24 -04:00
Gmer4Lfe 83508cfb1b Pull partner token ledgers into a RAM cache so fleet totals are fleet-wide 2026-08-04 18:14:38 -04:00
Gmer4Lfe 969a85f303 Record token usage per turn and show daily, weekly and all-time totals by host 2026-08-04 18:04:46 -04:00
Gmer4Lfe 0cf208b80a Remove the Open-WebUI retrieval bridge — Varaverk is the only Ollama client now 2026-08-04 17:11:46 -04:00
Gmer4Lfe 9c8eb8a2bc Refresh the AI index after a pull that changed tracked files
AI_INDEX_ON_PULL was designed, defaulted false, and never read by anything —
so nothing has ever rebuilt the index automatically. A pull is the only thing
that changes tracked files on a server, which makes it the only moment the
index can go stale, and a timer would do nothing 23 times a day while still
drifting from the pull that matters.

Staleness is invisible in the answers: the index keeps returning the old text
with confident, correct-looking citations. Gated on pull success, on
AI_INDEX_ON_PULL and on AI_ENABLED, and never fatal — a git pull must not fail
because an embedding call timed out.
2026-08-03 18:26:13 -04:00
Gmer4Lfe d0b3588f6c Give the assistant a standing memory file
A small operator-written file handed to the model at the start of every
conversation — who you are, how this install is set up, what has already been
decided. Injected ahead of the retrieved passages and marked operator-authored
so it outranks anything they contradict, and never cited as a source.

Deliberately not indexed and deliberately under DATA_DIR: it changes
constantly, vector similarity is the wrong way to retrieve things you were
told to remember, and gitignoring it keeps personal notes out of a pushed
repository. The character cap is a context budget — this text costs its share
of 16k on every single turn.
2026-08-03 17:11:27 -04:00
Gmer4Lfe 8896e05e35 Add a retrieval-only HTTP bridge so Open-WebUI can reach the index
Open-WebUI runs in its own container with no WebGUI session, and nginx applies
auth_request to everything it serves, so it cannot call the plugin API. Same
shape as the arr webhook listener: node outside nginx, shared secret, one
read-only verb. Serves chunks rather than answers because the caller already
has a model loaded.
2026-08-02 17:09:51 -04:00
Gmer4Lfe 2d07c7d80e Refuse a conf_upgrade that would keep nothing
Merging the raw host.conf.template against a live host conf matched HOSTN_
against HOST1_, classified all 140 real keys as deprecated and would have
removed every credential on the host. Also points the Ollama model default
at a tag that still exists.
2026-08-02 12:52:49 -04:00
Gmer4Lfe 6a959fb5e4 Add AI entry points, conf schema, and folder docs
ai_index.sh and ai_query.sh follow the usual conventions — fail-closed gate,
root check, lock, dry-run, status — with Node doing only the vector maths and
SQLite blobs, the same split api_cache_writer.sh uses for PHP.

AI_* and HOST*_OLLAMA_* land in both confs and both templates in this pass.
Everything ships off: AI_ENABLED false, every AI_ASSIST_* false, conf writes
disabled with an empty whitelist. Nothing in the ecosystem consults it.
2026-08-02 01:21:24 -04:00
Gmer4Lfe e2e76560a2 Fix Authelia config path and stale schedule comments in conf templates
The Authelia config default pointed at /mnt/user/appdata/Authelia, which does
not exist — the auth stack lives in the Critical-Data share so it gets synced.
Because conf_populate only fills empty fields, that wrong non-empty default
blocked auto-detection permanently and left the auth page's rules panel dead.

Also corrects three schedule comments that no longer matched varaverk.cron and
two references to scripts that have since been renamed or split.
2026-08-02 00:13:53 -04:00
Gmer4Lfe ab169a6f48 Document the Deployment folder
It was the only folder without docs, and it holds the only versioned record of what
configuration exists — the confs themselves are gitignored.
2026-08-01 23:12:18 -04:00
Gmer4Lfe 132a657f00 Refuse to auto-detect a container when the prefix is ambiguous
On this host "authelia" matches both Authelia and Authelia-Secondary, and taking the first
match writes the wrong instance into the conf every other script then trusts.
2026-08-01 22:44:23 -04:00
Gmer4Lfe 7669cd75b8 Install upgraded confs by atomic rename
A copy truncates the live conf and writes into it, so anything sourcing load_config.sh
during that window reads a partial file with empty path variables.
2026-08-01 22:44:23 -04:00
Gmer4Lfe 19a9f6410d Cache fanart.tv misses so the nightly art run stops re-querying artwork upstream has never had 2026-07-28 18:51:06 -04:00
Gmer4Lfe 74ea1a5218 Run the download orphan cleaner daily — a weekly batch of newly-eligible orphans can exceed the delete cap, which aborts the whole pass and rolls the backlog into an even larger next run 2026-07-27 17:24:40 -04:00
Gmer4Lfe 19036621dc Remove dangling emby-fallback example from host critical-sync template 2026-07-26 10:57:09 -04:00
Gmer4Lfe d96ddc8649 Condense conf redundancy: drop dead emby-fallback profile, no-op PROFILE_RETRY_COUNT, and default-valued profile rows; compose MEDIA patterns and Radarr protected patterns by reference 2026-07-26 10:51:31 -04:00
Gmer4Lfe 895347f20f Add weekly download orphan cleaner — nothing covered the SAB Completed folders and 755G of orphans accumulated since 2022 2026-07-26 09:55:17 -04:00
Gmer4Lfe 9425d16c19 Skip redundant restart for containers docker_update.sh already rebuilt
docker_update.sh rebuilds (stop+recreate) any container whose image
changed, in every mode — but for daily/weekly that was always followed
by the restart script's own unconditional pass, stopping and starting
the same container twice back to back. docker_update.sh now records
which containers it rebuilt this run to a file; docker_daily_restart.sh
and docker_weekly_restart.sh read it and skip those specifically,
still restarting everything else as before. A file older than
DOCKER_UPDATE_REBUILT_STALE_HOURS (default 12) is discarded rather
than trusted, so a missed or failed update run can't suppress a
restart indefinitely.
2026-07-19 16:09:55 -04:00
Gmer4Lfe 54919db13f Sync Sonarr/Radarr PROTECTED_PATTERNS template with live config
Template was missing theme.mkv/theme.mp4 and all trailer patterns,
including trailer-*.* for Trailarr's hyphenated TV trailer naming
(trailer-2.mkv) — a fresh install would hit the same false-positive
orphan bug just fixed live on prod.
2026-07-19 10:22:12 -04:00
Gmer4Lfe 2a09529e75 Document expanded FFPROBE_PATH_MAP coverage pattern in host.conf.template 2026-07-18 23:28:09 -04:00
Gmer4Lfe efa2f2edf8 Schedule arr_corruption_scan.sh --remediate weekly in master.conf.template 2026-07-18 22:54:22 -04:00
Gmer4Lfe 004bde2c62 Add strike system to arr_corruption_scan.sh — require repeat corrupt detections before remediating 2026-07-18 22:50:19 -04:00
Gmer4Lfe ab35fcd84d Add arr_corruption_scan.sh — sequential ffprobe-based corruption scan for Sonarr
Healarr does the same job but crashes on a Go concurrency bug (unsynchronized
map access) whenever multiple corruption events land close together — confirmed
via its own crash log, not fixable from our side. Processing one file at a time
here sidesteps the whole bug class instead of trying to work around it.

Delete + explicit EpisodeSearch rather than relying on Sonarr's own background
missing-search cycle, since that skips unmonitored episodes and this shouldn't.
2026-07-18 16:48:32 -04:00
Gmer4Lfe d5d1403ebf Raise Radarr cleanup safety threshold to 30GB
15GB was tripping on legitimate quality-upgrade orphan batches.
2026-07-18 12:05:21 -04:00
Gmer4Lfe 7bf47be106 Poll MoveMovie command to completion instead of trusting DB fields
Mirrors the Sonarr fix — same MoveMovieService one-at-a-time drain architecture,
never confirmed live on the Radarr side but the DB-instant/move-async split
is identical, so the same batch-verification race applies.
2026-07-17 23:00:08 -04:00
Gmer4Lfe 4f0cd1083a Poll MoveSeries command to completion instead of trusting DB fields
Confirmed live: DB fields flip instantly on the moveFiles=true PUT but the
physical move drains async, one command at a time — a batch could report
every series moved while most were still sitting at the old path.
2026-07-17 22:54:44 -04:00
Gmer4Lfe 1e1129687f Move arr_cache_prefill to intermediate tier, add classification scans to daily
arr_cache_prefill moved from critical (30min) to intermediate (4h) —
nothing in the critical tier reads this cache, and every actual
consumer already tolerates staleness well past 4h. Classification
scans run in daily, after the cleanup scripts (not before) — cleanup
frees disk space from orphans/junk first, so the searches --move
triggers for undownloaded relocated entries have headroom for the
resulting grabs.
2026-07-17 13:13:30 -04:00