From 1ed6b92118f3632a304c1cd5db5c05414851ef39 Mon Sep 17 00:00:00 2001 From: Gmer4Lfe Date: Sat, 1 Aug 2026 23:20:52 -0400 Subject: [PATCH] Document the Unraid platform layer The folder that translates the OS into Varaverk's vocabulary had no docs of its own, so the adapter contract and the three-layer web UI were only discoverable by reading the code. --- Plugin/unraid/README-unraid.md | 170 ++++++++++++++++++ .../README-System_Essentials.md | 131 ++++++++++++++ Plugin/unraid/Tools/README-Tools.md | 135 ++++++++++++++ 3 files changed, 436 insertions(+) create mode 100644 Plugin/unraid/README-unraid.md create mode 100644 Plugin/unraid/System_Essentials/README-System_Essentials.md create mode 100644 Plugin/unraid/Tools/README-Tools.md diff --git a/Plugin/unraid/README-unraid.md b/Plugin/unraid/README-unraid.md new file mode 100644 index 0000000..052fef1 --- /dev/null +++ b/Plugin/unraid/README-unraid.md @@ -0,0 +1,170 @@ +# ━━━━━ PLUGIN / UNRAID ━━━━━ + +The Unraid platform layer. Everything in this folder exists to translate one specific +operating system into the vocabulary Varaverk speaks — and to put a face on it in the Unraid +WebGUI. + +Varaverk itself knows nothing about Unraid. No script outside this folder references +`/etc/rc.d`, `/boot/config/plugins/dynamix`, `emhttp`, the mover, or a `.page` file. They call +`platform_*()` and read `$STATE_DIR`; this folder decides what those mean on Unraid. + +``` + adapter.sh 24 platform_*() functions — the entire OS contract + Varaverk.page WebGUI menu entry (Tasks:95) + VaraverkSettings.page WebGUI settings entry + event/ Unraid array lifecycle hooks + pages/ api/ include/ the three-layer web UI + System_Essentials/ Unraid-specific system scripts + Tools/ Unraid-specific operator tools + Watchdogs/System/ Unraid-specific watchdog (WebGUI health) + Partnership/ Unraid-specific container deploy/cleanup +``` + +> Swapping platforms means writing `Plugin/truenas/adapter.sh` implementing the same 24 +> function names. `load_config.sh` sources `Plugin/$PLATFORM/adapter.sh` — nothing else in +> the codebase changes. + +--- + +## ━━━ THE PROBLEM THAT BUILT THIS ━━━ + +**Every NAS OS Answers the Same Questions Differently** +"Is the array mounted?" "Restart this service." "What are the disk temperature thresholds?" +"Send a notification." Every platform has an answer, and every answer is different — a +different path, a different init system, a different config format. + +Scattering those differences through 115 scripts would mean every script carrying an `if +unraid ... elif truenas` branch, and every new platform touching all of them. + +**Varaverk Vars Are Not OS Vars** +Varaverk works in terms of `STATE_DIR`, `SCRIPTS_DIR`, `DOCKER_APPDATA_BASE`, +`WATCHDOG_CONTAINERS`. Unraid works in terms of `/boot/config/plugins/`, `emhttp`, +`/mnt/user`, dockerMan XML templates, and `dynamix.cfg`. Something has to map one onto the +other, in exactly one place, or the two vocabularies leak into each other. + +**A Headless Ecosystem Is Hard to Trust** +115 scripts running unattended produce a lot of state nobody can see. Without a UI the only +way to know what the watchdogs did last night is to read logs. The WebGUI exists so the +system can be inspected without SSH. + +--- + +## ━━━ THE ADAPTER — WHERE OS MEETS VARAVERK ━━━ + +`adapter.sh` is the whole contract. 24 functions, sourced by `load_config.sh` after +`common.sh`, so every script in the ecosystem has them. + +The most-called ones, by usage across the repo: + +| Function | Calls | What it hides | +|----------|-------|---------------| +| `platform_require_cmd` | 28 | Verifying a platform binary exists and is executable | +| `platform_restart_service` | 13 | `/etc/rc.d/rc.` init scripts | +| `platform_is_service_running` | 13 | Unraid's service process conventions | +| `platform_is_mover_running` | 10 | The Unraid mover — no other OS has one | +| `platform_get_os_version` | 10 | Where the version string lives | +| `platform_setup_db_path` | 9 | Where wizard/setup state persists | +| `platform_storage_path` | 7 | `/mnt/user` as the user-share root | + +Plus: disk states and temperature thresholds from `dynamix.cfg`, parity/maintenance detection, +native notifications through the dynamix notify script, container rebuild from dockerMan XML +templates, and the WebGUI install path. + +**The contract is strict, and it is what makes the layer work:** + +- **Every function returns 0/1 and never calls `exit`.** A missing capability is information + the caller needs, not a decision the adapter makes — a watchdog may want to skip where an + installer wants to abort. +- **Value-producing functions write to stdout**, captured with `$()`. Status travels in the + exit code. Keeping those separate is what lets callers use them in conditionals. +- **It reports; it does not remediate.** No retries, no escalation, no notification policy. + Burying those here would make identical calls behave differently per platform. + +--- + +## ━━━ THE WEB UI — THREE LAYERS ━━━ + +``` + pages/*.php what you look at ── requires ──► include/ + api/*.php what the page fetches ── requires ──► include/ + include/*.php the actual logic — vv_*() functions, shared by both +``` + +The split matters: a page and its API endpoint call **the same** `vv_*()` builders, so a +cached response and a live one cannot disagree in shape. Adding a metric means adding it in +`include/` once, and both the page and the endpoint get it. + +| Layer | Files | Role | +|-------|-------|------| +| `pages/` | 11 | One per WebGUI tab — monitor, docker, arrs, fallback, watchdog, rsync, partnership, scheduler, auth, settings, setup | +| `api/` | 50 | JSON endpoints the pages poll, plus action endpoints (run a script, stop a job, toggle a flag) | +| `include/` | 16 | Shared builders and helpers — `vv_monitor_*`, `vv_arrs_*`, `vv_docker_*`, config read/write, auth | + +**Caching.** Several endpoints serve from `/tmp/vv_cache` (tmpfs) rather than hitting live +APIs on every page view, refreshed by `Tools/api_cache_writer.sh`. `?live=1` bypasses the +cache. A missing or unparseable cache always falls back to a live call, so the cache can never +be the reason a page fails to load. + +--- + +## ━━━ ARRAY LIFECYCLE HOOKS ━━━ + +`event/` plugs Varaverk into Unraid's own array lifecycle. These are how the ecosystem starts +and stops with the array rather than with the OS. + +| Hook | Fires | Runs | +|------|-------|------| +| `disks_mounted/array_start_jobs` | Array started | `Orchestrators/array_started.sh` → `ARRAY_START_SCRIPTS` | +| `disks_mounted/rebuild_cron` | Array started | Regenerates `varaverk.cron` from the scheduler config | +| `unmounting_disks/array_stop_jobs` | Array stopping | `Orchestrators/array_stopping.sh` → `ARRAY_STOP_SCRIPTS` | + +Array-start rather than boot is the correct trigger: in flash storage mode `SCRIPTS_DIR` lives +in appdata, which does not exist until the array mounts. + +--- + +## ━━━ THE PLATFORM-SPECIFIC SCRIPT MIRRORS ━━━ + +These folders mirror the top-level Varaverk structure, but hold work that **cannot** be +platform-agnostic. A script lives here rather than in the root folder when it manipulates +Unraid itself. + +| Folder | Contents | Why it is platform-specific | +|--------|----------|----------------------------| +| `System_Essentials/` | 4 scripts | Mover, User Scripts plugin, PHP-FPM pool, unraid-api registry — all Unraid subsystems | +| `Tools/` | 5 files | Share `.cfg` recreation, storage-mode migration, API cache writers | +| `Watchdogs/System/` | `webgui_watchdog.sh` | emhttp / nginx / php-fpm health — there is no generic "WebGUI" | +| `Partnership/` | `containers.sh` | Container deploy from dockerMan CA XML templates | + +Each has its own README where the group is large enough to need one; single-script folders are +documented in the script header. + +--- + +## ━━━ HOW THIS FOLDER IS INSTALLED ━━━ + +Two paths, deliberately different: + +**Development** — `Plugin/plugin_setup.sh` symlinks this directory into the WebGUI plugin +location. Every edit is live immediately. This is the normal working mode. + +**Release** — `Plugin/build.sh` packages this directory into a Slackware `.txz` and rewrites +the version and checksum in `varaverk.plg`. That is what Unraid reinstalls from flash on every +boot. + +In flash storage mode `git_pull_execute.sh` syncs `Plugin/` back to `/boot/` after each pull, +so the WebGUI always serves current PHP even though `SCRIPTS_DIR` points at appdata. + +--- + +## ━━━ ADDING A NEW PLATFORM ━━━ + +1. Create `Plugin//adapter.sh` implementing all 24 `platform_*()` functions. +2. Set `PLATFORM=` in `varaverk.cfg`. + +That is the whole contract for the script ecosystem. `load_config.sh` sources +`Plugin/$PLATFORM/adapter.sh` and every script keeps working unchanged. + +The WebGUI is separate and optional — it is Unraid-specific by nature (`.page` files, emhttp +conventions). A new platform can run the entire ecosystem headless with only the adapter, and +add a UI later in whatever form that platform expects. diff --git a/Plugin/unraid/System_Essentials/README-System_Essentials.md b/Plugin/unraid/System_Essentials/README-System_Essentials.md new file mode 100644 index 0000000..f0f6580 --- /dev/null +++ b/Plugin/unraid/System_Essentials/README-System_Essentials.md @@ -0,0 +1,131 @@ +# ━━━━━ PLUGIN / UNRAID / SYSTEM ESSENTIALS ━━━━━ + +Four scripts that touch Unraid subsystems directly. They live here rather than in the +top-level `System_Essentials/` because each one manipulates something that only exists on +Unraid — the mover, the User Scripts plugin, the PHP-FPM pool serving emhttp, and the +unraid-api service registry. + +There is no platform-agnostic version of any of them. On TrueNAS there is no mover to stop. + +| Script | Runs | Role | +|--------|------|------| +| `php_fpm_max_children.sh` | Array **start** | Raise the PHP-FPM worker ceiling so the WebGUI stays responsive | +| `unraid_api_key_renew.sh` | Array **start** | Re-register Varaverk's key in the ephemeral unraid-api registry | +| `user_scripts_stop.sh` | Array **stop** | Stop User Scripts processes before the array goes down | +| `mover_stop.sh` | Array **stop** | Stop the mover gracefully before rsync or reboot | + +All four are wired through `ARRAY_START_SCRIPTS` / `ARRAY_STOP_SCRIPTS` in `master.conf` — +none are on a timer. + +--- + +## ━━━ WHY EACH ONE EXISTS ━━━ + +### 🐘 `php_fpm_max_children.sh` — the WebGUI gets slow, not broken + +Unraid's WebGUI runs through PHP-FPM, and the stock `pm.max_children` is very low (4–8). Under +real load — several browser tabs polling, Docker operations running, a dashboard open — every +worker saturates and new requests queue. The WebGUI becomes slow or stops answering, while +nothing is actually wrong with the server. + +Runs at array start because the setting **does not survive an Unraid update** — the OS +replaces the pool config. Idempotent: already at the target value means no write and no +restart, so a clean boot is silent. + +### 🔑 `unraid_api_key_renew.sh` — the registry is ephemeral + +Varaverk's enhanced monitoring authenticates against unraid-api. That registry is cleared by +OS updates and service restarts, so a key that worked yesterday can simply be gone. + +Re-registers unconditionally every array start rather than only when missing, because the +failure it repairs is precisely *a key present in the conf but absent from the registry* — +checking the conf would not detect it. Writes the resulting key into `HOST*_UNRAID_API_KEY`. + +### 🛑 `user_scripts_stop.sh` — stop new work before shutting down + +The User Scripts plugin spawns background processes that Varaverk does not own. During a +shutdown sequence those can still be starting operations while everything else is being torn +down. + +Runs **first** in `ARRAY_STOP_SCRIPTS` for that reason — stop new work before stopping the +things it would work on. Reports each process by **script name**, not just PID, because +"stopping 4 processes" tells an operator nothing they can act on. + +Matching is scoped strictly to processes the User Scripts plugin spawned. A broad pattern +would catch Varaverk's own scripts — including, during a reboot, the very script doing the +stopping. + +### 💾 `mover_stop.sh` — mover and rsync must not overlap + +The mover relocates files between cache and array. rsync reads those same paths. Both running +at once can produce a corrupt or half-moved snapshot on the remote. + +Ordering in the stop sequence is deliberate: `rsync_stop.sh` runs **before** `mover_stop.sh`, +because both write to the same paths and stopping the reader first is the safer order. + +Sends a wall warning first, then SIGTERM with a configurable window, then SIGKILL only if +ignored. The mover is mid-file-move by definition — giving it the chance to finish the current +file is the difference between a stopped transfer and a half-moved file. + +An absent mover exits 0. Callers use this as a precondition ("ensure the mover is not +running"), so treating "already stopped" as failure would abort every reboot on a quiet +system. + +--- + +## ━━━ WHERE THEY SIT IN THE LIFECYCLE ━━━ + +``` +Array starting + │ + ├── php_fpm_max_children.sh ──── WebGUI can handle the load that's about to arrive + └── unraid_api_key_renew.sh ──── monitoring can authenticate again + + ... normal operation ... + +Array stopping + │ + ├── user_scripts_stop.sh ─────── stop new work first + ├── fallback.sh --stop + ├── rsync_stop.sh ────────────── stop the reader + ├── mover_stop.sh ────────────── then the writer — same paths + └── docker_container_stop.sh ─── containers last, verified one at a time +``` + +`mover_stop.sh` is also called directly by `server_reboot.sh`, and both stop scripts appear in +the User Scripts master template for manual use. + +--- + +## ━━━ SAFEGUARDS ━━━ + +All four enforce root, take a lock, and support `--dry-run`. Specific to this group: + +**Graceful before forced.** `mover_stop.sh` and `user_scripts_stop.sh` both use SIGTERM with a +window before SIGKILL. Neither kills first. + +**Absent target is success.** No mover running, no User Scripts running — both exit 0. These +are preconditions, not commands that must find something to do. + +**Idempotent where it matters.** `php_fpm_max_children.sh` writes and restarts only when the +value is actually wrong, so array start does not restart PHP-FPM every single boot. + +**Targeted process matching.** `user_scripts_stop.sh` matches only what the User Scripts +plugin spawned — see above for why a broad pattern is dangerous here specifically. + +**Stop only, never start.** Neither stop script has a counterpart that restarts what it +stopped. Unraid's own schedule owns when the mover runs; these only remove it from the picture +for a window. + +--- + +## ━━━ CONFIGURATION ━━━ + +| Variable | Used by | Purpose | +|----------|---------|---------| +| `HOST*_UNRAID_API_KEY` | `unraid_api_key_renew.sh` | Written every array start — output, not input | +| `MOVER_STOP_TIMEOUT` | `mover_stop.sh` | SIGTERM grace window before escalating | +| `PHP_MAX_CHILDREN` | `php_fpm_max_children.sh` | Target worker ceiling | + +See each script's `CONFIGURATION` header section for the authoritative list — and +`Deployment/master.conf.template`, which is the versioned schema for all of them. diff --git a/Plugin/unraid/Tools/README-Tools.md b/Plugin/unraid/Tools/README-Tools.md new file mode 100644 index 0000000..e850f50 --- /dev/null +++ b/Plugin/unraid/Tools/README-Tools.md @@ -0,0 +1,135 @@ +# ━━━━━ PLUGIN / UNRAID / TOOLS ━━━━━ + +Platform-specific tooling for the WebGUI and the Unraid installation itself. Two of these keep +the UI fast, two are operator tools for specific recovery and migration situations. + +They live here rather than in the top-level `Tools/` because each one manipulates something +Unraid-specific — share `.cfg` files, the plugin's own PHP payload builders, or the +installation's storage location. + +| File | Type | Runs | +|------|------|------| +| `api_cache_writer.sh` + `.php` | UI cache | **Every minute** via cron | +| `remote_arr_cache_writer.sh` | UI cache | **Every 2 hours** via cron | +| `recreate_shares.sh` | Recovery | Manual — after a rebuild or fresh install | +| `storage_migrate.sh` | Migration | Manual — switching storage mode | + +--- + +## ━━━ THE CACHE WRITERS — WHY PAGES LOAD INSTANTLY ━━━ + +The monitor and arrs pages present a lot of state: per-core CPU, memory, network, GPUs, +containers, VMs, transcodes, arr library counts. Building that live on every page view means +dozens of API calls and `docker inspect` runs per refresh, from a WebGUI that is already the +first thing to slow down under load. + +Instead, both are written to `/tmp/vv_cache` (tmpfs — RAM-speed reads, cleared on reboot) and +the pages serve from there. + +### ⚡ `api_cache_writer.sh` — local payloads, every minute + +A deliberate one-line shim: `php api_cache_writer.php`. All logic is in the PHP because the +payload builders (`vv_monitor_*`, `vv_arrs_*`) are the **same functions the live API endpoints +call**. A bash reimplementation would be a second version of the same payload, free to drift. + +Nothing here is allowed to break the UI: + +- A failed run simply leaves the cache unrefreshed — pages fall back to live calls. Slower, + still correct. +- A missing or unparseable cache is treated as absent, never as empty data. +- `?live=1` bypasses the cache entirely. +- Runs unprivileged with no lock — a torn cache file is replaced within 60 seconds, and every + reader already has the live fallback. + +### 🌐 `remote_arr_cache_writer.sh` — partner payloads, every 2 hours + +SSHes to each partner and calls `vv_arrs_local_node()` **on their** PHP stack, caching the +result locally as `arrs_remote_.json`. + +The partner builds its own payload rather than this host querying the partner's arr APIs +directly. That matters: the partner already has working local URLs and API keys for its own +arrs, so no cross-host credentials and no path mapping are involved. This host never holds +keys for a remote's arrs. + +A partner whose Tailscale IP will not resolve is skipped, not fatal — the arrs page shows what +it has and falls back for the rest. + +--- + +## ━━━ THE OPERATOR TOOLS ━━━ + +### 🗂️ `recreate_shares.sh` — after a rebuild + +Recreates share **directories** on the correct disks by reading `/boot/config/shares/*.cfg` +and honouring each share's `shareInclude` disk list. + +The `.cfg` files are Unraid's own record of what a share is and which disks it spans — they +are the source of truth, not a list Varaverk maintains. Typically needed on a host after a +full disk replacement or fresh install, where the config survived but the folders did not. + +**Create only, never delete.** A directory that already exists is left alone. This runs at a +moment when the operator's picture of what should exist may be out of date, and removing +anything on that basis is how a recovery step becomes a data-loss step. + +**Refuses to run without `/mnt/user` mounted.** Creating share directories against an +unmounted array writes them into the underlying root filesystem, where they then shadow the +real shares once the array does mount. + +### 📦 `storage_migrate.sh` — moving the whole installation + +Relocates Varaverk between the two storage modes: + +| Mode | `SCRIPTS_DIR` | Trade-off | +|------|---------------|-----------| +| Internal | `/boot/config/plugins/varaverk` | Direct git pull/push. Available before the array mounts. | +| Flash | `/mnt/user/appdata/Varaverk` | Preserves USB flash lifetime. **Array must be started** for Varaverk to function at all. | + +This is the most destructive script in the folder — `rsync --delete`, `cp -a`, `rm -rf` — and +it rewrites the pointers everything else derives from: + +``` + varaverk.cfg SCRIPTS_DIR ← the authoritative path + master.conf TARGET_DIR + host*.conf HOST*_STORAGE_MODE_INTERNAL + varaverk.cron regenerated so job paths follow the new SCRIPTS_DIR +``` + +`STATE_DIR`, `DATA_DIR`, `PERSISTENT_CONF_CACHE` and every orchestrator job path are built +from `SCRIPTS_DIR`. Changing storage mode moves all of them at once, which is why the cron is +**regenerated** rather than edited. + +In flash mode, `git_pull_execute.sh` syncs `Plugin/` back to `/boot/` after each pull, so the +WebGUI keeps serving current PHP even though the scripts live in appdata. + +--- + +## ━━━ SAFEGUARDS ━━━ + +**The cache writers are deliberately unprivileged and lockless.** Their output is regenerable +within a minute and every consumer has a live fallback. There is no privileged operation to +gate and no state worth locking. This is documented in their headers so it is not "corrected" +later. + +**The operator tools are the opposite.** Both enforce root, take a lock, and support +`--dry-run`. `storage_migrate.sh` additionally exits cleanly if already in the requested mode, +and removes the old location only once the new one is confirmed in place. + +**`recreate_shares.sh` will not act on an unmounted array** — see above; this is the guard +that prevents shadow directories. + +--- + +## ━━━ RELATIONSHIP TO THE WEB UI ━━━ + +``` + pages/monitor.php ─┐ + ├─► include/monitor.php (vv_monitor_*) + api/monitor.php ───┘ ▲ + │ + Tools/api_cache_writer.php ─────┘ same builders → cache can't disagree with live + │ + └─► /tmp/vv_cache/monitor.json ──► served by api/monitor.php unless ?live=1 +``` + +Adding a metric means adding it in `include/` once. The page, the live endpoint, and the cache +all pick it up together.