Files

T

Gmer4Lfe 3a45c18555 Document today's restructures, and give data/ a README that ships with it

2026-08-09 00:01:05 -04:00

16 KiB

Raw Blame History

━━━━━ PLUGIN / UNRAID ━━━━━

The Unraid platform layer. Everything in this folder exists to translate one specific operating system into the vocabulary Varaverk speaks — and to put a face on it in the Unraid WebGUI.

Varaverk itself knows nothing about Unraid. No script outside this folder references /etc/rc.d, /boot/config/plugins/dynamix, emhttp, the mover, or a .page file. They call platform_*() and read $STATE_DIR; this folder decides what those mean on Unraid.

  adapter.sh              24 platform_*() functions — the entire OS contract
  Varaverk.page           WebGUI menu entry (Tasks:95)
  VaraverkSettings.page   WebGUI settings entry
  event/                  Unraid array lifecycle hooks
  pages/ api/ include/    the three-layer web UI
  System_Essentials/      Unraid-specific system scripts
  Tools/                  Unraid-specific operator tools
  Watchdogs/System/       Unraid-specific watchdog (WebGUI health)
  Partnership/            Unraid-specific container deploy/cleanup

Swapping platforms means writing Plugin/truenas/adapter.sh implementing the same 24 function names. load_config.sh sources Plugin/$PLATFORM/adapter.sh — nothing else in the codebase changes.

━━━ THE PROBLEM THAT BUILT THIS ━━━

Every NAS OS Answers the Same Questions Differently "Is the array mounted?" "Restart this service." "What are the disk temperature thresholds?" "Send a notification." Every platform has an answer, and every answer is different — a different path, a different init system, a different config format.

Scattering those differences through 115 scripts would mean every script carrying an if unraid ... elif truenas branch, and every new platform touching all of them.

Varaverk Vars Are Not OS Vars Varaverk works in terms of STATE_DIR, SCRIPTS_DIR, DOCKER_APPDATA_BASE, WATCHDOG_CONTAINERS. Unraid works in terms of /boot/config/plugins/, emhttp, /mnt/user, dockerMan XML templates, and dynamix.cfg. Something has to map one onto the other, in exactly one place, or the two vocabularies leak into each other.

A Headless Ecosystem Is Hard to Trust 115 scripts running unattended produce a lot of state nobody can see. Without a UI the only way to know what the watchdogs did last night is to read logs. The WebGUI exists so the system can be inspected without SSH.

━━━ THE ADAPTER — WHERE OS MEETS VARAVERK ━━━

adapter.sh is the whole contract. 24 functions, sourced by load_config.sh after common.sh, so every script in the ecosystem has them.

The most-called ones, by usage across the repo:

Function	Calls	What it hides
`platform_require_cmd`	28	Verifying a platform binary exists and is executable
`platform_restart_service`	13	`/etc/rc.d/rc.<name>` init scripts
`platform_is_service_running`	13	Unraid's service process conventions
`platform_is_mover_running`	10	The Unraid mover — no other OS has one
`platform_get_os_version`	10	Where the version string lives
`platform_setup_db_path`	9	Where wizard/setup state persists
`platform_storage_path`	7	`/mnt/user` as the user-share root

Plus: disk states and temperature thresholds from dynamix.cfg, parity/maintenance detection, native notifications through the dynamix notify script, container rebuild from dockerMan XML templates, and the WebGUI install path.

The contract is strict, and it is what makes the layer work:

Every function returns 0/1 and never calls exit. A missing capability is information the caller needs, not a decision the adapter makes — a watchdog may want to skip where an installer wants to abort.
Value-producing functions write to stdout, captured with $(). Status travels in the exit code. Keeping those separate is what lets callers use them in conditionals.
It reports; it does not remediate. No retries, no escalation, no notification policy. Burying those here would make identical calls behave differently per platform.

━━━ THE WEB UI — THREE LAYERS ━━━

  pages/*.php      what you look at        ── polls ─────►  api/
  api/*.php        what the page fetches   ── requires ──►  include/
  include/*.php    the actual logic — vv_*() functions

Most pages hold no logic at all. Eight of the eleven are pure view — markup, CSS and a poll loop, with every value arriving as JSON from api/. Only auth.php, monitor.php and scheduler.php require an include/ file directly, and then only to server-render their initial state; their mutations still go through endpoints.

That is why the layer split holds: a page and its endpoint cannot disagree about what a value means, because the page does not compute it. Adding a metric means adding it in include/ once, and the endpoint, the cache writer, and the page all pick it up together.

Layer	Files	Role
`pages/`	11	One per WebGUI tab — monitor, docker, arrs, fallback, watchdog, rsync, partnership, scheduler, auth, settings, setup
`api/`	50	JSON endpoints the pages poll, plus action endpoints (run a script, stop a job, toggle a flag)
`include/`	16	Shared builders and helpers — `vv_monitor_`, `vv_arrs_`, `vv_docker_*`, config read/write, auth

Caching. Several endpoints serve from $VV_CACHE_DIR (/tmp/varaverk/api, tmpfs) rather than hitting live APIs on every page view, refreshed by Tools/api_cache_writer.sh. ?live=1 bypasses the cache. A missing or unparseable cache always falls back to a live call, so the cache can never be the reason a page fails to load.

Anything expensive belongs in that one collection rather than in each consumer. vv_ai_stats() costs about a second — mostly waiting on Ollama and nvidia-smi — and the AI tab was paying it every 30 seconds per open tab; it is now written once to the ai cache and read by the tab, the Monitor row and the dock alike. Polling faster cannot make a figure newer, it only decides how soon a page notices the writer's update.

Tools/api_cache_writer.php rebuilds the monitor payload independently of api/monitor.php, and is what the page normally reads — the endpoint only assembles one on a cache miss. A key added to the endpoint and not to the writer leaves its card loading forever on every ordinary page load and working only on the request that happens to miss.

One widget, rendered twice. include/ai_chat.php owns the conversation surface — profile bar, transcript, composer, source viewer, stored-chat list — and both the AI tab and the Monitor tab's AI row construct it. Every id is composed from a prefix so instances can coexist, and each tears down the previous holder of its prefix, because Unraid swaps tab content by AJAX without unloading the old page's JavaScript.

The Scheduler tab's dock is deliberately not built on it: a one-line bar that follows the view you have open, with its own scope chip, fix flow and incident capture, is a different component that happens to share an endpoint. Folding it in would produce one widget with two personalities and a mode flag choosing between them.

Timezone. include/config.php adopts Unraid's own timeZone from ident.cfg for the whole PHP layer. PHP here has no date.timezone and therefore ran in UTC while the server ran local, so every date this layer produced was offset from every date the shell layer wrote — and the two are compared constantly. /etc/php.ini is the wrong fix twice over: /etc is a RAM filesystem so the edit dies at reboot, and it would retimezone every other PHP application on the box.

━━━ WHAT GUARDS THE API LAYER ━━━

Two layers, both supplied by the platform, neither implemented in this plugin.

1. nginx session auth. /etc/nginx/nginx.conf applies satisfy any; allow unix:; deny all; auth_request /auth-request.php; to everything it serves, /plugins/ included. A request without a valid WebGUI session never reaches PHP — it is redirected to the login page.

2. Unraid's CSRF prepend. /etc/php.ini sets auto_prepend_file=/usr/local/emhttp/webGui/include/local_prepend.php, which runs before the first line of any endpoint and, for every POST, requires a valid token as either a csrf_token body field or an X-CSRF-Token header. On a mismatch it logs to syslog and exits. php-fpm inherits this — there is no pool override in /etc/php-fpm.d/www.conf.

Anything that clears both can do everything this plugin can do: stop the array, power off the host, write master.conf and push it to every partner, create an lldap user, write and schedule a root-run script. That is the same trust level as the rest of the WebGUI, and it is the intended model.

The two consequences that actually matter

GET is not covered. The prepend inspects POSTs only. So any action that changes something must be POST — over GET it would run with no token check at all. Four actions were GET and have been converted: create_api_key.php, arrs.php?action=refresh_remote, cert.php?action=run and setup.php?action=ssh_generate. When adding an endpoint the rule is simply: if it changes state, it is POST. Read-only GETs (monitor, board, checklist, the browse pickers, api_test.php) are bounded by the session alone, which is correct for reads.

The webGUI's token injector is jQuery-only. $.ajaxPrefilter in dynamix's HeadInlineJS.php appends csrf_token to jQuery POSTs. Varaverk's pages use native fetch(), which it does not touch — so a fetch() POST carries no token and the prepend kills it silently: csrf_terminate() exits with an empty body, r.json() throws on the empty response, and the page's own .catch() swallows it. There is no error anywhere except a syslog line.

Varaverk.page therefore installs a small window.fetch shim, inline and above the tab content, that attaches X-CSRF-Token to same-origin /plugins/varaverk/ requests. The header form is deliberate: it works for FormData, URLSearchParams and raw JSON bodies alike, so no call site has to know about it and a new one cannot forget it. A cross-origin page cannot set a custom header without a preflight it will fail, which is what makes it a real defence rather than a formality.

Do not construct POSTs outside fetch() — a raw XMLHttpRequest or a generated form — without adding the token yourself; the shim only wraps fetch. A plain <form method="POST"> is already covered by a different platform mechanism: dynamix's BodyInlineJS.php appends a hidden csrf_token input to every form on the page, which is what makes VaraverkSettings.page work without any of this.

Never send a POST as multipart/form-data

FormData is banned in this plugin. A multipart/form-data POST to the plugin API hangs and never completes on this host — confirmed 2026-08-02.

The failure is completely silent, which is what makes it dangerous:

The browser sends it correctly, valid token and all (verified in the Network tab).
The Network row shows no status code at all — not 403, not 500. It never completes.
Server side there is nothing: no CSRF termination in syslog, no fatal in /var/log/phplog, and no output from a log statement that is the literal first line of the endpoint.

Use URLSearchParams. fetch sets application/x-www-form-urlencoded for it automatically, and it has the same append() / set() API, so it is a drop-in for any payload of strings:

const fd = new URLSearchParams();      // NOT new FormData()
fd.append('action', 'save');
fetch(url, { method: 'POST', body: fd });

The correlation across the plugin was exact: every page using URLSearchParams worked, every page using FormData hung. All 21 call sites were converted in one pass. Suspected to date from the Unraid 7.3.1→7.3.2 upgrade; root cause in nginx/php-fpm was never identified, only the workaround. If a POST ever hangs with no status code and leaves no server-side trace whatsoever, suspect the encoding first — not CSRF, not auth, not the endpoint.

api/webhook.php is dead code

It receives arr download events — but an arr has no WebGUI session and no CSRF token, so it is blocked by both layers above and cannot be called at all. The live path is Arrs_Stack/webhook_listener.js on WEBHOOK_PORT, running outside nginx entirely and validating WEBHOOK_SECRET itself. api/webhook.php cannot be made to work by adding a secret check — it would have to be served from outside /plugins/ first. Removing it is the other reasonable option.

Two endpoints read wider than the rest and are worth knowing about if the session boundary ever moves: api/api_test.php returns an API key prefix and the live GraphQL schema, and the browse actions in api/import_script.php and api/manual_sync.php list directory names anywhere on either host. All are read-only and none return file contents.

━━━ ARRAY LIFECYCLE HOOKS ━━━

event/ plugs Varaverk into Unraid's own array lifecycle. These are how the ecosystem starts and stops with the array rather than with the OS.

Hook	Fires	Runs
`disks_mounted/array_start_jobs`	Array started	`Orchestrators/array_started.sh` → `ARRAY_START_SCRIPTS`
`disks_mounted/rebuild_cron`	Array started	Regenerates `varaverk.cron` from the scheduler config
`unmounting_disks/array_stop_jobs`	Array stopping	`Orchestrators/array_stopping.sh` → `ARRAY_STOP_SCRIPTS`

Array-start rather than boot is the correct trigger: in flash storage mode SCRIPTS_DIR lives in appdata, which does not exist until the array mounts.

━━━ THE PLATFORM-SPECIFIC SCRIPT MIRRORS ━━━

These folders mirror the top-level Varaverk structure, but hold work that cannot be platform-agnostic. A script lives here rather than in the root folder when it manipulates Unraid itself.

Folder	Contents	Why it is platform-specific
`System_Essentials/`	4 scripts	Mover, User Scripts plugin, PHP-FPM pool, unraid-api registry — all Unraid subsystems
`Tools/`	5 files	Share `.cfg` recreation, storage-mode migration, API cache writers
`Watchdogs/System/`	`webgui_watchdog.sh`	emhttp / nginx / php-fpm health — there is no generic "WebGUI"
`Partnership/`	`containers.sh`	Container deploy from dockerMan CA XML templates

Each has its own README where the group is large enough to need one; single-script folders are documented in the script header.

━━━ HOW THIS FOLDER IS INSTALLED ━━━

Two paths, deliberately different:

Development — Plugin/plugin_setup.sh symlinks this directory into the WebGUI plugin location. Every edit is live immediately. This is the normal working mode.

Release — Plugin/build.sh packages this directory into a Slackware .txz and rewrites the version and checksum in varaverk.plg. That is what Unraid reinstalls from flash on every boot.

In flash storage mode git_pull_execute.sh syncs Plugin/ back to /boot/ after each pull, so the WebGUI always serves current PHP even though SCRIPTS_DIR points at appdata.

━━━ ADDING A NEW PLATFORM ━━━

Create Plugin/<platform>/adapter.sh implementing all 24 platform_*() functions.
Set PLATFORM=<platform> in varaverk.cfg.

That is the whole contract for the script ecosystem. load_config.sh sources Plugin/$PLATFORM/adapter.sh and every script keeps working unchanged.

The WebGUI is separate and optional — it is Unraid-specific by nature (.page files, emhttp conventions). A new platform can run the entire ecosystem headless with only the adapter, and add a UI later in whatever form that platform expects.

16 KiB Raw Blame History