Fill the single missing section in three scripts that had everything else

AI/ai_token_sync.sh

@@ -30,6 +30,31 @@
 # fetches its own data controls its own freshness rather than depending on the partner's cron.
 #
 # ==============================================================================================
+# DESIGN PRINCIPLES
+# ==============================================================================================
+#
+# A missing file means unknown, never zero.
+#   The whole point of the tab reporting "not collected here" is that it is a different claim
+#   from "this partner spent nothing". If a partner is dark, unreachable or has never run a turn,
+#   there is simply no cache file, and the reader is expected to say so rather than render a 0
+#   that looks like a measurement.
+#
+# The reader pulls; nobody pushes.
+#   conf_sync.sh pushes as well, because a partner that cannot reach us still needs our
+#   credentials. Nothing here is needed by anyone else, so a host that wants fleet totals fetches
+#   them and owns its own freshness instead of depending on someone else's cron having run.
+#
+# RAM, not flash.
+#   The cache lands in tmpfs. It is a copy of a file that already exists on the partner and is
+#   rebuilt on the next pass, so writing it to flash would cost wear for something that is never
+#   worth surviving a reboot.
+#
+# Same shape as conf_sync.sh, deliberately.
+#   Resolve over Tailscale, scp one small file into a tmpfs cache, let a missing file mean
+#   unknown. A second transport pattern for a second small file would be a second set of
+#   failure modes to learn.
+#
+# ==============================================================================================
 # OPERATIONAL SAFEGUARDS
 # ==============================================================================================
 #

Plugin/unraid/System_Essentials/pcie_aer_quiet.sh

@@ -24,6 +24,22 @@
 # list and reboot.
 #
 # ==============================================================================================
+# OPERATIONAL MODEL
+# ==============================================================================================
+#
+# One pass over PCIE_QUIET_DEVICES at array start. Each address is looked up under
+# /sys/bus/pci/devices, checked against the guards below, and removed through the kernel's own
+# per-device remove attribute. Nothing is masked and no kernel parameter is set.
+#
+# Idempotent, and silent when there is nothing to do. An address that is already gone is not an
+# error — that is the normal state on every array start after the first within one boot.
+#
+# Reapplied every array start rather than once, because a reboot re-enumerates the bus and the
+# devices come back. That is also the undo: clear PCIE_QUIET_DEVICES and reboot.
+#
+# PCIE_QUIET_ENABLED gates the whole run before any device is touched.
+#
+# ==============================================================================================
 # DESIGN PRINCIPLES
 # ==============================================================================================
 #

Plugin/unraid/Tools/ai_rpc.php

@@ -42,6 +42,11 @@
 //       stdin is an untrusted stream from another process. A malformed or endless payload must
 //       fail as a bad request, not as an out-of-memory.
 //
+// RUNTIME MODES
+//   Not invoked by hand. include/ai_rpc.php opens an SSH session to the owner and runs this
+//   file with the request as JSON on stdin; the response is JSON on stdout. There are no flags
+//   and no arguments — the action, the profile and the payload all arrive in the request body.
+//
 // DEPENDS ON
 //   include/ai_actions.php  vv_ai_dispatch() — the shared handlers
 //   include/config.php      vv_ai_is_owner()

Plugin/unraid/Tools/api_cache_writer.php

@@ -66,6 +66,13 @@
 //   VV_CACHE_DIR/arrs.json      consumed by api/arrs.php
 //   stdout                      one timing line, captured into the job log
 //
+// RUNTIME MODES
+//   api_cache_writer.php
+//       Build both payloads and write them to the cache. Takes no arguments.
+//
+//   Run every minute by Tools/api_cache_writer.sh, which is the cron entry — this file is the
+//   work, that one is the schedule, the lock and the log.
+//
 // DEPENDS ON
 //   include/monitor.php, include/common.php, include/unraid_api.php,
 //   include/vms.php, include/docker_folders.php, include/arrs.php

Plugin/unraid/Tools/auth_sweep.sh

@@ -28,6 +28,38 @@
 # finding for every hostname on the machine.
 #
 # ==============================================================================================
+# DESIGN PRINCIPLES
+# ==============================================================================================
+#
+# It answers the questions nobody was going to ask.
+#   Both checks were already possible from the Auth tab, and both required somebody to open it
+#   and press a button on the right row. That is why one host here returned nothing but 5xx for
+#   months. A check that depends on being run is a check that is not running.
+#
+# Files findings; never applies remedies.
+#   The fixes are "start a container", "edit a rule", "change a default policy". Each of those
+#   is a judgement about intent, and a sweep that guessed at intent would be repairing a
+#   deliberate configuration as often as a broken one.
+#
+# Down is measured in time, not in samples.
+#   A host must have been failing for longer than AUTH_SWEEP_DOWN_MIN before anything is filed.
+#   Counting failed samples instead would file a finding for every hostname on the machine every
+#   time it reboots, and a report that cries wolf after every restart stops being read.
+#
+# ==============================================================================================
+# OPERATIONAL SAFEGUARDS
+# ==============================================================================================
+#
+# Read-only against the auth stack. Nothing here starts a container, edits a proxy host, changes
+# a rule or alters a default policy — the two questions are answered by observation alone.
+#
+# --dry-run reports what it would file and writes no finding. --report only reads what is
+# already filed, and is silent when clean, so it cannot manufacture noise for the Sunday report.
+#
+# Findings are filed, not notified. A sweep that raised an alert per host would be its own
+# incident every time the array restarted.
+#
+# ==============================================================================================
 # RUNTIME MODES
 # ==============================================================================================
 #

Plugin/unraid/Tools/cert_history.sh

@@ -25,6 +25,40 @@
 # own created_on, which is a real date; nothing else is back-filled.
 #
 # ==============================================================================================
+# DESIGN PRINCIPLES
+# ==============================================================================================
+#
+# History is accumulated, never reconstructed.
+#   NPM holds only the present, so every past state this file knows about is one it observed at
+#   the time. Counts start at zero on first run and nothing is back-filled — first_seen is the
+#   single exception, seeded from NPM's own created_on because that is a real recorded date
+#   rather than an inference.
+#
+# A domain is retired on strikes, not on a single bad pass.
+#   A pass fails for a domain when it is absent from NPM's list or its expiry is already in the
+#   past — and NPM's list can come back short for reasons that have nothing to do with the
+#   certificate, such as an API hiccup or a restart mid-pass. CERT_HISTORY_STRIKES consecutive
+#   failures are required before a domain is retired, and it stays in the store afterwards, so
+#   one bad read can neither erase months of history nor hide a genuine expiry.
+#
+# The wrapper holds no logic.
+#   The work sits next to the NPM client it needs, because token handling lives in
+#   include/auth.php and a bash reimplementation would be a second copy of the thing most worth
+#   having only one of. Flags are forwarded verbatim.
+#
+# ==============================================================================================
+# OPERATIONAL SAFEGUARDS
+# ==============================================================================================
+#
+# The store is replaced atomically: temp file, verified, then renamed over the original. A pass
+# that fails partway leaves the previous history intact rather than a truncated file — this is
+# the only record of what these certificates did, and there is no second copy to restore from.
+#
+# --dry-run reports every change it would make and writes nothing. --status only reads.
+#
+# Read-only against NPM. Certificates are observed; nothing here renews, deletes or edits one.
+#
+# ==============================================================================================
 # RUNTIME MODES
 # ==============================================================================================
 #

Plugin/unraid/Tools/cert_triage.php

@@ -40,6 +40,21 @@
 //   downstream of something else here, and listing it alongside its own cause invites fixing the
 //   symptom.
 //
+// OPERATIONAL SAFEGUARDS
+//   Read-only. Opens certbot's logs and nothing else — no certificate is requested, renewed or
+//   deleted here, and no log is rotated or truncated.
+//
+//   Bounded by file count and by bytes per file, so a directory that has grown to 639 MB across
+//   1001 files cannot turn a page load into an unbounded read. Only the tail of each log is
+//   examined, because a run explains its failure at the end rather than the beginning.
+//
+//   An unrecognised failure is reported as unclassified, never folded into the nearest category.
+//   A tidy chart that is wrong sends the operator to fix a domain that was never broken.
+//
+//   The log directory is discovered from the NPM container rather than assumed, so a container
+//   path change surfaces as "no logs found" instead of an empty triage that reads as "no
+//   failures".
+//
 // RUNTIME MODES
 //   cert_triage.php            summary — categories, affected domains, and the causal reading
 //   cert_triage.php --json     the same as JSON, for the Certs tab

Plugin/unraid/Tools/cert_triage.sh

@@ -30,6 +30,42 @@
 # fails renewals for domains that have nothing wrong with them.
 #
 # ==============================================================================================
+# DESIGN PRINCIPLES
+# ==============================================================================================
+#
+# One invocation is one data point.
+#   A single failure writes its reason three times — the ACME response, the traceback, and
+#   certbot's summary. Counting lines would report it as three and inflate whichever category
+#   happens to be the most verbose, which is the opposite of what triage is for.
+#
+# Causes are separated from consequences.
+#   Rate limiting is nearly always downstream: retries against a hostname with no DNS record
+#   exhaust the allowance, which then fails renewals for domains that have nothing wrong with
+#   them. Reporting the rate limit as the problem sends the operator to fix the wrong domains.
+#
+# Rotation suffix is the clock, not mtime.
+#   Every file in this directory carries the same mtime because they arrive as a synced set, so
+#   ordering by mtime returns an arbitrary answer that looks authoritative.
+#
+# Bounded by design, not by hope.
+#   639 MB across 1001 files cannot be read in a page load. Only CERT_TRIAGE_FILES logs are
+#   opened, and only CERT_TRIAGE_MAX_BYTES from the end of each, because a failure explains
+#   itself at the end of the run rather than the start.
+#
+# ==============================================================================================
+# OPERATIONAL SAFEGUARDS
+# ==============================================================================================
+#
+# Read-only throughout. It opens certbot's logs and nothing else — no certificate is requested,
+# renewed or deleted, and no log is rotated or truncated by anything here.
+#
+# Diagnosis only. This names why renewals failed; acting on that is the operator's, and
+# cert_history.sh remains the thing that records what happened.
+#
+# The log directory is normally discovered from the NPM container rather than hardcoded, so a
+# container path change surfaces as "no logs found" rather than as a silently empty triage.
+#
+# ==============================================================================================
 # RUNTIME MODES
 # ==============================================================================================
 #

Plugin/unraid/Tools/mirror_folders.php

@@ -63,6 +63,22 @@
 //                                  "unfiled":[…]}
 //   import: one line per folder written, then a count
 //
+// RUNTIME MODES
+//   mirror_folders.php --export
+//       Print this host's folder layout as JSON, for the owner to compute a plan from.
+//
+//   mirror_folders.php --import
+//       Read a layout on stdin and apply it to this host's folder.view3.
+//
+//   mirror_folders.php --containers=<csv>
+//       Restrict the operation to these container names.
+//
+//   mirror_folders.php --fallback-only=<csv>
+//       Place only these into the partner's fallback folder, leaving the rest where they are.
+//
+//   --dry-run
+//       Compute and report the plan without writing folder.view3. Combines with any of the above.
+//
 // DEPENDS ON
 //   /boot/config/plugins/folder.view3/docker.json   the third-party layout, optional on import
 //   SCRIPTS_DIR/docker_folders.json                 Varaverk's own layout — what the card reads

Plugin/unraid/Tools/node_chat_receive.php

@@ -5,12 +5,25 @@
 //   vv_nc_deliver(), with the record on stdin.
 //
 // OPERATIONAL MODEL
-//   php node_chat_receive.php --channel=<id>          record on stdin, one JSON object
-//   php node_chat_receive.php --flush                 retry this host's own undelivered spool
-//
 //   Exit 0 means stored. The sender treats anything else as undelivered and spools for retry, so
 //   a non-zero exit here is a message that will arrive later rather than one that is lost.
 //
+//   Two jobs in one file because they are two halves of the same contract: --channel takes what
+//   a partner is delivering now, --flush pushes what this host failed to deliver earlier. A node
+//   is both a receiver and a sender, and splitting them would mean two files that must agree on
+//   the same spool layout.
+//
+// DESIGN PRINCIPLES
+//   The exit code is the receipt, and it is the only one.
+//       There is no acknowledgement message and no reply body. The sender is an SSH command that
+//       already has an exit status, so inventing a second channel to say the same thing would
+//       give the two ways to disagree.
+//
+//   Refusing is cheaper than being wrong.
+//       An unparseable record, an unknown channel or a failed write all exit non-zero and leave
+//       the message in the sender's spool. A retry costs one SSH round trip; a message accepted
+//       and dropped is gone with nothing recording that it existed.
+//
 // OPERATIONAL SAFEGUARDS
 //   Reached only over SSH with a key this mesh installed, so the caller already has root. This
 //   file therefore validates shape, not authority — there is no privilege here to protect that
@@ -22,6 +35,12 @@
 //   Storage is append-and-trim through vv_nc_append(), which is idempotent on message id — a
 //   retry of something that already landed is a no-op rather than a duplicate.
 //
+// RUNTIME MODES
+//   Not invoked by hand — the sending node's vv_nc_deliver() runs it over SSH.
+//
+//   php node_chat_receive.php --channel=<id>   store one JSON record, read from stdin
+//   php node_chat_receive.php --flush          retry this host's own undelivered spool
+//
 // DEPENDS ON
 //   include/node_chat.php   vv_nc_append(), vv_nc_channel(), vv_nc_flush_spool()
 // ═══════════════════════════════════════════════════════════════════════════════════════════════

Plugin/unraid/Tools/npm_access_stats.sh

@@ -22,6 +22,42 @@
 # passes are not counted. Running daily keeps that to whatever NPM rotates in a day.
 #
 # ==============================================================================================
+# DESIGN PRINCIPLES
+# ==============================================================================================
+#
+# Incremental by byte offset, never a re-read.
+#   475 MB across 41 files cannot be parsed on a page load, and re-parsing what was already
+#   counted would make each pass slower than the last. Each run records where it stopped and
+#   resumes there, so cost tracks new traffic rather than total traffic.
+#
+# A rotated log is detected, not assumed.
+#   A file smaller than the offset already recorded for it means NPM rotated it beneath us, so
+#   that file's offset restarts at zero and a rotation is counted. The totals are cumulative and
+#   are never reset by it — what rotated out between two passes is simply not counted, which is
+#   why this runs often enough to keep that gap to whatever NPM rotates in a day.
+#
+# Varaverk's own probes are excluded.
+#   uptime_probe.sh requests every proxied host every minute under Varaverk-Uptime/1.0. Counting
+#   those would make the monitor the busiest client of everything it monitors.
+#
+# ==============================================================================================
+# OPERATIONAL SAFEGUARDS
+# ==============================================================================================
+#
+# Read-only against NPM. Access logs are read and never rotated, truncated or removed — that is
+# NPM's business, and a stats collector that deleted its own inputs would be unrecoverable.
+#
+# --dry-run parses and reports without writing the store or advancing any offset, so a dry run
+# leaves the next real pass with exactly the same work to do.
+#
+# --reset forgets offsets and totals deliberately. It starts from the current end of each log,
+# so it discards history rather than double-counting it.
+#
+# The store is replaced atomically: written to a temp file, re-read and parsed to prove it is
+# valid JSON, then renamed over the original. A pass that dies midway — or produces something
+# unparseable — leaves the previous document untouched rather than a truncated one.
+#
+# ==============================================================================================
 # RUNTIME MODES
 # ==============================================================================================
 #

Plugin/unraid/Tools/uptime_probe.sh

@@ -25,6 +25,39 @@
 # that, this monitor becomes fifty thousand requests a day in the logs it reports on.
 #
 # ==============================================================================================
+# DESIGN PRINCIPLES
+# ==============================================================================================
+#
+# Probe what is served, not what was visited.
+#   The domain list comes from NPM's own proxy hosts, so a host nobody has requested is measured
+#   exactly like a busy one. Deriving the list from traffic would leave the quietest hosts — the
+#   ones most likely to be broken without anyone noticing — permanently unmonitored.
+#
+# The probe is excluded from the statistics it feeds.
+#   Every request carries Varaverk-Uptime/1.0, which npm_access_stats.sh filters out. Without
+#   that the monitor would be the largest single source of traffic in the logs it reports on,
+#   and every access figure would be measuring this script.
+#
+# The wrapper holds no logic.
+#   Flags are forwarded verbatim and nothing is interpreted here. Two places that both know what
+#   --dry-run means is two places that can disagree about it.
+#
+# ==============================================================================================
+# OPERATIONAL SAFEGUARDS
+# ==============================================================================================
+#
+# UPTIME_PROBE_ENABLED gates the whole run. Off means no probes and no writes.
+#
+# Bounded per domain by UPTIME_PROBE_TIMEOUT, so an unresponsive host costs one timeout rather
+# than stalling the pass — the probes run in parallel, so one slow domain never delays the rest.
+#
+# --dry-run probes and reports without writing. --status and --events read stored history and
+# probe nothing, so neither can alter what it is describing.
+#
+# Read-only against NPM. The domain list is read; no proxy host, certificate or setting is
+# touched by anything in this path.
+#
+# ==============================================================================================
 # RUNTIME MODES
 # ==============================================================================================
 #

Rsync/media_seed.sh

@@ -35,6 +35,25 @@
 # progress in the UI, and lets it be re-run or cancelled without touching onboard.
 #
 # ==============================================================================================
+# OPERATIONAL MODEL
+# ==============================================================================================
+#
+# One share at a time, in DAILY_SYNC_SHARES order, each handed to Rsync/rsync.sh --seed. The
+# transfer itself, its bandwidth cap and its resume behaviour are all rsync.sh's — this script
+# decides what to seed and in what order, never how.
+#
+# Sequential on purpose. The cap that matters is DEFAULT_RSYNC_OPTS' --bwlimit, which is a limit
+# per rsync rather than per host; running shares in parallel would multiply it by the number of
+# shares and saturate the link the rest of the ecosystem is still using.
+#
+# Refuses before it starts rather than partway. RSYNC_ENABLED must be true and DAILY_SYNC_SHARES
+# must be non-empty, both checked up front — a multi-week transfer is the wrong place to
+# discover that the global gate was off.
+#
+# Nothing waits on this. Phase 3 runs after the partnership is already established, so a seed
+# that is still copying weeks later blocks no flag, no status card and no job record.
+#
+# ==============================================================================================
 # DESIGN PRINCIPLES
 # ==============================================================================================
 #