Close the last four headers, and say what the destructive tools refuse to do

Deployment/migrate_data_layout.sh

@@ -28,6 +28,42 @@
 # pointing at a directory the data is not in is worse than not having started.
 #
 # ==============================================================================================
+# OPERATIONAL MODEL
+# ==============================================================================================
+#
+# Two halves, in order: move the files, then rewrite the conf keys that point at them. Doing it
+# the other way round would leave every path variable naming a location nothing had reached yet,
+# and any script that ran in between would create the old layout again underneath the new one.
+#
+# Idempotent. A path already under DATA_DIR is left alone, so a re-run after a partial migration
+# finishes the job rather than moving things twice or failing on what is already done.
+#
+# One-time by intent, not by a marker file. There is no "already migrated" flag — the check is
+# whether each individual path is already where it belongs, which is also what makes an
+# interrupted run safe to repeat.
+#
+# ==============================================================================================
+# DESIGN PRINCIPLES
+# ==============================================================================================
+#
+# Existing keys are rewritten, which is why conf_upgrade cannot do this.
+#   conf_upgrade adds keys the template has and the installation does not, and never rewrites a
+#   value the operator already holds — correct for it, and exactly why it is the wrong tool here.
+#   STATE_DIR, BANDWIDTH_LOG, AI_INDEX_DB and two dozen more are existing keys whose values must
+#   change, or they would go on naming the old layout forever while the new directory variables
+#   sat beside them unused.
+#
+# Move, never copy-and-hope.
+#   The data being relocated is the only copy — statistics, histories, the AI index, arr caches.
+#   Everything is moved and the source is gone afterwards, so there is no second location that
+#   might still be written to by something that missed the change.
+#
+# The conf rewrite is the last thing, and the riskiest thing.
+#   Until it happens the installation still works from the old layout. That ordering means an
+#   abort partway through leaves a system that runs, rather than one whose paths point at
+#   nothing.
+#
+# ==============================================================================================
 # OPERATIONAL SAFEGUARDS
 # ==============================================================================================
 #
@@ -49,6 +85,25 @@
 #   check costs nothing and the failure is silent otherwise.
 #
 # ==============================================================================================
+# CONFIGURATION
+# ==============================================================================================
+#
+# This script reads conf to find the old locations and rewrites conf to record the new ones. It
+# is the one script here whose purpose is to change these values rather than obey them.
+#
+#   Read to locate what moves
+#     STATE_DIR, BANDWIDTH_LOG, AI_INDEX_DB, AI_MEMORY_FILE, AI_TOKEN_DB, ARR_CLEANUP_STATS,
+#     ARR_SYNC_BLOCKLIST, CORRUPTION_SCAN_STATE_FILE, LIDARR_CACHE_FILE, ZFS_REPORT_LOG and the
+#     rest of the per-script path keys — roughly two dozen in total.
+#
+#   Written as the new roots
+#     DATA_DIR and the directories beneath it: DB_DIR, STATE_DIR, AI_DATA_DIR,
+#     CACHE_BACKUP_DIR, ARR_CACHE_BACKUP_DIR, CONF_CACHE_BACKUP_DIR, LOG_ARCHIVE_DIR.
+#
+#   Every rewritten value is expressed as ${DB_DIR}/… rather than an absolute path, so a later
+#   storage-mode migration moves them again by changing one variable.
+#
+# ==============================================================================================
 # RUNTIME MODES
 # ==============================================================================================
 #

Fallback/coverage_deploy.sh

@@ -42,7 +42,43 @@
 #       my-Foo.xml routinely holds a container called something else. Matching on the filename
 #       silently pushes the wrong template, or nothing at all.
 #
-# USAGE
+# OPERATIONAL SAFEGUARDS
+#   Only in NORMAL state. FALLBACK_STATE_FILE is read before anything is pushed or removed, and
+#   any other state refuses the action. A push during a live failover would deploy a second copy
+#   of a container the partner is currently running on our behalf; a remove would delete the one
+#   doing the covering.
+#
+#   --status is exempt from that gate, because it only reports. Refusing to answer "what is
+#   deployed over there" during a failover would withhold the information precisely when it is
+#   most wanted.
+#
+#   Every deploy is verified stopped, and a container that will not stay stopped is an error
+#   rather than a warning — see DESIGN PRINCIPLES. A second live instance against the same data
+#   is the failure this whole script exists inside.
+#
+#   Push and remove are explicit modes with no default. Running the script with no flag does
+#   nothing; neither action can be reached by accident, and neither is a side effect of editing
+#   the tier list.
+#
+#   --dry-run works in every mode and touches nothing on either host — no container is built,
+#   started, stopped or removed, and no template is written or deleted.
+#
+#   Remove deletes the container's appdata on the partner as well. That is deliberate and is the
+#   most destructive thing here; the NORMAL-state gate above is what keeps it away from a
+#   partner that is mid-handback.
+#
+# CONFIGURATION
+#   master.conf
+#     FALLBACK_<HOST>_TIER1..N   the covered container names — what --push deploys and --status
+#                                reports on. This script reads that list; it never edits it.
+#
+#   host*.conf
+#     FALLBACK_STATE_FILE        overrides where fallback.sh's state is read from. Defaults to
+#                                STATE_DIR/fallback_state.db. A missing file reads as NORMAL,
+#                                which is the correct default on a host where fallback has never
+#                                run.
+#
+# RUNTIME MODES
 #   coverage_deploy.sh --push            deploy every covered container onto the partner (stopped)
 #   coverage_deploy.sh --remove          stop, remove, and delete the pushed template on the partner
 #   coverage_deploy.sh --status          report, per covered container, whether it exists there

Monitors/uptime_report.sh

@@ -20,6 +20,42 @@
 # decision to notify is the exit code rather than this script parsing the text it just printed.
 #
 # ==============================================================================================
+# DESIGN PRINCIPLES
+# ==============================================================================================
+#
+# Silence is the normal output.
+#   A report that always says something is a report nobody reads. A perfect week prints nothing
+#   and notifies nothing, so anything that does appear in the Sunday report is worth the glance.
+#
+# The exit code is the decision, not the text.
+#   uptime_probe.php --report exits 1 when it has something to say and 0 when it does not. This
+#   script never parses the output it just printed to work out whether to notify — a report whose
+#   wording changed would otherwise silently stop notifying.
+#
+# It reads; it never probes.
+#   The measurements are already taken, once a minute, by Tools/uptime_probe.sh. Re-probing at
+#   report time would describe Sunday morning rather than the week being reported on.
+#
+# ==============================================================================================
+# OPERATIONAL SAFEGUARDS
+# ==============================================================================================
+#
+# Read-only. Reads the stored history and prints; records nothing, and cannot alter the data it
+# is reporting on.
+#
+# UPTIME_PROBE_ENABLED gates the whole run — with the probe off there is no history worth
+# reporting, and this says nothing rather than reporting an empty week as a perfect one.
+#
+# ==============================================================================================
+# RUNTIME MODES
+# ==============================================================================================
+#
+# uptime_report.sh    the weekly read. Silent when every domain was 100%.
+#
+# Called from COFFEE_REPORT_SCRIPTS; takes no arguments and has no other mode. For live figures
+# or a per-domain table, use Tools/uptime_probe.sh --status.
+#
+# ==============================================================================================
 # CONFIGURATION
 # ==============================================================================================
 #

Plugin/unraid/Tools/ai_log_check.php

@@ -8,10 +8,34 @@
 //   Reads ai_log_fixtures.txt and this host's /var/log/syslog*. Files nothing, writes nothing,
 //   and calls no part of the sweep beyond vv_ai_syslog_findings() on lines it supplies itself.
 //
-// EXIT
+// DESIGN PRINCIPLES
-//   0 when every fixture is recognised as written. Precision findings are reported but never
+//   Only recall can fail the run.
-//   fail the run: what a real syslog contains is a fact about the machine, not about the
+//       A missed fixture is a fact about the patterns and is always a defect. A precision hit is
-//   patterns, and a genuinely failing disk should not turn this into a red test.
+//       a fact about this machine — a genuinely failing disk should not turn this red, and if it
+//       did, the honest fix would be to stop having a failing disk rather than to edit a pattern.
+//
+//   Precision is replayed against real history, not a sample.
+//       The patterns that cause damage are the ones matching ordinary operation, and ordinary
+//       operation is exactly what a hand-written fixture file never contains. Only the machine's
+//       own syslog can show what a pattern fires on when nothing is wrong.
+//
+//   The sweep is never invoked, only its matcher.
+//       vv_ai_syslog_findings() is called on lines this file supplies. Running the real sweep
+//       would file findings, and a test that has to be cleaned up afterwards stops being run.
+//
+// OPERATIONAL SAFEGUARDS
+//   Read-only. Reads ai_log_fixtures.txt and this host's /var/log/syslog*; files no finding,
+//   writes no store, and touches no conf beyond the enable flag.
+//
+//   Exit 0 when every fixture is recognised as written. Precision findings are reported but
+//   never fail the run — see DESIGN PRINCIPLES.
+//
+// RUNTIME MODES
+//   php ai_log_check.php               both checks
+//   php ai_log_check.php --recall      fixtures only
+//   php ai_log_check.php --precision   replay this host's syslog history only
+//
+//   Not scheduled, and deliberately so. Run it after touching VV_AI_SYSLOG_PATTERNS.
 // ═══════════════════════════════════════════════════════════════════════════════════════════════
 require_once dirname(__DIR__) . '/include/ai_repair.php';
 

Plugin/unraid/Tools/ai_log_check.sh

@@ -15,6 +15,47 @@
 #
 # Run it after touching VV_AI_SYSLOG_PATTERNS. Nothing here writes: no findings are filed, no
 # conf is read for anything but the enable flag, and the sweep is never invoked.
+#
+# ==============================================================================================
+# OPERATIONAL MODEL
+# ==============================================================================================
+#
+# A wrapper. The work is in ai_log_check.php, next to the sweep's own matcher — the patterns and
+# vv_ai_syslog_findings() live in include/ai_repair.php, and a bash reimplementation of the
+# matching would be a second set of regexes to keep in step with the first.
+#
+# Flags are forwarded verbatim; nothing is interpreted here.
+#
+# Not scheduled and in no orchestrator. This is a development check that runs when the patterns
+# change, not on a timer — nothing on the running system depends on it.
+#
+# ==============================================================================================
+# DESIGN PRINCIPLES
+# ==============================================================================================
+#
+# Two checks, because the failure modes are opposite.
+#   Recall catches a pattern that stopped matching; precision catches one that matches too much.
+#   A single test would catch one and silently permit the other, and the second is the one that
+#   fills the findings store with noise until the operator stops reading it.
+#
+# Precision is measured against this machine's real history.
+#   A hand-written fixture file can show that a pattern matches what it should. Only a real
+#   syslog can show what it also matches when nothing is wrong.
+#
+# Only recall fails the run.
+#   What a real syslog contains is a fact about the machine, not about the patterns. A genuinely
+#   failing disk should not turn this red.
+#
+# ==============================================================================================
+# OPERATIONAL SAFEGUARDS
+# ==============================================================================================
+#
+# Read-only. No finding is filed, no store is written, and the repair sweep itself is never run
+# — only its matcher, on lines this check supplies.
+#
+# Safe to run on a live host at any time, including one that is currently faulting. It observes
+# the syslog it replays and changes nothing about it.
+#
 # ==============================================================================================
 # RUNTIME MODES
 # ==============================================================================================

Plugin/unraid/Tools/conf_widget_check.php

@@ -13,16 +13,39 @@
 //     php Tools/conf_widget_check.php            assertions, then the live summary
 //     php Tools/conf_widget_check.php --list     every live field and its inferred control
 //
-// WHY IT ASSERTS AGAINST SNIPPETS AND NOT THE LIVE CONF
+// DESIGN PRINCIPLES
-//   The live conf is the thing being described, so it cannot also be the thing that proves the
+//   Assertions run against snippets, never against the live conf.
-//   description right — an inference rule that silently stopped matching would keep passing as
+//       The live conf is the thing being described, so it cannot also be the thing that proves
-//   the conf drifted to suit it. The snippets are frozen copies of each convention as written,
+//       the description right — an inference rule that silently stopped matching would keep
-//   so a rule change that breaks one shows up here rather than as a wrong control on a page.
+//       passing as the conf drifted to suit it. The snippets are frozen copies of each
+//       convention as written, so a rule change that breaks one shows up here rather than as a
+//       wrong control on a page.
 //
-// WHAT AN INFERENCE IS NOT
+//   An inference is a drawing decision, not a validation.
-//   Consistent with confform.php, none of this validates. A number field carrying min and max is
+//       Consistent with confform.php, none of this validates. A number field carrying min and
-//   a courtesy to whoever is typing, not a promise the value is sensible — the consuming script
+//       max is a courtesy to whoever is typing, not a promise the value is sensible — the
-//   still owns that question.
+//       consuming script still owns that question.
+//
+//   The live pass reports, it does not assert.
+//       What this host's master.conf infers to is a description of that file, not a verdict on
+//       it. Turning the live summary into pass/fail would make an unusual but legitimate
+//       setting look like a defect.
+//
+// OPERATIONAL SAFEGUARDS
+//   Read-only. Parses conf and reports; writes no conf, no store and no page.
+//
+//   Exits non-zero only when a snippet assertion fails, so it can gate a commit without a real
+//   conf's contents ever being able to break the build.
+//
+//   Never renders. It reports which control would be drawn; the drawing stays in confform.php,
+//   so this cannot disagree with the page by construction.
+//
+// RUNTIME MODES
+//   php Tools/conf_widget_check.php            assertions, then the live summary
+//   php Tools/conf_widget_check.php --list     every live field and its inferred control
+//
+//   Hand-run. Not scheduled and in no orchestrator — run it after touching _vv_conf_widget(),
+//   after adding a conf convention, or when a setting draws as the wrong control.
 //
 // DEPENDS ON
 //   include/confform.php   _vv_conf_parse_field_range(), vv_conf_key_is_secret()

Plugin/unraid/Tools/ui_map_build.php

@@ -5,31 +5,49 @@
 //   to reach it. Generated so the assistant can answer "how do I change X" with a path through
 //   the pages instead of an instruction to open master.conf.
 //
-// WHY THE ASSISTANT NEEDS THIS AT ALL
-//   The retrieval index reads git-tracked files. PHP body markup is not indexed and would be
-//   useless if it were — a page is a pile of divs, not a description of itself — so the assistant
-//   has never had any way to know the UI exists. It could name a conf key and nothing more.
-//   pages/readme/*.md is the one directory the chunker classifies as kind='ui', which is why the
-//   output lands there and not in docs/.
-//
-// WHY IT IS GENERATED
-//   A hand-written map is a second description of the pages, and the moment a card moves it
-//   starts lying with total confidence — which is worse than saying nothing, because the
-//   assistant will repeat it. Everything here is derived from the same registries the pages
-//   themselves are built from: VV_SCRIPT_CONF_SECTIONS for what the Scheduler shows per script,
-//   VV_UI_SECTION_SURFACES for the pages that show sections by subject, and the conf files for
-//   the settings and their controls.
-//
 // OPERATIONAL MODEL
-//   Hand-run, and re-run after adding a conf section, a script mapping or a settings surface.
+//   Reads the section registries and the conf files, resolves each setting to the page and card
-//   Writes exactly one file and nothing else.
+//   that renders it, and writes the whole map in one pass. Nothing is merged with what is
+//   already there — the output is derived entirely from the registries, so a stale entry cannot
+//   survive a rebuild.
 //
+// DESIGN PRINCIPLES
+//   The assistant cannot see the UI any other way.
+//       The retrieval index reads git-tracked files. PHP body markup is not indexed and would be
+//       useless if it were — a page is a pile of divs, not a description of itself — so the
+//       assistant has never had any way to know the UI exists. It could name a conf key and
+//       nothing more. pages/readme/*.md is the one directory the chunker classifies as
+//       kind='ui', which is why the output lands there and not in docs/.
+//
+//   Generated, because a hand-written map lies with confidence.
+//       A second description of the pages starts being wrong the moment a card moves, and that
+//       is worse than saying nothing, because the assistant will repeat it. Everything here is
+//       derived from the same registries the pages themselves are built from:
+//       VV_SCRIPT_CONF_SECTIONS for what the Scheduler shows per script, VV_UI_SECTION_SURFACES
+//       for the pages that show sections by subject, and the conf files for the settings and
+//       their controls.
+//
+//   An unreachable section is reported, never dropped.
+//       A section no page renders is listed at the end rather than silently omitted. A setting
+//       with no route through the UI is a real finding, and this map is the only thing that
+//       would ever notice.
+//
+// OPERATIONAL SAFEGUARDS
+//   Writes exactly one file, pages/readme/ui-map.md, and nothing else. No conf is modified, no
+//   page is touched, and the registries it reads are only read.
+//
+//   --check reports what would change and writes nothing, so the map can be verified current in
+//   a commit without regenerating it.
+//
+//   Generated output only. Nothing hand-edited belongs in ui-map.md — an edit there is lost on
+//   the next run, which is the correct behaviour for a derived file and the reason the header
+//   says so.
+//
+// RUNTIME MODES
 //   php Tools/ui_map_build.php            write the map
 //   php Tools/ui_map_build.php --check    report what it would change, write nothing
 //
-//   Only sections that are genuinely reachable are listed. A section no page renders is reported
+//   Hand-run. Re-run after adding a conf section, a script mapping or a settings surface.
-//   at the end as unreachable rather than silently omitted — a setting with no route is a real
-//   finding, and the map is the only place that would notice.
 //
 // DEPENDS ON
 //   include/confform.php   the section registries, the parser, and the inferred controls

Tools/conf_container_prune.sh

@@ -23,6 +23,7 @@
 #       rsync would then stop nothing before copying a live database. They are excluded by name and
 #       must stay excluded.
 #
+# OPERATIONAL SAFEGUARDS
 #   A docker that cannot be read is not a docker with nothing in it.
 #       If `docker ps -a` fails, times out, or returns nothing on a host that is meant to run
 #       containers, EVERY name looks missing and one run would strike the entire configuration.
@@ -34,7 +35,24 @@
 #       verified by sourcing the result in a subshell and confirming the key still parses as an
 #       array with exactly one fewer element.
 #
-# USAGE
+#   Each removal is its own verified rewrite.
+#       Several names can reach the limit in one run, and each is removed and re-verified
+#       independently rather than batched into a single edit. A rewrite that fails verification
+#       therefore costs that one entry, not every entry the run intended to prune.
+#
+#   --dry-run records no strike. A dry run that advanced the counter would eventually prune
+#   through repetition alone, which is the opposite of what it is for.
+#
+# CONFIGURATION
+#   master.conf
+#     CONF_PRUNE_STRIKE_LIMIT   consecutive runs a name must be missing before it is removed.
+#                               Seeing the container again resets its strike to zero immediately,
+#                               so a rebuild costs one strike at most.
+#
+#   The keys this may prune are an explicit allow list in the script, deliberately not a conf
+#   value — see DESIGN PRINCIPLES for why a pattern is the wrong shape here.
+#
+# RUNTIME MODES
 #   conf_container_prune.sh              strike, and prune anything at the limit
 #   conf_container_prune.sh --dry-run    report what would be struck and pruned, write nothing
 #   conf_container_prune.sh --status     show current strikes and stop