Document what the proxy and cert collectors guarantee

2026-08-25 16:37:25 -04:00
parent 6b9fda76db
commit 68c835f33b
3 changed files with 103 additions and 0 deletions
@@ -22,6 +22,42 @@
 # passes are not counted. Running daily keeps that to whatever NPM rotates in a day.
 #
 # ==============================================================================================
+# DESIGN PRINCIPLES
+# ==============================================================================================
+#
+# Incremental by byte offset, never a re-read.
+#   475 MB across 41 files cannot be parsed on a page load, and re-parsing what was already
+#   counted would make each pass slower than the last. Each run records where it stopped and
+#   resumes there, so cost tracks new traffic rather than total traffic.
+#
+# A rotated log is detected, not assumed.
+#   A file smaller than the offset already recorded for it means NPM rotated it beneath us, so
+#   that file's offset restarts at zero and a rotation is counted. The totals are cumulative and
+#   are never reset by it — what rotated out between two passes is simply not counted, which is
+#   why this runs often enough to keep that gap to whatever NPM rotates in a day.
+#
+# Varaverk's own probes are excluded.
+#   uptime_probe.sh requests every proxied host every minute under Varaverk-Uptime/1.0. Counting
+#   those would make the monitor the busiest client of everything it monitors.
+#
+# ==============================================================================================
+# OPERATIONAL SAFEGUARDS
+# ==============================================================================================
+#
+# Read-only against NPM. Access logs are read and never rotated, truncated or removed — that is
+# NPM's business, and a stats collector that deleted its own inputs would be unrecoverable.
+#
+# --dry-run parses and reports without writing the store or advancing any offset, so a dry run
+# leaves the next real pass with exactly the same work to do.
+#
+# --reset forgets offsets and totals deliberately. It starts from the current end of each log,
+# so it discards history rather than double-counting it.
+#
+# The store is replaced atomically: written to a temp file, re-read and parsed to prove it is
+# valid JSON, then renamed over the original. A pass that dies midway — or produces something
+# unparseable — leaves the previous document untouched rather than a truncated one.
+#
+# ==============================================================================================
 # RUNTIME MODES
 # ==============================================================================================
 #