#!/bin/bash # ============================================================================================== # ================================ NPM Access Stats ============================================ # ============================================================================================== # # PURPOSE # ───────────────────────────────────────────────────────────────────────────── # Aggregates Nginx Proxy Manager's per-host access logs into DB_DIR/npm_access.json — requests, # bytes sent, status breakdown and last hit per proxy host. The Proxies tab reads it. # # NPM writes one access log per host and counts nothing. The logs here are 475 MB across 41 files, # so this cannot happen inside a page load; each pass reads only what arrived since the last one. # # ============================================================================================== # OPERATIONAL MODEL # ============================================================================================== # # A wrapper. The work is in npm_access_stats.php, next to the NPM client and conf helpers it uses. # Same split as api_cache_writer, ai_repair_sweep and cert_history. # # Totals are "since tracking began", not since the host existed: lines that rotate out between two # passes are not counted. Running daily keeps that to whatever NPM rotates in a day. # # ============================================================================================== # DESIGN PRINCIPLES # ============================================================================================== # # Incremental by byte offset, never a re-read. # 475 MB across 41 files cannot be parsed on a page load, and re-parsing what was already # counted would make each pass slower than the last. Each run records where it stopped and # resumes there, so cost tracks new traffic rather than total traffic. # # A rotated log is detected, not assumed. # A file smaller than the offset already recorded for it means NPM rotated it beneath us, so # that file's offset restarts at zero and a rotation is counted. The totals are cumulative and # are never reset by it — what rotated out between two passes is simply not counted, which is # why this runs often enough to keep that gap to whatever NPM rotates in a day. # # Varaverk's own probes are excluded. # uptime_probe.sh requests every proxied host every minute under Varaverk-Uptime/1.0. Counting # those would make the monitor the busiest client of everything it monitors. # # ============================================================================================== # OPERATIONAL SAFEGUARDS # ============================================================================================== # # Read-only against NPM. Access logs are read and never rotated, truncated or removed — that is # NPM's business, and a stats collector that deleted its own inputs would be unrecoverable. # # --dry-run parses and reports without writing the store or advancing any offset, so a dry run # leaves the next real pass with exactly the same work to do. # # --reset forgets offsets and totals deliberately. It starts from the current end of each log, # so it discards history rather than double-counting it. # # The store is replaced atomically: written to a temp file, re-read and parsed to prove it is # valid JSON, then renamed over the original. A pass that dies midway — or produces something # unparseable — leaves the previous document untouched rather than a truncated one. # # ============================================================================================== # RUNTIME MODES # ============================================================================================== # # npm_access_stats.sh one pass # npm_access_stats.sh --dry-run parse and report, write nothing # npm_access_stats.sh --status print the store # npm_access_stats.sh --reset forget offsets and totals, start again from the current logs # # ============================================================================================== # CONFIGURATION # ============================================================================================== # # NPM_LOG_DIR override the log directory; otherwise derived from the container's mounts # DB_DIR npm_access.json is written here # # ============================================================================================== SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" php "$SCRIPT_DIR/npm_access_stats.php" "$@"