80 lines
4.6 KiB
Bash
Executable File
80 lines
4.6 KiB
Bash
Executable File
#!/bin/bash
|
|
# ==============================================================================================
|
|
# ================================ NPM Access Stats ============================================
|
|
# ==============================================================================================
|
|
#
|
|
# PURPOSE
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# Aggregates Nginx Proxy Manager's per-host access logs into DB_DIR/npm_access.json — requests,
|
|
# bytes sent, status breakdown and last hit per proxy host. The Proxies tab reads it.
|
|
#
|
|
# NPM writes one access log per host and counts nothing. The logs here are 475 MB across 41 files,
|
|
# so this cannot happen inside a page load; each pass reads only what arrived since the last one.
|
|
#
|
|
# ==============================================================================================
|
|
# OPERATIONAL MODEL
|
|
# ==============================================================================================
|
|
#
|
|
# A wrapper. The work is in npm_access_stats.php, next to the NPM client and conf helpers it uses.
|
|
# Same split as api_cache_writer, ai_repair_sweep and cert_history.
|
|
#
|
|
# Totals are "since tracking began", not since the host existed: lines that rotate out between two
|
|
# passes are not counted. Running daily keeps that to whatever NPM rotates in a day.
|
|
#
|
|
# ==============================================================================================
|
|
# DESIGN PRINCIPLES
|
|
# ==============================================================================================
|
|
#
|
|
# Incremental by byte offset, never a re-read.
|
|
# 475 MB across 41 files cannot be parsed on a page load, and re-parsing what was already
|
|
# counted would make each pass slower than the last. Each run records where it stopped and
|
|
# resumes there, so cost tracks new traffic rather than total traffic.
|
|
#
|
|
# A rotated log is detected, not assumed.
|
|
# A file smaller than the offset already recorded for it means NPM rotated it beneath us, so
|
|
# that file's offset restarts at zero and a rotation is counted. The totals are cumulative and
|
|
# are never reset by it — what rotated out between two passes is simply not counted, which is
|
|
# why this runs often enough to keep that gap to whatever NPM rotates in a day.
|
|
#
|
|
# Varaverk's own probes are excluded.
|
|
# uptime_probe.sh requests every proxied host every minute under Varaverk-Uptime/1.0. Counting
|
|
# those would make the monitor the busiest client of everything it monitors.
|
|
#
|
|
# ==============================================================================================
|
|
# OPERATIONAL SAFEGUARDS
|
|
# ==============================================================================================
|
|
#
|
|
# Read-only against NPM. Access logs are read and never rotated, truncated or removed — that is
|
|
# NPM's business, and a stats collector that deleted its own inputs would be unrecoverable.
|
|
#
|
|
# --dry-run parses and reports without writing the store or advancing any offset, so a dry run
|
|
# leaves the next real pass with exactly the same work to do.
|
|
#
|
|
# --reset forgets offsets and totals deliberately. It starts from the current end of each log,
|
|
# so it discards history rather than double-counting it.
|
|
#
|
|
# The store is replaced atomically: written to a temp file, re-read and parsed to prove it is
|
|
# valid JSON, then renamed over the original. A pass that dies midway — or produces something
|
|
# unparseable — leaves the previous document untouched rather than a truncated one.
|
|
#
|
|
# ==============================================================================================
|
|
# RUNTIME MODES
|
|
# ==============================================================================================
|
|
#
|
|
# npm_access_stats.sh one pass
|
|
# npm_access_stats.sh --dry-run parse and report, write nothing
|
|
# npm_access_stats.sh --status print the store
|
|
# npm_access_stats.sh --reset forget offsets and totals, start again from the current logs
|
|
#
|
|
# ==============================================================================================
|
|
# CONFIGURATION
|
|
# ==============================================================================================
|
|
#
|
|
# NPM_LOG_DIR override the log directory; otherwise derived from the container's mounts
|
|
# DB_DIR npm_access.json is written here
|
|
#
|
|
# ==============================================================================================
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
php "$SCRIPT_DIR/npm_access_stats.php" "$@"
|