feat: slskd reconnect guard in downloaders_reset, mass v2 sync

- downloaders_reset: connection check block before slskd API sections; triggers PUT /api/v0/server reconnect if disconnected, polls 60s, gates Stuck Searches and Dead Transfer Records on SLSKD_CONNECTED - Sync all modified/new/deleted files from v2 refactor across Docker_Essentials, Media, Monitors, Partnership, Rsync, Tools, Transcodes, unRAID_Essentials, common.sh, master confs, and new Manual/README docs
2026-05-19 20:00:10 -04:00
parent 5cb16d4b18
commit e13f2fa14f
81 changed files with 12164 additions and 10656 deletions
@@ -1,55 +1,90 @@
 #!/bin/bash
 # ==============================================================================================
-# ================================= Certificate Monitor ========================================
+# ============================= Certificate Monitor ============================================
 # ==============================================================================================
-# Monitors SSL certificate expiry for all configured domains by connecting directly
-# via openssl — no dependency on NPM or any other service. Reads the actual certificate
-# the server is presenting to the outside world.
 #
-# ── WHY DIRECT OPENSSL ────────────────────────────────────────────────────────────────────────
-# Catches real-world cert issues that API-based checks miss:
-#   - Cert renewed in NPM but server not reloaded (old cert still serving)
-#   - Wrong cert being served to external clients
-#   - Cert chain issues not visible from the internal network
-#   - NPM reporting healthy while the world sees an expired cert
+# PURPOSE
+# ─────────────────────────────────────────────────────────────────────────────
+# SSL certificate expiry monitoring for all configured domains. Scheduled weekly
+# (Sunday 9am). Connects via openssl directly to each domain — not to NPM's API,
+# not to any internal check, but to the actual TLS handshake the outside world sees.
 #
-# ── BEHAVIOUR ─────────────────────────────────────────────────────────────────────────────────
-# Each domain is checked independently — they have independent certs.
-# Results per domain:
-#   HEALTHY  — > CERT_WARN_DAYS remaining — silent ✅
-#   WARNING  — <= CERT_WARN_DAYS remaining — notifies
-#   CRITICAL — <= CERT_CRIT_DAYS remaining — notifies with urgency
-#   FAILED   — could not connect or parse cert — notifies
+# Per domain: HEALTHY (> CERT_WARN_DAYS remaining, silent) | WARNING (≤ CERT_WARN_DAYS)
+# | CRITICAL (≤ CERT_CRIT_DAYS) | FAILED (could not connect or parse cert).
+# Notifications batched by severity — one message lists all WARNING domains, a
+# separate message lists all CRITICAL domains. Not one notification per domain.
 #
-# Notifications batched per severity — one message per severity level, not per domain.
-# This is a monitor script — SILENT_MODE=false — output is the point.
+# ==============================================================================================
+# DESIGN PRINCIPLES
+# ==============================================================================================
 #
-# ── HOST AWARENESS ────────────────────────────────────────────────────────────────────────────
-# detect_hosts() sets MY_ID and aliases HOST*_CERT_MONITOR_DOMAINS → CERT_MONITOR_DOMAINS.
-# Each server monitors its own domains — HOST1 monitors Gmer4Lfe.com etc.
+# Direct openssl, Not an API
+#   API-based cert checks ask the certificate manager whether the cert is valid.
+#   openssl checks ask the server what cert it is actually serving. These are not
+#   the same question and the answers can differ. Catches: cert renewed in NPM but
+#   server not reloaded (old cert still serving), wrong cert being served to external
+#   clients, chain issues visible externally but not internally, NPM reporting healthy
+#   while the outside world sees an expired cert.
 #
-# ── SAFEGUARDS ────────────────────────────────────────────────────────────────────────────────
-#   acquire_lock         — prevents concurrent runs
-#   detect_hosts()       — correct domain list per host via MY_ID aliases
-#   Empty array guard    — warns and exits cleanly if no domains configured
-#   CERT_TIMEOUT         — openssl connects are time-limited per domain
-#   validate_unraid_cmd  — openssl and notify validated before use
-#   Silent healthy certs — only problems produce visible output
+# ==============================================================================================
+# OPERATIONAL SAFEGUARDS
+# ==============================================================================================
 #
-# ── CONFIGURATION (master_host*.conf) ─────────────────────────────────────────────────────────
-#   HOST*_CERT_MONITOR_DOMAINS — domains checked by this host
-#   Aliased by detect_hosts() — script uses CERT_MONITOR_DOMAINS
+# Single Instance Lock
+#   acquire_lock prevents concurrent runs producing duplicate notifications.
 #
-# ── CONFIGURATION (master.conf) ───────────────────────────────────────────────────────────────
-#   CERT_WARN_DAYS  — warn when cert expires within this many days (default 30)
-#   CERT_CRIT_DAYS  — critical alert within this many days (default 7)
-#   CERT_TIMEOUT    — seconds per domain before giving up (default 10)
+# Per-Host Domain List
+#   detect_hosts() aliases HOST*_CERT_MONITOR_DOMAINS → CERT_MONITOR_DOMAINS.
+#   Each server monitors its own domains only.
+#
+# Empty Array Guard
+#   Warns and exits cleanly if CERT_MONITOR_DOMAINS is empty — no silent no-op.
+#
+# Connection Timeout
+#   CERT_TIMEOUT caps each openssl connection attempt. One unreachable domain
+#   does not block the remaining domains.
+#
+# Notification Validated
+#   validate_unraid_cmd confirms openssl and notify script are present before use.
+#
+# ==============================================================================================
+# CONFIGURATION
+# ==============================================================================================
+#
+# master_host*.conf
+#
+#   HOST*_CERT_MONITOR_DOMAINS
+#       Domains this host monitors. Each domain and subdomain is a separate entry —
+#       they have independent certs. Aliased by detect_hosts() → CERT_MONITOR_DOMAINS.
+#
+# master.conf
+#
+#   CERT_WARN_DAYS
+#       Days before expiry at which to send a warning notification. (default: 30)
+#
+#   CERT_CRIT_DAYS
+#       Days before expiry at which to send a critical notification. (default: 7)
+#
+#   CERT_TIMEOUT
+#       Seconds to wait per domain before declaring FAILED. (default: 10)
+#
+# ==============================================================================================
+# RUNTIME MODES
+# ==============================================================================================
+#
+# cert_monitor.sh
+#     Check all configured domains and notify on WARNING, CRITICAL, or FAILED.
+#     Silent when all domains are healthy.
+#
+# cert_monitor.sh --dry-run
+#     Check all domains and show results. No notifications sent regardless of result.
+#
+# cert_monitor.sh --status
+#     Show domain list, warning thresholds, and timeout. Then exit.
+#
+# cert_monitor.sh --log
+#     Verbose per-domain output during the run.
 #
-# ── USAGE ─────────────────────────────────────────────────────────────────────────────────────
-#   cert_monitor.sh              — normal run
-#   cert_monitor.sh --dry-run    — check certs and show results, no notifications
-#   cert_monitor.sh --log        — verbose output
-#   cert_monitor.sh --status     — show config and exit
 # ==============================================================================================

 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"