feat: slskd reconnect guard in downloaders_reset, mass v2 sync

- downloaders_reset: connection check block before slskd API sections;
  triggers PUT /api/v0/server reconnect if disconnected, polls 60s,
  gates Stuck Searches and Dead Transfer Records on SLSKD_CONNECTED
- Sync all modified/new/deleted files from v2 refactor across Docker_Essentials,
  Media, Monitors, Partnership, Rsync, Tools, Transcodes, unRAID_Essentials,
  common.sh, master confs, and new Manual/README docs
This commit is contained in:
Gmer4Lfe
2026-05-19 20:00:10 -04:00
parent 5cb16d4b18
commit e13f2fa14f
81 changed files with 12164 additions and 10656 deletions
+96 -42
View File
@@ -1,61 +1,115 @@
#!/bin/bash
# ==============================================================================================
# ================================= Health Digest ==============================================
# ============================= Health Digest ==================================================
# ==============================================================================================
# Aggregates system health data from across the ecosystem into a single digest report.
# Reads existing state files — no new writes.
#
# ── THREE PROFILES ────────────────────────────────────────────────────────────────────────────
# PURPOSE
# ─────────────────────────────────────────────────────────────────────────────
# Full ecosystem health aggregation from existing state files. Scheduled daily
# (8am). DIGEST_PROFILE controls when notifications actually send — the cron
# schedule never changes, only the profile in master.conf.
#
# Reads state files from across the system (watchdog strikes, fallback state,
# skip list, bandwidth history, transcode stats, cert status) and compiles them
# into a single digest. Reads only — writes nothing, changes nothing.
#
# ==============================================================================================
# OPERATIONAL MODEL
# ==============================================================================================
#
# Three profiles — switch by changing DIGEST_PROFILE in master.conf:
#
# always — sends every run regardless of findings
# schedule daily for a daily digest
# Use: daily digest of everything, even when healthy
#
# smart — sends only if something worth reporting was found
# runs every run but stays silent when all healthy
# DIGEST_SMART_ON_* toggles control what triggers a send
# smart — sends only when something worth reporting was found
# Stays silent on clean days. DIGEST_SMART_ON_* toggles control
# what triggers a send — all independently configurable.
#
# weekly — sends once per week on DIGEST_DAY regardless of schedule frequency
# run daily, digest only fires on DIGEST_DAY (default Sunday)
# weekly — sends once per week on DIGEST_DAY (default Sunday), silent all other days
# Use: one weekly summary without daily noise
#
# The cron schedule stays the same regardless of profile — change DIGEST_PROFILE in
# master.conf to switch behaviour. No cron changes needed.
#
# ── DATA SOURCES (reads only) ─────────────────────────────────────────────────────────────────
# FALLBACK_STATE_FILE — current fallback state
# SYS_WATCHDOG_FAILED_FILE — container skip list (manual intervention needed)
# WATCHDOG_STATE_FILE — active container watchdog strikes
# SYS_WATCHDOG_STATE_FILEactive system watchdog strikes
# BANDWIDTH_LOG — yesterday's transfer totals
# TRANSCODE_DAILY_LOG — weekly transcode statistics
# CERT_MONITOR_DOMAINS — live SSL cert check via openssl
# Data sources (reads only):
# FALLBACK_STATE_FILE — current fallback state
# SYS_WATCHDOG_FAILED_FILE — container skip list (manual intervention needed)
# WATCHDOG_STATE_FILE — active container watchdog strikes
# SYS_WATCHDOG_STATE_FILE — active system watchdog strikes
# BANDWIDTH_LOG — yesterday's transfer totals
# TRANSCODE_DAILY_LOG — weekly transcode statistics
# CERT_MONITOR_DOMAINS live SSL cert check via openssl
# RAMDISK_PATH / TRANSCODE_LINK — current transcode location and usage
#
# ── HOST AWARENESS ────────────────────────────────────────────────────────────────────────────
# detect_hosts() sets MY_ID and aliases CERT_MONITOR_DOMAINS, RAMDISK_WARN_GB,
# RAMDISK_SIZE, RAMDISK_LOW_GB and all other host-specific vars used in this report.
# ==============================================================================================
# OPERATIONAL SAFEGUARDS
# ==============================================================================================
#
# ── SAFEGUARDS ────────────────────────────────────────────────────────────────────────────────
# acquire_lock — report takes time, prevent duplicate runs
# detect_hosts() — correct vars per host
# validate_unraid_cmd — notify and openssl validated before use
# Per-section guards — missing state file skipped cleanly
# Silent smart profile — completely silent when nothing to report
# Single Instance Lock
# acquire_lock prevents duplicate reports — report generation takes time.
#
# Per-Host Variables
# detect_hosts() aliases CERT_MONITOR_DOMAINS, RAMDISK_WARN_GB, RAMDISK_SIZE,
# RAMDISK_LOW_GB, and all other host-specific vars used in the report.
#
# Per-Section Guards
# Each data source section checks whether its state file exists before reading.
# A missing state file is skipped cleanly — it does not abort the report.
#
# Silent Smart Profile
# smart profile produces no output and no notification when nothing worth
# reporting is found.
#
# Notifications Validated
# validate_unraid_cmd confirms notify and openssl are present before use.
#
# ==============================================================================================
# CONFIGURATION
# ==============================================================================================
#
# master.conf
#
# DIGEST_PROFILE
# Notification frequency: always | smart | weekly. (default: weekly)
#
# DIGEST_DAY
# Day name for weekly profile — must match `date +%A` output. (default: Sunday)
#
# DIGEST_SMART_ON_WATCHDOG
# Send smart profile notification if any active watchdog strikes. (default: true)
#
# DIGEST_SMART_ON_FALLBACK
# Send smart profile notification if fallback state is not NORMAL. (default: true)
#
# DIGEST_SMART_ON_CERT_WARN
# Send smart profile notification if any cert is within CERT_WARN_DAYS. (default: true)
#
# DIGEST_SMART_ON_BANDWIDTH
# Send smart profile notification if any transfer exceeded BANDWIDTH_WARN_GB. (default: true)
#
# ── CONFIGURATION (master.conf) ───────────────────────────────────────────────────────────────
# DIGEST_PROFILE — always | smart | weekly
# DIGEST_DAY — day name for weekly profile (e.g. Sunday)
# DIGEST_SMART_ON_WATCHDOG — send on active watchdog strikes
# DIGEST_SMART_ON_FALLBACK — send on non-NORMAL fallback state
# DIGEST_SMART_ON_CERT_WARN — send on cert warning
# DIGEST_SMART_ON_BANDWIDTH — send on high bandwidth day
# CERT_WARN_DAYS / CERT_CRIT_DAYS / CERT_TIMEOUT
# Cert check thresholds — shared with cert_monitor.sh.
#
# BANDWIDTH_WARN_GB
# High-transfer threshold — shared with bandwidth_monitor.sh.
#
# TRANSCODE_DAILY_LOG
# Path to the transcode statistics log.
#
# ==============================================================================================
# RUNTIME MODES
# ==============================================================================================
#
# weekly_health_digest.sh
# Run digest. DIGEST_PROFILE determines whether a notification is sent.
#
# weekly_health_digest.sh --dry-run
# Generate and display digest output. No notification sent regardless of profile.
#
# weekly_health_digest.sh --status
# Show profile, day, and smart trigger configuration. Then exit.
#
# weekly_health_digest.sh --log
# Verbose per-section output during digest generation.
#
# ── USAGE ─────────────────────────────────────────────────────────────────────────────────────
# weekly_health_digest.sh — normal run
# weekly_health_digest.sh --dry-run — generate report, no notification
# weekly_health_digest.sh --log — verbose output
# weekly_health_digest.sh --status — show config and exit
# ==============================================================================================
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"