feat: slskd reconnect guard in downloaders_reset, mass v2 sync

- downloaders_reset: connection check block before slskd API sections; triggers PUT /api/v0/server reconnect if disconnected, polls 60s, gates Stuck Searches and Dead Transfer Records on SLSKD_CONNECTED - Sync all modified/new/deleted files from v2 refactor across Docker_Essentials, Media, Monitors, Partnership, Rsync, Tools, Transcodes, unRAID_Essentials, common.sh, master confs, and new Manual/README docs
2026-05-19 20:00:10 -04:00
parent 5cb16d4b18
commit e13f2fa14f
81 changed files with 12164 additions and 10656 deletions
@@ -2,57 +2,92 @@
 # ==============================================================================================
 # ================================= WebGUI Watchdog ============================================
 # ==============================================================================================
-# Monitors the unRAID WebGUI and restarts services if unresponsive.
-# Uses a three-step escalating strategy — lightest fix first, heaviest last.
-# Run every 5-10 minutes via User Scripts plugin.
-# Silent when healthy — only produces output when something needs fixing.
 #
-# ── ESCALATION PATH ───────────────────────────────────────────────────────────────────────────
-#   Check WebGUI → responding → log() + exit 0 (completely silent ✅)
+# PURPOSE
+# ─────────────────────────────────────────────────────────────────────────────
+# Monitors the unRAID WebGUI and restarts services if unresponsive. Uses a
+# three-step escalating strategy — lightest fix first, heaviest last. Run
+# every 5–10 minutes via the User Scripts plugin. Silent when healthy.
+#
+# ==============================================================================================
+# OPERATIONAL MODEL
+# ==============================================================================================
+#
+# Escalation Path
+#   WebGUI responding → log() + exit 0 (completely silent ✅)
 #
 #   Not responding:
-#     Step 1 — Restart nginx
-#               Lightest fix — handles most transient WebGUI failures
-#               nginx crash, worker stuck, connection timeout
-#               Wait WEBGUI_NGINX_WAIT seconds → recheck
+#     Step 1 — nginx restart
+#               Lightest fix — handles most transient WebGUI failures:
+#               nginx crash, worker stuck, connection timeout.
+#               Wait WEBGUI_NGINX_WAIT seconds → recheck.
 #
-#     Step 2 — Restart php-fpm
-#               WebGUI runs through PHP-FPM — worker exhaustion causes silent failure
-#               php-fpm workers saturated → new requests queue → WebGUI appears frozen
-#               system_tuning_monitor.sh tracks usage — this recovers it
-#               Wait WEBGUI_PHP_WAIT seconds → recheck
+#     Step 2 — php-fpm restart
+#               WebGUI runs through PHP-FPM. Worker exhaustion causes silent
+#               failure — requests queue and the WebGUI appears frozen.
+#               Wait WEBGUI_PHP_WAIT seconds → recheck.
 #
-#     Step 3 — Restart emhttp
-#               Heaviest fix — emhttp is the unRAID management daemon
-#               Array, Docker, shares stay running — only WebGUI management restarts
-#               Takes longer to recover — WEBGUI_EMHTTP_WAIT gives it time
-#               Wait WEBGUI_EMHTTP_WAIT seconds → recheck
+#     Step 3 — emhttp restart
+#               Heaviest fix. emhttp is the unRAID management daemon.
+#               Array, Docker, and shares stay running — only WebGUI
+#               management restarts. Takes longer — WEBGUI_EMHTTP_WAIT.
+#               Wait WEBGUI_EMHTTP_WAIT seconds → recheck.
 #
-#     All three failed → notify warning, manual intervention needed → exit 1
+#     All three failed → notify, manual intervention needed → exit 1.
 #
-# ── HOST AWARENESS ────────────────────────────────────────────────────────────────────────────
-# detect_hosts() sets MY_ID — used in all notifications and summary.
-# Critical on two-server setup — which server's WebGUI failed?
+# ==============================================================================================
+# OPERATIONAL SAFEGUARDS
+# ==============================================================================================
 #
-# ── SAFEGUARDS ────────────────────────────────────────────────────────────────────────────────
-#   acquire_lock        — prevents concurrent runs double-restarting services
-#   detect_hosts()      — MY_ID in all notifications
-#   Process verify      — pgrep check after each service restart
-#   Silent healthy      — completely silent on healthy cycle ✅
-#   validate_unraid_cmd — notify validated before use
+# Root Required
+#   Service restart commands require root.
 #
-# ── CONFIGURATION (master.conf) ───────────────────────────────────────────────────────────────
-#   WEBGUI_URL          — URL to check (default http://localhost)
-#   WEBGUI_TIMEOUT      — curl timeout in seconds (default 5)
-#   WEBGUI_NGINX_WAIT   — seconds after nginx restart before rechecking (default 15)
-#   WEBGUI_PHP_WAIT     — seconds after php-fpm restart before rechecking (default 10)
-#   WEBGUI_EMHTTP_WAIT  — seconds after emhttp restart before rechecking (default 30)
+# Single Instance Lock
+#   acquire_lock prevents concurrent runs double-restarting services.
+#
+# Process Verify After Each Restart
+#   pgrep check after each rc.* command — errors if process not running.
+#
+# Silent When Healthy
+#   Completely silent on healthy cycles. Only produces output when recovering.
+#
+# ==============================================================================================
+# CONFIGURATION
+# ==============================================================================================
+#
+# master.conf
+#
+#   WEBGUI_URL
+#       URL to check for WebGUI response. (default: http://localhost)
+#
+#   WEBGUI_TIMEOUT
+#       curl timeout in seconds. (default: 5)
+#
+#   WEBGUI_NGINX_WAIT
+#       Seconds after nginx restart before rechecking. (default: 15)
+#
+#   WEBGUI_PHP_WAIT
+#       Seconds after php-fpm restart before rechecking. (default: 10)
+#
+#   WEBGUI_EMHTTP_WAIT
+#       Seconds after emhttp restart before rechecking. (default: 30)
+#
+# ==============================================================================================
+# RUNTIME MODES
+# ==============================================================================================
+#
+# webgui_restart.sh
+#     Check WebGUI. Escalate through nginx → php-fpm → emhttp if unresponsive.
+#
+# webgui_restart.sh --dry-run
+#     Show which services would be restarted. No restarts, no waits.
+#
+# webgui_restart.sh --status
+#     Show current WebGUI response state and nginx/php-fpm/emhttp process states.
+#
+# webgui_restart.sh --log
+#     Verbose output — show each check, each restart attempt, each wait.
 #
-# ── USAGE ─────────────────────────────────────────────────────────────────────────────────────
-#   webgui_restart.sh              — check and recover if needed
-#   webgui_restart.sh --dry-run    — show what would be restarted
-#   webgui_restart.sh --status     — show current WebGUI and service states
-#   webgui_restart.sh --log        — verbose output
 # ==============================================================================================

 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"