feat: slskd reconnect guard in downloaders_reset, mass v2 sync
- downloaders_reset: connection check block before slskd API sections; triggers PUT /api/v0/server reconnect if disconnected, polls 60s, gates Stuck Searches and Dead Transfer Records on SLSKD_CONNECTED - Sync all modified/new/deleted files from v2 refactor across Docker_Essentials, Media, Monitors, Partnership, Rsync, Tools, Transcodes, unRAID_Essentials, common.sh, master confs, and new Manual/README docs
This commit is contained in:
Gmer4Lfe
@@ -2,65 +2,105 @@
|
||||
# ==============================================================================================
|
||||
# ================================= Resource Manager ===========================================
|
||||
# ==============================================================================================
|
||||
# Pressure reduction layer — detects rising system load and reduces it intelligently.
|
||||
# Called by watchdog_orchestrator.sh every minute — single-pass, not a continuous loop.
|
||||
#
|
||||
# ── RESPONSIBILITY ────────────────────────────────────────────────────────────────────────────
|
||||
# Reduce system pressure before things break. NOT fixing broken containers (docker_watchdog)
|
||||
# and NOT rebooting (system_watchdog). The middle layer that keeps the system comfortable.
|
||||
# PURPOSE
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Pressure reduction layer — detects rising system load and reduces it before
|
||||
# things break. Called by watchdog_orchestrator.sh every minute as a single-
|
||||
# pass run. The middle layer between docker_watchdog.sh (fixes broken
|
||||
# containers) and system_watchdog.sh (reboots). Does neither of those things.
|
||||
#
|
||||
# "Pressure is rising — reduce load intelligently."
|
||||
# ==============================================================================================
|
||||
# OPERATIONAL MODEL
|
||||
# ==============================================================================================
|
||||
#
|
||||
# ── THREE-LEVEL PRESSURE RESPONSE ─────────────────────────────────────────────────────────────
|
||||
# Three-Level Pressure Response
|
||||
#
|
||||
# Level 1 — SOFT (RAM < RW_RAM_SOFT_GB OR load > RW_LOAD_SOFT_MULTIPLIER × cores):
|
||||
# Throttle SABnzbd download speed to RW_SABNZBD_SPEED_SOFT
|
||||
# Throttle qBittorrent download to RW_QBIT_DL_SOFT KB/s
|
||||
# Throttle SABnzbd download speed to RW_SABNZBD_SPEED_SOFT.
|
||||
# Throttle qBittorrent download to RW_QBIT_DL_SOFT KB/s.
|
||||
#
|
||||
# Level 2 — MEDIUM (RAM < RW_RAM_MEDIUM_GB OR load > RW_LOAD_MEDIUM_MULTIPLIER × cores):
|
||||
# Further throttle SABnzbd + qBittorrent to medium limits
|
||||
# docker pause RW_PAUSE_CONTAINERS — suspend without losing state, instant reversible
|
||||
# Further throttle SABnzbd + qBittorrent to medium limits.
|
||||
# docker pause RW_PAUSE_CONTAINERS — suspend without losing state, instantly reversible.
|
||||
#
|
||||
# Level 3 — HARD (RAM < RW_RAM_HARD_GB):
|
||||
# docker stop RW_STOP_CONTAINERS — optional/heavy services (games, LocalAI, etc.)
|
||||
# Write mem_shutdown_active=true — signals docker_watchdog to defer container restarts
|
||||
# docker stop RW_STOP_CONTAINERS — optional/heavy services (games, LocalAI, etc.).
|
||||
# Write mem_shutdown_active=true → signals docker_watchdog to defer container restarts.
|
||||
#
|
||||
# ── RECOVERY ──────────────────────────────────────────────────────────────────────────────────
|
||||
# Pressure must stay below current action threshold for RW_RECOVER_CYCLES consecutive runs
|
||||
# before restoring. De-escalates one level at a time to avoid re-triggering immediately.
|
||||
# Level 3 de-escalation additionally requires RAM >= RW_RAM_RECOVER_GB before un-stopping.
|
||||
# Recovery
|
||||
# Pressure must stay below current threshold for RW_RECOVER_CYCLES consecutive
|
||||
# runs before restoring. De-escalates one level at a time — prevents re-triggering
|
||||
# immediately after recovery. Level 3 additionally requires RAM >= RW_RAM_RECOVER_GB
|
||||
# before containers are un-stopped.
|
||||
#
|
||||
# ── COORDINATION WITH DOCKER WATCHDOG ─────────────────────────────────────────────────────────
|
||||
# At level 3: writes mem_shutdown_active=true to RW_STATE_FILE.
|
||||
# Coordination with docker_watchdog.sh
|
||||
# At level 3, writes mem_shutdown_active=true to RW_STATE_FILE.
|
||||
# docker_watchdog.sh reads this and defers all container restart logic.
|
||||
# Cleared when pressure fully resolves and containers are restarted.
|
||||
# This prevents docker_watchdog from restarting containers that RM just stopped to free RAM.
|
||||
# Without this, docker_watchdog would immediately restart containers that were
|
||||
# just stopped to free RAM — defeating the purpose of level 3.
|
||||
# Cleared when pressure resolves and containers are restarted.
|
||||
#
|
||||
# ── NOT RESPONSIBLE FOR ───────────────────────────────────────────────────────────────────────
|
||||
# Restarting broken containers — docker_watchdog.sh
|
||||
# Rebooting the system — system_watchdog.sh
|
||||
# Reacting to single data points — RW_RECOVER_CYCLES prevents flip-flopping
|
||||
# ==============================================================================================
|
||||
# OPERATIONAL SAFEGUARDS
|
||||
# ==============================================================================================
|
||||
#
|
||||
# ── CONFIGURATION (master.conf) ───────────────────────────────────────────────────────────────
|
||||
# Root Required
|
||||
# docker pause/stop require root.
|
||||
#
|
||||
# Single Instance Lock
|
||||
# acquire_lock prevents concurrent runs from racing on state file writes.
|
||||
#
|
||||
# RW_CRITICAL_CONTAINERS
|
||||
# Containers listed here are never paused or stopped regardless of pressure level.
|
||||
#
|
||||
# RW_ENABLED Flag
|
||||
# Set RW_ENABLED=false to disable the entire script without removing it from
|
||||
# the orchestrator schedule.
|
||||
#
|
||||
# ==============================================================================================
|
||||
# CONFIGURATION
|
||||
# ==============================================================================================
|
||||
#
|
||||
# master.conf
|
||||
# RW_ENABLED, RW_STATE_FILE
|
||||
# RW_RAM_SOFT_GB, RW_RAM_MEDIUM_GB, RW_RAM_HARD_GB, RW_RAM_RECOVER_GB
|
||||
# RW_LOAD_SOFT_MULTIPLIER, RW_LOAD_MEDIUM_MULTIPLIER
|
||||
# RW_RECOVER_CYCLES
|
||||
# RW_SABNZBD_ENABLED, RW_SABNZBD_SPEED_SOFT, RW_SABNZBD_SPEED_MEDIUM
|
||||
# RW_QBIT_ENABLED, RW_QBIT_DL_SOFT, RW_QBIT_DL_MEDIUM
|
||||
# RW_CRITICAL_CONTAINERS — never paused or stopped regardless of pressure
|
||||
# RW_CRITICAL_CONTAINERS — never paused or stopped regardless of pressure
|
||||
#
|
||||
# ── CONFIGURATION (master_host*.conf) ─────────────────────────────────────────────────────────
|
||||
# HOST*_RW_PAUSE_CONTAINERS — docker pause at medium pressure (aliased by detect_hosts)
|
||||
# HOST*_RW_STOP_CONTAINERS — docker stop at hard pressure (aliased by detect_hosts)
|
||||
# master_host*.conf (aliased by detect_hosts())
|
||||
# HOST*_RW_PAUSE_CONTAINERS — docker pause at medium pressure
|
||||
# HOST*_RW_STOP_CONTAINERS — docker stop at hard pressure
|
||||
# HOST*_SABNZBD_URL, HOST*_SABNZBD_API_KEY
|
||||
# HOST*_QBIT_URL, HOST*_QBIT_USERNAME, HOST*_QBIT_PASSWORD
|
||||
#
|
||||
# ── USAGE ─────────────────────────────────────────────────────────────────────────────────────
|
||||
# resource_watchdog.sh — normal run (via watchdog_orchestrator.sh)
|
||||
# resource_watchdog.sh --dry-run — show what would happen without acting
|
||||
# resource_watchdog.sh --status — current pressure level and active actions
|
||||
# resource_watchdog.sh --log — verbose per-check output
|
||||
# ==============================================================================================
|
||||
# STATE FILES
|
||||
# ==============================================================================================
|
||||
#
|
||||
# RW_STATE_FILE
|
||||
# Pressure level, recovery cycle count, stopped container list, and the
|
||||
# mem_shutdown_active coordination flag read by docker_watchdog.sh.
|
||||
#
|
||||
# ==============================================================================================
|
||||
# RUNTIME MODES
|
||||
# ==============================================================================================
|
||||
#
|
||||
# resource_watchdog.sh
|
||||
# Single-pass pressure check. Apply actions if threshold crossed. Silent if below.
|
||||
#
|
||||
# resource_watchdog.sh --dry-run
|
||||
# Show current pressure level and what would be throttled/paused/stopped. No changes.
|
||||
#
|
||||
# resource_watchdog.sh --status
|
||||
# Show current pressure level, active actions, recovery cycle count, stopped containers.
|
||||
#
|
||||
# resource_watchdog.sh --log
|
||||
# Verbose per-check output — show RAM, load, each threshold comparison, each action.
|
||||
#
|
||||
# ==============================================================================================
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
Reference in New Issue
Block a user