feat: slskd reconnect guard in downloaders_reset, mass v2 sync

- downloaders_reset: connection check block before slskd API sections;
  triggers PUT /api/v0/server reconnect if disconnected, polls 60s,
  gates Stuck Searches and Dead Transfer Records on SLSKD_CONNECTED
- Sync all modified/new/deleted files from v2 refactor across Docker_Essentials,
  Media, Monitors, Partnership, Rsync, Tools, Transcodes, unRAID_Essentials,
  common.sh, master confs, and new Manual/README docs
This commit is contained in:
Gmer4Lfe
2026-05-19 20:00:10 -04:00
parent 5cb16d4b18
commit e13f2fa14f
81 changed files with 12164 additions and 10656 deletions
+99 -40
View File
@@ -1,53 +1,112 @@
#!/bin/bash
# ==============================================================================================
# ================================= ZFS Memory Snapshot ========================================
# ============================= ZFS Memory Snapshot ============================================
# ==============================================================================================
# Weekly ZFS pool health and memory diagnostic report.
# Combines ZFS pool status, ARC statistics, memory summary, Docker memory usage
# and kernel pressure into a single report. Informational only — no action taken.
# system_watchdog.sh handles threshold-based intervention.
#
# ── WHAT IT REPORTS ───────────────────────────────────────────────────────────────────────────
# ZFS pool health — status, state, errors per pool (excluding ignored pools)
# ARC statistics — current size, max, utilization %, metadata pressure
# Memory status — total/free/available RAM vs thresholds
# Docker memory — top ZFS_REPORT_DOCKER_TOP containers by memory usage
# Kernel pressure — vmstat snapshot (3 samples)
# PURPOSE
# ─────────────────────────────────────────────────────────────────────────────
# Weekly ZFS pool health and memory diagnostic report. Scheduled Sunday 6am —
# first in the Sunday monitoring block, before other scripts run. Informational
# only — system_watchdog.sh handles threshold-based intervention.
#
# ── OUTPUT ────────────────────────────────────────────────────────────────────────────────────
# Output goes to both console and ZFS_REPORT_LOG for later review.
# In dry-run mode — console only, nothing written to log.
# Notifies if any warning thresholds are exceeded.
# Silent when all healthy — only problems produce output.
# Combines ZFS pool status, ARC statistics, Docker memory usage, and kernel
# memory pressure into a single snapshot. Output goes to both console (for User
# Scripts output log) and ZFS_REPORT_LOG for week-over-week comparison.
#
# ── HOST AWARENESS ────────────────────────────────────────────────────────────────────────────
# detect_hosts() sets MY_ID and aliases HOST*_ZFS_REPORT_IGNORE_POOLS → ZFS_REPORT_IGNORE_POOLS.
# Each server ignores its own single-disk ZFS array pools — not the peer's.
# ==============================================================================================
# OPERATIONAL MODEL
# ==============================================================================================
#
# ── SAFEGUARDS ────────────────────────────────────────────────────────────────────────────────
# acquire_lock — zpool + docker stats are slow, prevent duplicates
# detect_hosts() — correct pool ignore list per host
# validate_unraid_cmd — notify validated before use
# DOCKER_TIMEOUT — docker stats protected against hung daemon
# ZFS not available — skips pool and ARC sections gracefully
# Docker not available — skips container section gracefully
# Five report sections (each skips gracefully if its data source is unavailable):
#
# ── CONFIGURATION (master_host*.conf) ─────────────────────────────────────────────────────────
# HOST*_ZFS_REPORT_IGNORE_POOLS — pools excluded from health reporting
# Aliased by detect_hosts() — script uses ZFS_REPORT_IGNORE_POOLS
# ZFS pool health — status, state, errors per pool. Pools in
# ZFS_REPORT_IGNORE_POOLS excluded from the report
# (still fully monitored by unRAID — report-only exclusion).
# ARC statistics — current ARC vs max, metadata pressure, hit rate.
# Warns if ARC utilisation exceeds ZFS_REPORT_ARC_WARN_PCT.
# Memory status — total, free, available RAM.
# Warns if free < ZFS_REPORT_FREE_WARN_GB or
# available < ZFS_REPORT_AVAIL_WARN_GB.
# Docker memory — top ZFS_REPORT_DOCKER_TOP containers by memory usage.
# Useful for spotting containers approaching watchdog limits.
# Kernel pressure — vmstat snapshot (3 samples).
#
# ── CONFIGURATION (master.conf) ───────────────────────────────────────────────────────────────
# ZFS_REPORT_LOG — log file path for weekly reports
# ZFS_REPORT_ARC_WARN_PCT — warn if ARC using more than this % of max
# ZFS_REPORT_FREE_WARN_GB — warn if less than this GB free RAM
# ZFS_REPORT_AVAIL_WARN_GB — warn if less than this GB available RAM
# ZFS_REPORT_DOCKER_TOP — how many top Docker containers to show
# ==============================================================================================
# OPERATIONAL SAFEGUARDS
# ==============================================================================================
#
# Single Instance Lock
# acquire_lock prevents duplicate runs — zpool and docker stats are slow.
#
# Per-Host Pool Ignore List
# detect_hosts() aliases HOST*_ZFS_REPORT_IGNORE_POOLS → ZFS_REPORT_IGNORE_POOLS.
# Single-disk JBOD members excluded from report noise per server.
#
# ZFS Availability Guard
# Skips pool and ARC sections gracefully if ZFS is not available on this server.
#
# Docker Availability Guard
# Skips container memory section gracefully if Docker is not responding.
#
# Docker Stats Timeout
# DOCKER_TIMEOUT caps docker stats calls. A hung daemon does not block the report.
#
# Notification Validated
# validate_unraid_cmd confirms the notify script is present before use.
#
# ==============================================================================================
# STATE FILES
# ==============================================================================================
#
# ZFS_REPORT_LOG — /var/log/zfs-weekly-health.log (tmpfs, resets on reboot)
# Weekly report written here for comparison across runs. Open the log to
# see pool health trend week over week without remembering last week's values.
# In dry-run mode, console only — nothing written.
#
# ==============================================================================================
# CONFIGURATION
# ==============================================================================================
#
# master_host*.conf
#
# HOST*_ZFS_REPORT_IGNORE_POOLS
# Pools excluded from health reporting. Single-disk JBOD members generate
# expected high-usage warnings — exclude them to reduce report noise.
# Aliased by detect_hosts() → ZFS_REPORT_IGNORE_POOLS.
#
# master.conf
#
# ZFS_REPORT_LOG
# Log file path for weekly reports. (default: /var/log/zfs-weekly-health.log)
#
# ZFS_REPORT_ARC_WARN_PCT
# Warn if ARC is using more than this percentage of its configured max. (default: 90)
#
# ZFS_REPORT_FREE_WARN_GB
# Warn if free RAM is below this threshold in GB. (default: 10)
#
# ZFS_REPORT_AVAIL_WARN_GB
# Warn if available RAM is below this threshold in GB. (default: 20)
#
# ZFS_REPORT_DOCKER_TOP
# Number of top Docker containers by memory usage to include. (default: 10)
#
# ==============================================================================================
# RUNTIME MODES
# ==============================================================================================
#
# zfs_memory_snapshot.sh
# Generate report, write to ZFS_REPORT_LOG and console. Notify on warnings.
#
# zfs_memory_snapshot.sh --dry-run
# Generate report to console only. No log write, no notifications.
#
# zfs_memory_snapshot.sh --status
# Show pool ignore list and threshold configuration. Then exit.
#
# zfs_memory_snapshot.sh --log
# Verbose output during report generation.
#
# ── USAGE ─────────────────────────────────────────────────────────────────────────────────────
# zfs_memory_snapshot.sh — normal report (writes to log)
# zfs_memory_snapshot.sh --dry-run — console only, no log write
# zfs_memory_snapshot.sh --log — verbose output
# zfs_memory_snapshot.sh --status — show config and exit
# ==============================================================================================
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"