feat: slskd reconnect guard in downloaders_reset, mass v2 sync
- downloaders_reset: connection check block before slskd API sections; triggers PUT /api/v0/server reconnect if disconnected, polls 60s, gates Stuck Searches and Dead Transfer Records on SLSKD_CONNECTED - Sync all modified/new/deleted files from v2 refactor across Docker_Essentials, Media, Monitors, Partnership, Rsync, Tools, Transcodes, unRAID_Essentials, common.sh, master confs, and new Manual/README docs
This commit is contained in:
Gmer4Lfe
+118
-49
@@ -1,64 +1,133 @@
|
||||
#!/bin/bash
|
||||
# ==============================================================================================
|
||||
# ================================= Rsync Core Script ==========================================
|
||||
# ============================= Rsync Core Script ==============================================
|
||||
# ==============================================================================================
|
||||
# Core rsync script — called per share or per appdata profile.
|
||||
# Called by orchestrators (daily/weekly/critical sync) and directly for manual syncs.
|
||||
#
|
||||
# ── PROFILE SYSTEM ────────────────────────────────────────────────────────────────────────────
|
||||
# Profile is inferred from the directory basename (lowercased).
|
||||
# Override with --profile=name for explicit profile selection.
|
||||
# If no profile match found → all settings fall back to global defaults in master.conf.
|
||||
# PURPOSE
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Core rsync engine for the two-server ecosystem. Called per share or per
|
||||
# appdata profile by orchestrators (daily_sync_maintenance, weekly_sync_maintenance,
|
||||
# critical_sync_maintenance) and directly for manual or scheduled dirty syncs.
|
||||
#
|
||||
# Profiles define:
|
||||
# PROFILE_RSYNC_OPTS — rsync flags (does NOT inherit DEFAULT_RSYNC_OPTS)
|
||||
# PROFILE_BW_LIMIT — bandwidth limit in KB/s
|
||||
# PROFILE_RETRY_COUNT — retry attempts before giving up
|
||||
# PROFILE_SLEEP — seconds between retry attempts
|
||||
# PROFILE_CRITICAL_CONTAINER_NAMES — containers stopped both sides before sync
|
||||
# PROFILE_DELAYED_CONTAINERS — containers needing delay before starting after sync
|
||||
# PROFILE_CONTAINER_DELAY — seconds before starting delayed containers
|
||||
# PROFILE_EXCLUDE_DIRS — paths excluded from transfer
|
||||
# Profile is inferred from the directory basename (lowercased). Override with
|
||||
# --profile=name for explicit selection. If no profile matches, global defaults
|
||||
# from master.conf apply and no containers are stopped.
|
||||
#
|
||||
# After each sync, logs transfer data to bandwidth_monitor.sh for the weekly
|
||||
# bandwidth report. Silent on success — only failures produce visible output.
|
||||
#
|
||||
# ==============================================================================================
|
||||
# OPERATIONAL MODEL
|
||||
# ==============================================================================================
|
||||
#
|
||||
# Profiles define per-share behavior:
|
||||
# PROFILE_CRITICAL_CONTAINER_NAMES — containers stopped on both servers before sync
|
||||
# PROFILE_DELAYED_CONTAINERS — containers with a delay before restart after sync
|
||||
# PROFILE_CONTAINER_DELAY — seconds before delayed containers start
|
||||
# PROFILE_RSYNC_OPTS — rsync flags (does not inherit DEFAULT_RSYNC_OPTS)
|
||||
# PROFILE_BW_LIMIT — bandwidth limit in KB/s
|
||||
# PROFILE_RETRY_COUNT — retry attempts on failure
|
||||
# PROFILE_SLEEP — seconds between retry attempts
|
||||
# PROFILE_EXCLUDE_DIRS — paths excluded from transfer
|
||||
# PROFILE_REMOTE_RESTART_CONTAINERS — containers restarted on remote after dirty sync
|
||||
# (critical-fallback, emby-fallback profiles)
|
||||
# Was running → restart. Was stopped → leave stopped.
|
||||
#
|
||||
# ── RSYNC ENABLE/DISABLE ──────────────────────────────────────────────────────────────────────
|
||||
# Two-tier toggle system — checked at entry:
|
||||
# Tier 1: RSYNC_ENABLED=false → all rsync stops
|
||||
# Tier 2: Per-orchestrator flag (DAILY_RSYNC_ENABLED etc.) — checked by caller
|
||||
# Direct calls to rsync.sh only check Tier 1
|
||||
# Two-tier rsync enable/disable:
|
||||
# Tier 1: RSYNC_ENABLED=false → all rsync stops immediately (checked by this script)
|
||||
# Tier 2: per-orchestrator flag (DAILY_RSYNC_ENABLED etc.) → checked by caller
|
||||
#
|
||||
# ── BANDWIDTH LOGGING ─────────────────────────────────────────────────────────────────────────
|
||||
# After each sync logs to bandwidth_monitor.sh --log-transfer:
|
||||
# profile | duration_seconds | status | bytes_transferred
|
||||
# Bytes captured from rsync --stats output — version-proof parsing.
|
||||
# bandwidth_monitor.sh flags syncs exceeding BANDWIDTH_WARN_GB.
|
||||
# Bandwidth logging: after each sync, logs profile/duration/status/bytes to
|
||||
# bandwidth_monitor.sh --log-transfer. Bytes captured from rsync --stats via awk
|
||||
# using version-stable field names.
|
||||
#
|
||||
# ── DIRTY SYNC REMOTE RESTART ─────────────────────────────────────────────────────────────────
|
||||
# Profiles using dirty sync (critical-fallback, emby-fallback) define
|
||||
# PROFILE_REMOTE_RESTART_CONTAINERS — containers restarted on remote after sync completes.
|
||||
# This ensures the remote picks up config changes synced during the dirty window.
|
||||
# Was running → restart. Was stopped → leave stopped.
|
||||
# ==============================================================================================
|
||||
# OPERATIONAL SAFEGUARDS
|
||||
# ==============================================================================================
|
||||
#
|
||||
# ── SAFEGUARDS ────────────────────────────────────────────────────────────────────────────────
|
||||
# check_rsync_enabled() — Tier 1 gate before any operation
|
||||
# check_unraid_version_parity — refuses if servers on incompatible unRAID versions
|
||||
# check_remote_docker_daemon — verifies remote daemon before container operations
|
||||
# check_local_disk_temps() — temp check before transfer (exit 1=skip, 2=abort all)
|
||||
# check_connectivity() — verifies remote reachable
|
||||
# check_remote_rootfs() — aborts if remote rootfs nearly full
|
||||
# check_remote_share() — aborts if target directory missing or empty
|
||||
# check_remote_disks() — verifies all backing disks online on remote
|
||||
# acquire_rsync_lock() — per-profile lock + global concurrent limit
|
||||
# validate_unraid_cmd — notify validated before use
|
||||
# Silent by default — only failures produce visible output
|
||||
# Global Rsync Gate
|
||||
# check_rsync_enabled() — RSYNC_ENABLED=false exits cleanly before any operation.
|
||||
#
|
||||
# Partnership Blocklist
|
||||
# Refuses to sync if REMOTE_SERVER_NAME appears in the partnership blocklist.
|
||||
# Written at offboard — prevents stale access after a partnership ends.
|
||||
#
|
||||
# Version Parity
|
||||
# check_unraid_version_parity — refuses sync if servers on incompatible unRAID versions.
|
||||
#
|
||||
# Remote Health Pre-flights
|
||||
# check_connectivity() — Tailscale IP reachable before any SSH
|
||||
# check_remote_rootfs() — aborts if remote rootfs exceeds ROOTFS_WARN_PCT
|
||||
# check_remote_share() — aborts if target directory missing or empty on remote
|
||||
# check_remote_disks() — verifies all backing disks online on remote
|
||||
#
|
||||
# Drive Temperature Check
|
||||
# check_local_disk_temps() — runs before any transfer. Exit 1 = skip this profile,
|
||||
# exit 2 = abort all remaining syncs (CRITICAL temperature).
|
||||
#
|
||||
# Remote Docker Daemon Check
|
||||
# check_remote_docker_daemon — verified before any container stop/start operations.
|
||||
# If daemon unresponsive: container operations skipped, rsync proceeds without stopping.
|
||||
#
|
||||
# Per-Profile Concurrency Lock
|
||||
# acquire_rsync_lock() — per-profile lock prevents parallel runs of the same profile.
|
||||
# Global concurrent limit prevents too many simultaneous rsync processes.
|
||||
#
|
||||
# Notification Validated
|
||||
# validate_unraid_cmd confirms the notify script is present before use.
|
||||
#
|
||||
# ==============================================================================================
|
||||
# CONFIGURATION
|
||||
# ==============================================================================================
|
||||
#
|
||||
# master.conf
|
||||
#
|
||||
# RSYNC_ENABLED
|
||||
# Global on/off toggle for all rsync operations. (default: true)
|
||||
#
|
||||
# DEFAULT_RSYNC_OPTS
|
||||
# Base rsync flags for unproiled shares. Note: --delete is intentionally absent —
|
||||
# media shares spread files only, arr cleanup scripts own deletions. Profile-specific
|
||||
# opts set --delete explicitly where needed.
|
||||
#
|
||||
# BW_LIMIT
|
||||
# Default bandwidth cap in KB/s when no PROFILE_BW_LIMIT is set. (default: 0 = unlimited)
|
||||
#
|
||||
# RETRY_COUNT
|
||||
# Default retry attempts on rsync failure. (default: 3)
|
||||
#
|
||||
# SLEEP
|
||||
# Default seconds between retry attempts. (default: 60)
|
||||
#
|
||||
# ROOTFS_WARN_PCT
|
||||
# Abort threshold for remote rootfs percentage full. (default: 75)
|
||||
#
|
||||
# PROFILES["profile_KEY"]
|
||||
# Profile definitions — one entry per PROFILE_* key per profile name.
|
||||
# See OPERATIONAL MODEL above for all supported keys.
|
||||
#
|
||||
# BANDWIDTH_LOG / BANDWIDTH_WARN_GB
|
||||
# Shared with bandwidth_monitor.sh — set once, used by both.
|
||||
#
|
||||
# ==============================================================================================
|
||||
# RUNTIME MODES
|
||||
# ==============================================================================================
|
||||
#
|
||||
# rsync.sh /path/to/share
|
||||
# Sync the given path to the remote. Profile inferred from directory basename.
|
||||
#
|
||||
# rsync.sh /path/to/share --profile=name
|
||||
# Sync with explicit profile override — bypasses basename inference.
|
||||
#
|
||||
# rsync.sh /path/to/share --dry-run
|
||||
# Run all pre-flight checks and show what rsync would transfer. No transfer,
|
||||
# no container stops.
|
||||
#
|
||||
# rsync.sh /path/to/share --status
|
||||
# Show resolved profile, remote identity, and configuration. Then exit.
|
||||
#
|
||||
# rsync.sh /path/to/share --log
|
||||
# Verbose output throughout — every decision logged.
|
||||
#
|
||||
# ── USAGE ─────────────────────────────────────────────────────────────────────────────────────
|
||||
# rsync.sh /mnt/user/Movies — media share, global defaults
|
||||
# rsync.sh /mnt/user/appdata-Fallback/Arrs_Stack — matched to [arrs_stack] profile
|
||||
# rsync.sh /mnt/user/appdata-Fallback/Arrs_Stack --dry-run --log
|
||||
# rsync.sh /mnt/user/appdata-Fallback/Critical-Data --profile=critical-fallback
|
||||
# ==============================================================================================
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
Reference in New Issue
Block a user