FailedProxy/Varaverk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

audit and clean master.conf, common.sh, and emby sync tools

Browse Source 
master.conf:
- reorder ORCHESTRATORS section by run frequency; add missing entries
- remove heartbeat vars (watchdogs no longer run in continuous cycle)
- remove dead REMOTE_DOCKER_RETRY_WAIT and display-only watchdog interval vars
- move INTERMEDIATE_RSYNC_ENABLED into RSYNC two-tier block; reorder by frequency

common.sh:
- add missing info() and success() definitions — both were called throughout but never defined
- update output tier description to list all 5 functions
- fix double-icon in check_local_disk_temps() error/warn calls
- fix smashed curl/ssh command lines in notify_emby_scan() and check_remote_docker_daemon()
- update check_rsync_enabled() comment to include INTERMEDIATE, ordered by frequency

emby_to_radarr_sync.sh / emby_to_sonarr_sync.sh:
- wire up RADARR_EMBY_LIBRARIES / SONARR_EMBY_LIBRARIES; empty array now scans all libraries
This commit is contained in:
Gmer4Lfe
2026-05-23 09:09:25 -04:00
parent ee086f309e
commit 14a74939e6
4 changed files with 144 additions and 111 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Configurations/master.conf
+48 -62
View File
@@ -37,10 +37,14 @@
#
# ── ORCHESTRATORS ──────────────────────────────────────────────────────────────────────────
# ARRAY START Scripts launched at array start (array_started.sh)
# MONTHLY MAINTENANCE Uptime-triggered heavy tasks — ZFS scrub, SMART tests (monthly_maintenance.sh)
# ARRAY STOP Scripts run at planned shutdown (array_stopping.sh)
# WATCHDOG ORCHESTRATOR Per-minute watchdog runner (watchdog_orchestrator.sh)
# SYSTEM WATCHDOG Sub-scripts called by watchdog_orchestrator.sh
# CRITICAL SYNC MAINTENANCE 30-minute jobs + sync shares + partnership check (critical_sync_maintenance.sh)
# INTERMEDIATE SYNC MAINTENANCE Arr sync + optional mid-day rsync (intermediate_sync_maintenance.sh)
# DAILY SYNC MAINTENANCE Job list + media shares (daily_sync_maintenance.sh)
# WEEKLY SYNC MAINTENANCE Job list + sync shares + update toggles (weekly_sync_maintenance.sh)
# CRITICAL SYNC MAINTENANCE 30-minute jobs + sync shares + partnership check (critical_sync_maintenance.sh)
# MONTHLY MAINTENANCE Uptime-triggered heavy tasks — ZFS scrub, SMART tests (monthly_maintenance.sh)
#
# ── RSYNC ──────────────────────────────────────────────────────────────────────────────────
# RSYNC ENABLE/DISABLE Two-tier toggle system — global gate + per-orchestrator
@@ -138,7 +142,6 @@
# Scripts that issue remote container commands check the remote daemon first.
# Below limit → skip operation this run | At limit → notify critical + exit
REMOTE_DOCKER_STRIKE_LIMIT=3 # consecutive failures before critical notify
REMOTE_DOCKER_RETRY_WAIT=30 # seconds to wait before retrying after failure
# ==============================================================================================
# ── PARTNERSHIP ───────────────────────────────────────────────────────────────────────────────
@@ -259,18 +262,6 @@
# No changes to orchestrator scripts needed when adding or removing jobs.
# Sections ordered by run frequency: array events first, then shortest interval to longest.
# ━━━ Array Stop ━━━
# Scripts run by array_stopping.sh for a planned shutdown — stops everything cleanly in order.
# Run sequentially (foreground) — each must complete before the next starts.
# Order matters: user scripts first (prevents new ops), then data movement, then containers.
ARRAY_STOP_SCRIPTS=(
"unRAID_Essentials/user_scripts_stop.sh" # stop background scripts before they start new ops
"Fallback/fallback.sh --stop" # gracefully stop fallback (not caught by user_scripts_stop)
"unRAID_Essentials/rsync_stop.sh --rsync-only" # kill rsync; skip container recovery (handled below)
"unRAID_Essentials/mover_stop.sh" # stop mover after rsync (they conflict on same files)
"Docker_Essentials/docker_container_stop.sh" # stop all containers last
)
# ━━━ Array Start ━━━
# Scripts launched by array_started.sh when the array comes online.
# Launched in order — each as a background process.
@@ -287,6 +278,18 @@
"Fallback/fallback.sh" # mutual failover — continuous
)
# ━━━ Array Stop ━━━
# Scripts run by array_stopping.sh for a planned shutdown — stops everything cleanly in order.
# Run sequentially (foreground) — each must complete before the next starts.
# Order matters: user scripts first (prevents new ops), then data movement, then containers.
ARRAY_STOP_SCRIPTS=(
"unRAID_Essentials/user_scripts_stop.sh" # stop background scripts before they start new ops
"Fallback/fallback.sh --stop" # gracefully stop fallback (not caught by user_scripts_stop)
"unRAID_Essentials/rsync_stop.sh --rsync-only" # kill rsync; skip container recovery (handled below)
"unRAID_Essentials/mover_stop.sh" # stop mover after rsync (they conflict on same files)
"Docker_Essentials/docker_container_stop.sh" # stop all containers last
)
# ━━━ Watchdog Orchestrator ━━━
# watchdog_orchestrator.sh runs WATCHDOG_ORCHESTRATOR_SCRIPTS in order each cron cycle.
# Schedule: * * * * * (every minute)
@@ -300,9 +303,6 @@
"Watchdogs/stability_watchdog.sh" # reboot if all else fails — last line of defense
)
WATCHDOG_ORCHESTRATOR_HEARTBEAT=true
WATCHDOG_ORCHESTRATOR_HEARTBEAT_HOURS=1
# ━━━ System Watchdog ━━━
# system_watchdog.sh runs SYSTEM_WATCHDOG_SCRIPTS sequentially each cycle.
# Called by watchdog_orchestrator.sh — not scheduled directly.
@@ -312,23 +312,6 @@
"Watchdogs/System/network_watchdog.sh" # internet, DDNS, Tailscale, NPM proxy
)
# ━━━ Monthly Maintenance ━━━
# monthly_maintenance.sh fires only when BOTH gates pass:
# 1. Server uptime >= MONTHLY_UPTIME_THRESHOLD_DAYS days
# 2. Last run was >= MONTHLY_RUN_INTERVAL_DAYS days ago (or never run)
# Cron: 0 3 * * * (daily 3am check — script self-gates, calling daily is safe)
# NOT in WATCHDOG_ORCHESTRATOR_SCRIPTS — has its own cron entry.
# Add scripts that require a long-stable settled system — scrubs, extended drive tests.
# State file on /boot/config — survives reboots (interval gate independent of uptime gate).
MONTHLY_MAINTENANCE_SCRIPTS=(
#"Tools/zfs_pool_scrub.sh" # ZFS pool integrity scrub — not yet built
#"Tools/smart_long_test.sh" # SMART extended drive health test — not yet built
)
MONTHLY_UPTIME_THRESHOLD_DAYS=30 # minimum uptime in days before maintenance fires
MONTHLY_RUN_INTERVAL_DAYS=30 # minimum days since last run before running again
MONTHLY_LAST_RUN_FILE="/boot/config/monthly_maintenance_last_run.db"
# ━━━ Critical Sync Maintenance ━━━
# critical_sync_maintenance.sh runs every 30 minutes.
# Order: CRITICAL_MAINTENANCE_SCRIPTS (jobs) → CRITICAL_SYNC_SHARES (rsync) → partnership --check
@@ -348,7 +331,6 @@
# and optional mid-day rsync for any shares that need sub-daily propagation.
# Schedule: 0 */4 * * *
# INTERMEDIATE_SYNC_SHARES is host-specific — configure HOST*_INTERMEDIATE_SYNC_SHARES in host*.conf.
INTERMEDIATE_RSYNC_ENABLED=true # set false to disable mid-day rsync without removing shares
INTERMEDIATE_MAINTENANCE_SCRIPTS=(
"Media/arrs_failed_stalled_recovery.sh" # blocklist + re-search failed/stalled arr queue items
@@ -423,6 +405,23 @@
WEEKLY_SYNC_UPDATES=true # pull container updates locally during weekly window
WEEKLY_SYNC_UPDATES_REMOTE=true # pull container updates on remote via SSH
# ━━━ Monthly Maintenance ━━━
# monthly_maintenance.sh fires only when BOTH gates pass:
# 1. Server uptime >= MONTHLY_UPTIME_THRESHOLD_DAYS days
# 2. Last run was >= MONTHLY_RUN_INTERVAL_DAYS days ago (or never run)
# Cron: 0 3 * * * (daily 3am check — script self-gates, calling daily is safe)
# NOT in WATCHDOG_ORCHESTRATOR_SCRIPTS — has its own cron entry.
# Add scripts that require a long-stable settled system — scrubs, extended drive tests.
# State file on /boot/config — survives reboots (interval gate independent of uptime gate).
MONTHLY_MAINTENANCE_SCRIPTS=(
#"Tools/zfs_pool_scrub.sh" # ZFS pool integrity scrub — not yet built
#"Tools/smart_long_test.sh" # SMART extended drive health test — not yet built
)
MONTHLY_UPTIME_THRESHOLD_DAYS=30 # minimum uptime in days before maintenance fires
MONTHLY_RUN_INTERVAL_DAYS=30 # minimum days since last run before running again
MONTHLY_LAST_RUN_FILE="/boot/config/monthly_maintenance_last_run.db"
# ==============================================================================================
# ── RSYNC ─────────────────────────────────────────────────────────────────────────────────────
# ==============================================================================================
@@ -439,18 +438,20 @@
# Use when: rebuilding secondary, testing, per-window bandwidth management
#
# Example — HOST2 data rebuild:
# RSYNC_ENABLED=true ← rsync works, individual scripts run fine
# DAILY_RSYNC_ENABLED=false ← skip daily HDD syncs during rebuild
# WEEKLY_RSYNC_ENABLED=true ← Emby + Critical-Data still sync (NVMe)
# CRITICAL_RSYNC_ENABLED=true ← 30min auth stack sync still runs
# FALLBACK_RSYNC_ENABLED=true ← handback writeback still works
# RSYNC_ENABLED=true ← rsync works, individual scripts run fine
# CRITICAL_RSYNC_ENABLED=true ← 30min auth stack sync still runs
# INTERMEDIATE_RSYNC_ENABLED=false ← skip 4h arr/mid-day rsync during rebuild
# DAILY_RSYNC_ENABLED=false ← skip daily HDD syncs during rebuild
# WEEKLY_RSYNC_ENABLED=true ← Emby + Critical-Data still sync (NVMe)
# FALLBACK_RSYNC_ENABLED=true ← handback writeback still works
# → Run individual: bash Rsync/rsync.sh /mnt/user/Movies
# → When ready: DAILY_RSYNC_ENABLED=true
RSYNC_ENABLED=true # Tier 1 — global gate, overrides everything below
DAILY_RSYNC_ENABLED=false # Tier 2 — HOST2 rebuild in progress, re-enable when ready
WEEKLY_RSYNC_ENABLED=true # Tier 2 — weekly_sync_maintenance.sh rsync section
CRITICAL_RSYNC_ENABLED=false # Tier 2 — disabled during HOST2 rebuild, re-enable when ready
FALLBACK_RSYNC_ENABLED=true # Tier 2 — fallback.sh writeback jobs on handback
# → When ready: INTERMEDIATE_RSYNC_ENABLED=true DAILY_RSYNC_ENABLED=true
RSYNC_ENABLED=true # Tier 1 — global gate, overrides everything below
CRITICAL_RSYNC_ENABLED=false # Tier 2 — disabled during HOST2 rebuild, re-enable when ready
INTERMEDIATE_RSYNC_ENABLED=true # Tier 2 — intermediate_sync_maintenance.sh rsync section
DAILY_RSYNC_ENABLED=false # Tier 2 — HOST2 rebuild in progress, re-enable when ready
WEEKLY_RSYNC_ENABLED=true # Tier 2 — weekly_sync_maintenance.sh rsync section
FALLBACK_RSYNC_ENABLED=true # Tier 2 — fallback.sh writeback jobs on handback
# ━━━ Rsync Defaults ━━━
# Global fallback values used when no profile match is found.
@@ -696,13 +697,6 @@
RESP_FAIL_LIMIT=2 # consecutive failed checks before restart
CURL_TIMEOUT=5 # seconds per check before timeout
# Watchdog cycle interval
DOCKER_WATCHDOG_INTERVAL=900 # seconds between watchdog cycles (15 minutes)
# Heartbeat — proof of life logged periodically even when everything is healthy
DOCKER_WATCHDOG_HEARTBEAT=true
DOCKER_WATCHDOG_HEARTBEAT_HOURS=1
# Tier 2 master toggle
WATCHDOG_SCAN_ALL=true # false = only WATCHDOG_CONTAINERS + required containers
@@ -1332,19 +1326,11 @@
# Single spikes (one bad reading) are ignored — sustained problems trigger reboot.
SYS_WATCHDOG_STRIKE_LIMIT=2 # consecutive failures before reboot trigger
# How often checks run — 300s = 5 minutes.
# At STRIKE_LIMIT=2 and INTERVAL=300: problem must persist 10min before reboot.
SYSTEM_WATCHDOG_INTERVAL=300
# Reboot loop protection — if system keeps rebooting something is seriously wrong.
# After REBOOT_LIMIT reboots in REBOOT_WINDOW_HRS → shutdown instead of reboot.
SYS_WATCHDOG_REBOOT_LIMIT=3
SYS_WATCHDOG_REBOOT_WINDOW_HRS=12
# Heartbeat — proof of life logged periodically even when everything is healthy.
SYSTEM_WATCHDOG_HEARTBEAT=true
SYSTEM_WATCHDOG_HEARTBEAT_HOURS=1
# ━━━ RAM Reboot Threshold ━━━
# Reboot trigger only — warn/shutdown/recover handled by resource_watchdog.sh
# RW_RAM_HARD_GB > SYS_WATCHDOG_MEM_GB always (RM acts before watchdog reboots)