Structural reorganization: watchdog taxonomy + server_reboot integration

Watchdog renames and moves:
  system_watchdog.sh → stability_watchdog.sh (last line of defense — reboots)
  storage_watchdog.sh → Watchdogs/System/storage_watchdog.sh
  webgui_restart.sh → Watchdogs/System/webgui_watchdog.sh (renamed to match folder convention)

New thin orchestrator:
  Watchdogs/system_watchdog.sh — runs SYSTEM_WATCHDOG_SCRIPTS from master.conf
  Sits between docker_watchdog and stability_watchdog in the orchestrator tier chain
  System/ subfolder is the growth seam for future system component watchdogs

master.conf:
  WATCHDOG_ORCHESTRATOR_SCRIPTS updated — storage removed, system_watchdog added as tier
  SYSTEM_WATCHDOG_SCRIPTS array added — storage + webgui

server_reboot.sh:
  Calls array_stopping.sh before VM shutdown for guaranteed safe array stop
  Removed raw rc.docker stop and exit trap — orchestrator owns container shutdown
This commit is contained in:
Gmer4Lfe
2026-05-22 20:17:22 -04:00
parent 670f25fbbd
commit cf21efaeea
18 changed files with 962 additions and 824 deletions
+8 -8
View File
@@ -25,7 +25,7 @@
# weekly_sync_maintenance.sh 2:30am Sunday — clean sync + image updates
# sunday_morning_coffee_report.sh 7am Sunday — full weekly digest
# weekly_health_digest.sh 8am daily — profile-controlled health notification
# webgui_restart.sh every 10 min — WebGUI availability watchdog
# webgui_watchdog.sh every 10 min — WebGUI availability watchdog
# system_tuning_monitor.sh every 6 hours — inotify + php-fpm trend tracking
# clear_logs.sh 5am Sunday — size-threshold log cleanup
# [Sunday monitor block] 6am-11am Sun — certs, SMART, backups, reports
@@ -401,7 +401,7 @@
# 4. Restart emhttp (core daemon) wait WEBGUI_EMHTTP_WAIT=30s, recheck
# 5. All failed → notify warning, manual intervention needed
#
# bash /mnt/user/appdata/unraid_scripts/unRAID_Essentials/webgui_restart.sh
# bash /mnt/user/appdata/unraid_scripts/Watchdogs/System/webgui_watchdog.sh
# ── SYSTEM TUNING MONITOR ────────────────────────────────────────────────────────────────────
@@ -733,8 +733,8 @@
# Reboot loop protection: N reboots in X hours → shutdown instead.
# State file heartbeat: writes watchdog_cycle=N every cycle (docker_watchdog stale guard).
#
# bash /mnt/user/appdata/unraid_scripts/Watchdogs/system_watchdog.sh --status
# bash /mnt/user/appdata/unraid_scripts/Watchdogs/system_watchdog.sh --dry-run
# bash /mnt/user/appdata/unraid_scripts/Watchdogs/stability_watchdog.sh --status
# bash /mnt/user/appdata/unraid_scripts/Watchdogs/stability_watchdog.sh --dry-run
# resource_watchdog.sh — three-level pressure reduction layer (single-pass, called by watchdog_orchestrator)
# Reduces system load intelligently BEFORE docker_watchdog attempts container restarts.
@@ -787,9 +787,9 @@
# bash /mnt/user/appdata/unraid_scripts/unRAID_Essentials/docker_syslog_filter.sh --status
# bash /mnt/user/appdata/unraid_scripts/unRAID_Essentials/docker_syslog_filter.sh
# webgui_restart.sh — WebGUI availability watchdog (see ORCHESTRATORS section above)
# bash /mnt/user/appdata/unraid_scripts/unRAID_Essentials/webgui_restart.sh --dry-run
# bash /mnt/user/appdata/unraid_scripts/unRAID_Essentials/webgui_restart.sh --status
# webgui_watchdog.sh — WebGUI availability watchdog (see ORCHESTRATORS section above)
# bash /mnt/user/appdata/unraid_scripts/Watchdogs/System/webgui_watchdog.sh --dry-run
# bash /mnt/user/appdata/unraid_scripts/Watchdogs/System/webgui_watchdog.sh --status
# clear_logs.sh — size-threshold log cleanup (see ORCHESTRATORS section above)
# bash /mnt/user/appdata/unraid_scripts/unRAID_Essentials/clear_logs.sh --dry-run
@@ -1301,7 +1301,7 @@
# Orchestrators/transcode_management.sh
#
# */10 * * * * every 10 minutes:
# unRAID_Essentials/webgui_restart.sh
# Watchdogs/System/webgui_watchdog.sh
#
# */15 * * * * every 15 minutes:
# Orchestrators/critical_sync_maintenance.sh