Commit Graph
28 Commits
Author SHA1 Message Date
Gmer4Lfe cf21efaeea Structural reorganization: watchdog taxonomy + server_reboot integration
Watchdog renames and moves:
  system_watchdog.sh → stability_watchdog.sh (last line of defense — reboots)
  storage_watchdog.sh → Watchdogs/System/storage_watchdog.sh
  webgui_restart.sh → Watchdogs/System/webgui_watchdog.sh (renamed to match folder convention)

New thin orchestrator:
  Watchdogs/system_watchdog.sh — runs SYSTEM_WATCHDOG_SCRIPTS from master.conf
  Sits between docker_watchdog and stability_watchdog in the orchestrator tier chain
  System/ subfolder is the growth seam for future system component watchdogs

master.conf:
  WATCHDOG_ORCHESTRATOR_SCRIPTS updated — storage removed, system_watchdog added as tier
  SYSTEM_WATCHDOG_SCRIPTS array added — storage + webgui

server_reboot.sh:
  Calls array_stopping.sh before VM shutdown for guaranteed safe array stop
  Removed raw rc.docker stop and exit trap — orchestrator owns container shutdown
2026-05-22 20:17:22 -04:00
Gmer4Lfe ec79a926e8 Revert orchestrator + skip_list_manager to original folders
watchdog_orchestrator.sh stays in Orchestrators/ — it's an orchestrator, not a watchdog.
watchdog_skip_list_manager.sh stays in Tools/ — it's a management utility.

Only the 4 watchdog scripts belong in Watchdogs/:
  docker_watchdog.sh, resource_watchdog.sh, storage_watchdog.sh, system_watchdog.sh
2026-05-22 17:10:53 -04:00
Gmer4Lfe 95151c2278 Watchdogs/ folder + host conf rename
Move all watchdog scripts to a dedicated Watchdogs/ folder:
  Docker_Essentials/docker_watchdog.sh   → Watchdogs/
  unRAID_Essentials/system_watchdog.sh   → Watchdogs/
  unRAID_Essentials/resource_watchdog.sh → Watchdogs/
  Orchestrators/watchdog_orchestrator.sh → Watchdogs/
  Tools/watchdog_skip_list_manager.sh    → Watchdogs/

Rename host config files:
  master_host1.conf → host1.conf
  master_host2.conf → host2.conf

Update all references across the ecosystem:
  master.conf: WATCHDOG_ORCHESTRATOR_SCRIPTS paths → Watchdogs/
  load_config.sh: host*.conf glob + all comments
  git_pull_execute.sh: sparse checkout glob + all comments
  Partnership/ssh_setup.sh: HOST_CONF path construction
  user_script_plug-in.sh: all script paths + per-host conf path
  common.sh, README.md, README-User_Script_Plug-in.md: comment refs
  All Partnership, Fallback, Monitors, Transcodes, Tools scripts: comment refs
2026-05-22 17:08:36 -04:00
Gmer4Lfe aa6d165725 feat: add fallback.sh --stop for clean array shutdown
Fallback runs as a bare subprocess (cmdline: bash .../Fallback/fallback.sh),
not via /tmp/user.scripts, so user_scripts_stop.sh cannot find it.
Adding --stop mode closes that gap in the array shutdown sequence.

fallback.sh --stop: reads lock file (/tmp/unraid_locks/fallback.lock) to
get the running PID, sends SIGTERM (hits the existing trap that sets
FALLBACK_RUNNING=false), waits up to 10s, SIGKILLs if still alive.

master.conf: insert Fallback/fallback.sh --stop as step 2 in
ARRAY_STOP_SCRIPTS (after user_scripts_stop, before rsync_stop).

user_script_plug-in.sh: update ARRAY STOPPING block to show 5-step order
and add v2.5 changelog entry.
2026-05-18 22:04:07 -04:00
Gmer4LfeandClaude Sonnet 4.6 75510cfd54 docs: remove arrs_failed_stalled_recovery from orchestrators header list
It's a Media/ individual script, not an orchestrator — belongs in the
individual scripts section only.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 21:44:47 -04:00
Gmer4LfeandClaude Sonnet 4.6 6fdc27b00b docs: update user_script_plug-in.sh to v2.4
Add array_stopping.sh (At Stopping of Array) and watchdog_orchestrator.sh
(every minute) as new scheduled orchestrators. Add docker_container_stop.sh
and resource_watchdog.sh individual script entries. Update system_watchdog
and docker_watchdog descriptions to reflect single-pass architecture. Fix
paths: git_pull_execute.sh (root-level), continuous_scripts_status.sh (Tools/).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 21:41:59 -04:00
Gmer4LfeandClaude Sonnet 4.6 f16c962ac0 feat: rename array orchestrators, add docker_update remainder mode
- Rename array_start.sh → array_started.sh, array_stop.sh → array_stopping.sh
  to clarify these are event-driven (array has started/is stopping), not imperative
- Update all references across 9 files (master.conf, user_script_plug-in.sh,
  watchdogs, continuous_scripts_status.sh, claude_startup.sh)
- Add --remainder mode to docker_update.sh: updates all running containers
  excluding daily containers, weekly sync-window containers (emby + critical-data),
  and fallback coverage containers (FALLBACK_${MY_ID}_COVERS_${REMOTE_ID}_TIER*)
  Fallback containers excluded because the remote server owns their version —
  independent updates risk writeback incompatibility on handback
- weekly_sync_maintenance.sh calls docker_update.sh --remainder as final step
- git_pull_execute.sh: add safe.directory config to fix dubious ownership error
  when running as root on a directory owned by uid 1000

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 22:12:08 -04:00
Gmer4LfeandClaude Sonnet 4.6 a03280d6d3 docs(user_script_plug-in): v2.3 — document all new scripts and fix moved paths
New orchestrator: intermediate_sync_maintenance.sh (0 */4 * * *)
New scripts: docker_update, docker_update_remaining, downloaders_reset,
arr_sync, lidarr_missing_art, radarr_tmdb_removed, sonarr_tvdb_removed
New section: Initial_run/ with ssh_setup.sh and partnership_onboard.sh
Moved: arrs_failed_stalled_recovery → Media/, fallback_state_reset +
watchdog_skip_list_manager → Tools/
Renamed: partnership_manage.sh → partnership_manager.sh

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 21:36:37 -04:00
Gmer4LfeandClaude Sonnet 4.6 b65f572367 feat: arr-native distributed media sync + config architecture fixes
Arr sync (new):
- Media/arr_sync.sh — full mesh bidirectional sync across all HOST* nodes
  - Lidarr (MusicBrainz), Sonarr (TVDB), Radarr (TMDB) all handled in one script
  - Remote API keys read live from config.xml via SSH — never stored in conf files
  - Shared blocklist (DATA_DIR/arr_sync_blocklist.tsv) merged from all nodes at runtime
  - Graceful skip if arr not configured locally or not reachable on a remote node
  - --blocklist-add / --blocklist-remove / --blocklist-list management flags
- daily_sync_maintenance.sh — arr sync runs as explicit phase before rsync
- partnership_onboard.sh — Step 3 bootstraps merged library on both sides at onboard
- master.conf — ARR_SYNC_* config block, DOCKER_APPDATA_BASE

Rsync / cleanup:
- DEFAULT_RSYNC_OPTS — removed --delete; arr_cleanup.sh owns orphan enforcement
- lidarr_cleanup.sh — removed HOST1-only guard; runs on any node with Lidarr configured

Config architecture:
- HOST1/HOST2 hostnames moved from master_host*.conf → master.conf (not credentials)
- Sparse checkout now works correctly: each server only needs its own host conf
- detect_hosts() still resolves MY_ID + REMOTE_ID via master.conf hostname values

Bug fix:
- common.sh line 493 — watchdog toggle eval had broken quoting; all SYS_WATCHDOG_CHECK_*
  globals were silently set to empty instead of their configured values

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 09:53:40 -04:00
Gmer4LfeandClaude Sonnet 4.6 009820e981 refactor: rename failover/HA → fallback across entire codebase
Removes all references to "failover" and "HA" (high availability)
terminology from variable names, config keys, state values, rsync
profile names, directory paths, and user-visible strings.

Mapping:
  FAILOVER_*              → FALLBACK_*
  FAILOVER_HOST*_RUNS_FOR → FALLBACK_HOST*_COVERS
  critical-failover       → critical-fallback
  emby-failover           → emby-fallback
  appdata-Failover/       → appdata-Fallback/
  "FAILOVER" state value  → "FALLBACK"
  failover_start key      → fallback_start
  Failover/ directory     → Fallback/
  failover.sh             → fallback.sh
  failover_state.db       → fallback_state.db
  -Failover folder suffix → -Fallback

State machine: NORMAL | FALLBACK | DARK (unchanged)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 19:28:21 -04:00
FailedProxy ec7de648dc massive update. Master conf split, now modular with a load sceriprt to drive all configs to scripts. with unraid scpecific safeguard tests , and improved standardized ux. including dynamic host detect, who am i who else it there. EVERY SINGLE SCRIPT UPDATED. DEBATING THAT THIS IS ACUALLY V2 2026-05-03 17:16:49 -04:00
FailedProxy 7496c49bf5 updated master conf vars 2026-04-29 20:40:08 -04:00
FailedProxy 11872b802b big logic update to rsync and rsync called scripts 2026-04-28 17:04:09 -04:00
FailedProxy e9a42c2aee updated weekly health and sunday coffee report, all needed monitor now in both 2026-04-26 14:42:23 -04:00
FailedProxy 6e95a65bc6 master conf var update 2026-04-26 14:14:36 -04:00
FailedProxy e03c531159 added inotify script and updted downloader reset to match our standard 2026-04-26 14:03:43 -04:00
FailedProxy b3d53b792b added /mnt/user/appdata/unraid_scripts/Monitors/continuous_scripts_status.sh 2026-04-26 02:30:12 -04:00
FailedProxy 3df4542351 added manual neuclear option to arrs cleanup 2026-04-26 01:05:22 -04:00
FailedProxy a3f90ed175 created loop for both watchdogs and consolidated orch lists 2026-04-23 22:44:56 -04:00
FailedProxy 39f2525d0f added a arrs atalled failled handler 2026-04-22 18:22:29 -04:00
FailedProxy 3b08aa30c1 added container update ability be fore sync. i mean its already down 2026-04-21 22:04:18 -04:00
FailedProxy 2467183a68 orch naming restructure with the scripts 2026-04-21 19:46:04 -04:00
FailedProxy 107b9d99fe minor change not worth adressing 2026-04-21 19:17:10 -04:00
FailedProxy d4bf31a5f4 true bidirectional with split truth modle set up. all scripts are fully bidirectional 2026-04-21 18:20:47 -04:00
FailedProxy cbfdc488c3 incorp, host profiles into scripts that needed it 2026-04-19 17:39:11 -04:00
FailedProxy ed9e927c41 added a readme specific to unraids user plug in setup 2026-04-19 13:06:05 -04:00
FailedProxy 2ee3d607fb fixed reADME AND USER SCRIPT PLUGIN 2026-04-19 12:12:49 -04:00
FailedProxy 804a435b31 fixed ttypos 2026-04-19 11:44:58 -04:00