Commit Graph
26 Commits
Author SHA1 Message Date
Gmer4Lfe e932936acc Add docker availability check to all scripts that use docker commands
22 scripts used docker without verifying it's present. Consistent with the
pattern already established in Docker_Essentials scripts. Added after acquire_lock
and before detect_hosts in each script's setup section.
2026-05-20 18:35:34 -04:00
Gmer4Lfe e13f2fa14f feat: slskd reconnect guard in downloaders_reset, mass v2 sync
- downloaders_reset: connection check block before slskd API sections;
  triggers PUT /api/v0/server reconnect if disconnected, polls 60s,
  gates Stuck Searches and Dead Transfer Records on SLSKD_CONNECTED
- Sync all modified/new/deleted files from v2 refactor across Docker_Essentials,
  Media, Monitors, Partnership, Rsync, Tools, Transcodes, unRAID_Essentials,
  common.sh, master confs, and new Manual/README docs
2026-05-19 20:00:10 -04:00
Gmer4LfeandClaude Sonnet 4.6 9a5f6f88f2 refactor: rename resource_manager → resource_watchdog, RM_ → RW_
Consistent naming with the watchdog family (system_watchdog, docker_watchdog).
File renamed, all RM_ config variables and internal references updated to RW_
across master.conf, master_host1.conf, common.sh, docker_watchdog.sh,
system_watchdog.sh, watchdog_orchestrator.sh, and sunday_morning_coffee_report.sh.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 18:31:09 -04:00
Gmer4LfeandClaude Sonnet 4.6 309546e615 feat: watchdog architecture v2 — resource manager + single-pass orchestrator
Introduce a four-layer self-healing stack replacing the continuous-loop watchdogs:

- resource_manager.sh (new): single-pass pressure reduction layer; throttles
  SABnzbd/qBit at level 1, docker-pauses background containers at level 2,
  docker-stops optional containers and signals docker_watchdog to defer at
  level 3; graduated recovery with hysteresis

- watchdog_orchestrator.sh (new, Orchestrators/): runs resource_manager →
  docker_watchdog → system_watchdog in sequence; intended for per-minute cron
  via User Scripts; startup grace, acquire_lock to prevent pile-up, heartbeat

- docker_watchdog.sh: de-looped to single-pass; daemon strikes persisted to
  state file across runs; cross-script coordination reads RM_STATE_FILE instead
  of SYS_WATCHDOG_STATE_FILE

- system_watchdog.sh: de-looped to single-pass; stripped of all container
  management (shutdown_non_essential_containers removed); reboot-only last resort

- master.conf: removed system_watchdog and docker_watchdog from
  ARRAY_START_SCRIPTS; added WATCHDOG ORCHESTRATOR and RESOURCE MANAGER sections

- master_host1.conf: added RM_PAUSE_CONTAINERS and RM_STOP_CONTAINERS arrays

- common.sh: aliased RM_PAUSE_CONTAINERS and RM_STOP_CONTAINERS via detect_hosts()

- continuous_scripts_status.sh: moved to Tools/ (preserved for future use)

- sunday_morning_coffee_report.sh: watchdog section updated to use state file
  mtime checks instead of is_running; added Resource Manager subsection;
  fixed mem_shutdown grep filter pointing to wrong state file

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 17:41:59 -04:00
Gmer4LfeandClaude Sonnet 4.6 f16c962ac0 feat: rename array orchestrators, add docker_update remainder mode
- Rename array_start.sh → array_started.sh, array_stop.sh → array_stopping.sh
  to clarify these are event-driven (array has started/is stopping), not imperative
- Update all references across 9 files (master.conf, user_script_plug-in.sh,
  watchdogs, continuous_scripts_status.sh, claude_startup.sh)
- Add --remainder mode to docker_update.sh: updates all running containers
  excluding daily containers, weekly sync-window containers (emby + critical-data),
  and fallback coverage containers (FALLBACK_${MY_ID}_COVERS_${REMOTE_ID}_TIER*)
  Fallback containers excluded because the remote server owns their version —
  independent updates risk writeback incompatibility on handback
- weekly_sync_maintenance.sh calls docker_update.sh --remainder as final step
- git_pull_execute.sh: add safe.directory config to fix dubious ownership error
  when running as root on a directory owned by uid 1000

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 22:12:08 -04:00
Gmer4LfeandClaude Sonnet 4.6 0ae31b5fa6 feat: Tailscale resolution hardening, partnership offboard completion, Emby provisioning
common.sh:
- Add resolve_tailscale_ip() helper — tries `tailscale ip -4` first, falls back to
  parsing `tailscale status` output; handles hosts where MagicDNS short-name resolution
  is not active
- Add PARTNERSHIP_OWN_CONTAINERS alias in detect_hosts()
- Add aliasing for 4 Emby provisioning vars (PARTNERSHIP_PROVISION_EMBY_ADMIN,
  PARTNERSHIP_EMBY_ADMIN_USER, PARTNERSHIP_EMBY_ADMIN_PASS, PARTNERSHIP_EMBY_PORT)

Partnership/partnership_manager.sh:
- Replace 9 bare `tailscale ip -4` calls with resolve_tailscale_ip()
- Add read_remote_conf_var() and read_remote_conf_array() — SSH to mirror, source its
  own load_config.sh + detect_hosts(), return aliased variable; solves sparse-checkout
  problem where HOST1 cannot read master_host2.conf directly
- Add derive_short_name() — strips unraid- prefix, capitalises first char
- Add cleanup_partner_containers() — removes partner containers via FolderView3 folder
  if enabled, else falls back to FALLBACK_*_COVERS_*_TIER* arrays
- Add cleanup_owner_containers_on_mirror() — SSH to mirror, stops and removes containers
  matching *-${OWNER_SHORT} naming convention
- Add start_own_stack() and start_mirror_own_stack() — restart own containers locally
  or on mirror via SSH using PARTNERSHIP_OWN_CONTAINERS
- Add provision_emby_admin() — reads mirror credentials via read_remote_conf_var, checks
  for username collision, creates user + sets password + grants admin policy via Emby API
- Add revoke_emby_admin() — looks up mirror username on local Emby, deletes via REST API
- Wire offboard paths (both mirror-initiated and owner-initiated) to call container
  cleanup and stack restart; update --check finalisation paths accordingly
- Fix write_state_file in --onboard not gated on DRY_RUN (was writing ACTIVE state on
  dry runs)

master_host1.conf:
- Add HOST1_PARTNERSHIP_OWN_CONTAINERS array
- Add partnership Emby provisioning config (toggle + port + per-host credentials)

master_host2.conf:
- Add HOST2_PARTNERSHIP_OWN_CONTAINERS array
- Add HOST2_PARTNERSHIP_EMBY_ADMIN_USER and HOST2_PARTNERSHIP_EMBY_ADMIN_PASS

Tailscale fix applied to:
- Initial_run/ssh_setup.sh (2 callsites)
- unRAID_Essentials/rsync_stop.sh (1 callsite)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 20:09:13 -04:00
Gmer4LfeandClaude Sonnet 4.6 5526bc1eca v2 post-migration bug fixes, lidarr_missing_art port, and Partnership SSH/FolderView3 enhancements
Bug fixes across the ecosystem after v1→v2 architecture migration and Unraid 7.2.5 upgrade:

- common.sh: fix _alias_array() phantom empty element (removed [@]:-} pattern), fix
  resolve_remote_ip() with Tailscale FQDN lowercase + awk fallback, add FANART/LASTFM
  key aliases in detect_hosts(), global [@]:-} sweep across 10+ scripts
- webgui_restart.sh: fix emhttp detection (pgrep emhttpd) and restart command for 7.2.5
  (/usr/local/sbin/emhttp stop && start — rc.emhttp removed in 7.2.5)
- bandwidth_monitor.sh, continuous_scripts_status.sh: fix 'local' keyword outside function
- backup_verify.sh: fix resolve_remote_ip() called before detect_hosts()
- Orchestrators: fix script display duplication bug in status output (${entry##*/})
- rsync_stop.sh, git_pull_execute.sh, partnership_manager.sh, coffee_report: lowercase all
  tailscale ip -4 call sites to match Tailscale's lowercase device names
- master_host1.conf: fix SSH key path (gmer4lfe_rsync_automation), add FANART/LASTFM keys
- master_host2.conf: add FANART/LASTFM API keys

New: Media/lidarr_missing_art.sh
- Full ecosystem port of standalone Lidarr artwork fetcher
- Fetches missing album art via fanart.tv + Last.fm APIs
- @tsv batch extraction: 1 jq call per API response vs N*albums (8050 albums in 24s)
- HOST guard (HOST1 only), --status, --dry-run, acquire_lock

New: Initial_run/ssh_setup.sh
- Generates {hostname}_rsync_automation ed25519 keypair (skip if exists, --force to regen)
- ssh-copy-id to remote via Tailscale IP, auto-updates master_host*.conf
- --validate mode: strike tracking (SSH_MAX_STRIKES, SSH_STRIKE_RESET_HRS),
  notify at limit — Tailscale-unreachable remote does NOT count as SSH strike

New: Initial_run/partnership_onboard.sh
- Orchestrator: ssh_setup.sh then partnership_manager.sh --onboard in one command

Partnership/partnership_manager.sh: FolderView3 integration
- Derive partner folder name at runtime (strip unraid- prefix case-insensitively)
- --onboard: create {Mirror}-Failover folder with failover tier containers
- --offboard (both paths): stop + rm containers in folder, remove JSON entry
- --check: calls ssh_setup.sh --validate when IP resolves but SSH state empty
- --status: shows FolderView3 folder state and containers inline

master.conf: SSH_MAX_STRIKES, SSH_STRIKE_RESET_HRS, PARTNERSHIP_FOLDERVIEW3,
PARTNERSHIP_FOLDERVIEW3_URL added to Partnership section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 18:20:23 -04:00
FailedProxy ec7de648dc massive update. Master conf split, now modular with a load sceriprt to drive all configs to scripts. with unraid scpecific safeguard tests , and improved standardized ux. including dynamic host detect, who am i who else it there. EVERY SINGLE SCRIPT UPDATED. DEBATING THAT THIS IS ACUALLY V2 2026-05-03 17:16:49 -04:00
FailedProxy 94f90a4579 padded partnership set up 2026-04-29 20:09:28 -04:00
FailedProxy 95ce62e663 massive update to rsync stop 2026-04-28 17:40:24 -04:00
FailedProxy e03c531159 added inotify script and updted downloader reset to match our standard 2026-04-26 14:03:43 -04:00
FailedProxy dc43b9844b quick update to watchdogs 2026-04-26 02:44:59 -04:00
FailedProxy f826672b80 added more descriptions to master.conf 2026-04-24 22:52:32 -04:00
FailedProxy 63101fdb83 system watchdog update to minitor for unheathy pids 2026-04-24 20:57:45 -04:00
FailedProxy b3f72417a4 fixed arrs clean up showing only orphans 2026-04-24 17:22:34 -04:00
FailedProxy 6df0bafc09 fixed lock logic. if a continuious app hits a lock it skips to the next instead of error 2026-04-24 16:26:35 -04:00
FailedProxy a3f90ed175 created loop for both watchdogs and consolidated orch lists 2026-04-23 22:44:56 -04:00
FailedProxy e83473ab8e added safe guards to all scripts that need it. all reference comm.sh like it should be 2026-04-20 17:18:25 -04:00
FailedProxy 8c5667b1df added all ai generated Readme files, added the last of the tools scripts 2026-04-18 15:40:41 -04:00
FailedProxy 6f2e19265b updated my variables in master.conf 2026-04-14 17:38:44 -04:00
FailedProxy b080e0ffdc naming fix 2026-04-11 18:09:32 -04:00
FailedProxy 55930a1166 added failover script and other sytem scripts 2026-04-11 17:46:11 -04:00
FailedProxy 44915887c2 yeah 2026-04-10 18:47:05 -04:00
FailedProxy b7706f4ab4 Added system watchdog. and way to many other changes 2026-04-10 18:11:16 -04:00
FailedProxy 6cc26c8fb9 added notify to all needed in unraid essentials 2026-04-09 17:20:09 -04:00
FailedProxy 9a7a28055f all scripts are now current with common.sh v2.1 2026-04-08 17:36:58 -04:00