Commit Graph
20 Commits
Author SHA1 Message Date
Gmer4Lfe d5cf3db2ec Close every orchestrator the same way, and make skipped work a visible outcome
A gated-off section left nothing failed, so the weekly could run for hours and report "all
complete" beside "0 shares synced"; skipped is now derived from what was expected rather than
self-reported, and the verdict degrades to PARTIAL instead of flattering.
2026-08-23 16:38:58 -04:00
Gmer4Lfe e8b114094a Bring script headers onto the template and close safeguard gaps
Headers claimed protections the code never had, and several destructive paths had no
guard against a collapsed config value.
2026-08-01 20:37:59 -04:00
Gmer4Lfe 2a062e5140 Standardize orchestrator child-script execution and logging
Every orchestrator invoked its children differently — four near-duplicate
run_job() copies, a differently-shaped run_watchdog(), or plain inline bash
calls, each with its own take on path resolution, pass/fail naming, and
dry-run threading. Extracted one shared run_orch_child() into common.sh so
there's a single place to fix or extend this behavior going forward.

Along the way: watchdog_orchestrator.sh and monthly_maintenance.sh were
checking $VERBOSE, a variable nothing in the codebase ever assigns, so --log
silently did nothing beyond basic logging on those two. Fixed to
$ENABLE_LOGGING. watchdog_orchestrator.sh and array_started.sh had no
trailing exit, so their exit codes reflected whatever the last command
happened to return rather than actual success/failure. transcode_management.sh
had no failure notification and no summary at all. Also made
transcode_management.sh's two-script pipeline config-driven
(TRANSCODE_MANAGEMENT_SCRIPTS in master.conf) instead of hardcoded, for room
to extend it later without editing the orchestrator itself.
2026-07-03 10:57:52 -04:00
Gmer4Lfe b4bc9267e9 Move arr stack scripts from Media/ to Arrs_Stack/
Media/ now holds only media-level scripts (cleaner, permissions, play_state_sync).
All arr management scripts (cleanup, discovery, sync, webhooks, release fixer) live in Arrs_Stack/.
2026-06-27 18:39:33 -04:00
Gmer4Lfe f92ee4064b Add full banner headers to all scripts across the codebase
Every script now has the established header format: PURPOSE with ─────── separator,
OPERATIONAL MODEL, DESIGN PRINCIPLES, OPERATIONAL SAFEGUARDS, CONFIGURATION, and
RUNTIME MODES — structured with full ====== banner sections throughout.

Orchestrators converted from compact ── inline format to full banners. Stale
emby-fallback and dirty sync references removed from Plugin/user_script_plug-in.sh.
2026-06-26 18:50:05 -04:00
Gmer4Lfe 3964f6fb46 audit echo vs log across all scripts — outcomes always visible, verbose for per-item loops 2026-06-14 12:40:15 -04:00
Gmer4Lfe 27bfc21cb0 Platform-agnostic refactor: eliminate OS-specific hardcodes from core scripts
All bash scripts are now platform-neutral. Unraid-specific paths, commands,
and service checks moved to Plugin/unraid/adapter.sh. Core scripts call
platform_*() functions exclusively — no direct OS paths in runtime logic.

New adapter functions: platform_storage_path, platform_webui_install_path,
platform_scripts_dir_probe_cmd, platform_setup_db_path, platform_storage_healthy,
platform_is_service_enabled, platform_get_temp_thresholds, platform_disk_states_path,
platform_rebuild_container, platform_push_conf, platform_push_setup_state,
platform_get_templates_dir, platform_send_os_notification.

Partnership services stack (Emby/Jellyfin/Seerr/SeerrFin) added as third
onboarding stack alongside auth and arr stacks.
2026-06-14 00:59:19 -04:00
Gmer4Lfe 369a9e6c19 Platform adapter: rename System_Essentials, add Plugin/unraid/adapter.sh, wire call sites
- Rename unRAID_Essentials/ → System_Essentials/ (git detects as rename)
- Add Plugin/unraid/adapter.sh: 13 platform_*() functions providing OS-agnostic API
  for storage health, service management, mover, user scripts, notifications,
  disk temps, and platform command validation
- Update load_config.sh: detect PLATFORM (unraid/truenas/unknown), export SCRIPTS_DIR,
  auto-source Plugin/$PLATFORM/adapter.sh after common.sh
- Wire all call sites: replace direct rc.d, pgrep/pkill, var.ini, dynamix.cfg,
  disks.ini, and validate_unraid_cmd calls with platform_*() functions across
  watchdogs, orchestrators, and System_Essentials scripts
- Update all documentation: rename refs, update webgui escalation logic,
  add platform adapter section to Plugin README, update main README with
  portability vision and corrected self-healing stack description
2026-06-04 18:14:34 -04:00
Gmer4Lfe cb7d7a688d rename: failover → fallback throughout codebase — this system is Fallback, state is FALLBACK; preserve conventional failover comparisons 2026-06-01 20:04:17 -04:00
Gmer4Lfe 95151c2278 Watchdogs/ folder + host conf rename
Move all watchdog scripts to a dedicated Watchdogs/ folder:
  Docker_Essentials/docker_watchdog.sh   → Watchdogs/
  unRAID_Essentials/system_watchdog.sh   → Watchdogs/
  unRAID_Essentials/resource_watchdog.sh → Watchdogs/
  Orchestrators/watchdog_orchestrator.sh → Watchdogs/
  Tools/watchdog_skip_list_manager.sh    → Watchdogs/

Rename host config files:
  master_host1.conf → host1.conf
  master_host2.conf → host2.conf

Update all references across the ecosystem:
  master.conf: WATCHDOG_ORCHESTRATOR_SCRIPTS paths → Watchdogs/
  load_config.sh: host*.conf glob + all comments
  git_pull_execute.sh: sparse checkout glob + all comments
  Partnership/ssh_setup.sh: HOST_CONF path construction
  user_script_plug-in.sh: all script paths + per-host conf path
  common.sh, README.md, README-User_Script_Plug-in.md: comment refs
  All Partnership, Fallback, Monitors, Transcodes, Tools scripts: comment refs
2026-05-22 17:08:36 -04:00
Gmer4Lfe ef8ef436cf Orchestrators echo/log audit pass
Status conclusions, clean-cycle summaries, skipped-feature lines, and phase
completion results → echo across all nine orchestrator scripts. Per-share,
per-job, and per-script detail lines inside loops remain log. README-Orchestrators:
added Output Tiers section.
2026-05-21 17:25:29 -04:00
Gmer4Lfe e932936acc Add docker availability check to all scripts that use docker commands
22 scripts used docker without verifying it's present. Consistent with the
pattern already established in Docker_Essentials scripts. Added after acquire_lock
and before detect_hosts in each script's setup section.
2026-05-20 18:35:34 -04:00
Gmer4LfeandClaude Sonnet 4.6 b65f572367 feat: arr-native distributed media sync + config architecture fixes
Arr sync (new):
- Media/arr_sync.sh — full mesh bidirectional sync across all HOST* nodes
  - Lidarr (MusicBrainz), Sonarr (TVDB), Radarr (TMDB) all handled in one script
  - Remote API keys read live from config.xml via SSH — never stored in conf files
  - Shared blocklist (DATA_DIR/arr_sync_blocklist.tsv) merged from all nodes at runtime
  - Graceful skip if arr not configured locally or not reachable on a remote node
  - --blocklist-add / --blocklist-remove / --blocklist-list management flags
- daily_sync_maintenance.sh — arr sync runs as explicit phase before rsync
- partnership_onboard.sh — Step 3 bootstraps merged library on both sides at onboard
- master.conf — ARR_SYNC_* config block, DOCKER_APPDATA_BASE

Rsync / cleanup:
- DEFAULT_RSYNC_OPTS — removed --delete; arr_cleanup.sh owns orphan enforcement
- lidarr_cleanup.sh — removed HOST1-only guard; runs on any node with Lidarr configured

Config architecture:
- HOST1/HOST2 hostnames moved from master_host*.conf → master.conf (not credentials)
- Sparse checkout now works correctly: each server only needs its own host conf
- detect_hosts() still resolves MY_ID + REMOTE_ID via master.conf hostname values

Bug fix:
- common.sh line 493 — watchdog toggle eval had broken quoting; all SYS_WATCHDOG_CHECK_*
  globals were silently set to empty instead of their configured values

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 09:53:40 -04:00
Gmer4LfeandClaude Sonnet 4.6 5526bc1eca v2 post-migration bug fixes, lidarr_missing_art port, and Partnership SSH/FolderView3 enhancements
Bug fixes across the ecosystem after v1→v2 architecture migration and Unraid 7.2.5 upgrade:

- common.sh: fix _alias_array() phantom empty element (removed [@]:-} pattern), fix
  resolve_remote_ip() with Tailscale FQDN lowercase + awk fallback, add FANART/LASTFM
  key aliases in detect_hosts(), global [@]:-} sweep across 10+ scripts
- webgui_restart.sh: fix emhttp detection (pgrep emhttpd) and restart command for 7.2.5
  (/usr/local/sbin/emhttp stop && start — rc.emhttp removed in 7.2.5)
- bandwidth_monitor.sh, continuous_scripts_status.sh: fix 'local' keyword outside function
- backup_verify.sh: fix resolve_remote_ip() called before detect_hosts()
- Orchestrators: fix script display duplication bug in status output (${entry##*/})
- rsync_stop.sh, git_pull_execute.sh, partnership_manager.sh, coffee_report: lowercase all
  tailscale ip -4 call sites to match Tailscale's lowercase device names
- master_host1.conf: fix SSH key path (gmer4lfe_rsync_automation), add FANART/LASTFM keys
- master_host2.conf: add FANART/LASTFM API keys

New: Media/lidarr_missing_art.sh
- Full ecosystem port of standalone Lidarr artwork fetcher
- Fetches missing album art via fanart.tv + Last.fm APIs
- @tsv batch extraction: 1 jq call per API response vs N*albums (8050 albums in 24s)
- HOST guard (HOST1 only), --status, --dry-run, acquire_lock

New: Initial_run/ssh_setup.sh
- Generates {hostname}_rsync_automation ed25519 keypair (skip if exists, --force to regen)
- ssh-copy-id to remote via Tailscale IP, auto-updates master_host*.conf
- --validate mode: strike tracking (SSH_MAX_STRIKES, SSH_STRIKE_RESET_HRS),
  notify at limit — Tailscale-unreachable remote does NOT count as SSH strike

New: Initial_run/partnership_onboard.sh
- Orchestrator: ssh_setup.sh then partnership_manager.sh --onboard in one command

Partnership/partnership_manager.sh: FolderView3 integration
- Derive partner folder name at runtime (strip unraid- prefix case-insensitively)
- --onboard: create {Mirror}-Failover folder with failover tier containers
- --offboard (both paths): stop + rm containers in folder, remove JSON entry
- --check: calls ssh_setup.sh --validate when IP resolves but SSH state empty
- --status: shows FolderView3 folder state and containers inline

master.conf: SSH_MAX_STRIKES, SSH_STRIKE_RESET_HRS, PARTNERSHIP_FOLDERVIEW3,
PARTNERSHIP_FOLDERVIEW3_URL added to Partnership section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 18:20:23 -04:00
FailedProxy ec7de648dc massive update. Master conf split, now modular with a load sceriprt to drive all configs to scripts. with unraid scpecific safeguard tests , and improved standardized ux. including dynamic host detect, who am i who else it there. EVERY SINGLE SCRIPT UPDATED. DEBATING THAT THIS IS ACUALLY V2 2026-05-03 17:16:49 -04:00
FailedProxy 94f90a4579 padded partnership set up 2026-04-29 20:09:28 -04:00
FailedProxy 1c765b7260 added rsync disable in master global tier 1, advanced selection tier 2 2026-04-28 17:07:44 -04:00
FailedProxy 11872b802b big logic update to rsync and rsync called scripts 2026-04-28 17:04:09 -04:00
FailedProxy 3df4542351 added manual neuclear option to arrs cleanup 2026-04-26 01:05:22 -04:00
FailedProxy a3f90ed175 created loop for both watchdogs and consolidated orch lists 2026-04-23 22:44:56 -04:00