Commit Graph
260 Commits
Author SHA1 Message Date
Gmer4LfeandClaude Sonnet 4.6 f277ce92d5 feat: intermediate orchestrator + arr_sync blocklist end-to-end + mesh sync
Add intermediate_sync_maintenance.sh (0 */4 * * *) — arr library sync as
fixed first step, optional mid-day rsync (INTERMEDIATE_SYNC_SHARES, empty
by default), then INTERMEDIATE_MAINTENANCE_SCRIPTS. Wired with full rsync
infrastructure (temperature abort, check_rsync_enabled, INTERMEDIATE_RSYNC_ENABLED
toggle) matching daily/weekly pattern. lidarr_missing_art.sh scheduled here.

arr_sync.sh: blocklist --add now end-to-end — looks up real display name
from local arr API, writes TSV, deletes from local arr API (deleteFiles=false),
SSHes each remote node and deletes from their arr API. Files become orphans
for arr_cleanup safety pass.

master.conf: DEFAULT_RSYNC_OPTS updated (--partial, --timeout=60,
--numeric-ids; --no-whole-file removed). Media share comments updated to
mesh model.

master_host1.conf + master_host2.conf: both hosts now push all media shares
bidirectionally (true mesh — no ownership per share, arr_cleanup enforces truth).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 15:24:35 -04:00
Gmer4LfeandClaude Sonnet 4.6 6674dd4c17 Add SSH key revocation to offboard flow
On offboard, neither server should retain SSH access to the other.
Keys are now revoked as the final step before Tailscale removal — after
all state pushes and syncs are complete so no SSH operation is cut short.

Revocation runs in both owner-initiated offboard and when --check
finalises a mirror-requested offboard. Both directions are handled:
  - Our pubkey removed from remote's authorized_keys via SSH (identified
    by key comment keyname@hostname set by ssh_setup.sh at keygen time)
  - Remote's pubkey removed from local authorized_keys (matched by
    @REMOTE_SERVER_NAME suffix — unique across the authorized_keys file)

Non-fatal: SSH failure logs a manual revocation instruction; local
cleanup still runs. Summary shows per-direction revocation status.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 10:21:01 -04:00
Gmer4LfeandClaude Sonnet 4.6 4821dbdfb4 config: update HOST1 Sonarr API key
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 10:06:35 -04:00
Gmer4LfeandClaude Sonnet 4.6 b65f572367 feat: arr-native distributed media sync + config architecture fixes
Arr sync (new):
- Media/arr_sync.sh — full mesh bidirectional sync across all HOST* nodes
  - Lidarr (MusicBrainz), Sonarr (TVDB), Radarr (TMDB) all handled in one script
  - Remote API keys read live from config.xml via SSH — never stored in conf files
  - Shared blocklist (DATA_DIR/arr_sync_blocklist.tsv) merged from all nodes at runtime
  - Graceful skip if arr not configured locally or not reachable on a remote node
  - --blocklist-add / --blocklist-remove / --blocklist-list management flags
- daily_sync_maintenance.sh — arr sync runs as explicit phase before rsync
- partnership_onboard.sh — Step 3 bootstraps merged library on both sides at onboard
- master.conf — ARR_SYNC_* config block, DOCKER_APPDATA_BASE

Rsync / cleanup:
- DEFAULT_RSYNC_OPTS — removed --delete; arr_cleanup.sh owns orphan enforcement
- lidarr_cleanup.sh — removed HOST1-only guard; runs on any node with Lidarr configured

Config architecture:
- HOST1/HOST2 hostnames moved from master_host*.conf → master.conf (not credentials)
- Sparse checkout now works correctly: each server only needs its own host conf
- detect_hosts() still resolves MY_ID + REMOTE_ID via master.conf hostname values

Bug fix:
- common.sh line 493 — watchdog toggle eval had broken quoting; all SYS_WATCHDOG_CHECK_*
  globals were silently set to empty instead of their configured values

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 09:53:40 -04:00
Gmer4LfeandClaude Sonnet 4.6 009820e981 refactor: rename failover/HA → fallback across entire codebase
Removes all references to "failover" and "HA" (high availability)
terminology from variable names, config keys, state values, rsync
profile names, directory paths, and user-visible strings.

Mapping:
  FAILOVER_*              → FALLBACK_*
  FAILOVER_HOST*_RUNS_FOR → FALLBACK_HOST*_COVERS
  critical-failover       → critical-fallback
  emby-failover           → emby-fallback
  appdata-Failover/       → appdata-Fallback/
  "FAILOVER" state value  → "FALLBACK"
  failover_start key      → fallback_start
  Failover/ directory     → Fallback/
  failover.sh             → fallback.sh
  failover_state.db       → fallback_state.db
  -Failover folder suffix → -Fallback

State machine: NORMAL | FALLBACK | DARK (unchanged)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 19:28:21 -04:00
Gmer4LfeandClaude Sonnet 4.6 4e22f5d1f7 feat(docker): add docker_update_remaining.sh for weekly catch-all image pulls
Pulls the latest image for every running container not already covered
by the daily update (DAILY_RESTART_CONTAINERS) or weekly restart
(WEEKLY_RESTART_CONTAINERS) lists. Runs at the end of WEEKLY_MAINTENANCE_SCRIPTS.

Exclusion set is derived automatically — no list to maintain.
Together with docker_update.sh, every deployed container gets at least
one image pull per week without relying on the CA Update Applications plugin.

Toggle: WEEKLY_REMAINING_UPDATES=true/false in master.conf.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 19:13:50 -04:00
Gmer4LfeandClaude Sonnet 4.6 0adfe04173 feat(docker): add docker_update.sh for daily container image pulls
Pulls the latest image for each container in DAILY_RESTART_CONTAINERS
before docker_daily_restart.sh runs. Containers stay running during the
pull — no extra downtime. Mirrors the weekly update pattern exactly.

Toggle: DAILY_CONTAINER_UPDATES=true/false in master.conf.
Pull failures are non-fatal — the daily restart proceeds regardless.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 19:12:00 -04:00
Gmer4LfeandClaude Sonnet 4.6 69464ba385 feat(partnership): add blocklist to block access after offboard
After offboard, the former partner's hostname is written to
/boot/config/partnership_blocklist.db. This blocks passive reconnection:
  - --check skips the remote entirely (no auto-reconnect noise)
  - rsync.sh refuses to sync with a blocklisted host
  - --onboard warns about the previous offboard but proceeds,
    and clears the block on success (onboard = deliberate intent)

New --unblock <hostname> mode removes an entry to permit re-onboarding
without running a full --onboard first. --status shows the full blocklist.

Blocklist file survives reboots (/boot/config) and Tailscale reconnections —
application-layer guard that complements Tailscale device removal.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 18:59:29 -04:00
Gmer4LfeandClaude Sonnet 4.6 09e9cec0c3 feat(rsync): print status message when transfer starts
rsync output is fully captured — nothing visible while it runs.
Adds a "Rsync running — this may take a while..." line before each
attempt so the Unraid log viewer shows transfer progress clearly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 18:53:41 -04:00
Gmer4LfeandClaude Sonnet 4.6 10da8108e5 fix: detect_hosts() aborts when REMOTE_ID is empty on single-server sparse checkout
${!REMOTE_ID:-unknown} with REMOTE_ID="" is a fatal bash error that terminates
the entire function — not just the line. All subsequent aliases (EMBY_CONTAINER,
TRANSCODE_SSD, RAMDISK_LOW_GB, SSH_KEY) were never set, causing awk syntax errors
and "SSD fallback path missing" warnings in every transcode script invocation.

Guard with an explicit empty check instead of relying on the :- fallback,
which bash does not apply when the indirect expansion itself is invalid.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 18:41:48 -04:00
Gmer4LfeandClaude Sonnet 4.6 5526bc1eca v2 post-migration bug fixes, lidarr_missing_art port, and Partnership SSH/FolderView3 enhancements
Bug fixes across the ecosystem after v1→v2 architecture migration and Unraid 7.2.5 upgrade:

- common.sh: fix _alias_array() phantom empty element (removed [@]:-} pattern), fix
  resolve_remote_ip() with Tailscale FQDN lowercase + awk fallback, add FANART/LASTFM
  key aliases in detect_hosts(), global [@]:-} sweep across 10+ scripts
- webgui_restart.sh: fix emhttp detection (pgrep emhttpd) and restart command for 7.2.5
  (/usr/local/sbin/emhttp stop && start — rc.emhttp removed in 7.2.5)
- bandwidth_monitor.sh, continuous_scripts_status.sh: fix 'local' keyword outside function
- backup_verify.sh: fix resolve_remote_ip() called before detect_hosts()
- Orchestrators: fix script display duplication bug in status output (${entry##*/})
- rsync_stop.sh, git_pull_execute.sh, partnership_manager.sh, coffee_report: lowercase all
  tailscale ip -4 call sites to match Tailscale's lowercase device names
- master_host1.conf: fix SSH key path (gmer4lfe_rsync_automation), add FANART/LASTFM keys
- master_host2.conf: add FANART/LASTFM API keys

New: Media/lidarr_missing_art.sh
- Full ecosystem port of standalone Lidarr artwork fetcher
- Fetches missing album art via fanart.tv + Last.fm APIs
- @tsv batch extraction: 1 jq call per API response vs N*albums (8050 albums in 24s)
- HOST guard (HOST1 only), --status, --dry-run, acquire_lock

New: Initial_run/ssh_setup.sh
- Generates {hostname}_rsync_automation ed25519 keypair (skip if exists, --force to regen)
- ssh-copy-id to remote via Tailscale IP, auto-updates master_host*.conf
- --validate mode: strike tracking (SSH_MAX_STRIKES, SSH_STRIKE_RESET_HRS),
  notify at limit — Tailscale-unreachable remote does NOT count as SSH strike

New: Initial_run/partnership_onboard.sh
- Orchestrator: ssh_setup.sh then partnership_manager.sh --onboard in one command

Partnership/partnership_manager.sh: FolderView3 integration
- Derive partner folder name at runtime (strip unraid- prefix case-insensitively)
- --onboard: create {Mirror}-Failover folder with failover tier containers
- --offboard (both paths): stop + rm containers in folder, remove JSON entry
- --check: calls ssh_setup.sh --validate when IP resolves but SSH state empty
- --status: shows FolderView3 folder state and containers inline

master.conf: SSH_MAX_STRIKES, SSH_STRIKE_RESET_HRS, PARTNERSHIP_FOLDERVIEW3,
PARTNERSHIP_FOLDERVIEW3_URL added to Partnership section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 18:20:23 -04:00
FailedProxy 7cecda6b90 added lidar missing art checker 2026-05-08 16:26:22 -04:00
FailedProxy 35707a4191 common var fix 2026-05-04 18:03:51 -04:00
FailedProxy ec7de648dc massive update. Master conf split, now modular with a load sceriprt to drive all configs to scripts. with unraid scpecific safeguard tests , and improved standardized ux. including dynamic host detect, who am i who else it there. EVERY SINGLE SCRIPT UPDATED. DEBATING THAT THIS IS ACUALLY V2 2026-05-03 17:16:49 -04:00
FailedProxy 2691a35e80 same 2026-04-30 20:57:23 -04:00
FailedProxy 84b54ccb1d same 2026-04-30 20:54:42 -04:00
FailedProxy d520e94180 same 2026-04-30 20:51:34 -04:00
FailedProxy a2ce4318c0 same 2026-04-29 21:27:54 -04:00
FailedProxy 112793055f master conf var update 2026-04-29 21:19:15 -04:00
FailedProxy 18a4c6f3c7 updated disk checking logic in rsync on remote check 2026-04-29 20:44:11 -04:00
FailedProxy 7496c49bf5 updated master conf vars 2026-04-29 20:40:08 -04:00
FailedProxy 94f90a4579 padded partnership set up 2026-04-29 20:09:28 -04:00
FailedProxy 95ce62e663 massive update to rsync stop 2026-04-28 17:40:24 -04:00
FailedProxy 1c765b7260 added rsync disable in master global tier 1, advanced selection tier 2 2026-04-28 17:07:44 -04:00
FailedProxy 11872b802b big logic update to rsync and rsync called scripts 2026-04-28 17:04:09 -04:00
FailedProxy f7074cfba0 updated variables for coffee report. 2026-04-26 16:38:38 -04:00
FailedProxy 4f621192a5 update to incorrect output in temps on coffee report 2026-04-26 15:58:15 -04:00
FailedProxy e9a42c2aee updated weekly health and sunday coffee report, all needed monitor now in both 2026-04-26 14:42:23 -04:00
FailedProxy 6e95a65bc6 master conf var update 2026-04-26 14:14:36 -04:00
FailedProxy e03c531159 added inotify script and updted downloader reset to match our standard 2026-04-26 14:03:43 -04:00
FailedProxy 0f19913c52 another qbit fix time stamps 2026-04-26 12:28:00 -04:00
FailedProxy 4d94ad8863 i age retention fix for qbit in downloaders reset 2026-04-26 12:21:27 -04:00
FailedProxy 516f6996bc added and testin a downloar reset script 2026-04-26 12:11:24 -04:00
FailedProxy a638035a2f updated docker network logic added a custome network check 2026-04-26 04:29:09 -04:00
FailedProxy ad0546c398 master var change 2026-04-26 03:45:38 -04:00
FailedProxy 76d81180d2 master var change 2026-04-26 03:42:18 -04:00
FailedProxy aa16374455 added watchdog excluded list 2026-04-26 03:05:18 -04:00
FailedProxy 097e9037ba typo fix 2026-04-26 02:49:06 -04:00
FailedProxy dc43b9844b quick update to watchdogs 2026-04-26 02:44:59 -04:00
FailedProxy b3d53b792b added /mnt/user/appdata/unraid_scripts/Monitors/continuous_scripts_status.sh 2026-04-26 02:30:12 -04:00
FailedProxy d3906a8efa added more protected file types 2026-04-26 02:05:22 -04:00
FailedProxy 1a9162ed80 added version check to arrs. checks version that checks its api version 2026-04-26 01:54:27 -04:00
FailedProxy fb42461e2f fixed protected lists in all arrs 2026-04-26 01:41:00 -04:00
FailedProxy e44b01364f added trailer exclusions into sonarr and radarr 2026-04-26 01:29:08 -04:00
FailedProxy e6a6bfd5e9 update radar and sonarr enpoints 2026-04-26 01:19:32 -04:00
FailedProxy 3df4542351 added manual neuclear option to arrs cleanup 2026-04-26 01:05:22 -04:00
FailedProxy f826672b80 added more descriptions to master.conf 2026-04-24 22:52:32 -04:00
FailedProxy 63101fdb83 system watchdog update to minitor for unheathy pids 2026-04-24 20:57:45 -04:00
FailedProxy 5f940a7181 now i fixed arrs ckean, maybe? 2026-04-24 17:44:11 -04:00
FailedProxy b3f72417a4 fixed arrs clean up showing only orphans 2026-04-24 17:22:34 -04:00