feat: arr-native distributed media sync + config architecture fixes

Arr sync (new):
- Media/arr_sync.sh — full mesh bidirectional sync across all HOST* nodes
  - Lidarr (MusicBrainz), Sonarr (TVDB), Radarr (TMDB) all handled in one script
  - Remote API keys read live from config.xml via SSH — never stored in conf files
  - Shared blocklist (DATA_DIR/arr_sync_blocklist.tsv) merged from all nodes at runtime
  - Graceful skip if arr not configured locally or not reachable on a remote node
  - --blocklist-add / --blocklist-remove / --blocklist-list management flags
- daily_sync_maintenance.sh — arr sync runs as explicit phase before rsync
- partnership_onboard.sh — Step 3 bootstraps merged library on both sides at onboard
- master.conf — ARR_SYNC_* config block, DOCKER_APPDATA_BASE

Rsync / cleanup:
- DEFAULT_RSYNC_OPTS — removed --delete; arr_cleanup.sh owns orphan enforcement
- lidarr_cleanup.sh — removed HOST1-only guard; runs on any node with Lidarr configured

Config architecture:
- HOST1/HOST2 hostnames moved from master_host*.conf → master.conf (not credentials)
- Sparse checkout now works correctly: each server only needs its own host conf
- detect_hosts() still resolves MY_ID + REMOTE_ID via master.conf hostname values

Bug fix:
- common.sh line 493 — watchdog toggle eval had broken quoting; all SYS_WATCHDOG_CHECK_*
  globals were silently set to empty instead of their configured values

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Gmer4Lfe
2026-05-09 09:53:40 -04:00
co-authored by Claude Sonnet 4.6
parent 009820e981
commit b65f572367
21 changed files with 1548 additions and 272 deletions
+6 -6
View File
@@ -203,7 +203,7 @@
# bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh
# ── RSYNC EMBY FAILOVER ───────────────────────────────────────────────────────────────────────
# ── RSYNC EMBY FALLBACK ───────────────────────────────────────────────────────────────────────
# Schedule: */30 * * * * (every 30 minutes)
# Background: YES
#
@@ -375,14 +375,14 @@
# ──────────────────────────────────────────────────────────────────────────────────────────────
# FAILOVER
# FALLBACK
# ──────────────────────────────────────────────────────────────────────────────────────────────
# fallback.sh — [continuous] mutual fallback state machine
# Started by array_start.sh on both servers independently.
# Every FALLBACK_CHECK_INTERVAL (120s) pings: remote Tailscale IP + 8.8.8.8
# States: NORMAL / FAILOVER / NO_INTERNET / DARK
# FAILOVER: starts remote containers in tiers across 24 hours:
# States: NORMAL / FALLBACK / NO_INTERNET / DARK
# FALLBACK: starts remote containers in tiers across 24 hours:
# Tier 1 immediate: DDNS + Emby + auth (NPM/Authelia/LLDAP) + VaultWarden + Live TV
# Tier 2 at 4hr: NextCloud + Immich + Jellyseerr
# Tier 3 at 12hr: AdGuard + Gitea + uptime monitoring
@@ -399,7 +399,7 @@
# ALWAYS run --dry-run first. Live test starts/stops real containers — brief service interruption.
# iptables safety trap removes the block rule on ANY exit including ctrl-c and crashes.
# Remote is always reachable after the test completes regardless of what happened.
# 8 phases: pre-flight → block remote (iptables) → detect FAILOVER → verify Tier 1 started
# 8 phases: pre-flight → block remote (iptables) → detect FALLBACK → verify Tier 1 started
# → restore remote → wait handback → verify NORMAL restored → full pass/fail report.
# FALLBACK_TEST_BLOCK_WAIT must be > FALLBACK_CHECK_INTERVAL + buffer (default: 150s).
# FALLBACK_TEST_HANDBACK_WAIT must cover: strike confirmation + rsync + container start (default: 360s).
@@ -409,7 +409,7 @@
# fallback_state_reset.sh — reset fallback state file to NORMAL
# Resets state file ONLY — does NOT start or stop any containers.
# Use when: state stuck in FAILOVER after testing, after killing fallback.sh mid-cycle,
# Use when: state stuck in FALLBACK after testing, after killing fallback.sh mid-cycle,
# after manual intervention left state inconsistent, after a failed handback.
# Shows current state file before asking for confirmation.
# Verify before resetting: right containers on right server, DDNS correct,