feat: intermediate orchestrator + arr_sync blocklist end-to-end + mesh sync

Add intermediate_sync_maintenance.sh (0 */4 * * *) — arr library sync as
fixed first step, optional mid-day rsync (INTERMEDIATE_SYNC_SHARES, empty
by default), then INTERMEDIATE_MAINTENANCE_SCRIPTS. Wired with full rsync
infrastructure (temperature abort, check_rsync_enabled, INTERMEDIATE_RSYNC_ENABLED
toggle) matching daily/weekly pattern. lidarr_missing_art.sh scheduled here.

arr_sync.sh: blocklist --add now end-to-end — looks up real display name
from local arr API, writes TSV, deletes from local arr API (deleteFiles=false),
SSHes each remote node and deletes from their arr API. Files become orphans
for arr_cleanup safety pass.

master.conf: DEFAULT_RSYNC_OPTS updated (--partial, --timeout=60,
--numeric-ids; --no-whole-file removed). Media share comments updated to
mesh model.

master_host1.conf + master_host2.conf: both hosts now push all media shares
bidirectionally (true mesh — no ownership per share, arr_cleanup enforces truth).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Gmer4Lfe
2026-05-09 15:24:35 -04:00
co-authored by Claude Sonnet 4.6
parent 6674dd4c17
commit f277ce92d5
5 changed files with 500 additions and 18 deletions
+29 -6
View File
@@ -280,6 +280,23 @@
# "Fallback/fallback.sh" # mutual failover — enable when HOST2 ready
)
# ━━━ Intermediate Sync Maintenance ━━━
# intermediate_sync_maintenance.sh runs every 4 hours — arr library sync, artwork fetch,
# and optional mid-day rsync for any shares that need sub-daily propagation.
# Schedule: 0 */4 * * *
INTERMEDIATE_SYNC_SHARES=(
# Add shares here to enable mid-day rsync — empty = rsync section skipped entirely.
# Uses DEFAULT_RSYNC_OPTS (no --delete). Full media sync stays in the daily window.
# Example: "/mnt/user/Emby_Metadata"
)
INTERMEDIATE_RSYNC_ENABLED=true # set false to disable mid-day rsync without removing shares
INTERMEDIATE_MAINTENANCE_SCRIPTS=(
"Media/lidarr_missing_art.sh" # fetch missing album/artist artwork (HOST1 only — self-guards)
)
# arr_sync.sh runs as a fixed first step in intermediate_sync_maintenance.sh — not listed here.
# It is controlled by ARR_SYNC_ENABLED (see Arr Sync section above).
# ━━━ Daily Sync Maintenance ━━━
# daily_sync_maintenance.sh runs media share sync first, then iterates
# DAILY_MAINTENANCE_SCRIPTS for all jobs.
@@ -305,9 +322,10 @@
# Media shares synced daily by daily_sync_maintenance.sh.
# Defined per-host in master_host*.conf — HOST1_DAILY_SYNC_SHARES and HOST2_DAILY_SYNC_SHARES.
# Each server syncs only the shares it owns — direction is automatic.
# HOST1 pushes its shares to HOST2. HOST2 pushes its shares to HOST1.
# Never both pushing the same share — one server is always the truth holder.
# Mesh model: every node pushes every media share. rsync has no --delete so pushes are additive.
# arr_sync (union) ensures all arr libraries converge first. arr_cleanup removes true orphans.
# Any node can download content to any share — it propagates to all nodes on the next cycle.
# Adding HOST3: list every media share in HOST3_DAILY_SYNC_SHARES. No ownership to track.
# These shares use DEFAULT_RSYNC_OPTS — no profile entry needed.
# For shares needing custom options or container stops — create a profile in RSYNC section.
@@ -398,11 +416,16 @@
CONTAINER_DELAY=5 # seconds before starting delayed containers
EXCLUDE_DIRS=() # directories excluded from transfer — profiles override
# --inplace writes directly to destination — better for large files
# --no-whole-file forces delta transfer — sends only changed blocks
# --inplace writes directly to destination — delta against existing file, better for large media
# --partial keep partial file on interrupted transfer so next run resumes, not re-transfers
# --timeout kill stalled transfers instead of hanging indefinitely
# --numeric-ids use UIDs/GIDs numerically — prevents ownership mismatches between servers
# --delete intentionally omitted — arr_cleanup.sh enforces media truth post-rsync.
# Media shares use this default; rsync spreads files only, never removes them.
DEFAULT_RSYNC_OPTS=(-av --info=progress2 --human-readable --bwlimit="$BW_LIMIT" --inplace --no-whole-file)
# Note: --no-whole-file removed — redundant over SSH (delta transfer is already the default).
# Note: arr_cleanup only catches true orphans under the union model — intentional removals
# require arr_sync.sh --blocklist-add first, then manual file deletion.
DEFAULT_RSYNC_OPTS=(-av --info=progress2 --human-readable --bwlimit="$BW_LIMIT" --inplace --partial --timeout=60 --numeric-ids)
# ━━━ Remote Health Checks ━━━
# Pre-flight — aborts if remote rootfs (/) usage is at or above this percentage.