From ee086f309ee810c37f00781db5bcade6a97051eb Mon Sep 17 00:00:00 2001 From: Gmer4Lfe Date: Fri, 22 May 2026 22:22:21 -0400 Subject: [PATCH] =?UTF-8?q?rename=20appdata-Failover=E2=86=92appdata-Fallb?= =?UTF-8?q?ack=20in=20docs;=20critical=20sync=2015=E2=86=9230min;=20transc?= =?UTF-8?q?ode=203=E2=86=927min?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit appdata-Failover → appdata-Fallback: All .md files updated — scripts and confs already used appdata-Fallback. Share was renamed; docs were the only stale references. critical_sync_maintenance.sh: */15 → */30, 96/day → 48/day: critical_sync_maintenance.sh header, master.conf, host1/2.conf comments, Orchestrators README, User Script Plug-in README, Partnership README/manual/script, Docker_Essentials README/downloaders_reset.sh, Fallback README, weekly_sync_maintenance.sh, user_script_plug-in.sh all updated. rsync_emby_failover User Script superseded: Emby dirty sync was already in HOST1_CRITICAL_SYNC_SHARES — it now runs inside critical_sync_maintenance.sh every 30min alongside auth sync and partnership check. Section in User Script Plug-in README marked for deletion from plugin. transcode_management.sh: */3 → */7: transcode_management.sh, transcode_manager.sh, transcode_cleanup.sh headers, Transcodes README/manual, Orchestrators README, User Script Plug-in README, user_script_plug-in.sh all updated. --- Configurations/host1.conf | 6 +- Configurations/host2.conf | 4 +- Configurations/master.conf | 18 ++--- Docker_Essentials/README-Docker_Essentials.md | 6 +- Docker_Essentials/downloaders_reset.sh | 4 +- Fallback/Manual-Fallback.md | 14 ++-- Fallback/README-Fallback.md | 2 +- Orchestrators/README-Orchestrators.md | 28 ++++---- Orchestrators/critical_sync_maintenance.sh | 16 ++--- Orchestrators/transcode_management.sh | 6 +- Orchestrators/weekly_sync_maintenance.sh | 2 +- Partnership/Manual-Partnership.md | 8 +-- Partnership/README-Partnership.md | 10 +-- Partnership/partnership_manager.sh | 10 +-- README-User_Script_Plug-in.md | 71 ++++++------------- README.md | 18 ++--- Rsync/Manual-Rsync.md | 22 +++--- Transcodes/Manual-Transcoding.md | 6 +- Transcodes/README-Transcoding.md | 6 +- Transcodes/transcode_cleanup.sh | 4 +- Transcodes/transcode_manager.sh | 4 +- user_script_plug-in.sh | 20 +++--- 22 files changed, 129 insertions(+), 156 deletions(-) diff --git a/Configurations/host1.conf b/Configurations/host1.conf index f77e87e..172909b 100644 --- a/Configurations/host1.conf +++ b/Configurations/host1.conf @@ -23,7 +23,7 @@ # ── RSYNC ────────────────────────────────────────────────────────────────────────────────── # DAILY SYNC SHARES media shares HOST1 owns and pushes to HOST2 # WEEKLY SYNC SHARES appdata shares synced weekly (Sunday 2:30am) -# CRITICAL SYNC SHARES appdata shares synced every 15 minutes +# CRITICAL SYNC SHARES appdata shares synced every 30 minutes # BACKUP VERIFY shares for checksum verification against remote # HOST1 RSYNC PROFILE host1-appdata profile for HOST1-specific appdata syncs # @@ -221,7 +221,7 @@ ) # ━━━ Critical Sync Shares ━━━ -# Appdata shares synced every 15 minutes by critical_sync_maintenance.sh. +# Appdata shares synced every 30 minutes by critical_sync_maintenance.sh. # Format: "/path/to/share" or "/path/to/share|profile-name" # Order matters — Critical-Data first (auth stack), then Emby dirty sync. HOST1_CRITICAL_SYNC_SHARES=( @@ -558,7 +558,7 @@ # Add one entry per root folder in arr Settings → Media Management → Root Folders. # ━━━ Downloaders ━━━ -# Used by downloaders_reset.sh — runs every 15min via CRITICAL_MAINTENANCE_SCRIPTS. +# Used by downloaders_reset.sh — runs every 30min via CRITICAL_MAINTENANCE_SCRIPTS. # Clears stuck states, purges old history, prepares each client for a clean cycle. # slskd — clears stuck searches, dead transfers, purges expired failed imports. diff --git a/Configurations/host2.conf b/Configurations/host2.conf index 481caf9..f775a97 100644 --- a/Configurations/host2.conf +++ b/Configurations/host2.conf @@ -27,7 +27,7 @@ # ── RSYNC ────────────────────────────────────────────────────────────────────────────────── # DAILY SYNC SHARES media shares HOST2 owns and pushes to HOST1 # WEEKLY SYNC SHARES appdata shares synced weekly (Sunday 2:30am) -# CRITICAL SYNC SHARES appdata shares synced every 15 minutes +# CRITICAL SYNC SHARES appdata shares synced every 30 minutes # BACKUP VERIFY shares for checksum verification against remote # HOST2 RSYNC PROFILE host2-appdata profile for HOST2-specific appdata syncs # @@ -204,7 +204,7 @@ ) # ━━━ Critical Sync Shares ━━━ -# Appdata shares synced every 15 minutes by critical_sync_maintenance.sh. +# Appdata shares synced every 30 minutes by critical_sync_maintenance.sh. # Format: "/path/to/share" or "/path/to/share|profile-name" HOST2_CRITICAL_SYNC_SHARES=( # fill in when HOST2 is back online diff --git a/Configurations/master.conf b/Configurations/master.conf index 9bef021..943e481 100644 --- a/Configurations/master.conf +++ b/Configurations/master.conf @@ -40,7 +40,7 @@ # MONTHLY MAINTENANCE Uptime-triggered heavy tasks — ZFS scrub, SMART tests (monthly_maintenance.sh) # DAILY SYNC MAINTENANCE Job list + media shares (daily_sync_maintenance.sh) # WEEKLY SYNC MAINTENANCE Job list + sync shares + update toggles (weekly_sync_maintenance.sh) -# CRITICAL SYNC MAINTENANCE 15-minute jobs + sync shares + partnership check (critical_sync_maintenance.sh) +# CRITICAL SYNC MAINTENANCE 30-minute jobs + sync shares + partnership check (critical_sync_maintenance.sh) # # ── RSYNC ────────────────────────────────────────────────────────────────────────────────── # RSYNC ENABLE/DISABLE Two-tier toggle system — global gate + per-orchestrator @@ -154,7 +154,7 @@ # /boot/config/partnership_HOST2.db ← HOST2 writes only # Propagated via SSH — no rsync needed # -# critical_sync_maintenance.sh runs --check every 15min: +# critical_sync_maintenance.sh runs --check every 30min: # Reads both state files via SSH # Detects offboard requests → finalises from owner side # Increments offline counter → auto-offboards after threshold @@ -330,15 +330,15 @@ MONTHLY_LAST_RUN_FILE="/boot/config/monthly_maintenance_last_run.db" # ━━━ Critical Sync Maintenance ━━━ -# critical_sync_maintenance.sh runs every 15 minutes. +# critical_sync_maintenance.sh runs every 30 minutes. # Order: CRITICAL_MAINTENANCE_SCRIPTS (jobs) → CRITICAL_SYNC_SHARES (rsync) → partnership --check # partnership --check always runs last regardless of rsync gate. # Comment out entries to disable without removing. CRITICAL_MAINTENANCE_SCRIPTS=( - "Docker_Essentials/downloaders_reset.sh" # clear stuck download states every 15min + "Docker_Essentials/downloaders_reset.sh" # clear stuck download states every 30min ) -# Shares synced every 15 minutes — defined per host in host*.conf. +# Shares synced every 30 minutes — defined per host in host*.conf. # HOST1_CRITICAL_SYNC_SHARES / HOST2_CRITICAL_SYNC_SHARES # Format: "/path/to/share" or "/path/to/share|profile-name" # Order matters — Critical-Data first (auth stack), then Emby dirty sync. @@ -442,7 +442,7 @@ # RSYNC_ENABLED=true ← rsync works, individual scripts run fine # DAILY_RSYNC_ENABLED=false ← skip daily HDD syncs during rebuild # WEEKLY_RSYNC_ENABLED=true ← Emby + Critical-Data still sync (NVMe) -# CRITICAL_RSYNC_ENABLED=true ← 15min auth stack sync still runs +# CRITICAL_RSYNC_ENABLED=true ← 30min auth stack sync still runs # FALLBACK_RSYNC_ENABLED=true ← handback writeback still works # → Run individual: bash Rsync/rsync.sh /mnt/user/Movies # → When ready: DAILY_RSYNC_ENABLED=true @@ -493,14 +493,14 @@ # critical-data — auth stack — full stop both sides, Authelia delayed start # called by weekly_sync_maintenance.sh — full clean sync weekly # critical-fallback — dirty sync — auth stays running both sides, WAL excluded -# called by critical_sync_maintenance.sh every 15min +# called by critical_sync_maintenance.sh every 30min # host1-appdata — HOST1 server-specific appdata — defined in host1.conf # host2-appdata — HOST2 server-specific appdata — defined in host2.conf # important-data — NextCloud + Postgres — NextCloud delayed start after Postgres # emby — weekly clean sync — both Emby stopped, full mirror # called by weekly_sync_maintenance.sh only — do NOT schedule separately # emby-fallback — dirty sync — Emby stays running, WAL excluded -# called by critical_sync_maintenance.sh every 15min +# called by critical_sync_maintenance.sh every 30min declare -A PROFILE_RSYNC_OPTS=( [arrs_stack]="-av --info=progress2 --human-readable --bwlimit=$BW_LIMIT --delete --inplace" @@ -632,7 +632,7 @@ # ============================================================================================== # ━━━ Downloaders Reset ━━━ -# Runs every 15 minutes via CRITICAL_MAINTENANCE_SCRIPTS. +# Runs every 30 minutes via CRITICAL_MAINTENANCE_SCRIPTS. # Clears stuck states, purges old history, prepares each download client for a clean cycle. # Per-host URLs and API keys live in host*.conf. DOWNLOADER_RETENTION_DAYS=7 # days — purge history older than this diff --git a/Docker_Essentials/README-Docker_Essentials.md b/Docker_Essentials/README-Docker_Essentials.md index f54543a..eb7afe7 100644 --- a/Docker_Essentials/README-Docker_Essentials.md +++ b/Docker_Essentials/README-Docker_Essentials.md @@ -197,7 +197,7 @@ array start. Silent when everything is correct. Notifies when it has to create s ### 🧹 Downloader Hygiene — `downloaders_reset.sh` -Maintenance reset for all download clients (slskd, SABnzbd, qBittorrent) every 15 +Maintenance reset for all download clients (slskd, SABnzbd, qBittorrent) every 30 minutes. Clears stuck searches, dead transfers, failed imports, and stale queue entries that download clients accumulate but never clean up themselves. Never touches active or in-progress downloads. @@ -233,7 +233,7 @@ full coordination model between all four watchdogs. | `docker_update_remaining.sh` | Image update + prune for all containers not in managed lists | End of weekly maintenance window | | `docker_network_connect.sh` | Network existence + container connection enforcement | Every array start | | `docker_container_stop.sh` | Ordered container shutdown — graceful then forced | Called by `array_stopping.sh` | -| `downloaders_reset.sh` | Download client hygiene — slskd / SABnzbd / qBittorrent | Every 15min via `critical_sync_maintenance.sh` | +| `downloaders_reset.sh` | Download client hygiene — slskd / SABnzbd / qBittorrent | Every 30min via `critical_sync_maintenance.sh` | > `watchdog_skip_list_manager.sh` — manual recovery tool for the skip list. Lives in `Tools/` because it's an operator utility, not a lifecycle script. @@ -262,7 +262,7 @@ Weekly maintenance window (2:30am Sunday): ├── docker_update.sh --remainder ─ update containers not in managed lists └── docker_update_remaining.sh ── prune dangling images -Critical maintenance window (every 15min): +Critical maintenance window (every 30min): critical_sync_maintenance.sh └── downloaders_reset.sh ──────── clear stuck downloads / stale queue entries diff --git a/Docker_Essentials/downloaders_reset.sh b/Docker_Essentials/downloaders_reset.sh index d832b71..af18902 100644 --- a/Docker_Essentials/downloaders_reset.sh +++ b/Docker_Essentials/downloaders_reset.sh @@ -9,7 +9,7 @@ # state that download clients generate but never clean up themselves — stuck # searches, dead transfers, failed imports, stale queue entries, completed history. # -# Called every 15 minutes by critical_sync_maintenance.sh via +# Called every 30 minutes by critical_sync_maintenance.sh via # CRITICAL_MAINTENANCE_SCRIPTS. Can also be run manually for ad hoc cleanup. # If a downloader is not configured for this host, that section skips cleanly. # @@ -137,7 +137,7 @@ if [[ "$EUID" -ne 0 ]]; then exit 1 fi -# Lock first — wait mode since this runs every 15min and previous may still be finishing +# Lock first — wait mode since this runs every 30min and previous may still be finishing acquire_lock "wait" # detect_hosts() sets MY_ID and aliases all HOST*_SLSKD_*, HOST*_SABNZBD_*, HOST*_QBIT_* vars diff --git a/Fallback/Manual-Fallback.md b/Fallback/Manual-Fallback.md index 4fc0f14..973bdb6 100644 --- a/Fallback/Manual-Fallback.md +++ b/Fallback/Manual-Fallback.md @@ -242,16 +242,16 @@ HOST1_TIER1_WRITEBACK_DELAY=60 # Writeback paths — synced back to HOST1 on handback FALLBACK_HOST1_WRITEBACK_TIER1=( - "/mnt/user/appdata-Failover/Critical-Data" # auth stack — Authelia + NPM + certs + "/mnt/user/appdata-Fallback/Critical-Data" # auth stack — Authelia + NPM + certs "/mnt/user/Media_Server/Emby" # Emby userdata — watch history, playstates ) FALLBACK_HOST1_WRITEBACK_TIER2=( - "/mnt/user/appdata-Failover/Important-Data" # NextCloud + Postgres + Immich + "/mnt/user/appdata-Fallback/Important-Data" # NextCloud + Postgres + Immich ) FALLBACK_HOST1_WRITEBACK_TIER3=( - "/mnt/user/appdata-Failover/Gmer4Lfe" # secondary appdata accumulated changes + "/mnt/user/appdata-Fallback/Gmer4Lfe" # secondary appdata accumulated changes ) # Tier 4 writeback uses HOST1_DAILY_SYNC_SHARES automatically — no list needed here. @@ -355,15 +355,15 @@ ssh root@[HOST2-tailscale-ip] "docker inspect Emby --format '{{.State.Status}}'" These shares must exist on HOST2 with current data from HOST1 before failover is needed: ``` -/mnt/user/appdata-Failover/Critical-Data # auth stack — NPM, LLDAP, Authelia, certs -/mnt/user/appdata-Failover/Important-Data # NextCloud + Postgres + Immich +/mnt/user/appdata-Fallback/Critical-Data # auth stack — NPM, LLDAP, Authelia, certs +/mnt/user/appdata-Fallback/Important-Data # NextCloud + Postgres + Immich /mnt/user/Media_Server/Emby # Emby userdata — watch history, playstates -/mnt/user/appdata-Failover/Gmer4Lfe # server appdata +/mnt/user/appdata-Fallback/Gmer4Lfe # server appdata ``` ```bash # Verify data is current — check modification times -ssh root@[HOST2-tailscale-ip] "ls -la /mnt/user/appdata-Failover/Critical-Data/" +ssh root@[HOST2-tailscale-ip] "ls -la /mnt/user/appdata-Fallback/Critical-Data/" ``` Sync is maintained continuously by `daily_sync_maintenance.sh` critical-data profile. diff --git a/Fallback/README-Fallback.md b/Fallback/README-Fallback.md index ee7c289..d623bb0 100644 --- a/Fallback/README-Fallback.md +++ b/Fallback/README-Fallback.md @@ -102,7 +102,7 @@ know the full library — arr_sync kept them in union while both servers were on NPM, LLDAP, and Authelia run actively on both servers at all times. HOST2 needs them running to serve his own users daily — this is not a fallback-only configuration. HOST1 is source of -truth: all changes mirror to HOST2 every 15 minutes via critical sync. +truth: all changes mirror to HOST2 every 30 minutes via critical sync. When DNS cuts over, auth is already running on the covering server. The 30-60 second dead zone where auth is coming up after DNS has already switched does not exist. diff --git a/Orchestrators/README-Orchestrators.md b/Orchestrators/README-Orchestrators.md index 20962fb..9aad197 100644 --- a/Orchestrators/README-Orchestrators.md +++ b/Orchestrators/README-Orchestrators.md @@ -49,7 +49,7 @@ tells you which job and what happened — no correlation needed. ### 🔴 Transcode Manager Triggering Unnecessary SSD Flips -`transcode_manager.sh` ran every 3 minutes on its own. It checked ramdisk usage — +`transcode_manager.sh` ran every 7 minutes on its own. It checked ramdisk usage — saw 6.8GB used, threshold is 6.5GB, flipped sessions to SSD. One minute later `transcode_cleanup.sh` ran and removed 4GB of stale segment files from ended sessions. Actual usage was 2.8GB. Sessions were now on SSD for no reason. Users experiencing @@ -102,14 +102,14 @@ The User Scripts plugin contains exactly these entries: array_started.sh # Cron — one entry per maintenance window: -*/3 * * * * transcode_management.sh +*/7 * * * * transcode_management.sh 0 */6 * * * arrs_failed_stalled_recovery.sh -*/30 * * * * rsync.sh emby-fallback ← not an orchestrator, direct call +*/30 * * * * critical_sync_maintenance.sh ← auth + Emby dirty sync + partnership 0 1 * * * daily_sync_maintenance.sh 30 2 * * 0 weekly_sync_maintenance.sh # Manual only (not scheduled): -failover_test.sh, emby_database_repair.sh, repair tools +fallback_test.sh, emby_database_repair.sh, repair tools ``` Every job list is configured in the `ORCHESTRATORS` section of `master.conf`. @@ -168,7 +168,7 @@ without `--log`. |--------|---------------------|----------| | `array_started.sh` | All array startup scripts in order | At Startup of Array | | `watchdog_orchestrator.sh` | resource → docker → system → stability watchdogs | Every minute | -| `transcode_management.sh` | Cleanup then manager — order critical | Every 3 minutes | +| `transcode_management.sh` | Cleanup then manager — order critical | Every 7 minutes | | `arrs_failed_stalled_recovery.sh` | Failed import + stalled download recovery | Every 6 hours | | `daily_sync_maintenance.sh` | git pull → sync → media maintenance → restarts | 1am daily | | `weekly_sync_maintenance.sh` | Stop → update → clean sync → start → weekly restarts | 2:30am Sunday | @@ -292,11 +292,11 @@ array_started.sh --status ## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Runs `transcode_cleanup.sh` then `transcode_manager.sh` in the correct order every -3 minutes. Exists because the order is not optional — the manager must always see +7 minutes. Exists because the order is not optional — the manager must always see post-cleanup usage to make accurate flip decisions. ```bash -# Scheduled: */3 * * * * (every 3 minutes) +# Scheduled: */7 * * * * (every 7 minutes) ``` --- @@ -371,7 +371,7 @@ post-cleanup usage to make accurate flip decisions. ### ── Usage ─────────────────────────────────────────────────────────────────── ```bash -transcode_management.sh # normal run (every 3 min via cron) +transcode_management.sh # normal run (every 7 min via cron) transcode_management.sh --dry-run # passes --dry-run to both child scripts transcode_management.sh --status # show config, current state, daily log stats transcode_management.sh --log # verbose output from both child scripts @@ -730,7 +730,7 @@ maintenance block before the 7am coffee report. # ───────────────────────────────────────────────────────────────────────────── # Two Emby syncs run in parallel — dirty and clean: # -# emby-fallback dirty sync (every 30 minutes, Emby running): +# emby-fallback dirty sync (every 30 minutes via critical_sync_maintenance.sh, Emby running): # watch states, library deltas, user activity — continuous coverage # WAL files excluded — safe to copy while Emby writes # HOST2 always within 30 minutes of HOST1 on playback state @@ -814,7 +814,7 @@ maintenance block before the 7am coffee report. # master.conf WEEKLY_SYNC_SHARES=( "/mnt/user/Media_Server/Emby" # emby profile — full clean mirror - "/mnt/user/appdata-Failover/Critical-Data" # critical-data — auth stack clean state + "/mnt/user/appdata-Fallback/Critical-Data" # critical-data — auth stack clean state ) WEEKLY_MAINTENANCE_SCRIPTS=( @@ -943,14 +943,14 @@ media_management.sh --status # show configured job list and exit array_started.sh # ───────────────────────────────────────────────────────────────────────────── -# Frequent — every 3 minutes: +# Frequent — every 7 minutes: # ───────────────────────────────────────────────────────────────────────────── -*/3 * * * * transcode_management.sh +*/7 * * * * transcode_management.sh # ───────────────────────────────────────────────────────────────────────────── -# Periodic — not an orchestrator, direct rsync call: +# Every 30 minutes — auth stack + Emby dirty sync + partnership check: # ───────────────────────────────────────────────────────────────────────────── -*/30 * * * * rsync.sh /mnt/user/Media_Server/Emby --profile=emby-fallback +*/30 * * * * critical_sync_maintenance.sh # ───────────────────────────────────────────────────────────────────────────── # Every 6 hours: diff --git a/Orchestrators/critical_sync_maintenance.sh b/Orchestrators/critical_sync_maintenance.sh index 8605455..06a86e3 100644 --- a/Orchestrators/critical_sync_maintenance.sh +++ b/Orchestrators/critical_sync_maintenance.sh @@ -2,9 +2,9 @@ # ============================================================================================== # ============================= Critical Sync Maintenance ====================================== # ============================================================================================== -# Orchestrator for time-sensitive syncs that run every 15 minutes. +# Orchestrator for time-sensitive syncs that run every 30 minutes. # Keeps the mirror current between the less frequent daily and weekly windows. -# Schedule: */15 * * * * (every 15 minutes via User Scripts plugin) +# Schedule: */30 * * * * (every 30 minutes via User Scripts plugin) # # ── EXECUTION ORDER ─────────────────────────────────────────────────────────────────────────── # 1. Critical-Data rsync — auth stack, NPM config, certs (containers stopped both sides) @@ -12,8 +12,8 @@ # 3. CRITICAL_MAINTENANCE_SCRIPTS — any scripts configured for critical window # 4. partnership --check — read both state files, detect changes, act accordingly # -# ── WHY EVERY 15 MINUTES ────────────────────────────────────────────────────────────────────── -# Auth stack changes (new users, proxy rules, certs) propagate within 15min ✅ +# ── WHY EVERY 30 MINUTES ────────────────────────────────────────────────────────────────────── +# Auth stack changes (new users, proxy rules, certs) propagate within 30min ✅ # Emby watch states stay in sync — mirror users see correct playback position ✅ # Partnership state changes detected and acted on quickly ✅ # Lock prevents: daily rsync doing Critical-Data mid-critical window ✅ @@ -24,17 +24,17 @@ # partnership --check always runs regardless — state check doesn't need rsync # # ── LOCK BEHAVIOUR ──────────────────────────────────────────────────────────────────────────── -# acquire_lock "strict" — if previous 15min run still going, skip this cycle entirely -# Critical-Data taking > 15min is a problem worth knowing about +# acquire_lock "strict" — if previous 30min run still going, skip this cycle entirely +# Critical-Data taking > 30min is a problem worth knowing about # Strict mode prevents pile-up without waiting — log and move on ✅ # # ── SILENT WHEN HEALTHY ─────────────────────────────────────────────────────────────────────── -# Runs 96 times per day — clean runs must produce zero output ✅ +# Runs 48 times per day — clean runs must produce zero output ✅ # Only failures and notable events produce visible output # # ── CONFIGURATION (master.conf) ─────────────────────────────────────────────────────────────── # CRITICAL_RSYNC_ENABLED — enable/disable rsync section -# CRITICAL_SYNC_SHARES — shares synced every 15min (HOST*_CRITICAL_SYNC_SHARES) +# CRITICAL_SYNC_SHARES — shares synced every 30min (HOST*_CRITICAL_SYNC_SHARES) # CRITICAL_MAINTENANCE_SCRIPTS — scripts run in critical window (optional) # PARTNERSHIP_ENABLED — enable/disable partnership check # diff --git a/Orchestrators/transcode_management.sh b/Orchestrators/transcode_management.sh index 17e7e5a..1804228 100644 --- a/Orchestrators/transcode_management.sh +++ b/Orchestrators/transcode_management.sh @@ -4,7 +4,7 @@ # ============================================================================================== # Orchestrator — runs transcode_cleanup.sh then transcode_manager.sh in the correct order. # Replace individual transcode_manager and transcode_cleanup cron entries with this. -# Schedule: */3 * * * * (every 3 minutes via User Scripts plugin) +# Schedule: */7 * * * * (every 7 minutes via User Scripts plugin) # # ── WHY CLEANUP BEFORE MANAGER ──────────────────────────────────────────────────────────────── # Cleanup runs first — removes stale segment files from ended sessions. @@ -47,7 +47,7 @@ # All TRANSCODE_* threshold vars — see master.conf Transcode Manager section # # ── USAGE ───────────────────────────────────────────────────────────────────────────────────── -# transcode_management.sh — normal run (every 3 minutes via cron) +# transcode_management.sh — normal run (every 7 minutes via cron) # transcode_management.sh --dry-run — preview without changes (passed to children) # transcode_management.sh --status — show configuration and current state # transcode_management.sh --log — verbose output from both child scripts @@ -92,7 +92,7 @@ if [[ "$SHOW_STATUS" == true ]]; then echo "$ICON_RAM Ramdisk: $RAMDISK_PATH ($RAMDISK_SIZE)" echo "$ICON_DISK SSD fallback: $TRANSCODE_SSD" echo "$ICON_LINK Symlink: $TRANSCODE_LINK" - echo "$ICON_TIME Schedule: every 3 minutes" + echo "$ICON_TIME Schedule: every 7 minutes" echo "" echo "━━━ Child Scripts ━━━" [[ -f "$CLEANUP_SCRIPT" ]] && \ diff --git a/Orchestrators/weekly_sync_maintenance.sh b/Orchestrators/weekly_sync_maintenance.sh index d4d86a7..fba8f20 100644 --- a/Orchestrators/weekly_sync_maintenance.sh +++ b/Orchestrators/weekly_sync_maintenance.sh @@ -20,7 +20,7 @@ # Emby builds a warm image cache on HOST2 throughout the week. # Syncing nightly resets cache — cold loads every morning for users. # Weekly sync: cache stays warm 6 days, resets Sunday night while users sleep. -# emby-fallback dirty sync covers watch states + library every 15min between weekly syncs. +# emby-fallback dirty sync covers watch states + library every 30min between weekly syncs. # # ── CONTAINER UPDATES ───────────────────────────────────────────────────────────────────────── # Containers already stopped for sync — updates pull at zero extra downtime. diff --git a/Partnership/Manual-Partnership.md b/Partnership/Manual-Partnership.md index ffa762e..e489c83 100644 --- a/Partnership/Manual-Partnership.md +++ b/Partnership/Manual-Partnership.md @@ -18,20 +18,20 @@ NPM proxy rules NPM serving mirror's domain LLDAP users LLDAP — same users Authelia policies Authelia — same policies Certs Certs — mirrored, valid -Emby Emby — dirty-synced every 15min +Emby Emby — dirty-synced every 30min -Changes made here ──→ Propagated every 15 minutes +Changes made here ──→ Propagated every 30 minutes WebUI management ──→ Redirected to HOST1 via Tailscale Config → git push ──→ Received via git pull on next cycle ``` **Mirror's daily experience:** open unRAID Docker UI, click NginxProxyManager, browser opens HOST1's NPM WebUI automatically via Tailscale. Make a proxy rule -change. 15 minutes later it's live on HOST2. The mirror operator never manages +change. 30 minutes later it's live on HOST2. The mirror operator never manages auth directly — every WebUI redirect is transparent and automatic. **Owner's daily experience:** manage auth as normal. Opens NPM → adds a proxy -rule → 15min later live on both servers. Never needs to SSH to HOST2 or think +rule → 30min later live on both servers. Never needs to SSH to HOST2 or think about HOST2 during normal operation. --- diff --git a/Partnership/README-Partnership.md b/Partnership/README-Partnership.md index 4541237..03cbcfd 100644 --- a/Partnership/README-Partnership.md +++ b/Partnership/README-Partnership.md @@ -34,7 +34,7 @@ in one place and forgotten in the other. **The fix:** one auth stack with a managed mirror. HOST1 owns the configuration. HOST2 runs a warm copy that stays current via 15-minute sync. HOST2's operator makes zero auth management decisions — clicking an auth container opens HOST1's WebUI via -Tailscale. Changes happen there, propagate to HOST2 in 15 minutes. One place to +Tailscale. Changes happen there, propagate to HOST2 in 30 minutes. One place to manage everything for both households. --- @@ -107,7 +107,7 @@ Called by `partnership_manager.sh --offboard` but runnable directly. **Lifecycle management** (`partnership_manager.sh`) — dispatcher and monitor. - Manually: `--onboard`, `--offboard` (delegates to offboard script), `--transfer`, `--status`, `--unblock` -- Automatically: `--check` called every 15 minutes by `critical_sync_maintenance.sh` +- Automatically: `--check` called every 30 minutes by `critical_sync_maintenance.sh` **SSH management** (`ssh_setup.sh`) — generates the keypair for rsync automation, installs it on the remote, and tracks auth failures with a configurable strike system. @@ -119,7 +119,7 @@ re-keying. ## ━━━ RELATIONSHIP TO OTHER FOLDERS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ **Orchestrators/** — `critical_sync_maintenance.sh` calls `partnership_manager.sh --check` -every 15 minutes, passing `--remote-seen` or `--remote-unseen` based on whether the +every 30 minutes, passing `--remote-seen` or `--remote-unseen` based on whether the rsync to the partner succeeded. The rsync outcome is the connectivity signal — no separate ping needed. @@ -140,7 +140,7 @@ independently at 4-hour cadence. |--------|------|--------------| | `partnership_onboard.sh` | One-time setup — SSH keys, stack deploy, arr bootstrap | Manually, once per server per partnership | | `partnership_offboard.sh` | Clean separation — both paths, both roles | Via `partnership_manager.sh --offboard`; or directly | -| `partnership_manager.sh` | Dispatcher + monitor — onboard WebUIs, health check, transfer, status | `--check` every 15min; all other modes manually | +| `partnership_manager.sh` | Dispatcher + monitor — onboard WebUIs, health check, transfer, status | `--check` every 30min; all other modes manually | | `ssh_setup.sh` | SSH key generation, remote install, auth validation | Called by onboard; manually for re-keying or validation | --- @@ -166,7 +166,7 @@ HOST1 (owner) runs: ├─ partnership_manager.sh --onboard reconfigures WebUIs, writes ACTIVE state └─ arr_sync.sh bootstraps full library on both servers -ONGOING OPERATION (every 15min) +ONGOING OPERATION (every 30min) ───────────────────────────────────────────────────────────────────────────── critical_sync_maintenance.sh ├─ Critical-Data rsync keeps auth appdata current on mirror diff --git a/Partnership/partnership_manager.sh b/Partnership/partnership_manager.sh index 6cf8084..bec32f5 100644 --- a/Partnership/partnership_manager.sh +++ b/Partnership/partnership_manager.sh @@ -7,7 +7,7 @@ # ───────────────────────────────────────────────────────────────────────────── # Manages the full lifecycle of a two-server partnership — onboard, daily health # monitoring, offboard, and ownership transfer. Called by partnership_onboard.sh -# during initial setup, and by critical_sync_maintenance.sh every 15 minutes for +# during initial setup, and by critical_sync_maintenance.sh every 30 minutes for # the --check mode. All other modes are run manually. # # PARTNERSHIP_OWNER_HOST flips to "HOST2" after a successful --transfer. @@ -34,7 +34,7 @@ # Reconfigures both servers, flips PARTNERSHIP_OWNER_HOST in master.conf # Requires confirmation string + consecutive health check passes # -# --check (called every 15min by critical_sync_maintenance.sh) +# --check (called every 30min by critical_sync_maintenance.sh) # --remote-seen: rsync succeeded → reset offline counter, read remote state # --remote-unseen: rsync failed → increment offline counter → auto-offboard at threshold # Silent when healthy — never noisy on clean runs @@ -172,7 +172,7 @@ # Flip ownership — owner only. Requires exact confirmation string. # # partnership_manager.sh --check --remote-seen|--remote-unseen -# Called by critical_sync_maintenance.sh every 15min — do not run manually +# Called by critical_sync_maintenance.sh every 30min — do not run manually # # partnership_manager.sh --status # Show state files, blocklist, SSH key status from both servers @@ -1300,7 +1300,7 @@ fi # ============================================================================================== # ━━━ Check ━━━ -# Called every 15min by critical_sync_maintenance.sh — must be silent when healthy +# Called every 30min by critical_sync_maintenance.sh — must be silent when healthy # ============================================================================================== if [[ "$MODE" == "check" ]]; then @@ -1316,7 +1316,7 @@ if [[ "$MODE" == "check" ]]; then OFFLINE_COUNT=$(( OFFLINE_COUNT + 1 )) echo "$OFFLINE_COUNT" > "$OFFLINE_COUNTER" - # Auto-offboard threshold: threshold_days × 96 intervals/day (every 15min) + # Auto-offboard threshold: threshold_days × 48 intervals/day (every 30min) THRESHOLD_INTERVALS=$(( ${PARTNERSHIP_OFFLINE_THRESHOLD:-30} * 96 )) if [[ "$OFFLINE_COUNT" -ge "$THRESHOLD_INTERVALS" ]]; then warn "Remote offline for ${PARTNERSHIP_OFFLINE_THRESHOLD} days — triggering auto-offboard" diff --git a/README-User_Script_Plug-in.md b/README-User_Script_Plug-in.md index a287775..0e0f3bf 100644 --- a/README-User_Script_Plug-in.md +++ b/README-User_Script_Plug-in.md @@ -127,7 +127,7 @@ the array stops. Check with: `pgrep -f "stability_watchdog\|docker_watchdog\|fai ``` Category: Orchestrators Name: transcode_management -Schedule: */3 * * * * (every 3 minutes) +Schedule: */7 * * * * (every 7 minutes) Background: YES ``` @@ -138,7 +138,7 @@ Background: YES **What it does:** Runs transcode_cleanup.sh then transcode_manager.sh — in that exact order every -3 minutes. Order is non-negotiable: cleanup removes stale files first so the manager +7 minutes. Order is non-negotiable: cleanup removes stale files first so the manager sees real active session usage, not inflated numbers from ended sessions. **transcode_cleanup.sh:** @@ -167,7 +167,7 @@ auto-flip to SSD, missing transcoding-temp → recreate immediately. ``` Category: Orchestrators Name: critical_sync_maintenance -Schedule: */15 * * * * (every 15 minutes) +Schedule: */30 * * * * (every 30 minutes) Background: YES ``` @@ -177,13 +177,13 @@ Background: YES ``` **What it does:** -Keeps the auth stack and Emby watch state current on HOST2 — 15 minutes is the -maximum staleness for failover-critical data. +Keeps the auth stack and Emby watch state current on HOST2 — 30 minutes is the +maximum staleness for fallback-critical data. **Jobs in order:** ``` rsync Critical-Data — pushes auth stack to HOST2 (NPM rules, certs, LLDAP users, - Authelia config). HOST2's auth is always within 15min of HOST1. + Authelia config). HOST2's auth is always within 30min of HOST1. rsync Emby dirty sync — pushes watch states, user activity, library delta to HOST2 with Emby running. WAL files excluded — safe mid-write. HOST2 Emby restarts after sync to pick up config changes. @@ -191,7 +191,7 @@ partnership --check — reads remote state file, increments offline counter unreachable, triggers auto-offboard at PARTNERSHIP_OFFLINE_THRESHOLD ``` -**Silent when healthy:** no output on clean cycles — runs 96 times per day. +**Silent when healthy:** no output on clean cycles — runs 48 times per day. --- @@ -286,35 +286,14 @@ in one orchestrated sequence. ### ── rsync_emby_failover ───────────────────────────────────────────────────── -``` -Category: Orchestrators -Name: rsync_emby_failover -Schedule: */30 * * * * (every 30 minutes) -Background: YES -``` +> **Superseded by `critical_sync_maintenance`.** Delete this User Script from the +> plugin. The emby-fallback dirty sync now runs as step 2 inside +> `critical_sync_maintenance.sh` every 30 minutes alongside the auth stack sync +> and partnership check. Running both produces duplicate syncs. -```bash -#!/bin/bash -/mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \ - /mnt/user/Media_Server/Emby --profile=emby-fallback -``` - -**What it does:** -Keeps HOST2's Emby within 30 minutes of HOST1 on watch states, play positions, and -library changes — with Emby running on both sides. Not a full sync (that's weekly) — -this is continuous coverage of what matters for failover. - -**What syncs (emby-fallback profile):** -``` -Syncs: users.db, library.db, authentication.db, config/ -Skips: *.wal *.shm — WAL files being written by Emby, unsafe to copy mid-write - transcodes/ — volatile, never needed on HOST2 - logs/ — local diagnostics only - cache/ — rebuilt automatically -``` - -**HOST2 Emby restarts after each sync** to pick up config changes. -**Result:** if HOST1 goes down, users resume from a position at most 30min stale. +The Emby dirty sync (`/mnt/user/Media_Server/Emby --profile=emby-fallback`) is +configured in `HOST1_CRITICAL_SYNC_SHARES` in host1.conf. It runs automatically +when critical_sync_maintenance.sh runs. --- @@ -346,7 +325,7 @@ updates at zero extra downtime, weekly container restarts. 5. Pull images remotely — if WEEKLY_SYNC_UPDATES_REMOTE=true (SSH to remote) 6. rsync WEEKLY_SYNC_SHARES: /mnt/user/Media_Server/Emby — full clean mirror, databases checkpointed - /mnt/user/appdata-Failover/Critical-Data — auth stack clean state + /mnt/user/appdata-Fallback/Critical-Data — auth stack clean state 7. Start remote containers— dependency order, new image, settle + verify each 8. Start local containers — same 9. docker_weekly_restart.sh — restarts less-critical services that benefit from @@ -774,7 +753,7 @@ Background: YES ```bash #!/bin/bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \ - /mnt/user/appdata-Failover/Critical-Data + /mnt/user/appdata-Fallback/Critical-Data ``` **What syncs:** auth stack — NPM proxy rules, TLS certificates, LLDAP user data, @@ -816,7 +795,7 @@ Background: YES ```bash #!/bin/bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \ - /mnt/user/appdata-Failover/Arrs_Stack + /mnt/user/appdata-Fallback/Arrs_Stack ``` **What syncs:** all arr appdata — Sonarr, Radarr, Lidarr, Prowlarr, Bazarr, Pinchflat @@ -837,7 +816,7 @@ Background: YES ```bash #!/bin/bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \ - /mnt/user/appdata-Failover/Important-Data + /mnt/user/appdata-Fallback/Important-Data ``` **What syncs:** NextCloud, Postgres-NextCloud, Immich, PostgreSQL-Immich appdata. @@ -857,7 +836,7 @@ Background: YES ```bash #!/bin/bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \ - /mnt/user/appdata-Failover/Gmer4Lfe + /mnt/user/appdata-Fallback/Gmer4Lfe ``` **What syncs:** HOST1 server appdata backup — everything in the Gmer4Lfe appdata share. @@ -1175,7 +1154,7 @@ Current usage on both ramdisk and SSD. ``` Category: Transcodes Name: transcode_manager_manual -Schedule: None — run manually (runs via transcode_management.sh every 3 min) +Schedule: None — run manually (runs via transcode_management.sh every 7 min) Background: NO ``` @@ -1641,17 +1620,11 @@ Wrong startup setup (old approach): At Startup of Array: array_start BG: YES — single entry, handles everything -*/3 * * * * (every 3 min): +*/7 * * * * (every 7 min): transcode_management BG: YES — cleanup then manager -*/10 * * * * (every 10 min): - webgui_watchdog BG: YES — silent when healthy - -*/15 * * * * (every 15 min): - critical_sync_maintenance BG: YES — auth stack + Emby dirty + partnership check - */30 * * * * (every 30 min): - rsync_emby_failover BG: YES — Emby watch state sync + critical_sync_maintenance BG: YES — auth stack + Emby dirty sync + partnership 0 */6 * * * (every 6 hours): arrs_failed_stalled_recovery BG: YES — blocklist + re-search failed imports diff --git a/README.md b/README.md index 40b7728..d1187cb 100644 --- a/README.md +++ b/README.md @@ -243,14 +243,14 @@ where determines whether you need encryption and whether sync failure matters. # services to work correctly when they start on HOST2. # /mnt/user/Media_Server/Emby # Emby userdata — watch history, playstates - /mnt/user/appdata-Failover/Critical-Data # auth stack — NPM rules, certs, user accounts + /mnt/user/appdata-Fallback/Critical-Data # auth stack — NPM rules, certs, user accounts /mnt/user/Movies # media library content /mnt/user/Tv_Shows # media library content /mnt/user/Music # media library content # # Sync failure here = failover runs but with stale data (acceptable) or missing # data (problematic). These sync every night via daily_sync_maintenance.sh. -# Critical-Data and Emby also sync every 15 minutes via critical_sync_maintenance.sh. +# Critical-Data and Emby also sync every 30 minutes via critical_sync_maintenance.sh. # ───────────────────────────────────────────────────────────────────────────── ``` @@ -266,8 +266,8 @@ where determines whether you need encryption and whether sync failure matters. # Content is not sensitive — anyone with access to HOST2 could read it. # # Examples: - /mnt/user/appdata-Failover/Gmer4Lfe # Gmer4Lfe server appdata backup - /mnt/user/appdata-Failover/Important-Data # NextCloud, Immich appdata + /mnt/user/appdata-Fallback/Gmer4Lfe # Gmer4Lfe server appdata backup + /mnt/user/appdata-Fallback/Important-Data # NextCloud, Immich appdata /mnt/user/Kids_Movies # kids content — not sensitive /mnt/user/Sports # sports content — not sensitive # @@ -345,7 +345,7 @@ This is not standby. This is mutual coverage between peers. # # "Minimal disruption" does NOT mean: # Zero downtime (DDNS TTL = 1 minute, brief gap expected) -# Perfect consistency (Emby userdata is 15min stale at most) +# Perfect consistency (Emby userdata is 30min stale at most) # All services covered immediately (tiered start over 24 hours) # # The system is designed around the real experience of a household: @@ -363,7 +363,7 @@ This is not standby. This is mutual coverage between peers. ```bash # ───────────────────────────────────────────────────────────────────────────── # Things conventional failover provides that this does NOT: -# Zero RPO (recovery point objective) — there is ~15min of potential staleness +# Zero RPO (recovery point objective) — there is ~30min of potential staleness # Synchronous state replication — changes don't reach HOST2 instantly # Automatic failure detection in seconds — fallback.sh checks every 2 minutes # Transparent IP migration — DNS TTL means up to 1 minute of routing to old IP @@ -426,7 +426,7 @@ server's own domain. # # HOST1 is source of truth for the auth configuration. All changes to NPM proxy # rules, LLDAP users, Authelia policies, and certs happen on HOST1 and propagate -# to HOST2 via Critical-Data sync every 15 minutes. +# to HOST2 via Critical-Data sync every 30 minutes. # # HOST2 operator's experience: click NginxProxyManager in Docker UI → # browser opens HOST1's NPM via Tailscale redirect → make changes there → @@ -451,7 +451,7 @@ Always running: Source of truth: Media Movies, Tv_Shows, Music ←── Anime_Movies, Anime_Shows ───────────────────────→ (bidirectional daily push) - Config auth stack config ──→ mirrored every 15 min + Config auth stack config ──→ mirrored every 30 min Emby state watch history, playstates──→ dirty sync every 30 min Failover coverage: @@ -621,7 +621,7 @@ Throughout the day (every minute via watchdog_orchestrator.sh): Throughout the day: fallback.sh watching: remote server, internet connectivity transcode_management.sh managing: ramdisk ↔ SSD, session cleanup (every 3min) - critical_sync_maintenance.sh keeping: auth stack + Emby current (every 15min) + critical_sync_maintenance.sh keeping: auth stack + Emby current (every 30min) 1am: git pull (latest scripts first) diff --git a/Rsync/Manual-Rsync.md b/Rsync/Manual-Rsync.md index d0fe276..b867c23 100644 --- a/Rsync/Manual-Rsync.md +++ b/Rsync/Manual-Rsync.md @@ -261,7 +261,7 @@ HOST2_DAILY_SYNC_SHARES=( # Synced during the Sunday 2:30am window — containers stopped both sides. WEEKLY_SYNC_SHARES=( "/mnt/user/Media_Server/Emby" - "/mnt/user/appdata-Failover/Critical-Data" + "/mnt/user/appdata-Fallback/Critical-Data" ) ``` @@ -296,13 +296,13 @@ Override with `--profile=name`. ### Profile Matching ```bash -# rsync.sh /mnt/user/appdata-Failover/Arrs_Stack +# rsync.sh /mnt/user/appdata-Fallback/Arrs_Stack # basename: Arrs_Stack → lowercased: arrs_stack → matches [arrs_stack] profile # # rsync.sh /mnt/user/Movies # basename: Movies → no profile match → global defaults (no containers stopped) # -# rsync.sh /mnt/user/appdata-Failover/Critical-Data --profile=critical-data +# rsync.sh /mnt/user/appdata-Fallback/Critical-Data --profile=critical-data # explicit override ``` @@ -492,7 +492,7 @@ Test with `--dry-run` first — all pre-flight checks run, no changes made. ```bash bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \ - /mnt/user/appdata-Failover/Arrs_Stack --dry-run --log + /mnt/user/appdata-Fallback/Arrs_Stack --dry-run --log ``` Expected output (healthy): @@ -521,7 +521,7 @@ bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh -- ```bash bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \ - /mnt/user/appdata-Failover/Arrs_Stack --status + /mnt/user/appdata-Fallback/Arrs_Stack --status ``` --- @@ -619,13 +619,13 @@ ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@[HOST2-ip] "docker ps --format '{{.Nam ```bash # Profile key = directory basename lowercased -# /mnt/user/appdata-Failover/Arrs_Stack → key: arrs_stack +# /mnt/user/appdata-Fallback/Arrs_Stack → key: arrs_stack # Override explicitly: -bash rsync.sh /mnt/user/appdata-Failover/My_Stuff --profile=arrs_stack +bash rsync.sh /mnt/user/appdata-Fallback/My_Stuff --profile=arrs_stack # Verify what profile resolved: -bash rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --status +bash rsync.sh /mnt/user/appdata-Fallback/Arrs_Stack --status ``` --- @@ -705,7 +705,7 @@ stops, no bandwidth log entry. Safe to run at any time. ```bash rsync.sh /mnt/user/Movies --dry-run -rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --dry-run --log +rsync.sh /mnt/user/appdata-Fallback/Arrs_Stack --dry-run --log daily_sync_maintenance.sh --dry-run ``` @@ -716,7 +716,7 @@ used — then exits. No pre-flight checks, no rsync. Use to verify configuration loaded correctly. ```bash -rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --status +rsync.sh /mnt/user/appdata-Fallback/Arrs_Stack --status ``` ### --log @@ -730,6 +730,6 @@ Override profile selection. Bypasses basename inference. Use when the directory name doesn't match any profile key, or when testing a specific profile. ```bash -rsync.sh /mnt/user/appdata-Failover/Critical-Data --profile=critical-data +rsync.sh /mnt/user/appdata-Fallback/Critical-Data --profile=critical-data rsync.sh /mnt/user/Media_Server/Emby --profile=emby-fallback ``` diff --git a/Transcodes/Manual-Transcoding.md b/Transcodes/Manual-Transcoding.md index 589d3e7..97cd56b 100644 --- a/Transcodes/Manual-Transcoding.md +++ b/Transcodes/Manual-Transcoding.md @@ -26,11 +26,11 @@ All three scripts follow a two-tier output model: `echo` lines are always visibl **ramdisk_setup.sh** — one-shot at array start. Without `--log`, section headers and the final summary are visible. Per-step creation detail suppressed. -**transcode_manager.sh** — runs every 3 minutes. Without `--log`, only state +**transcode_manager.sh** — runs every 7 minutes. Without `--log`, only state transitions (flips, warnings, errors) and active session display are shown. When nothing changes, a single one-line confirmation is printed. Per-check detail suppressed. -**transcode_cleanup.sh** — runs every 3 minutes. Without `--log`, the cleanup +**transcode_cleanup.sh** — runs every 7 minutes. Without `--log`, the cleanup summary (files removed, space freed) is always visible. Per-file deletion detail suppressed. @@ -376,7 +376,7 @@ At Startup of Array (via array_started.sh in unRAID_Essentials/): ramdisk_setup.sh — creates ramdisk, symlink, transcoding-temp Must run BEFORE Emby starts -Every 3 minutes (via transcode_management.sh in Orchestrators/): +Every 7 minutes (via transcode_management.sh in Orchestrators/): 1. transcode_cleanup.sh — remove stale files, check flip-back 2. transcode_manager.sh — check usage, flip if needed, display sessions diff --git a/Transcodes/README-Transcoding.md b/Transcodes/README-Transcoding.md index e047b52..5bd9eb5 100644 --- a/Transcodes/README-Transcoding.md +++ b/Transcodes/README-Transcoding.md @@ -49,7 +49,7 @@ it at mount time. Both protections together prevent this permanently. **lsof Per File on a Live TV System** Early cleanup called `lsof filename` per file to check if anything had it open. On a busy Live TV night with 5 simultaneous streams, the ramdisk contains thousands of HLS segment -files — thousands of subprocess calls every 3 minutes. +files — thousands of subprocess calls every 7 minutes. Fix: lsof called once per location to build a complete open-file map. All subsequent checks are O(1) lookups against that map. @@ -82,7 +82,7 @@ unRAID_Essentials/ Orchestrators/ transcode_management.sh ──── cleanup first ──► transcode_cleanup.sh ──── then manager ──► transcode_manager.sh - (every 3 minutes — order non-negotiable) + (every 7 minutes — order non-negotiable) Monitors/ weekly_health_digest.sh ◄─── reads ──────────── TRANSCODE_DAILY_LOG @@ -121,7 +121,7 @@ Emby starts, reads transcode path from config All new sessions write to: /mnt/ram-transcode → /mnt/ramdisk_transcodes/ -Every 3 minutes (transcode_management.sh): +Every 7 minutes (transcode_management.sh): │ ├─ transcode_cleanup.sh │ Remove files older than TRANSCODE_MAX_AGE, not open by any process diff --git a/Transcodes/transcode_cleanup.sh b/Transcodes/transcode_cleanup.sh index b699fd2..3b98f99 100644 --- a/Transcodes/transcode_cleanup.sh +++ b/Transcodes/transcode_cleanup.sh @@ -20,7 +20,7 @@ # # lsof Called Once, Not Per File # On a busy Live TV system the ramdisk contains thousands of HLS segment files. -# Calling lsof once per file creates thousands of subprocess calls every 3 minutes. +# Calling lsof once per file creates thousands of subprocess calls every 7 minutes. # lsof is called once per location to build a complete open-file map. All subsequent # checks are O(1) lookups against that map — thousands of files, one lsof call. # @@ -59,7 +59,7 @@ # `! -name "transcoding-temp"` in the find command — protected unconditionally. # # Silent by Default -# Runs every 3 minutes — must not produce noise when healthy. +# Runs every 7 minutes — must not produce noise when healthy. # # ============================================================================================== # CONFIGURATION diff --git a/Transcodes/transcode_manager.sh b/Transcodes/transcode_manager.sh index b6d75b7..3bb94c2 100644 --- a/Transcodes/transcode_manager.sh +++ b/Transcodes/transcode_manager.sh @@ -6,7 +6,7 @@ # PURPOSE # ───────────────────────────────────────────────────────────────────────────── # Monitors ramdisk usage and manages the transcode symlink direction. Called by -# transcode_management.sh (Orchestrators/) every 3 minutes — always after +# transcode_management.sh (Orchestrators/) every 7 minutes — always after # transcode_cleanup.sh runs first. Must be fast, non-blocking, and silent when # nothing has changed. # @@ -59,7 +59,7 @@ # validate_unraid_cmd confirms the notify script is present before use. # # Silent by Default -# Runs every 3 minutes — only speaks when something changes or needs attention. +# Runs every 7 minutes — only speaks when something changes or needs attention. # # ============================================================================================== # CONFIGURATION diff --git a/user_script_plug-in.sh b/user_script_plug-in.sh index fff46f3..5c2f24e 100644 --- a/user_script_plug-in.sh +++ b/user_script_plug-in.sh @@ -17,8 +17,8 @@ # array_started.sh at array start — launches ALL startup scripts in order # array_stopping.sh at array stop — graceful shutdown sequence # watchdog_orchestrator.sh every minute — resource + docker + system watchdog -# transcode_management.sh every 3 min — cleanup then manager (order critical) -# critical_sync_maintenance.sh every 15 min — auth + Emby dirty sync + partnership +# transcode_management.sh every 7 min — cleanup then manager (order critical) +# critical_sync_maintenance.sh every 30 min — auth + Emby dirty sync + partnership # intermediate_sync_maintenance.sh every 4 hours — arr library sync + artwork fetch # daily_sync_maintenance.sh 1am daily — git + rsync + media + restart # rsync.sh --profile=emby-fallback every 30 min — Emby watch state dirty sync @@ -174,7 +174,7 @@ # ── TRANSCODE MANAGEMENT ────────────────────────────────────────────────────────────────────── -# Schedule: */3 * * * * (every 3 minutes) +# Schedule: */7 * * * * (every 7 minutes) # Background: YES # # Cleanup first, then manager — order is non-negotiable. Stale segment files are removed @@ -202,15 +202,15 @@ # ── CRITICAL SYNC MAINTENANCE ───────────────────────────────────────────────────────────────── -# Schedule: */15 * * * * (every 15 minutes) +# Schedule: */30 * * * * (every 30 minutes) # Background: YES # -# Keeps failover-critical data current on HOST2. 15 minutes is the maximum staleness +# Keeps fallback-critical data current on HOST2. 30 minutes is the maximum staleness # for the auth stack and Emby watch states. Run on HOST1 — bidirectional via detect_hosts(). # # rsync Critical-Data: # Auth stack to HOST2: NPM proxy rules, TLS certs, LLDAP user accounts, -# Authelia config and policies. HOST2 auth always within 15min of HOST1. +# Authelia config and policies. HOST2 auth always within 30min of HOST1. # # rsync Emby dirty sync: # Watch states, user activity, library delta — Emby stays running on both sides. @@ -221,7 +221,7 @@ # Reads remote state file. Increments offline counter on failed sync. # Triggers auto-offboard at PARTNERSHIP_OFFLINE_THRESHOLD (30 days) unreachable. # -# Silent on clean cycles: runs 96 times per day, no output when healthy. +# Silent on clean cycles: runs 48 times per day, no output when healthy. # # bash /mnt/user/appdata/unraid_scripts/Orchestrators/critical_sync_maintenance.sh @@ -684,7 +684,7 @@ # bash /mnt/user/appdata/unraid_scripts/Docker_Essentials/docker_update_remaining.sh # downloaders_reset.sh — maintenance reset for all download clients on this server -# Called every 15 minutes by critical_sync_maintenance.sh via CRITICAL_MAINTENANCE_SCRIPTS. +# Called every 30 minutes by critical_sync_maintenance.sh via CRITICAL_MAINTENANCE_SCRIPTS. # Host-aware: sections skip gracefully if a downloader URL is empty for this server. # # slskd: @@ -1297,13 +1297,13 @@ # * * * * * every minute: # Orchestrators/watchdog_orchestrator.sh # -# */3 * * * * every 3 minutes: +# */7 * * * * every 7 minutes: # Orchestrators/transcode_management.sh # # */10 * * * * every 10 minutes: # Watchdogs/System/webgui_watchdog.sh # -# */15 * * * * every 15 minutes: +# */30 * * * * every 30 minutes: # Orchestrators/critical_sync_maintenance.sh # # */30 * * * * every 30 minutes: