refactor: rename failover/HA → fallback across entire codebase

Removes all references to "failover" and "HA" (high availability)
terminology from variable names, config keys, state values, rsync
profile names, directory paths, and user-visible strings.

Mapping:
  FAILOVER_*              → FALLBACK_*
  FAILOVER_HOST*_RUNS_FOR → FALLBACK_HOST*_COVERS
  critical-failover       → critical-fallback
  emby-failover           → emby-fallback
  appdata-Failover/       → appdata-Fallback/
  "FAILOVER" state value  → "FALLBACK"
  failover_start key      → fallback_start
  Failover/ directory     → Fallback/
  failover.sh             → fallback.sh
  failover_state.db       → fallback_state.db
  -Failover folder suffix → -Fallback

State machine: NORMAL | FALLBACK | DARK (unchanged)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Gmer4Lfe
2026-05-08 19:28:21 -04:00
co-authored by Claude Sonnet 4.6
parent 4e22f5d1f7
commit 009820e981
19 changed files with 517 additions and 416 deletions
+28 -28
View File
@@ -19,7 +19,7 @@
# critical_sync_maintenance.sh every 15 min — auth + Emby dirty sync + partnership
# arrs_failed_stalled_recovery.sh every 6 hours — blocklist + re-search failed imports
# daily_sync_maintenance.sh 1am daily — git + rsync + media + restart
# rsync.sh --profile=emby-failover every 30 min — Emby watch state dirty sync
# rsync.sh --profile=emby-fallback every 30 min — Emby watch state dirty sync
# weekly_sync_maintenance.sh 2:30am Sunday — clean sync + image updates
# sunday_morning_coffee_report.sh 7am Sunday — full weekly digest
# weekly_health_digest.sh 8am daily — profile-controlled health notification
@@ -100,7 +100,7 @@
# docker_network_connect.sh ensure custom networks exist and containers are connected
# system_watchdog.sh [continuous] three-tier server stability watchdog
# docker_watchdog.sh [continuous] two-tier container self-healing watchdog
# failover.sh [continuous] mutual failover state machine
# fallback.sh [continuous] mutual fallback state machine
#
# bash /mnt/user/appdata/unraid_scripts/Orchestrators/array_start.sh
@@ -216,7 +216,7 @@
# Result: if HOST1 fails, users resume from at most 30 minutes stale.
#
# bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
# /mnt/user/Media_Server/Emby --profile=emby-failover
# /mnt/user/Media_Server/Emby --profile=emby-fallback
# ── WEEKLY SYNC MAINTENANCE ───────────────────────────────────────────────────────────────────
@@ -284,7 +284,7 @@
# bandwidth event exceeded BANDWIDTH_WARN_GB
# weekly notify once per week on DIGEST_DAY (Sunday), silent all other days
#
# Data sources: failover state, watchdog strike files, container skip list,
# Data sources: fallback state, watchdog strike files, container skip list,
# bandwidth log, transcode daily log, cert expiry (live openssl check)
#
# bash /mnt/user/appdata/unraid_scripts/Monitors/weekly_health_digest.sh
@@ -378,9 +378,9 @@
# FAILOVER
# ──────────────────────────────────────────────────────────────────────────────────────────────
# failover.sh — [continuous] mutual failover state machine
# fallback.sh — [continuous] mutual fallback state machine
# Started by array_start.sh on both servers independently.
# Every FAILOVER_CHECK_INTERVAL (120s) pings: remote Tailscale IP + 8.8.8.8
# Every FALLBACK_CHECK_INTERVAL (120s) pings: remote Tailscale IP + 8.8.8.8
# States: NORMAL / FAILOVER / NO_INTERNET / DARK
# FAILOVER: starts remote containers in tiers across 24 hours:
# Tier 1 immediate: DDNS + Emby + auth (NPM/Authelia/LLDAP) + VaultWarden + Live TV
@@ -391,33 +391,33 @@
# DDNS: one domain, one DDNS container active at all times. Split-brain prevented by design.
# Handback: stop remote DDNS → rsync writeback → start containers → start local DDNS.
#
# bash /mnt/user/appdata/unraid_scripts/Failover/failover.sh
# bash /mnt/user/appdata/unraid_scripts/Failover/failover.sh --status
# bash /mnt/user/appdata/unraid_scripts/Failover/failover.sh --dry-run
# bash /mnt/user/appdata/unraid_scripts/Fallback/fallback.sh
# bash /mnt/user/appdata/unraid_scripts/Fallback/fallback.sh --status
# bash /mnt/user/appdata/unraid_scripts/Fallback/fallback.sh --dry-run
# failover_test.sh — controlled full-lifecycle failover test using iptables simulation
# fallback_test.sh — controlled full-lifecycle fallback test using iptables simulation
# ALWAYS run --dry-run first. Live test starts/stops real containers — brief service interruption.
# iptables safety trap removes the block rule on ANY exit including ctrl-c and crashes.
# Remote is always reachable after the test completes regardless of what happened.
# 8 phases: pre-flight → block remote (iptables) → detect FAILOVER → verify Tier 1 started
# → restore remote → wait handback → verify NORMAL restored → full pass/fail report.
# FAILOVER_TEST_BLOCK_WAIT must be > FAILOVER_CHECK_INTERVAL + buffer (default: 150s).
# FAILOVER_TEST_HANDBACK_WAIT must cover: strike confirmation + rsync + container start (default: 360s).
# FALLBACK_TEST_BLOCK_WAIT must be > FALLBACK_CHECK_INTERVAL + buffer (default: 150s).
# FALLBACK_TEST_HANDBACK_WAIT must cover: strike confirmation + rsync + container start (default: 360s).
#
# bash /mnt/user/appdata/unraid_scripts/Failover/failover_test.sh --dry-run
# bash /mnt/user/appdata/unraid_scripts/Failover/failover_test.sh
# bash /mnt/user/appdata/unraid_scripts/Fallback/fallback_test.sh --dry-run
# bash /mnt/user/appdata/unraid_scripts/Fallback/fallback_test.sh
# failover_state_reset.sh — reset failover state file to NORMAL
# fallback_state_reset.sh — reset fallback state file to NORMAL
# Resets state file ONLY — does NOT start or stop any containers.
# Use when: state stuck in FAILOVER after testing, after killing failover.sh mid-cycle,
# Use when: state stuck in FAILOVER after testing, after killing fallback.sh mid-cycle,
# after manual intervention left state inconsistent, after a failed handback.
# Shows current state file before asking for confirmation.
# Verify before resetting: right containers on right server, DDNS correct,
# both servers Tailscale visible, failover.sh not currently running.
# both servers Tailscale visible, fallback.sh not currently running.
#
# bash /mnt/user/appdata/unraid_scripts/Failover/failover_state_reset.sh --status
# bash /mnt/user/appdata/unraid_scripts/Failover/failover_state_reset.sh
# bash /mnt/user/appdata/unraid_scripts/Failover/failover_state_reset.sh --force
# bash /mnt/user/appdata/unraid_scripts/Fallback/fallback_state_reset.sh --status
# bash /mnt/user/appdata/unraid_scripts/Fallback/fallback_state_reset.sh
# bash /mnt/user/appdata/unraid_scripts/Fallback/fallback_state_reset.sh --force
# ──────────────────────────────────────────────────────────────────────────────────────────────
@@ -436,21 +436,21 @@
# important-data Postgres-NextCloud + delayed: NextCloud
# arrs_stack Sonarr, Radarr, Lidarr, Prowlarr, Bazarr, Pinchflat
# emby Emby both sides (weekly clean sync — both instances stopped)
# emby-failover nothing stopped (Emby stays running — dirty sync, WAL/SHM excluded)
# emby-fallback nothing stopped (Emby stays running — dirty sync, WAL/SHM excluded)
# [no profile] no containers stopped (media shares, plain data)
#
# bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
# /mnt/user/appdata-Failover/Critical-Data
# /mnt/user/appdata-Fallback/Critical-Data
# bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
# /mnt/user/appdata-Failover/Important-Data
# /mnt/user/appdata-Fallback/Important-Data
# bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
# /mnt/user/appdata-Failover/Arrs_Stack
# /mnt/user/appdata-Fallback/Arrs_Stack
# bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
# /mnt/user/Media_Server/Emby --profile=emby
# bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
# /mnt/user/Media_Server/Emby --profile=emby-failover
# /mnt/user/Media_Server/Emby --profile=emby-fallback
# bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
# /mnt/user/appdata-Failover/Gmer4Lfe
# /mnt/user/appdata-Fallback/Gmer4Lfe
# bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/Movies
# bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/Tv_Shows
# bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/Music
@@ -897,7 +897,7 @@
# Running state, PID, uptime, approximate cycle count
# Active strikes per check. Required container states. Memory-monitored container usage.
# Skip list contents (any entry needs human attention).
# Failover state, active tiers, outage duration, remote Tailscale visibility.
# Fallback state, active tiers, outage duration, remote Tailscale visibility.
# No schedule — on demand only. Run any time you want a complete status snapshot.
#
# bash /mnt/user/appdata/unraid_scripts/Monitors/continuous_scripts_status.sh
@@ -1021,7 +1021,7 @@
# Orchestrators/critical_sync_maintenance.sh
#
# */30 * * * * every 30 minutes:
# Rsync/rsync.sh /mnt/user/Media_Server/Emby --profile=emby-failover
# Rsync/rsync.sh /mnt/user/Media_Server/Emby --profile=emby-fallback
#
# 0 */6 * * * every 6 hours:
# Orchestrators/arrs_failed_stalled_recovery.sh