FailedProxy/Varaverk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix widespread naming errors across all docs — array_started, fallback, FALLBACK_

Browse Source 
array_start.sh → array_started.sh (bulk rename — actual file has always been
array_started.sh; docs consistently used the wrong name across 9 files)

failover.sh → fallback.sh (bulk rename — script is Fallback/fallback.sh;
docs used the old name throughout)

FAILOVER_HOST2_RUNS_FOR_HOST1_TIER1 → FALLBACK_HOST2_COVERS_HOST1_TIER1
(README.md — wrong variable name, actual vars use FALLBACK_ prefix and
COVERS_ not RUNS_FOR_)

docker_watchdog.sh attribution: "started by array_started.sh" → "called every
minute by watchdog_orchestrator.sh" (docker_watchdog is not in ARRAY_START_SCRIPTS)

server_reboot.sh shutdown sequence: step 7 "rc.docker stop" → calls
array_stopping.sh (user_scripts, fallback, rsync, mover, docker_container_stop)

Rsync manual ARRAY_START_SCRIPTS list: removed stability_watchdog and
docker_watchdog (now in watchdog_orchestrator); failover → fallback
This commit is contained in:
Gmer4Lfe
2026-05-22 21:53:32 -04:00
parent 8d72fbf0e2
commit 070013cb4f
10 changed files with 46 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Fallback/Manual-Fallback.md
+1 -1
View File
@@ -591,7 +591,7 @@ errors are always visible regardless of `--log`.
### fallback.sh
`fallback.sh`
Normal start — continuous loop. Start via User Scripts plugin or array_start.sh. **Do NOT
Normal start — continuous loop. Start via User Scripts plugin or array_started.sh. **Do NOT
stop by killing the process** — state file may be left inconsistent. Stop via User Scripts
Abort only.
Fallback/README-Fallback.md
+2 -2
View File
@@ -242,7 +242,7 @@ movement required. The fallback and rsync scripts are configuration-driven.
| Script | Role | When It Runs |
|--------|------|--------------|
| `fallback.sh` | Continuous state machine — monitors remote, manages fallback and handback | Continuously (started by `array_start.sh`) |
| `fallback.sh` | Continuous state machine — monitors remote, manages fallback and handback | Continuously (started by `array_started.sh`) |
| `fallback_test.sh` | 7-phase test harness — validates the entire fallback lifecycle via iptables simulation | On demand — maintenance window only |
---
@@ -250,7 +250,7 @@ movement required. The fallback and rsync scripts are configuration-driven.
## ━━━ HOW THE SCRIPTS RELATE ━━━
```
array_start.sh
array_started.sh
│
└── starts fallback.sh (continuous loop)
│
Orchestrators/README-Orchestrators.md
+9 -9
View File
@@ -86,7 +86,7 @@ filter might run after containers have already created veth interfaces. PHP-FPM
tuning might run after the WebGUI has already served its first requests. Each
script competes for the same startup slot with no guaranteed order.
The fix: `array_start.sh` is the only "At Startup of Array" entry. It launches
The fix: `array_started.sh` is the only "At Startup of Array" entry. It launches
every startup script in a defined order, with one-second settle between each, and
reports which succeeded and which failed. Order is guaranteed. Nothing starts before
its dependency. Everything is visible in a single summary.
@@ -99,7 +99,7 @@ The User Scripts plugin contains exactly these entries:
```bash
# At Startup of Array — single entry for all startup scripts:
array_start.sh
array_started.sh
# Cron — one entry per maintenance window:
*/3 * * * * transcode_management.sh
@@ -178,7 +178,7 @@ without `--log`.
---
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
## 🚀 array_start.sh
## 🚀 array_started.sh
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Single "At Startup of Array" entry for the entire ecosystem. Launches every startup
@@ -265,16 +265,16 @@ ARRAY_START_SCRIPTS=(
```bash
# ─────────────────────────────────────────────────────────────────────────────
# Normal — called by User Scripts at array start. Never run manually in production.
# array_start.sh runs once and exits — the continuous scripts it launched
# array_started.sh runs once and exits — the continuous scripts it launched
# keep running as background processes.
# ─────────────────────────────────────────────────────────────────────────────
array_start.sh
array_started.sh
# ─────────────────────────────────────────────────────────────────────────────
# Dry run — show what would be launched, in order, without launching anything.
# Use to verify the ARRAY_START_SCRIPTS list before an array restart.
# ─────────────────────────────────────────────────────────────────────────────
array_start.sh --dry-run
array_started.sh --dry-run
# ─────────────────────────────────────────────────────────────────────────────
# Status — show each configured script with its current running state.
@@ -282,7 +282,7 @@ array_start.sh --dry-run
# not running — one-shot that has completed, or continuous not yet started
# FILE NOT FOUND — script path wrong or missing
# ─────────────────────────────────────────────────────────────────────────────
array_start.sh --status
array_started.sh --status
```
---
@@ -660,7 +660,7 @@ DAILY_MAINTENANCE_SCRIPTS=(
```bash
# ─────────────────────────────────────────────────────────────────────────────
# The same HOST*_DAILY_SYNC_SHARES lists are used by failover.sh for Tier 4
# The same HOST*_DAILY_SYNC_SHARES lists are used by fallback.sh for Tier 4
# writeback — but in the opposite direction.
#
# Normal (daily_sync_maintenance.sh):
@@ -896,7 +896,7 @@ media_management.sh --status # show configured job list and exit
# ─────────────────────────────────────────────────────────────────────────────
# At Startup of Array — single entry:
# ─────────────────────────────────────────────────────────────────────────────
array_start.sh
array_started.sh
# ─────────────────────────────────────────────────────────────────────────────
# Frequent — every 3 minutes:
README-User_Script_Plug-in.md
+9 -9
View File
@@ -95,7 +95,7 @@ Background: YES — required (starts continuous background processes)
```bash
#!/bin/bash
/mnt/user/appdata/unraid_scripts/Orchestrators/array_start.sh
/mnt/user/appdata/unraid_scripts/Orchestrators/array_started.sh
```
**What it does:**
@@ -113,7 +113,7 @@ ramdisk_setup.sh — creates /mnt/ramdisk_transcodes and symlink before
docker_network_connect.sh— ensures custom networks exist + containers connected
stability_watchdog.sh ← last-resort — three-tier server stability watchdog
docker_watchdog.sh ← continuous — two-tier container healing watchdog
failover.sh ← continuous — mutual failover state machine
fallback.sh ← continuous — mutual failover state machine
```
**After running:**
@@ -659,7 +659,7 @@ that survive rsync updates without parser changes.
## 🔄 FAILOVER
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
> **failover.sh is started by array_start.sh — not scheduled separately.**
> **fallback.sh is started by array_started.sh — not scheduled separately.**
> These entries exist for manual control and testing only.
---
@@ -869,10 +869,10 @@ No profile container stop required. Incremental — safe to run at any time.
## 🐳 DOCKER_ESSENTIALS
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
> **docker_watchdog.sh is started by array_start.sh — not scheduled separately.**
> **docker_watchdog.sh is called every minute by watchdog_orchestrator.sh — not scheduled separately.**
> docker_daily_restart.sh is called by daily_sync_maintenance.sh.
> docker_weekly_restart.sh is called by weekly_sync_maintenance.sh.
> docker_network_connect.sh is called by array_start.sh.
> docker_network_connect.sh is called by array_started.sh.
> These entries exist for manual control only.
---
@@ -969,7 +969,7 @@ Dispatcharr-Basic, ErsatzTV-Emby — services that degrade with uptime.
```
Category: Docker_Essentials
Name: docker_network_connect_manual
Schedule: None — run manually (runs automatically via array_start.sh)
Schedule: None — run manually (runs automatically via array_started.sh)
Background: NO
```
@@ -1143,7 +1143,7 @@ Host-aware: HOST1 manages Movies, HOST2 manages Anime_Movies.
## 🎬 TRANSCODES
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
> **ramdisk_setup.sh is called by array_start.sh — not scheduled separately.**
> **ramdisk_setup.sh is called by array_started.sh — not scheduled separately.**
> **transcode_manager.sh and transcode_cleanup.sh are called by transcode_management.sh.**
> These entries exist for manual status checks and emergency use.
@@ -1197,7 +1197,7 @@ flip count, session split ramdisk vs SSD).
> **stability_watchdog.sh is called by watchdog_orchestrator.sh — not scheduled separately.**
> **webgui_watchdog.sh, system_tuning_monitor.sh, clear_logs.sh are scheduled below.**
> **inotify_tuning.sh, php_fpm_max_children.sh, docker_syslog_filter.sh run via array_start.sh.**
> **inotify_tuning.sh, php_fpm_max_children.sh, docker_syslog_filter.sh run via array_started.sh.**
---
@@ -1616,7 +1616,7 @@ automatically at 1am as the first job in the window.
**Only one entry needed at startup:** `array_start`. It handles everything.
`array_start.sh` is the single "At Startup of Array" entry. It reads ARRAY_START_SCRIPTS
`array_started.sh` is the single "At Startup of Array" entry. It reads ARRAY_START_SCRIPTS
from master.conf and launches each in order. No other startup entries. No dragging to
reorder individual scripts. Change the order by editing master.conf.
README.md
+10 -10
View File
@@ -70,7 +70,7 @@ break silently.
Dispatcharr # shared Live TV scheduler
```
When `failover.sh` on HOST2 reads `FAILOVER_HOST2_RUNS_FOR_HOST1_TIER1=(..."Emby"...)`,
When `fallback.sh` on HOST2 reads `FALLBACK_HOST2_COVERS_HOST1_TIER1=(..."Emby"...)`,
it looks for a container called exactly `Emby` and starts it. HOST2 must have a
container called `Emby` — not `Emby-Gmer4Lfe`, not `Emby-HOST1`, just `Emby` — with
its volume mounts pointing at the mirrored share paths. This is what makes one
@@ -107,7 +107,7 @@ the suffix makes the owner unambiguous. Scripts targeting `Emby` and scripts tar
# The entire ecosystem uses container names as identifiers.
# docker_watchdog.sh: WATCHDOG_REQUIRED_CONTAINERS=("NginxProxyManager" "Authelia")
# → looks for exactly "NginxProxyManager" on the running server
# failover.sh: FAILOVER_HOST2_RUNS_FOR_HOST1_TIER1=("Emby" "VaultWarden")
# fallback.sh: FALLBACK_HOST2_COVERS_HOST1_TIER1=("Emby" "VaultWarden")
# → looks for exactly "Emby" and "VaultWarden" on HOST2
# docker_daily_restart.sh: DAILY_RESTART_CONTAINERS=("Dispatcharr" "Authelia")
# → looks for exactly those names in docker ps
@@ -365,7 +365,7 @@ This is not standby. This is mutual coverage between peers.
# Things conventional failover provides that this does NOT:
# Zero RPO (recovery point objective) — there is ~15min of potential staleness
# Synchronous state replication — changes don't reach HOST2 instantly
# Automatic failure detection in seconds — failover.sh checks every 2 minutes
# Automatic failure detection in seconds — fallback.sh checks every 2 minutes
# Transparent IP migration — DNS TTL means up to 1 minute of routing to old IP
#
# What you get instead:
@@ -380,7 +380,7 @@ This is not standby. This is mutual coverage between peers.
## ━━━ HOW FAILOVER ACTUALLY WORKS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
`failover.sh` runs on both servers as a continuous background process. Every 2 minutes
`fallback.sh` runs on both servers as a continuous background process. Every 2 minutes
it pings two things: the other server's Tailscale IP, and `8.8.8.8`. The combination
of those two answers determines the current state.
@@ -465,7 +465,7 @@ Failover coverage:
Monitoring:
Both servers resource/docker/system/ resource/docker/system/
stability_watchdog.sh stability_watchdog.sh
failover.sh failover.sh
fallback.sh fallback.sh
Sunday morning coffee report Sunday morning coffee report
```
@@ -558,13 +558,13 @@ Arbiter runs only orchestrators — never individual scripts directly.
```
At Startup of Array:
array_start.sh launches all startup scripts in order
array_started.sh launches all startup scripts in order
→ inotify_tuning.sh raise kernel limits before containers start
→ docker_syslog_filter.sh suppress veth noise before containers create interfaces
→ php_fpm_max_children.sh WebGUI tuning before first request
→ ramdisk_setup.sh create ramdisk before Emby starts
→ docker_network_connect.sh connect containers to extra networks
→ failover.sh continuous — mutual failover state machine
→ fallback.sh continuous — mutual failover state machine
Every minute:
watchdog_orchestrator.sh fires each watchdog in sequence
@@ -619,7 +619,7 @@ Throughout the day (every minute via watchdog_orchestrator.sh):
stability_watchdog.sh watching: RAM, CPU temp, rootfs, kernel, daemon
Throughout the day:
failover.sh watching: remote server, internet connectivity
fallback.sh watching: remote server, internet connectivity
transcode_management.sh managing: ramdisk ↔ SSD, session cleanup (every 3min)
critical_sync_maintenance.sh keeping: auth stack + Emby current (every 15min)
@@ -671,7 +671,7 @@ Problem: Docker daemon hung
→ If still hung: immediate reboot (no strikes needed — daemon can't be managed)
Problem: HOST1 loses power
→ failover.sh on HOST2 detects at next 2-minute check
→ fallback.sh on HOST2 detects at next 2-minute check
→ Tier 1 starts immediately: DDNS, Emby, VaultWarden, Live TV
→ Household barely notices. Live TV may buffer for < 2 minutes.
```
@@ -732,7 +732,7 @@ Things that are different from what you might expect:
# ─────────────────────────────────────────────────────────────────────────────
# 1. Arbiter has very few entries.
# Everything runs through orchestrators. Individual scripts are never
# scheduled directly. array_start.sh is the only "At Startup of Array" entry.
# scheduled directly. array_started.sh is the only "At Startup of Array" entry.
# 2. Scripts are silent when healthy.
# stability_watchdog.sh runs 288 times per day. It produces zero output on clean
Rsync/Manual-Rsync.md
+4 -3
View File
@@ -404,7 +404,7 @@ zfs change-key \
-o keyformat=raw \
poolname/Gmer4Lfe-Personal
# Add to array_start.sh or ramdisk_setup.sh:
# Add to array_started.sh or ramdisk_setup.sh:
zfs load-key poolname/Gmer4Lfe-Personal
zfs mount poolname/Gmer4Lfe-Personal
@@ -432,14 +432,15 @@ Individual scripts are never scheduled directly except the 30-minute Emby sync.
```bash
#!/bin/bash
bash /mnt/user/appdata/unraid_scripts/Orchestrators/array_start.sh
bash /mnt/user/appdata/unraid_scripts/Orchestrators/array_started.sh
# Schedule: At Startup of Array
# Run as: Background Task
```
This single entry launches everything defined in ARRAY_START_SCRIPTS from master.conf:
inotify_tuning, docker_syslog_filter, php_fpm_max_children, ramdisk_setup,
docker_network_connect, stability_watchdog, docker_watchdog, failover.
docker_network_connect, fallback.
Watchdogs (resource, docker, system, stability) run separately via watchdog_orchestrator.
### Cron Schedule
Transcodes/Manual-Transcoding.md
+2 -2
View File
@@ -179,7 +179,7 @@ ls /mnt/ramdisk_transcodes/
### Usage
```bash
ramdisk_setup.sh # normal run (called by array_start.sh)
ramdisk_setup.sh # normal run (called by array_started.sh)
ramdisk_setup.sh --dry-run # show what would be created without creating
ramdisk_setup.sh --status # show current ramdisk, symlink, and SSD state
ramdisk_setup.sh --log # verbose — show each creation step
@@ -372,7 +372,7 @@ TRANSCODE_SERVERS=(
## Schedule
```
At Startup of Array (via array_start.sh in unRAID_Essentials/):
At Startup of Array (via array_started.sh in unRAID_Essentials/):
ramdisk_setup.sh — creates ramdisk, symlink, transcoding-temp
Must run BEFORE Emby starts
Transcodes/README-Transcoding.md
+2 -2
View File
@@ -77,7 +77,7 @@ path is a symlink whose target is managed at runtime. Sessions in progress never
```
unRAID_Essentials/
array_start.sh ──────────────────────────────► ramdisk_setup.sh (at array start)
array_started.sh ──────────────────────────────► ramdisk_setup.sh (at array start)
Orchestrators/
transcode_management.sh ──── cleanup first ──► transcode_cleanup.sh
@@ -97,7 +97,7 @@ Both are called by `transcode_management.sh` in the correct order.
| Script | Role | When It Runs |
|--------|------|-------------|
| `ramdisk_setup.sh` | Create tmpfs, SSD fallback dir, symlink, transcoding-temp | At array start (via array_start.sh) |
| `ramdisk_setup.sh` | Create tmpfs, SSD fallback dir, symlink, transcoding-temp | At array start (via array_started.sh) |
| `transcode_cleanup.sh` | Remove stale files, check for flip-back opportunity | Every 3 min via transcode_management.sh — runs first |
| `transcode_manager.sh` | Check usage, flip symlink, safety checks, session display, daily log | Every 3 min via transcode_management.sh — runs second |
unRAID_Essentials/Manual-Unraid_Essentials.md
+6 -1
View File
@@ -414,7 +414,12 @@ user_scripts_stop.sh --dry-run # show what would be stopped
6. /etc/rc.d/rc.libvirt stop (VM Manager)
7. /etc/rc.d/rc.docker stop (all containers stop)
7. Orchestrators/array_stopping.sh — safe ordered array stop:
- user_scripts_stop.sh stop background User Scripts
- fallback.sh --stop graceful fallback teardown
- rsync_stop.sh --rsync-only kill active rsync transfers
- mover_stop.sh stop mover
- docker_container_stop.sh stop all containers gracefully
8. sync (flush filesystem buffers to disk)
unRAID_Essentials/README-Unraid_Essentials.md
+1 -1
View File
@@ -91,7 +91,7 @@ Graceful operations mover_stop.sh — clean mover stop
```
Orchestrators/
array_start.sh ─────────────────────────► inotify_tuning.sh (first in sequence)
array_started.sh ─────────────────────────► inotify_tuning.sh (first in sequence)
─────────────────────────► docker_syslog_filter.sh (second)
─────────────────────────► php_fpm_max_children.sh
weekly_maintenance.sh ──────────────────► clear_logs.sh