docs: add Manual.md — end-to-end setup guide
Step-by-step deployment guide: prerequisites, Tailscale, git clone, sparse checkout, master.conf/host*.conf config, SSH keys via ssh_setup.sh, Arbiter cron table, initial sync, fallback enable, partnership onboard, verification, and common first-run troubleshooting. Fixes the broken README-Rsync_Setup.md reference in README.md.
This commit is contained in:
@@ -0,0 +1,670 @@
|
|||||||
|
# ━━━━━ VARAVERK — Setup Manual ━━━━━
|
||||||
|
|
||||||
|
Getting a fresh two-server ecosystem running from scratch.
|
||||||
|
For system overview see [README.md](README.md). For individual subsystem detail see folder READMEs and script headers.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ━━━ BEFORE YOU START ━━━
|
||||||
|
|
||||||
|
Things that must be in place before you touch any scripts.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Two unRAID Servers ───────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
Both servers need to be up and have their arrays started. Hardware does not need to
|
||||||
|
match. The scripts work via `/mnt/user/` — the same share name on both servers resolves
|
||||||
|
correctly regardless of the hardware underneath.
|
||||||
|
|
||||||
|
Decide which server is **HOST1 (owner)** and which is **HOST2 (mirror)**. This is just a
|
||||||
|
label — both servers run their own full stack. The owner manages the shared auth
|
||||||
|
configuration. Pick the more capable or more reliably-online server as owner.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Plugins — install on both servers ────────────────────────────────────────
|
||||||
|
|
||||||
|
Install these via **Apps** (Community Applications) on both servers:
|
||||||
|
|
||||||
|
| Plugin | Why |
|
||||||
|
|--------|-----|
|
||||||
|
| **Tailscale** | Encrypted VPN between servers — all script traffic travels over it |
|
||||||
|
| **User Scripts** | Schedules the orchestrators (replaces per-script cron entries) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── CLI Tools — install on both servers ────────────────────────────────────
|
||||||
|
|
||||||
|
Required by multiple scripts. Install via **Tools → NerdPack** (install NerdPack first
|
||||||
|
from Apps if not present):
|
||||||
|
|
||||||
|
| Tool | Why |
|
||||||
|
|------|-----|
|
||||||
|
| `git` | Pulling script updates, version tracking |
|
||||||
|
| `jq` | JSON parsing — arr APIs, Tailscale API, Docker inspect |
|
||||||
|
| `curl` | API calls — arrs, Emby, Tailscale, Discord webhooks |
|
||||||
|
| `openssl` | Certificate monitoring (`cert_monitor.sh`) |
|
||||||
|
|
||||||
|
Verify after install:
|
||||||
|
```bash
|
||||||
|
git --version && jq --version && curl --version | head -1 && openssl version
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ━━━ STEP 1: TAILSCALE ━━━
|
||||||
|
|
||||||
|
Both servers must be visible to each other on the same Tailscale network.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Connect both servers ────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
1. Open the Tailscale plugin on HOST1. Log in with your Tailscale account.
|
||||||
|
2. Open the Tailscale plugin on HOST2. Log in with the **same account** — or have the
|
||||||
|
HOST2 owner share their machine into your tailnet (Tailscale → Share → invite by email).
|
||||||
|
3. Confirm both devices appear in your Tailscale admin panel.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Verify reachability ──────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
On HOST1:
|
||||||
|
```bash
|
||||||
|
# Get your Tailscale IP
|
||||||
|
tailscale ip -4
|
||||||
|
|
||||||
|
# Ping HOST2 by its Tailscale hostname (the unRAID server name, e.g. unRAID-Jayred365)
|
||||||
|
ping -c 3 unRAID-Jayred365
|
||||||
|
```
|
||||||
|
|
||||||
|
On HOST2 — same in reverse. Both pings must succeed before continuing.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Note the Tailscale hostnames ──────────────────────────────────────────────
|
||||||
|
|
||||||
|
The scripts use Tailscale DNS to resolve IPs — no hardcoded addresses anywhere.
|
||||||
|
For this to work, the Tailscale device name must match the unRAID server hostname exactly.
|
||||||
|
|
||||||
|
Check the unRAID hostname: **Settings → System Identification → Server Name**
|
||||||
|
Check the Tailscale device name: Tailscale admin panel → Machines
|
||||||
|
|
||||||
|
If they differ, rename the Tailscale device to match the unRAID hostname. Case matters.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# On each server — confirm the hostname the scripts will use
|
||||||
|
hostname
|
||||||
|
# Should return e.g. "unRAID-Gmer4Lfe"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ━━━ STEP 2: CLONE THE REPOSITORY ━━━
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Clone on HOST1 ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone <your_repo_url> /mnt/user/appdata/unraid_scripts
|
||||||
|
cd /mnt/user/appdata/unraid_scripts
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Clone on HOST2 ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone <your_repo_url> /mnt/user/appdata/unraid_scripts
|
||||||
|
cd /mnt/user/appdata/unraid_scripts
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Sparse checkout — keep credentials separate ────────────────────────────
|
||||||
|
|
||||||
|
`host1.conf` contains HOST1's SSH keys, API keys, and passwords.
|
||||||
|
`host2.conf` contains HOST2's equivalents. Neither server should hold the other's
|
||||||
|
credentials. Set up sparse checkout so each server only receives its own conf file.
|
||||||
|
|
||||||
|
**On HOST1** — exclude host2.conf:
|
||||||
|
```bash
|
||||||
|
cd /mnt/user/appdata/unraid_scripts
|
||||||
|
git sparse-checkout init --no-cone
|
||||||
|
git sparse-checkout set '/*' '!/Configurations/host2.conf'
|
||||||
|
git checkout
|
||||||
|
```
|
||||||
|
|
||||||
|
**On HOST2** — exclude host1.conf:
|
||||||
|
```bash
|
||||||
|
cd /mnt/user/appdata/unraid_scripts
|
||||||
|
git sparse-checkout init --no-cone
|
||||||
|
git sparse-checkout set '/*' '!/Configurations/host1.conf'
|
||||||
|
git checkout
|
||||||
|
```
|
||||||
|
|
||||||
|
Verify: `ls Configurations/` — each server should see `master.conf` and only its own `host*.conf`.
|
||||||
|
|
||||||
|
> If you are not using git (deploying manually) skip this and ensure you only copy
|
||||||
|
> each server's own host*.conf when deploying.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ━━━ STEP 3: CONFIGURE ━━━
|
||||||
|
|
||||||
|
Configuration lives in `Configurations/`. Three files:
|
||||||
|
- `master.conf` — shared, both servers see it — thresholds, toggles, schedules
|
||||||
|
- `host1.conf` — HOST1 only — credentials, share lists, container names
|
||||||
|
- `host2.conf` — HOST2 only — same structure, different values
|
||||||
|
|
||||||
|
Edit these with any text editor. Everything in the scripts is controlled through these
|
||||||
|
files — never edit scripts directly to change behaviour.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── 3a. master.conf — server hostnames (do this first) ─────────────────────
|
||||||
|
|
||||||
|
Open `Configurations/master.conf`. Find the HOST IDENTITIES section near the top:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
HOST1="unRAID-Gmer4Lfe"
|
||||||
|
HOST2="unRAID-Jayred365"
|
||||||
|
```
|
||||||
|
|
||||||
|
Replace these with your actual unRAID server hostnames. These must match exactly —
|
||||||
|
`detect_hosts()` compares the running server's hostname against these two values to
|
||||||
|
know which server it is on. Everything else in the ecosystem flows from this.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── 3b. host1.conf — HOST1 identity ────────────────────────────────────────
|
||||||
|
|
||||||
|
Open `Configurations/host1.conf`. Fill in the IDENTITY section:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
HOST1_SSH_KEY="/root/.ssh/gmer4lfe_rsync_automation" # path — ssh_setup.sh will create this
|
||||||
|
HOST1_OWNER="gmer4lfe" # short name, lowercase, no spaces
|
||||||
|
HOST1_OWNER_EMAIL="you@example.com" # for partnership notifications
|
||||||
|
```
|
||||||
|
|
||||||
|
The SSH key does not exist yet — `ssh_setup.sh` in Step 4 will create it and fill in
|
||||||
|
the path automatically. Leave the path as-is for now; it will be overwritten.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── 3c. host2.conf — HOST2 identity ────────────────────────────────────────
|
||||||
|
|
||||||
|
Open `Configurations/host2.conf`. Fill in the same fields with HOST2's values:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
HOST2_SSH_KEY="/root/.ssh/jayred365_rsync_automation"
|
||||||
|
HOST2_OWNER="jayred365"
|
||||||
|
HOST2_OWNER_EMAIL="them@example.com"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── 3d. Remaining configuration ────────────────────────────────────────────
|
||||||
|
|
||||||
|
The scripts will not break if the remaining fields are empty — they self-guard
|
||||||
|
when things are not configured. Fill in sections as you enable each feature. The
|
||||||
|
minimum set needed to get a working sync + fallback:
|
||||||
|
|
||||||
|
**In host1.conf (and host2.conf mirror):**
|
||||||
|
```bash
|
||||||
|
# Shares HOST1 pushes to HOST2 nightly
|
||||||
|
HOST1_DAILY_SYNC_SHARES=(
|
||||||
|
"/mnt/user/Movies"
|
||||||
|
"/mnt/user/Tv_Shows"
|
||||||
|
"/mnt/user/Music"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Shares synced every 30 minutes (auth config + Emby watch state)
|
||||||
|
HOST1_CRITICAL_SYNC_SHARES=(
|
||||||
|
"/mnt/user/appdata-Fallback/Critical-Data|critical-data"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Containers fallback.sh starts on HOST2 when HOST1 goes down (Tier 1 = immediate)
|
||||||
|
HOST2_FALLBACK_HOST1_COVERS_HOST2_TIER1=() # HOST2's own covers for HOST1, and vice versa
|
||||||
|
HOST1_FALLBACK_HOST2_COVERS_HOST1_TIER1=(
|
||||||
|
"Emby"
|
||||||
|
"VaultWarden"
|
||||||
|
"NginxProxyManager"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
For full configuration reference see `Manual-Fallback.md`, `Manual-Rsync.md`,
|
||||||
|
and `Manual-Watchdogs.md`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ━━━ STEP 4: SSH KEYS ━━━
|
||||||
|
|
||||||
|
The servers need passwordless SSH access to each other. `ssh_setup.sh` handles
|
||||||
|
key generation and remote installation in one step.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Run on HOST2 first ────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
The mirror runs first — it generates its own key and copies it to HOST1. You will be
|
||||||
|
prompted for HOST1's root password once (for the initial key copy). After that,
|
||||||
|
all SSH is keyless.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /mnt/user/appdata/unraid_scripts
|
||||||
|
bash Partnership/ssh_setup.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
When prompted, enter HOST1's root password. The script:
|
||||||
|
- Generates `/root/.ssh/jayred365_rsync_automation` (ed25519)
|
||||||
|
- Copies the public key to HOST1's `~/.ssh/authorized_keys`
|
||||||
|
- Updates `host2.conf` with the key path
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Run on HOST1 ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /mnt/user/appdata/unraid_scripts
|
||||||
|
bash Partnership/ssh_setup.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Enter HOST2's root password when prompted. Same steps, opposite direction.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Verify both directions ────────────────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# On HOST1 — SSH to HOST2 should succeed without a password
|
||||||
|
ssh -i /root/.ssh/gmer4lfe_rsync_automation root@unRAID-Jayred365 "hostname"
|
||||||
|
# Expected: unRAID-Jayred365
|
||||||
|
|
||||||
|
# On HOST2 — SSH to HOST1 should succeed without a password
|
||||||
|
ssh -i /root/.ssh/jayred365_rsync_automation root@unRAID-Gmer4Lfe "hostname"
|
||||||
|
# Expected: unRAID-Gmer4Lfe
|
||||||
|
```
|
||||||
|
|
||||||
|
If either fails:
|
||||||
|
```bash
|
||||||
|
# Check status and fingerprints
|
||||||
|
bash Partnership/ssh_setup.sh --status
|
||||||
|
|
||||||
|
# Re-run with --force to regenerate and re-copy
|
||||||
|
bash Partnership/ssh_setup.sh --force
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Tailscale SSH note ──────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
The scripts SSH via Tailscale hostnames, not local IPs. The first connection to a
|
||||||
|
new host requires accepting the host key. Accept it now so scripts never block on an
|
||||||
|
interactive prompt:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# On HOST1
|
||||||
|
ssh -o StrictHostKeyChecking=accept-new root@unRAID-Jayred365 "echo ok"
|
||||||
|
|
||||||
|
# On HOST2
|
||||||
|
ssh -o StrictHostKeyChecking=accept-new root@unRAID-Gmer4Lfe "echo ok"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ━━━ STEP 5: ARBITER — USER SCRIPTS SETUP ━━━
|
||||||
|
|
||||||
|
Arbiter (the User Scripts plugin) is the only thing Arbiter runs — never individual
|
||||||
|
scripts directly. Each entry is an orchestrator that calls everything in its window.
|
||||||
|
|
||||||
|
Open **Settings → User Scripts** in the unRAID WebGUI.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Required entries (both servers) ────────────────────────────────────────
|
||||||
|
|
||||||
|
Create one User Script entry for each row. Set the schedule in the cron field and
|
||||||
|
paste the command. Name the entry to match the script name.
|
||||||
|
|
||||||
|
| Script | Cron schedule | Command |
|
||||||
|
|--------|--------------|---------|
|
||||||
|
| `array_started` | At Array Start | `bash /mnt/user/appdata/unraid_scripts/Orchestrators/array_started.sh` |
|
||||||
|
| `watchdog_orchestrator` | `* * * * *` | `bash /mnt/user/appdata/unraid_scripts/Orchestrators/watchdog_orchestrator.sh` |
|
||||||
|
| `transcode_management` | `*/7 * * * *` | `bash /mnt/user/appdata/unraid_scripts/Orchestrators/transcode_management.sh` |
|
||||||
|
| `critical_sync_maintenance` | `*/30 * * * *` | `bash /mnt/user/appdata/unraid_scripts/Orchestrators/critical_sync_maintenance.sh` |
|
||||||
|
| `emby_fallback_sync` | `*/30 * * * *` | `bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/Media_Server/Emby --profile=emby-fallback` |
|
||||||
|
| `intermediate_sync_maintenance` | `0 */4 * * *` | `bash /mnt/user/appdata/unraid_scripts/Orchestrators/intermediate_sync_maintenance.sh` |
|
||||||
|
| `daily_sync_maintenance` | `0 1 * * *` | `bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh` |
|
||||||
|
| `weekly_sync_maintenance` | `30 2 * * 0` | `bash /mnt/user/appdata/unraid_scripts/Orchestrators/weekly_sync_maintenance.sh` |
|
||||||
|
| `weekly_health_digest` | `0 8 * * *` | `bash /mnt/user/appdata/unraid_scripts/Monitors/weekly_health_digest.sh` |
|
||||||
|
|
||||||
|
> `array_started` uses the built-in "At Array Start" schedule option in User Scripts —
|
||||||
|
> it is not a cron expression.
|
||||||
|
|
||||||
|
> `emby_fallback_sync` is the only script with a direct Arbiter entry. It is separate
|
||||||
|
> from `critical_sync_maintenance` because it needs its own 30-minute slot and Emby
|
||||||
|
> must stay running during this sync (dirty sync — no container stop).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Schedule note ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
`transcode_management` and `critical_sync_maintenance` both run every 7 and 30 minutes
|
||||||
|
respectively. In User Scripts, concurrent runs of the same script are prevented by the
|
||||||
|
lock system (`acquire_lock`) — if the previous run is still active the new one exits
|
||||||
|
immediately. No need to stagger them manually.
|
||||||
|
|
||||||
|
For the complete schedule and what each orchestrator runs, see
|
||||||
|
[README-Orchestrators.md](Orchestrators/README-Orchestrators.md).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ━━━ STEP 6: INITIAL SYNC ━━━
|
||||||
|
|
||||||
|
Before enabling fallback, HOST2 needs a copy of HOST1's data. This first sync
|
||||||
|
will take a while depending on library size.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Dry run first ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# On HOST1 — preview what would sync, check paths and profiles
|
||||||
|
bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh --dry-run --log
|
||||||
|
```
|
||||||
|
|
||||||
|
Review the output. Check that:
|
||||||
|
- Share paths resolve correctly on both sides
|
||||||
|
- No pre-flight errors (disk space, rootfs, connectivity)
|
||||||
|
- Profiles look correct for each share
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Run the initial sync ─────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# On HOST1 — run the full daily sync (this includes rsync for all DAILY_SYNC_SHARES)
|
||||||
|
bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh --log
|
||||||
|
```
|
||||||
|
|
||||||
|
This will take longer than future nightly runs — it is transferring everything for
|
||||||
|
the first time. Monitor progress via the User Scripts output or:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Watch transfer in real time
|
||||||
|
watch -n5 'ls -lh /mnt/user/Movies/ | tail -5'
|
||||||
|
```
|
||||||
|
|
||||||
|
After it finishes, also run the critical sync to populate auth and Emby state:
|
||||||
|
```bash
|
||||||
|
bash /mnt/user/appdata/unraid_scripts/Orchestrators/critical_sync_maintenance.sh --log
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ━━━ STEP 7: ENABLE FALLBACK ━━━
|
||||||
|
|
||||||
|
`fallback.sh` is disabled by default — it should only be enabled when both servers
|
||||||
|
are correctly synced and the data on HOST2 is current.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Enable in master.conf ────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
FALLBACK_ENABLED=true
|
||||||
|
```
|
||||||
|
|
||||||
|
Commit and pull on both servers so both pick up the change.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Fallback starts automatically on array start ────────────────────────────
|
||||||
|
|
||||||
|
`array_started.sh` launches `fallback.sh` as a background process. Once
|
||||||
|
`FALLBACK_ENABLED=true` is in master.conf, restarting the array starts it.
|
||||||
|
|
||||||
|
To start it now without a reboot:
|
||||||
|
```bash
|
||||||
|
bash /mnt/user/appdata/unraid_scripts/Fallback/fallback.sh &
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Verify it is running ──────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pgrep -a -f fallback.sh
|
||||||
|
# Should show the running process
|
||||||
|
```
|
||||||
|
|
||||||
|
Check current state:
|
||||||
|
```bash
|
||||||
|
cat /boot/config/fallback_state.db
|
||||||
|
# state=NORMAL — both servers up
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ━━━ STEP 8: TEST FAILOVER ━━━
|
||||||
|
|
||||||
|
Before relying on the system, confirm it actually triggers. `fallback_test.sh`
|
||||||
|
simulates an outage using `iptables` — no real downtime, no real data changes.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# On HOST2 — dry run first (sees the sequence, no container starts)
|
||||||
|
bash /mnt/user/appdata/unraid_scripts/Fallback/fallback_test.sh --dry-run --log
|
||||||
|
|
||||||
|
# When ready — real test (uses iptables to simulate HOST1 unreachable)
|
||||||
|
bash /mnt/user/appdata/unraid_scripts/Fallback/fallback_test.sh --log
|
||||||
|
```
|
||||||
|
|
||||||
|
The test runs in phases — blocks HOST1's Tailscale IP, waits for `fallback.sh` to
|
||||||
|
detect it and start Tier 1 containers, then unblocks and waits for handback.
|
||||||
|
|
||||||
|
See [Manual-Fallback.md](Fallback/Manual-Fallback.md) for what each phase does and
|
||||||
|
how to interpret the output.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ━━━ STEP 9: PARTNERSHIP ONBOARD ━━━
|
||||||
|
|
||||||
|
The partnership is the optional layer that shares HOST1's auth stack with HOST2 —
|
||||||
|
NPM, Authelia, LLDAP, MariaDB, Redis all warm on HOST2, managed centrally from HOST1.
|
||||||
|
|
||||||
|
Skip this step if HOST2 does not need HOST1's auth stack (it runs fully independent auth).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Prerequisites ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
Before running partnership onboard:
|
||||||
|
|
||||||
|
- Steps 1–8 complete on both servers
|
||||||
|
- HOST1's auth containers are running and healthy
|
||||||
|
- `PARTNERSHIP_AUTH_STACK` in `host1.conf` lists the auth container XML templates
|
||||||
|
- `PARTNERSHIP_ARR_STACK` in `host1.conf` lists arr container XML templates (if sharing arrs)
|
||||||
|
- FolderView3 plugin installed on both servers (if using `PARTNERSHIP_FOLDERVIEW3=true`)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Run on HOST2 first (mirror) ────────────────────────────────────────────
|
||||||
|
|
||||||
|
HOST2 runs first — it generates its SSH key and waits. The owner completes setup remotely.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# On HOST2
|
||||||
|
bash /mnt/user/appdata/unraid_scripts/Partnership/partnership_onboard.sh --dry-run --log
|
||||||
|
# Review output, then:
|
||||||
|
bash /mnt/user/appdata/unraid_scripts/Partnership/partnership_onboard.sh --log
|
||||||
|
```
|
||||||
|
|
||||||
|
HOST2's work is done after Step 1 (SSH key setup). The rest happens on HOST1.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Run on HOST1 (owner) ────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# On HOST1
|
||||||
|
bash /mnt/user/appdata/unraid_scripts/Partnership/partnership_onboard.sh --dry-run --log
|
||||||
|
# Review output — this will deploy auth + arr stacks to HOST2 remotely
|
||||||
|
bash /mnt/user/appdata/unraid_scripts/Partnership/partnership_onboard.sh --log
|
||||||
|
```
|
||||||
|
|
||||||
|
The owner path deploys containers to HOST2 via SSH, configures WebUI targets, runs
|
||||||
|
an initial arr library sync, and writes the partnership state files on both servers.
|
||||||
|
|
||||||
|
For full detail on what each step does see
|
||||||
|
[Manual-Partnership.md](Partnership/Manual-Partnership.md).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ━━━ STEP 10: VERIFY THE ECOSYSTEM ━━━
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Check script output ────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
Run the main orchestrators manually with `--log` to see verbose output:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Watchdogs — should find everything healthy
|
||||||
|
bash Orchestrators/watchdog_orchestrator.sh --log
|
||||||
|
|
||||||
|
# Daily sync — dry run to confirm share paths and profiles
|
||||||
|
bash Orchestrators/daily_sync_maintenance.sh --dry-run --log
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Check fallback state ────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cat /boot/config/fallback_state.db
|
||||||
|
# Expected:
|
||||||
|
# state=NORMAL
|
||||||
|
# fallback_start=0
|
||||||
|
# handback_strikes=0
|
||||||
|
# tier2_started=false
|
||||||
|
# tier3_started=false
|
||||||
|
# tier4_started=false
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Check sync logs ─────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
Bandwidth monitor records every sync:
|
||||||
|
```bash
|
||||||
|
# Latest transfer log
|
||||||
|
ls -lt /mnt/user/appdata/unraid_scripts/data/bandwidth_monitor/
|
||||||
|
cat /mnt/user/appdata/unraid_scripts/data/bandwidth_monitor/<latest_log>
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Run status on key scripts ─────────────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash Fallback/fallback.sh --status
|
||||||
|
bash Partnership/ssh_setup.sh --status
|
||||||
|
bash Monitors/cert_monitor.sh --status
|
||||||
|
bash Watchdogs/resource_watchdog.sh --status
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ━━━ COMMON FIRST-RUN ISSUES ━━━
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Script exits silently / does nothing ────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run with --log to see verbose output
|
||||||
|
bash <script> --log
|
||||||
|
|
||||||
|
# Check if the script is skipping due to a toggle being false
|
||||||
|
grep "ENABLED\|_TOGGLE\|ENABLED=" Configurations/master.conf | grep "false"
|
||||||
|
```
|
||||||
|
|
||||||
|
Most scripts have an `ENABLED=true/false` toggle in `master.conf`. Check that the
|
||||||
|
feature you are testing is enabled.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── SSH connection refused or times out ────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Verify Tailscale is running and the remote is reachable
|
||||||
|
tailscale status
|
||||||
|
ping -c 3 <remote_hostname>
|
||||||
|
|
||||||
|
# Verify the key is authorised on the remote
|
||||||
|
ssh -v -i /root/.ssh/<your_key> root@<remote_hostname> "echo ok"
|
||||||
|
|
||||||
|
# Re-run ssh_setup.sh if the key is not installed
|
||||||
|
bash Partnership/ssh_setup.sh --force
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── detect_hosts() exits — MY_ID unknown ────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# The running server's hostname doesn't match HOST1 or HOST2 in master.conf
|
||||||
|
hostname
|
||||||
|
# Compare to master.conf
|
||||||
|
grep "^ HOST[12]=" Configurations/master.conf
|
||||||
|
# They must match exactly — case sensitive
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Fallback triggers immediately on startup ────────────────────────────────
|
||||||
|
|
||||||
|
The remote server is unreachable on the first check. Common causes:
|
||||||
|
- Tailscale not running on the remote
|
||||||
|
- Remote server is actually offline
|
||||||
|
- `HOST1`/`HOST2` hostnames in `master.conf` don't match Tailscale device names
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Confirm which state fallback is in
|
||||||
|
cat /boot/config/fallback_state.db
|
||||||
|
# If stuck in FAILOVER after remote comes back: reset state
|
||||||
|
bash Tools/fallback_state_reset.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ── Rsync fails — remote share not found ─────────────────────────────────────
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check the share exists on the remote
|
||||||
|
ssh -i /root/.ssh/<your_key> root@<remote_hostname> "ls /mnt/user/"
|
||||||
|
|
||||||
|
# Share names must be identical on both servers
|
||||||
|
# /mnt/user/Movies on HOST1 → /mnt/user/Movies on HOST2
|
||||||
|
# A mismatch = rsync writes to a non-existent path or aborts pre-flight
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ━━━ WHAT'S NEXT ━━━
|
||||||
|
|
||||||
|
Once the core is running, enable and configure the remaining features at your own pace.
|
||||||
|
Each subsystem has a README with the design decisions and a Manual with the config
|
||||||
|
reference and troubleshooting. The `--status` flag on any script shows the current
|
||||||
|
configuration and state.
|
||||||
|
|
||||||
|
```
|
||||||
|
Watchdogs/ → README-Watchdogs.md configure memory limits, container lists
|
||||||
|
Media/ → README-Media.md enable arr cleanup, discovery scripts
|
||||||
|
Monitors/ → README-Monitors.md cert monitoring, SMART, health digest
|
||||||
|
Transcodes/ → README-Transcoding.md ramdisk setup for Emby transcode
|
||||||
|
Partnership/ → README-Partnership.md auth stack sharing, full lifecycle
|
||||||
|
```
|
||||||
@@ -686,7 +686,7 @@ Problem: HOST1 loses power
|
|||||||
If you're setting this up from scratch on two servers:
|
If you're setting this up from scratch on two servers:
|
||||||
|
|
||||||
```
|
```
|
||||||
1. Follow README-Rsync_Setup.md completely — end-to-end setup guide
|
1. Follow Manual.md — end-to-end setup guide
|
||||||
Tailscale → SSH keys → git clone → master.conf → Arbiter schedule
|
Tailscale → SSH keys → git clone → master.conf → Arbiter schedule
|
||||||
|
|
||||||
2. Set your container names correctly before anything else
|
2. Set your container names correctly before anything else
|
||||||
@@ -718,7 +718,7 @@ If you're setting this up from scratch on two servers:
|
|||||||
| [README-Transcoding.md](Transcodes/README-Transcoding.md) | Ramdisk design, symlink architecture, Docker mount requirement, SSD fallback |
|
| [README-Transcoding.md](Transcodes/README-Transcoding.md) | Ramdisk design, symlink architecture, Docker mount requirement, SSD fallback |
|
||||||
| [README-Monitors.md](Monitors/README-Monitors.md) | Cert monitoring, SMART health, bandwidth tracking, health digest profiles |
|
| [README-Monitors.md](Monitors/README-Monitors.md) | Cert monitoring, SMART health, bandwidth tracking, health digest profiles |
|
||||||
| [README-Partnership.md](Partnership/README-Partnership.md) | Auth stack sharing, onboard/offboard/transfer lifecycle, deferred offboard |
|
| [README-Partnership.md](Partnership/README-Partnership.md) | Auth stack sharing, onboard/offboard/transfer lifecycle, deferred offboard |
|
||||||
| [README-Rsync_Setup.md](Rsync/README-Rsync_Setup.md) | Complete setup guide — Tailscale, SSH keys, git clone, master.conf, Arbiter |
|
| [Manual.md](Manual.md) | Complete setup guide — Tailscale, SSH keys, git clone, master.conf, Arbiter |
|
||||||
| [README-Tools.md](Tools/README-Tools.md) | failover_state_reset, skip list manager, emby_database_repair, container export |
|
| [README-Tools.md](Tools/README-Tools.md) | failover_state_reset, skip list manager, emby_database_repair, container export |
|
||||||
| [README-Unraid_Essentials.md](unRAID_Essentials/README-Unraid_Essentials.md) | `git_pull_execute.sh`, `server_reboot.sh`, `mover_stop.sh`, `user_scripts_stop.sh` |
|
| [README-Unraid_Essentials.md](unRAID_Essentials/README-Unraid_Essentials.md) | `git_pull_execute.sh`, `server_reboot.sh`, `mover_stop.sh`, `user_scripts_stop.sh` |
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user