created loop for both watchdogs and consolidated orch lists

This commit is contained in:
2026-04-23 22:44:56 -04:00
parent 8f60e5dde4
commit a3f90ed175
11 changed files with 2530 additions and 1467 deletions
+374 -197
View File
@@ -1,6 +1,6 @@
# Rsync Setup Guide
> **Status:** Work in Progress
> For the unRAID Rsync Ecosystem — `common.sh` · `Master.conf` · `rsync.sh` · `daily_sync.sh`
> For the unRAID Script Ecosystem — `Master.conf` · `common.sh` · `rsync.sh` · `daily_sync_maintenance.sh` · `weekly_sync_maintenance.sh`
---
@@ -8,11 +8,13 @@
This guide walks through setting up the rsync ecosystem on both your primary and secondary unRAID 7.x servers. By the end you will have:
- A Gitea repository cloned to both servers
- SSH keys configured for server-to-server communication
- Tailscale running on both servers for secure networking
- Scripts scheduled and running via the User Scripts plugin
- Automated daily sync of media shares and appdata profiles
- A Gitea repository cloned to both servers
- All scripts scheduled via the User Scripts plugin
- Automated daily sync of media shares driven by orchestrators
- Automated weekly clean sync of critical appdata (Emby + auth stack)
- Optional personal encrypted shares synced for offsite backup
---
@@ -21,141 +23,130 @@ This guide walks through setting up the rsync ecosystem on both your primary and
Both servers need the following before starting:
- unRAID 7.x
- [Community Applications plugin](https://forums.unraid.net/topic/38582-plug-in-community-applications/) installed
- [User Scripts plugin](https://forums.unraid.net/topic/48286-plugin-user-scripts/) installed via Community Applications
- [Tailscale plugin](https://forums.unraid.net/topic/136889-tailscale-plugin/) installed via Community Applications
- Access to a Gitea instance (self-hosted or remote)
- Terminal access to both servers (via unRAID UI → Tools → Terminal, or SSH)
- Access to a Gitea instance (self-hosted recommended — Gitea runs as a Docker container on HOST1)
- Terminal access to both servers (unRAID UI → Tools → Terminal, or SSH)
---
## Step 1 — Tailscale Setup
Tailscale provides the secure network tunnel between your two servers. The scripts resolve the remote server's IP via Tailscale at runtime.
Tailscale provides the secure network tunnel between your two servers. Scripts resolve the remote server's IP via Tailscale at runtime — no hardcoded IPs needed.
### On Both Servers
1. Open **Apps** in the unRAID UI
2. Search for **Tailscale** and install the plugin
3. Once installed go to **Settings → Tailscale**
3. Go to **Settings → Tailscale**
4. Click **Connect** and authenticate with your Tailscale account
5. Verify both servers appear in your [Tailscale admin console](https://login.tailscale.com/admin/machines)
### Verify Connectivity
Run this on the primary to confirm it can see the secondary:
Run this on HOST1 to confirm it can reach HOST2:
```bash
tailscale ip -4 unRAID-Jayred365
```
You should get back a `100.x.x.x` IP. If not, check that both machines are authenticated and connected in the Tailscale admin console.
You should get back a `100.x.x.x` IP. If not, check both machines are authenticated in the Tailscale admin console.
> **Note:** The hostnames used in `Master.conf` (`HOST1` and `HOST2`) must match the Tailscale machine names exactly — these are case sensitive.
> **Important:** The hostnames in `Master.conf` (`HOST1` and `HOST2`) must match the Tailscale machine names exactly — case sensitive.
---
## Step 2 — Generate SSH Keys
## Step 2 — Enable SSH on unRAID
unRAID 7.x has SSH disabled by default. Enable it on both servers so scripts can connect between them.
1. Go to **Settings → Management Access**
2. Under **Secure Shell** set **SSH** to `Enabled`
3. Set **SSH port** to `22`
4. Click **Apply**
> SSH is only exposed on your local network and Tailscale interface. Scripts connect via Tailscale IP — traffic is encrypted end-to-end.
---
## Step 3 — Generate SSH Keys
The scripts use SSH keys for two purposes:
- **Server-to-server rsync** — primary authenticates to secondary (and vice versa)
- **Server-to-server rsync and failover** — each server authenticates to the other
- **Gitea access** — both servers pull from the git repository
### 2a — Server-to-Server Keys
### 3a — Server-to-Server Keys
Run the following on **each server** to generate its rsync key. Replace the filename with the appropriate server name.
**On Primary (unRAID-Gmer4Lfe):**
**On HOST1 (unRAID-Gmer4Lfe):**
```bash
ssh-keygen -t ed25519 -f /root/.ssh/Gmer4Lfe-rsync-key -C "gmer4lfe-rsync" -N ""
```
**On Secondary (unRAID-Jayred365):**
**On HOST2 (unRAID-Jayred365):**
```bash
ssh-keygen -t ed25519 -f /root/.ssh/Jayred365-rsync-key -C "jayred365-rsync" -N ""
```
### 2b — Copy Public Keys to Each Server
### 3b — Authorise Keys on Each Server
The primary's public key must be authorised on the secondary, and vice versa.
HOST1's public key must be authorised on HOST2, and vice versa.
**Copy primary key → secondary:**
**Copy HOST1 key → HOST2:**
```bash
# Run on primary
# On HOST1 — print the public key
cat /root/.ssh/Gmer4Lfe-rsync-key.pub
```
Copy the output. Then on the secondary:
```bash
# Run on secondary
# On HOST2 — paste and authorise
mkdir -p /root/.ssh
echo "PASTE_PUBLIC_KEY_HERE" >> /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys
```
**Copy secondary key → primary:**
**Copy HOST2 key → HOST1:**
```bash
# Run on secondary
# On HOST2 — print the public key
cat /root/.ssh/Jayred365-rsync-key.pub
```
Copy the output. Then on the primary:
```bash
# Run on primary
mkdir -p /root/.ssh
# On HOST1 — paste and authorise
echo "PASTE_PUBLIC_KEY_HERE" >> /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys
```
### 2c — Test the Connection
### 3c — Test the Connection
From the primary, test that it can SSH to the secondary without a password prompt:
From HOST1, verify it can SSH to HOST2 without a password prompt:
```bash
ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@$(tailscale ip -4 unRAID-Jayred365) "echo connected"
```
You should see `connected`. If prompted for a password the key was not authorised correctly — recheck Step 2b.
You should see `connected`. If prompted for a password the key was not authorised correctly — recheck Step 3b.
### 2d — Gitea SSH Key
### 3d — Gitea SSH Key
Generate a separate key for Gitea access on each server:
```bash
ssh-keygen -t ed25519 -f /root/.ssh/id_gitea_rsync -C "unraid-gitea" -N ""
ssh-keygen -t ed25519 -f /root/.ssh/unraid_gitea -C "unraid-gitea" -N ""
```
Add the public key to your Gitea account:
```bash
cat /root/.ssh/id_gitea_rsync.pub
cat /root/.ssh/unraid_gitea.pub
```
Copy the output and add it in Gitea under **Settings → SSH / GPG Keys → Add Key**.
---
## Step 3 — Enable SSH on unRAID
unRAID 7.x has SSH disabled by default. Enable it on both servers so the scripts can connect between them.
1. Go to **Settings → Management Access**
2. Under **Secure Shell** set **SSH** to `Enabled`
3. Set **SSH port** to `22` (default)
4. Click **Apply**
> **Security note:** SSH is only exposed on your local network and Tailscale interface. The rsync scripts connect via the Tailscale IP so traffic is encrypted end-to-end.
---
## Step 4 — Clone the Git Repository
The scripts live in a Gitea repository. Both servers clone from the same repo so updates propagate everywhere via a single git pull.
Both servers clone from the same Gitea repository. Updates pushed to the repo propagate to both servers on the next daily git pull.
### On Both Servers
@@ -164,12 +155,12 @@ The scripts live in a Gitea repository. Both servers clone from the same repo so
mkdir -p /mnt/user/appdata/unraid_scripts
# Clone the repository
GIT_SSH_COMMAND="ssh -i /root/.ssh/id_gitea_rsync" \
git clone git@YOUR_GITEA_HOST:YOUR_USER/Unraid_Scripts.git \
GIT_SSH_COMMAND="ssh -i /root/.ssh/unraid_gitea" \
git clone git@YOUR_GITEA_HOST:FailedProxy/Unraid_Scripts.git \
/mnt/user/appdata/unraid_scripts
```
Replace `YOUR_GITEA_HOST` and `YOUR_USER` with your Gitea server address and username.
Replace `YOUR_GITEA_HOST` with your Gitea server address and port.
### Verify the Structure
@@ -182,252 +173,385 @@ You should see:
```
Master.conf
common.sh
Rsync/
rsync.sh
Orchestrators/
daily_sync.sh
Rsync/
Failover/
Docker_Essentials/
unRAID_Essentials/
Media/
Transcodes/
Monitors/
Tools/
recreate_shares.sh
```
### Make Scripts Executable
```bash
chmod +x /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh
chmod +x /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync.sh
chmod +x /mnt/user/appdata/unraid_scripts/Tools/recreate_shares.sh
find /mnt/user/appdata/unraid_scripts -name "*.sh" -exec chmod +x {} \;
```
---
## Step 5 — Configure Master.conf
All user configuration lives in `Master.conf`. Open it and adjust the following to match your setup:
All user configuration lives in `Master.conf`. Open it and fill in your values:
```bash
nano /mnt/user/appdata/unraid_scripts/Master.conf
```
### Required Changes
| Variable | Description | Example |
|---|---|---|
| `HOST1` | Hostname of your primary server | `unRAID-Gmer4Lfe` |
| `HOST2` | Hostname of your secondary server | `unRAID-Jayred365` |
| `HOST1_SSH_KEY` | Path to primary's rsync private key | `/root/.ssh/Gmer4Lfe-rsync-key` |
| `HOST2_SSH_KEY` | Path to secondary's rsync private key | `/root/.ssh/Jayred365-rsync-key` |
| `REPO_SSH` | SSH URL of your Gitea repository | `git@192.168.50.2:User/Unraid_Scripts.git` |
| `GITEA_SSH_KEY` | Path to Gitea private key | `/root/.ssh/id_gitea_rsync` |
| `BW_LIMIT` | Global bandwidth limit in KB/s | `12500` |
| `ROOTFS_WARN` | Remote rootfs % threshold before aborting | `75` |
### Daily Sync Shares
Add the full paths of all media shares you want synced nightly:
### Host Configuration
```bash
DAILY_SYNC_SHARES=(
HOST1="unRAID-Gmer4Lfe" # must match Tailscale machine name exactly
HOST2="unRAID-Jayred365"
HOST1_SSH_KEY="/root/.ssh/Gmer4Lfe-rsync-key"
HOST2_SSH_KEY="/root/.ssh/Jayred365-rsync-key"
HOST1_EMBY_CONTAINER="Emby"
HOST1_EMBY_URL="http://localhost:8096"
HOST1_EMBY_API_KEY="your-host1-emby-api-key" # Emby Dashboard → API Keys → + New Key
HOST2_EMBY_CONTAINER="Emby-Jayred365"
HOST2_EMBY_URL="http://localhost:8096"
HOST2_EMBY_API_KEY="your-host2-emby-api-key"
```
### Git / Repo
```bash
GITEA_CONTAINER="Gitea" # exact Docker container name
GITEA_REPO_PATH="FailedProxy/Unraid_Scripts.git"
GITEA_DOMAIN="" # optional public domain fallback
TARGET_DIR="/mnt/user/appdata/unraid_scripts"
GITEA_SSH_KEY="/root/.ssh/unraid_gitea"
SSH_PORT=221
```
### Orchestrators — Daily Sync Shares
Define which shares each server owns. Each server only pushes the shares it is source of truth for — the other server mirrors and treats them as read-only.
```bash
HOST1_DAILY_SYNC_SHARES=(
/mnt/user/Movies
/mnt/user/Tv_Shows
/mnt/user/Music
# add more here
# add all HOST1-managed shares here
)
HOST2_DAILY_SYNC_SHARES=(
/mnt/user/Anime_Shows
/mnt/user/Anime_Movies
# add all HOST2-managed shares here
)
```
### Profiles
> Never put the same share in both lists. One server is always the truth holder for each share.
Profiles control per-share rsync behaviour for your frequently synced appdata shares. Each profile is matched by the directory basename (lowercased) — or overridden with `--profile=name`.
### Orchestrators — Weekly Sync Jobs
Shares synced during the Sunday maintenance window with containers stopped both sides:
```bash
# One array drives both local and remote container stops
# Local stops first (flush databases) then remote stops (clean receive)
# Same container names on both HOST1 and HOST2 — no duplication needed
# Containers not found on a server are skipped gracefully
declare -A PROFILE_CRITICAL_CONTAINER_NAMES=(
[critical-data]="Mariadb-Authelia Mariadb-Authelia-Secondary Redis-Authelia Redis-Authelia-Secondary Lldap-Gmer4Lfe NginxProxyManager Authelia Authelia-Secondary"
[important-data]="Postgres-NextCloud NextCloud"
[emby]="Emby" # nightly clean sync — Emby stopped both sides
[emby-failover]="" # dirty sync — Emby stays running
WEEKLY_SYNC_JOBS=(
"/mnt/user/Media_Server/Emby" # full clean Emby mirror
"/mnt/user/appdata-Failover/Critical-Data" # auth stack
)
```
Any share with no matching profile falls through to the global `DEFAULT_RSYNC_OPTS`. Containers not found on a server are skipped gracefully — only containers that were actually running get restarted.
---
**Two Emby profiles:**
## Step 6 — Rsync Profiles
Profiles control per-share rsync behaviour for appdata syncs. The profile key is matched automatically by the basename of the directory passed to `rsync.sh` (lowercased). Override with `--profile=name`.
One array drives both local and remote container stops. Same container names on both servers — consistent naming is a requirement of this ecosystem.
### Current Profiles
| Profile | Purpose | Containers Stopped |
|---|---|---|
| `arrs_stack` | Arr databases | Sonarr, Radarr, Lidarr, Prowlarr, Bazarr, Pinchflat |
| `critical-data` | Auth stack | Mariadb, Redis, LLDAP, NPM, Authelia (delayed start) |
| `important-data` | NextCloud + Postgres | Postgres, NextCloud (delayed start) |
| `gmer4lfe` | Server-specific appdata | Organizr, UptimeKuma, VaultWarden |
| `emby` | Weekly clean sync | Emby both sides — WAL checkpointed |
| `emby-failover` | Frequent dirty sync | None — Emby stays running |
### Two Emby Profiles
```
emby-failover — frequent dirty sync (every 30-60min):
Emby stays running on both sides
emby-failover — every 30-60min, Emby stays running:
WAL and SHM excluded — safe while Emby is active
Only critical failover data: users.db, library.db, authentication.db, config/
Fast, small dataset, high bandwidth
This is also what gets written back during failover handback
Critical failover data only: users.db, library.db, authentication.db, config/
Fast, small dataset — users continue watching without interruption on failover
Also used for failover writeback on handback
emby — weekly clean sync (via nightly_critical_full_sync.sh Sunday 2:30am):
Emby stopped on both sides — WAL checkpointed on shutdown
emby — weekly Sunday 2:30am, both Emby instances stopped:
WAL checkpointed on shutdown — full consistent mirror
Full mirror: metadata, plugins, config all included
Minimal excludes: logs, transcodes, cache, crash files only
Complete faithful state pushed once per week
Cache stays warm on HOST2 all week — only reset on Sunday
emby-failover handles the critical state between weekly syncs
```
Usage with profile override:
```bash
# Dirty sync — Emby stays running
bash rsync.sh /mnt/user/Media_Server/Emby --profile=emby-failover
# Clean sync — called by nightly_critical_full_sync.sh, Emby stopped via profile
bash rsync.sh /mnt/user/Media_Server/Emby
emby-failover covers the critical state between weekly syncs
```
---
## Step 6Set Up User Scripts
## Step 7Personal Encrypted Shares
The User Scripts plugin is how unRAID schedules and runs the scripts. Each sync job is its own script entry in the plugin.
Personal shares can be synced to the remote server for offsite backup. ZFS encrypts at the dataset level — the remote server receives encrypted blocks and cannot read the content without your passphrase or keyfile.
### Frequent Sync Jobs (Scheduled Individually)
### ZFS Encryption Setup (unRAID 7)
Create one script entry per appdata profile. Go to **Plugins → User Scripts → Add New Script**.
**Step 1 — Create an encrypted dataset:**
Name it descriptively — e.g. `rsync appdata arrs_stack`.
1. In the unRAID UI go to **Main** → click your ZFS pool name
2. Click **+ Dataset** to create a new dataset
3. Name it — e.g. `Gmer4Lfe-Personal`
4. Enable **Encryption** → set your passphrase
> ⚠️ Write your passphrase down — if lost, data is unrecoverable
In the script body paste:
**Step 2 — Create the share:**
1. Go to **Settings → Shares → Add Share**
2. Set the share path to your new encrypted dataset
3. Set **Use cache:** `Only` — keeps data on ZFS pool, not array
**Step 3 — Verify encryption is active before syncing:**
```bash
zfs get encryption poolname/Gmer4Lfe-Personal
# Should show: encryption aes-256-gcm
```
**Step 4 — Auto-unlock on boot (keyfile approach — optional):**
```bash
# Create keyfile — on HOST1 only, never sync this file
dd if=/dev/urandom bs=32 count=1 | base64 > /root/.zfs-keys/personal.key
chmod 600 /root/.zfs-keys/personal.key
# Set dataset to use keyfile
zfs change-key -o keylocation=file:///root/.zfs-keys/personal.key \
-o keyformat=raw poolname/Gmer4Lfe-Personal
# Add to array start (via array_start.sh or User Scripts):
zfs load-key poolname/Gmer4Lfe-Personal
zfs mount poolname/Gmer4Lfe-Personal
```
**Manual unlock alternative (most secure):**
```bash
zfs load-key poolname/Gmer4Lfe-Personal # prompts for passphrase
zfs mount poolname/Gmer4Lfe-Personal
```
**Step 5 — Add to Master.conf:**
```bash
HOST1_PERSONAL_SHARES=(
/mnt/user/Gmer4Lfe-Personal
)
```
Personal shares sync automatically with the daily media share sync in `daily_sync_maintenance.sh`. The remote server receives encrypted blocks — content is unreadable without your key.
---
## Step 8 — Set Up User Scripts
The ecosystem uses a single orchestrator entry for array startup plus a small number of scheduled scripts.
### At Startup of Array
Create one script entry named `array start`:
```bash
#!/bin/bash
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/appdata-Failover/Arrs_Stack
bash /mnt/user/appdata/unraid_scripts/Orchestrators/array_start.sh
```
Set the schedule to match your desired frequency:
Set schedule to: **At Startup of Array**
| Profile | Schedule | Notes |
This single entry launches everything configured in `ARRAY_START_SCRIPTS` in `Master.conf`:
- `ramdisk_setup.sh` — creates ramdisk before Emby starts
- `docker_syslog_filter.sh` — suppresses veth log noise
- `php_fpm_max_children.sh` — WebGUI tuning
- `docker_network_connect.sh` — connects containers to extra networks
- `system_watchdog.sh` — continuous system health monitor
- `docker_watchdog.sh` — continuous container health monitor
- `failover.sh` — continuous mutual failover
### Cron Schedules
| Script | Schedule | Purpose |
|---|---|---|
| `emby-failover` | Every 30-60 min | Dirty sync — Emby running, critical data only |
| `emby` | Weekly Sunday via `nightly_critical_full_sync.sh` | Clean sync — Emby stopped, full mirror, cache stays warm all week |
| `Critical-Data` | Weekly Sunday via `nightly_critical_full_sync.sh` | Auth stack — clean weekly sync |
| `Important-Data` | Every 6-12 hours | NextCloud file changes |
| `Arrs_Stack` | Every 12-24 hours | Arr databases |
| `Gmer4Lfe` | Daily or weekly | Personal appdata, rarely changes |
| `transcode_management.sh` | `*/3 * * * *` | Transcode cleanup + manager |
| `arrs_failed_stalled_recovery.sh` | `0 */6 * * *` | Blocklist + re-search failed imports |
| `rsync.sh ... --profile=emby-failover` | `*/30 * * * *` | Emby dirty sync |
| `daily_sync_maintenance.sh` | `0 1 * * *` | Full daily maintenance window |
| `weekly_sync_maintenance.sh` | `30 2 * * 0` | Weekly sync + updates + restarts |
| `weekly_health_digest.sh` | `0 8 * * 6` | Saturday morning health digest |
> **Important:** Set each script to run as a **Background Task** — this ensures output streams correctly to the log rather than buffering in the browser.
### Emby Failover Dirty Sync
### Daily Sync Orchestrator
Create one more script entry for the daily media sync:
Name it `daily media sync`.
In the script body paste:
Create a separate script entry named `rsync emby failover`:
```bash
#!/bin/bash
bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync.sh
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
/mnt/user/Media_Server/Emby --profile=emby-failover
```
Set the schedule to **Daily at 01:00**.
Set schedule to: `*/30 * * * *`
### Appdata Profile Syncs
Create one entry per appdata profile you want on a schedule:
```bash
#!/bin/bash
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
/mnt/user/appdata-Failover/Arrs_Stack
```
| Profile | Recommended Schedule |
|---|---|
| `Arrs_Stack` | Every 12-24 hours |
| `Important-Data` | Every 6-12 hours |
| `Gmer4Lfe` | Daily or weekly |
| `emby` | Via `weekly_sync_maintenance.sh` only — do NOT schedule separately |
| `Critical-Data` | Via `weekly_sync_maintenance.sh` only — do NOT schedule separately |
> Set all scripts to run as **Background Task** — output streams correctly rather than buffering in the browser.
---
## Step 7 — Verify the Setup
## Step 9 — Verify the Setup
Before letting the scheduled jobs run, do a manual test from the terminal on the primary:
Before letting scheduled jobs run, test manually from the terminal on HOST1:
```bash
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --log
# Test a single appdata profile sync
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
/mnt/user/appdata-Failover/Arrs_Stack --dry-run --log
# Test the daily sync orchestrator
bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh --dry-run
```
A healthy run will show:
```
━━━ ⚙️ Setup ━━━
[INFO] 🖥️ Host: unRAID-Gmer4Lfe → unRAID-Jayred365
[INFO] 🌐 Remote IP: 100.x.x.x
️ Host: unRAID-Gmer4Lfe → unRAID-Jayred365
️ Remote IP: 100.x.x.x
━━━ 🛡️ Pre-flight Checks ━━━
️ [INFO] 📡 unRAID-Jayred365 is reachable
️ [INFO] 🩺 Remote rootfs: 12% used (threshold: 75%)
️ [INFO] 🩺 Remote share verified: ...
️ [INFO] 💾 disk1 🟢 — share present
✅ [OK] All disks backing share are online
✅ Remote reachable
Remote rootfs: 12% (threshold: 75%)
✅ All pre-flight checks passed
```
If any pre-flight check fails the script will abort with a clear error and hint before touching anything.
If any pre-flight check fails the script aborts with a clear error before touching anything.
---
## Step 8 — Secondary Server Initial Setup
## Step 10 — Secondary Server Initial Sync
If setting up the secondary from scratch (no existing data):
If setting up HOST2 from scratch with empty shares:
1. Complete Steps 15 on the secondary
1. Complete Steps 18 on HOST2
2. Start the array and create your shares in the unRAID UI
3. Run the share recreation tool to create disk directories from your cfg files:
3. Run the share recreation tool to create disk directories from cfg files:
```bash
bash /mnt/user/appdata/unraid_scripts/Tools/recreate_shares.sh
```
4. Temporarily remove `--delete` from `DEFAULT_RSYNC_OPTS` in `Master.conf`
5. Run the initial push from the primary — the `.recovery` marker files allow rsync to populate empty shares without aborting
6. Once complete, restore `--delete` to `Master.conf`
7. The next nightly run will clean up the `.recovery` marker files automatically
4. Run the initial push from HOST1 — this populates HOST2's empty shares:
```bash
bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh --log
```
5. Once complete, scheduled runs take over automatically.
---
## Naming Consistency — Required
The ecosystem is built on the assumption that containers and shares have identical names on both servers. This is not optional — it is what makes one codebase work on both servers without modification.
```
Container names must match exactly:
Emby ← HOST1 and HOST2
NginxProxyManager ← HOST1 and HOST2
Mariadb-Authelia ← HOST1 and HOST2
Share paths must match exactly:
/mnt/user/Movies ← HOST1 and HOST2
/mnt/user/Tv_Shows ← HOST1 and HOST2
```
If a container or share has a different name on one server the script skips it gracefully — but it will not do what you expect. Diverge from consistent naming and every script that touches containers or shares needs custom logic for each server. Keep naming consistent and one codebase covers both servers automatically.
---
## Troubleshooting
### SSH connection refused
- Verify SSH is enabled on the target server (Step 3)
- Check the correct key is referenced in `Master.conf`
- Confirm the Tailscale IP resolves: `tailscale ip -4 HOSTNAME`
- Verify SSH is enabled (Step 2)
- Confirm the correct key is referenced in `Master.conf`
- Test Tailscale: `tailscale ip -4 HOSTNAME`
### Pre-flight aborts on rootfs
- Remote rootfs is above `ROOTFS_WARN` threshold
- Check if the remote array is started and drives are mounted
- `df /` on the remote to see current usage
- Remote rootfs above `ROOTFS_WARN` threshold
- Check remote array is started and drives are mounted
- Run `df /` on the remote to see current usage
### Pre-flight aborts on empty share
- Share exists but has no content — drives may not be mounted
- Run `recreate_shares.sh` if setting up fresh
- Check array status on the remote server
### Pre-flight aborts on disk check
- One or more disks backing the share are not mounted
- Check **Main → Array Devices** on the remote for offline disks
- Verify disk assignments are correct after any hardware changes
### Script not found
- Verify the repo was cloned to `/mnt/user/appdata/unraid_scripts/`
- Check scripts are executable: `chmod +x /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh`
### Containers not stopping/starting
- Verify container names in `Master.conf` match exactly what Docker shows
- Check SSH key has access to run docker commands on the remote
- Verify container names in `Master.conf` match Docker exactly — case sensitive
- Test manually: `ssh -i /root/.ssh/KEY root@REMOTE_IP "docker ps"`
### Script not found
- Verify repo was cloned to `/mnt/user/appdata/unraid_scripts/`
- Make scripts executable: `find /mnt/user/appdata/unraid_scripts -name "*.sh" -exec chmod +x {} \;`
### Profile not matching
- Profile key is matched by directory basename lowercased
- `/mnt/user/appdata-Failover/Arrs_Stack` → basename `Arrs_Stack` → key `arrs_stack`
- Override with `--profile=name` if basename doesn't match
---
## Available Flags
All scripts support the following flags:
All scripts support:
| Flag | Description |
|---|---|
| `--dry-run` or `-n` | Run without making any changes |
| `--dry-run` | Run without making any changes |
| `--log` | Enable verbose logging output |
| `--no-log` | Disable logging (overrides Master.conf) |
| `--no-log` | Disable logging |
| `--status` | Print resolved configuration and exit |
Example:
```bash
# Preview what would be synced without transferring anything
# Preview what would be synced
bash rsync.sh /mnt/user/Movies --dry-run --log
# Check what profile and settings resolved for a share
# Check resolved profile settings
bash rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --status
# Test daily orchestrator without changes
bash daily_sync_maintenance.sh --dry-run
```
---
@@ -436,16 +560,69 @@ bash rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --status
```
Unraid_Scripts/
├── Master.conf # All user configuration — edit this file only
├── common.sh # Shared library — functions used by all scripts
├── Rsync/
│ └── rsync.sh # Core rsync script — called per share
├── Master.conf # All user configuration — edit this file only
├── common.sh # Shared library — functions used by all scripts
├── Orchestrators/
── daily_sync.sh # Daily media sync orchestrator
── array_start.sh # Single array-start entry point
│ ├── daily_sync_maintenance.sh # Daily maintenance window orchestrator
│ ├── weekly_sync_maintenance.sh # Weekly maintenance window orchestrator
│ ├── media_management.sh # Permissions + cleaners + arr cleanup
│ └── transcode_management.sh # Transcode cleanup + manager
├── Rsync/
│ └── rsync.sh # Core rsync script — called per share
├── Failover/
│ ├── failover.sh # Mutual container failover — continuous loop
│ ├── failover_test.sh # Controlled failover simulation
│ └── failover_state_reset.sh # Reset failover state manually
├── Docker_Essentials/
│ ├── docker_watchdog.sh # Two-tier container monitor — continuous loop
│ ├── docker_daily_restart.sh # Daily container restarts
│ ├── docker_weekly_restart.sh # Weekly container restarts
│ └── docker_network_connect.sh # Connect containers to extra networks
├── unRAID_Essentials/
│ ├── system_watchdog.sh # System health monitor — continuous loop
│ ├── ramdisk_setup.sh # Creates ramdisk + symlink at array start
│ ├── docker_syslog_filter.sh # Suppress veth log noise
│ ├── php_fpm_max_children.sh # WebGUI performance tuning
│ ├── server_reboot.sh # Graceful scheduled reboot
│ ├── mover_stop.sh # Stop mover cleanly
│ ├── clear_logs.sh # Weekly log cleanup
│ ├── webgui_restart.sh # nginx + emhttp restart escalation
│ └── git_pull_execute.sh # Pull latest scripts from Gitea
├── Media/
│ ├── media_shares_permissions.sh # Apply permissions to media shares
│ ├── media_cleaner.sh # Remove junk files from media shares
│ ├── lidarr_cleanup.sh # Remove orphaned music files
│ ├── sonarr_cleanup.sh # Remove orphaned TV files
│ ├── radarr_cleanup.sh # Remove orphaned movie files
│ └── arrs_failed_stalled_recovery.sh # Blocklist + re-search failed imports
├── Transcodes/
│ ├── transcode_manager.sh # Symlink direction management
│ ├── transcode_cleanup.sh # Remove old inactive transcode files
│ └── ramdisk_setup.sh # (also in unRAID_Essentials — symlinked)
├── Monitors/
│ ├── cert_monitor.sh # SSL certificate expiry monitoring
│ ├── backup_verify.sh # Checksum verification against remote
│ ├── smart_health.sh # Drive SMART attribute monitoring
│ ├── zfs_memory_snapshot.sh # Weekly ZFS health + memory report
│ ├── bandwidth_monitor.sh # Rsync transfer logging + weekly summary
│ ├── weekly_health_digest.sh # Aggregated health digest email
│ ├── emby_session_report.sh # Weekly Emby usage statistics
│ └── emby_database_repair.sh # Emby SQLite database repair
└── Tools/
── recreate_shares.sh # Share directory recreation from cfg files
```
---
*Guide version aligned with common.sh v1.6*
── recreate_shares.sh # Recreate share directories from cfg files
├── bulk_permissions_repair.sh # One-shot permission repair
├── watchdog_skip_list_manager.sh # Manage docker watchdog skip list
├── rsync_stop.sh # Stop active rsync jobs cleanly
├── user_scripts_stop.sh # Stop running user scripts
└── container_data_export.sh # Export container configuration
```