354 lines
11 KiB
Markdown
354 lines
11 KiB
Markdown
This guide is a work in progress and generated by ai, dont have time for guides. so this works
|
||
|
||
Rsync Setup Guide
|
||
|
||
Status: Work in Progress
|
||
For the unRAID Rsync Ecosystem — common.sh · Master.conf · rsync.sh · daily_sync.sh
|
||
|
||
|
||
|
||
Overview
|
||
|
||
This guide walks through setting up the rsync ecosystem on both your primary and secondary unRAID 7.x servers. By the end you will have:
|
||
|
||
A Gitea repository cloned to both servers
|
||
SSH keys configured for server-to-server communication
|
||
Tailscale running on both servers for secure networking
|
||
Scripts scheduled and running via the User Scripts plugin
|
||
Automated daily sync of media shares and appdata profiles
|
||
|
||
Prerequisites
|
||
|
||
Both servers need the following before starting:
|
||
|
||
unRAID 7.x
|
||
User Scripts plugin installed via Community Applications
|
||
Tailscale plugin installed via Community Applications
|
||
Access to a Gitea instance (self-hosted or remote)
|
||
Terminal access to both servers (via unRAID UI → Tools → Terminal, or SSH)
|
||
|
||
Step 1 — Tailscale Setup
|
||
|
||
Tailscale provides the secure network tunnel between your two servers. The scripts resolve the remote server's IP via Tailscale at runtime.
|
||
On Both Servers
|
||
|
||
Open Apps in the unRAID UI
|
||
Search for Tailscale and install the plugin
|
||
Once installed go to Settings → Tailscale
|
||
Click Connect and authenticate with your Tailscale account
|
||
Verify both servers appear in your Tailscale admin console
|
||
|
||
Verify Connectivity
|
||
|
||
Run this on the primary to confirm it can see the secondary:
|
||
bash
|
||
|
||
tailscale ip -4 unRAID-Jayred365
|
||
|
||
You should get back a 100.x.x.x IP. If not, check that both machines are authenticated and connected in the Tailscale admin console.
|
||
|
||
Note: The hostnames used in Master.conf (HOST1 and HOST2) must match the Tailscale machine names exactly — these are case sensitive.
|
||
|
||
Step 2 — Generate SSH Keys
|
||
|
||
The scripts use SSH keys for two purposes:
|
||
|
||
Server-to-server rsync — primary authenticates to secondary (and vice versa)
|
||
Gitea access — both servers pull from the git repository
|
||
|
||
2a — Server-to-Server Keys
|
||
|
||
Run the following on each server to generate its rsync key. Replace the filename with the appropriate server name.
|
||
|
||
On Primary (unRAID-Gmer4Lfe):
|
||
bash
|
||
|
||
ssh-keygen -t ed25519 -f /root/.ssh/Gmer4Lfe-rsync-key -C "gmer4lfe-rsync" -N ""
|
||
|
||
On Secondary (unRAID-Jayred365):
|
||
bash
|
||
|
||
ssh-keygen -t ed25519 -f /root/.ssh/Jayred365-rsync-key -C "jayred365-rsync" -N ""
|
||
|
||
2b — Copy Public Keys to Each Server
|
||
|
||
The primary's public key must be authorised on the secondary, and vice versa.
|
||
|
||
Copy primary key → secondary:
|
||
bash
|
||
|
||
# Run on primary
|
||
cat /root/.ssh/Gmer4Lfe-rsync-key.pub
|
||
|
||
Copy the output. Then on the secondary:
|
||
bash
|
||
|
||
# Run on secondary
|
||
mkdir -p /root/.ssh
|
||
echo "PASTE_PUBLIC_KEY_HERE" >> /root/.ssh/authorized_keys
|
||
chmod 600 /root/.ssh/authorized_keys
|
||
|
||
Copy secondary key → primary:
|
||
bash
|
||
|
||
# Run on secondary
|
||
cat /root/.ssh/Jayred365-rsync-key.pub
|
||
|
||
Copy the output. Then on the primary:
|
||
bash
|
||
|
||
# Run on primary
|
||
mkdir -p /root/.ssh
|
||
echo "PASTE_PUBLIC_KEY_HERE" >> /root/.ssh/authorized_keys
|
||
chmod 600 /root/.ssh/authorized_keys
|
||
|
||
2c — Test the Connection
|
||
|
||
From the primary, test that it can SSH to the secondary without a password prompt:
|
||
bash
|
||
|
||
ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@$(tailscale ip -4 unRAID-Jayred365) "echo connected"
|
||
|
||
You should see connected. If prompted for a password the key was not authorised correctly — recheck Step 2b.
|
||
2d — Gitea SSH Key
|
||
|
||
Generate a separate key for Gitea access on each server:
|
||
bash
|
||
|
||
ssh-keygen -t ed25519 -f /root/.ssh/id_gitea_rsync -C "unraid-gitea" -N ""
|
||
|
||
Add the public key to your Gitea account:
|
||
bash
|
||
|
||
cat /root/.ssh/id_gitea_rsync.pub
|
||
|
||
Copy the output and add it in Gitea under Settings → SSH / GPG Keys → Add Key.
|
||
Step 3 — Enable SSH on unRAID
|
||
|
||
unRAID 7.x has SSH disabled by default. Enable it on both servers so the scripts can connect between them.
|
||
|
||
Go to Settings → Management Access
|
||
Under Secure Shell set SSH to Enabled
|
||
Set SSH port to 22 (default)
|
||
Click Apply
|
||
|
||
Security note: SSH is only exposed on your local network and Tailscale interface. The rsync scripts connect via the Tailscale IP so traffic is encrypted end-to-end.
|
||
|
||
Step 4 — Clone the Git Repository
|
||
|
||
The scripts live in a Gitea repository. Both servers clone from the same repo so updates propagate everywhere via a single git pull.
|
||
On Both Servers
|
||
bash
|
||
|
||
# Create the target directory
|
||
mkdir -p /mnt/user/appdata/unraid_scripts
|
||
|
||
# Clone the repository
|
||
GIT_SSH_COMMAND="ssh -i /root/.ssh/id_gitea_rsync" \
|
||
git clone git@YOUR_GITEA_HOST:YOUR_USER/Unraid_Scripts.git \
|
||
/mnt/user/appdata/unraid_scripts
|
||
|
||
Replace YOUR_GITEA_HOST and YOUR_USER with your Gitea server address and username.
|
||
Verify the Structure
|
||
bash
|
||
|
||
ls /mnt/user/appdata/unraid_scripts
|
||
|
||
You should see:
|
||
|
||
Master.conf
|
||
common.sh
|
||
Rsync/
|
||
rsync.sh
|
||
Orchestrators/
|
||
daily_sync.sh
|
||
Tools/
|
||
recreate_shares.sh
|
||
|
||
Make Scripts Executable
|
||
bash
|
||
|
||
chmod +x /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh
|
||
chmod +x /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync.sh
|
||
chmod +x /mnt/user/appdata/unraid_scripts/Tools/recreate_shares.sh
|
||
|
||
Step 5 — Configure Master.conf
|
||
|
||
All user configuration lives in Master.conf. Open it and adjust the following to match your setup:
|
||
bash
|
||
|
||
nano /mnt/user/appdata/unraid_scripts/Master.conf
|
||
|
||
Required Changes
|
||
Variable Description Example
|
||
HOST1 Hostname of your primary server unRAID-Gmer4Lfe
|
||
HOST2 Hostname of your secondary server unRAID-Jayred365
|
||
HOST1_SSH_KEY Path to primary's rsync private key /root/.ssh/Gmer4Lfe-rsync-key
|
||
HOST2_SSH_KEY Path to secondary's rsync private key /root/.ssh/Jayred365-rsync-key
|
||
REPO_SSH SSH URL of your Gitea repository git@192.168.50.2:User/Unraid_Scripts.git
|
||
GITEA_SSH_KEY Path to Gitea private key /root/.ssh/id_gitea_rsync
|
||
BW_LIMIT Global bandwidth limit in KB/s 12500
|
||
ROOTFS_WARN Remote rootfs % threshold before aborting 75
|
||
Daily Sync Shares
|
||
|
||
Add the full paths of all media shares you want synced nightly:
|
||
bash
|
||
|
||
DAILY_SYNC_SHARES=(
|
||
/mnt/user/Movies
|
||
/mnt/user/Tv_Shows
|
||
/mnt/user/Music
|
||
# add more here
|
||
)
|
||
|
||
Profiles
|
||
|
||
Profiles control per-share rsync behaviour for your frequently synced appdata shares. Each profile is matched by the directory basename (lowercased). Add a key to each profile array for any share that needs custom settings:
|
||
bash
|
||
|
||
declare -A PROFILE_CRITICAL_CONTAINER_NAMES=(
|
||
[arrs_stack]="Sonarr Radarr Lidarr Prowlarr"
|
||
[important-data]="Postgres-NextCloud NextCloud"
|
||
)
|
||
|
||
Any share with no matching profile falls through to the global DEFAULT_RSYNC_OPTS.
|
||
Step 6 — Set Up User Scripts
|
||
|
||
The User Scripts plugin is how unRAID schedules and runs the scripts. Each sync job is its own script entry in the plugin.
|
||
Frequent Sync Jobs (Scheduled Individually)
|
||
|
||
Create one script entry per appdata profile. Go to Plugins → User Scripts → Add New Script.
|
||
|
||
Name it descriptively — e.g. rsync appdata arrs_stack.
|
||
|
||
In the script body paste:
|
||
bash
|
||
|
||
#!/bin/bash
|
||
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/appdata-Failover/Arrs_Stack
|
||
|
||
Set the schedule to match your desired frequency:
|
||
Profile Recommended Schedule
|
||
Emby Every 1 hour
|
||
Critical-Data Every 2 hours
|
||
Important-Data Every 6 hours
|
||
Arrs_Stack Every 12 hours
|
||
|
||
Important: Set each script to run as a Background Task — this ensures output streams correctly to the log rather than buffering in the browser.
|
||
|
||
Daily Sync Orchestrator
|
||
|
||
Create one more script entry for the daily media sync:
|
||
|
||
Name it daily media sync.
|
||
|
||
In the script body paste:
|
||
bash
|
||
|
||
#!/bin/bash
|
||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync.sh
|
||
|
||
Set the schedule to Daily at 01:00.
|
||
Step 7 — Verify the Setup
|
||
|
||
Before letting the scheduled jobs run, do a manual test from the terminal on the primary:
|
||
bash
|
||
|
||
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --log
|
||
|
||
A healthy run will show:
|
||
|
||
━━━ ⚙️ Setup ━━━
|
||
ℹ️ [INFO] 🖥️ Host: unRAID-Gmer4Lfe → unRAID-Jayred365
|
||
ℹ️ [INFO] 🌐 Remote IP: 100.x.x.x
|
||
|
||
━━━ 🛡️ Pre-flight Checks ━━━
|
||
ℹ️ [INFO] 📡 unRAID-Jayred365 is reachable
|
||
ℹ️ [INFO] 🩺 Remote rootfs: 12% used (threshold: 75%)
|
||
ℹ️ [INFO] 🩺 Remote share verified: ...
|
||
ℹ️ [INFO] 💾 disk1 🟢 — share present
|
||
✅ [OK] All disks backing share are online
|
||
|
||
If any pre-flight check fails the script will abort with a clear error and hint before touching anything.
|
||
Step 8 — Secondary Server Initial Setup
|
||
|
||
If setting up the secondary from scratch (no existing data):
|
||
|
||
Complete Steps 1–5 on the secondary
|
||
Start the array and create your shares in the unRAID UI
|
||
Run the share recreation tool to create disk directories from your cfg files:
|
||
|
||
bash
|
||
|
||
bash /mnt/user/appdata/unraid_scripts/Tools/recreate_shares.sh
|
||
|
||
Temporarily remove --delete from DEFAULT_RSYNC_OPTS in Master.conf
|
||
Run the initial push from the primary — the .recovery marker files allow rsync to populate empty shares without aborting
|
||
Once complete, restore --delete to Master.conf
|
||
The next nightly run will clean up the .recovery marker files automatically
|
||
|
||
Troubleshooting
|
||
SSH connection refused
|
||
|
||
Verify SSH is enabled on the target server (Step 3)
|
||
Check the correct key is referenced in Master.conf
|
||
Confirm the Tailscale IP resolves: tailscale ip -4 HOSTNAME
|
||
|
||
Pre-flight aborts on rootfs
|
||
|
||
Remote rootfs is above ROOTFS_WARN threshold
|
||
Check if the remote array is started and drives are mounted
|
||
df / on the remote to see current usage
|
||
|
||
Pre-flight aborts on empty share
|
||
|
||
Share exists but has no content — drives may not be mounted
|
||
Run recreate_shares.sh if setting up fresh
|
||
Check array status on the remote server
|
||
|
||
Pre-flight aborts on disk check
|
||
|
||
One or more disks backing the share are not mounted
|
||
Check Main → Array Devices on the remote for offline disks
|
||
Verify disk assignments are correct after any hardware changes
|
||
|
||
Script not found
|
||
|
||
Verify the repo was cloned to /mnt/user/appdata/unraid_scripts/
|
||
Check scripts are executable: chmod +x /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh
|
||
|
||
Containers not stopping/starting
|
||
|
||
Verify container names in Master.conf match exactly what Docker shows
|
||
Check SSH key has access to run docker commands on the remote
|
||
Test manually: ssh -i /root/.ssh/KEY root@REMOTE_IP "docker ps"
|
||
|
||
Available Flags
|
||
|
||
All scripts support the following flags:
|
||
Flag Description
|
||
--dry-run or -n Run without making any changes
|
||
--log Enable verbose logging output
|
||
--no-log Disable logging (overrides Master.conf)
|
||
--status Print resolved configuration and exit
|
||
|
||
Example:
|
||
bash
|
||
|
||
# Preview what would be synced without transferring anything
|
||
bash rsync.sh /mnt/user/Movies --dry-run --log
|
||
|
||
# Check what profile and settings resolved for a share
|
||
bash rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --status
|
||
|
||
Repository Structure
|
||
|
||
Unraid_Scripts/
|
||
├── Master.conf # All user configuration — edit this file only
|
||
├── common.sh # Shared library — functions used by all scripts
|
||
├── Rsync/
|
||
│ └── rsync.sh # Core rsync script — called per share
|
||
├── Orchestrators/
|
||
│ └── daily_sync.sh # Daily media sync orchestrator
|
||
└── Tools/
|
||
└── recreate_shares.sh # Share directory recreation from cfg files |