guide adjustment needs more in the futur
This commit is contained in:
+79
-212
@@ -2,197 +2,119 @@ This guide is a work in progress and generated by ai, dont have time for guides.
|
||||
|
||||
Rsync Setup Guide
|
||||
|
||||
Status: Work in Progress
|
||||
For the unRAID Rsync Ecosystem — common.sh · Master.conf · rsync.sh · daily_sync.sh
|
||||
|
||||
|
||||
Status: Work in Progress
|
||||
For the unRAID Rsync Ecosystem — common.sh · Master.conf · rsync.sh · daily_sync.sh
|
||||
|
||||
Overview
|
||||
|
||||
This guide walks through setting up the rsync ecosystem on both your primary and secondary unRAID 7.x servers. By the end you will have:
|
||||
|
||||
A Gitea repository cloned to both servers
|
||||
SSH keys configured for server-to-server communication
|
||||
Tailscale running on both servers for secure networking
|
||||
Scripts scheduled and running via the User Scripts plugin
|
||||
Automated daily sync of media shares and appdata profiles
|
||||
|
||||
SSH keys configured for server-to-server communication
|
||||
Tailscale running on both servers for secure networking
|
||||
Scripts scheduled and running via the User Scripts plugin
|
||||
Automated daily sync of media shares and appdata profiles
|
||||
Prerequisites
|
||||
|
||||
Both servers need the following before starting:
|
||||
|
||||
unRAID 7.x
|
||||
User Scripts plugin installed via Community Applications
|
||||
Tailscale plugin installed via Community Applications
|
||||
Access to a Gitea instance (self-hosted or remote)
|
||||
Terminal access to both servers (via unRAID UI → Tools → Terminal, or SSH)
|
||||
|
||||
unRAID 7.x
|
||||
User Scripts plugin installed via Community Applications
|
||||
Tailscale plugin installed via Community Applications
|
||||
Terminal access to both servers (via unRAID UI → Tools → Terminal, or SSH)
|
||||
Step 1 — Tailscale Setup
|
||||
|
||||
Tailscale provides the secure network tunnel between your two servers. The scripts resolve the remote server's IP via Tailscale at runtime.
|
||||
|
||||
On Both Servers
|
||||
|
||||
Open Apps in the unRAID UI
|
||||
Search for Tailscale and install the plugin
|
||||
Once installed go to Settings → Tailscale
|
||||
Click Connect and authenticate with your Tailscale account
|
||||
Verify both servers appear in your Tailscale admin console
|
||||
Open Apps in the unRAID UI
|
||||
Search for Tailscale and install the plugin
|
||||
Once installed, go to Settings → Tailscale
|
||||
Click Connect and authenticate with your Tailscale account
|
||||
Verify both servers appear in your Tailscale admin console
|
||||
|
||||
Verify Connectivity
|
||||
|
||||
Run this on the primary to confirm it can see the secondary:
|
||||
bash
|
||||
|
||||
tailscale ip -4 unRAID-Jayred365
|
||||
|
||||
You should get back a 100.x.x.x IP. If not, check that both machines are authenticated and connected in the Tailscale admin console.
|
||||
You should get back a 100.x.x.x IP. If not, check both servers in the Tailscale admin console.
|
||||
|
||||
Note: The hostnames used in Master.conf (HOST1 and HOST2) must match the Tailscale machine names exactly — these are case sensitive.
|
||||
Note: The hostnames used in Master.conf (HOST1 and HOST2) must match the Tailscale machine names exactly — these are case sensitive.
|
||||
|
||||
Step 2 — Generate SSH Keys
|
||||
Step 2 — Generate SSH Keys (Server-to-Server)
|
||||
|
||||
The scripts use SSH keys for two purposes:
|
||||
|
||||
Server-to-server rsync — primary authenticates to secondary (and vice versa)
|
||||
Gitea access — both servers pull from the git repository
|
||||
|
||||
2a — Server-to-Server Keys
|
||||
|
||||
Run the following on each server to generate its rsync key. Replace the filename with the appropriate server name.
|
||||
The scripts use SSH keys for server-to-server rsync.
|
||||
|
||||
On Primary (unRAID-Gmer4Lfe):
|
||||
bash
|
||||
|
||||
ssh-keygen -t ed25519 -f /root/.ssh/Gmer4Lfe-rsync-key -C "gmer4lfe-rsync" -N ""
|
||||
|
||||
On Secondary (unRAID-Jayred365):
|
||||
bash
|
||||
|
||||
ssh-keygen -t ed25519 -f /root/.ssh/Jayred365-rsync-key -C "jayred365-rsync" -N ""
|
||||
Copy Public Keys to Each Server
|
||||
|
||||
2b — Copy Public Keys to Each Server
|
||||
Primary → Secondary
|
||||
|
||||
The primary's public key must be authorised on the secondary, and vice versa.
|
||||
|
||||
Copy primary key → secondary:
|
||||
bash
|
||||
|
||||
# Run on primary
|
||||
# On primary
|
||||
cat /root/.ssh/Gmer4Lfe-rsync-key.pub
|
||||
|
||||
Copy the output. Then on the secondary:
|
||||
bash
|
||||
Copy the output. Then on secondary:
|
||||
|
||||
# Run on secondary
|
||||
mkdir -p /root/.ssh
|
||||
echo "PASTE_PUBLIC_KEY_HERE" >> /root/.ssh/authorized_keys
|
||||
chmod 600 /root/.ssh/authorized_keys
|
||||
|
||||
Copy secondary key → primary:
|
||||
bash
|
||||
Secondary → Primary
|
||||
|
||||
# Run on secondary
|
||||
# On secondary
|
||||
cat /root/.ssh/Jayred365-rsync-key.pub
|
||||
|
||||
Copy the output. Then on the primary:
|
||||
bash
|
||||
Copy the output. Then on primary:
|
||||
|
||||
# Run on primary
|
||||
mkdir -p /root/.ssh
|
||||
echo "PASTE_PUBLIC_KEY_HERE" >> /root/.ssh/authorized_keys
|
||||
chmod 600 /root/.ssh/authorized_keys
|
||||
Test the Connection
|
||||
|
||||
2c — Test the Connection
|
||||
|
||||
From the primary, test that it can SSH to the secondary without a password prompt:
|
||||
bash
|
||||
From the primary, test SSH access to the secondary:
|
||||
|
||||
ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@$(tailscale ip -4 unRAID-Jayred365) "echo connected"
|
||||
|
||||
You should see connected. If prompted for a password the key was not authorised correctly — recheck Step 2b.
|
||||
2d — Gitea SSH Key
|
||||
You should see connected. If prompted for a password, recheck the key authorization.
|
||||
|
||||
Generate a separate key for Gitea access on each server:
|
||||
bash
|
||||
|
||||
ssh-keygen -t ed25519 -f /root/.ssh/id_gitea_rsync -C "unraid-gitea" -N ""
|
||||
|
||||
Add the public key to your Gitea account:
|
||||
bash
|
||||
|
||||
cat /root/.ssh/id_gitea_rsync.pub
|
||||
|
||||
Copy the output and add it in Gitea under Settings → SSH / GPG Keys → Add Key.
|
||||
Step 3 — Enable SSH on unRAID
|
||||
|
||||
unRAID 7.x has SSH disabled by default. Enable it on both servers so the scripts can connect between them.
|
||||
unRAID 7.x has SSH disabled by default. Enable it on both servers so the scripts can connect:
|
||||
|
||||
Go to Settings → Management Access
|
||||
Under Secure Shell set SSH to Enabled
|
||||
Set SSH port to 22 (default)
|
||||
Click Apply
|
||||
Go to Settings → Management Access
|
||||
Under Secure Shell, set SSH to Enabled
|
||||
Set SSH port to 22 (default)
|
||||
Click Apply
|
||||
|
||||
Security note: SSH is only exposed on your local network and Tailscale interface. The rsync scripts connect via the Tailscale IP so traffic is encrypted end-to-end.
|
||||
Security note: SSH is only exposed on your local network and Tailscale interface. The rsync scripts connect via Tailscale IP, so traffic is encrypted end-to-end.
|
||||
|
||||
Step 4 — Clone the Git Repository
|
||||
|
||||
The scripts live in a Gitea repository. Both servers clone from the same repo so updates propagate everywhere via a single git pull.
|
||||
On Both Servers
|
||||
bash
|
||||
|
||||
# Create the target directory
|
||||
mkdir -p /mnt/user/appdata/unraid_scripts
|
||||
|
||||
# Clone the repository
|
||||
GIT_SSH_COMMAND="ssh -i /root/.ssh/id_gitea_rsync" \
|
||||
git clone git@YOUR_GITEA_HOST:YOUR_USER/Unraid_Scripts.git \
|
||||
/mnt/user/appdata/unraid_scripts
|
||||
|
||||
Replace YOUR_GITEA_HOST and YOUR_USER with your Gitea server address and username.
|
||||
Verify the Structure
|
||||
bash
|
||||
|
||||
ls /mnt/user/appdata/unraid_scripts
|
||||
|
||||
You should see:
|
||||
|
||||
Master.conf
|
||||
common.sh
|
||||
Rsync/
|
||||
rsync.sh
|
||||
Orchestrators/
|
||||
daily_sync.sh
|
||||
Tools/
|
||||
recreate_shares.sh
|
||||
|
||||
Make Scripts Executable
|
||||
bash
|
||||
|
||||
chmod +x /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh
|
||||
chmod +x /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync.sh
|
||||
chmod +x /mnt/user/appdata/unraid_scripts/Tools/recreate_shares.sh
|
||||
|
||||
Step 5 — Configure Master.conf
|
||||
Step 4 — Configure Master.conf
|
||||
|
||||
All user configuration lives in Master.conf. Open it and adjust the following to match your setup:
|
||||
bash
|
||||
|
||||
nano /mnt/user/appdata/unraid_scripts/Master.conf
|
||||
|
||||
Required Changes
|
||||
Required Changes:
|
||||
|
||||
Variable Description Example
|
||||
HOST1 Hostname of your primary server unRAID-Gmer4Lfe
|
||||
HOST2 Hostname of your secondary server unRAID-Jayred365
|
||||
HOST1_SSH_KEY Path to primary's rsync private key /root/.ssh/Gmer4Lfe-rsync-key
|
||||
HOST2_SSH_KEY Path to secondary's rsync private key /root/.ssh/Jayred365-rsync-key
|
||||
REPO_SSH SSH URL of your Gitea repository git@192.168.50.2:User/Unraid_Scripts.git
|
||||
GITEA_SSH_KEY Path to Gitea private key /root/.ssh/id_gitea_rsync
|
||||
BW_LIMIT Global bandwidth limit in KB/s 12500
|
||||
ROOTFS_WARN Remote rootfs % threshold before aborting 75
|
||||
|
||||
Daily Sync Shares
|
||||
|
||||
Add the full paths of all media shares you want synced nightly:
|
||||
bash
|
||||
|
||||
DAILY_SYNC_SHARES=(
|
||||
/mnt/user/Movies
|
||||
@@ -203,8 +125,7 @@ DAILY_SYNC_SHARES=(
|
||||
|
||||
Profiles
|
||||
|
||||
Profiles control per-share rsync behaviour for your frequently synced appdata shares. Each profile is matched by the directory basename (lowercased). Add a key to each profile array for any share that needs custom settings:
|
||||
bash
|
||||
Profiles control per-share rsync behaviour for your frequently synced appdata shares.
|
||||
|
||||
declare -A PROFILE_CRITICAL_CONTAINER_NAMES=(
|
||||
[arrs_stack]="Sonarr Radarr Lidarr Prowlarr"
|
||||
@@ -212,137 +133,83 @@ declare -A PROFILE_CRITICAL_CONTAINER_NAMES=(
|
||||
)
|
||||
|
||||
Any share with no matching profile falls through to the global DEFAULT_RSYNC_OPTS.
|
||||
Step 6 — Set Up User Scripts
|
||||
|
||||
The User Scripts plugin is how unRAID schedules and runs the scripts. Each sync job is its own script entry in the plugin.
|
||||
Frequent Sync Jobs (Scheduled Individually)
|
||||
Step 5 — Set Up User Scripts
|
||||
|
||||
Create one script entry per appdata profile. Go to Plugins → User Scripts → Add New Script.
|
||||
The User Scripts plugin schedules and runs the scripts. Each sync job is its own script entry.
|
||||
|
||||
Name it descriptively — e.g. rsync appdata arrs_stack.
|
||||
|
||||
In the script body paste:
|
||||
bash
|
||||
Frequent Sync Jobs
|
||||
|
||||
Create one script entry per appdata profile (Plugins → User Scripts → Add New Script)
|
||||
Example for ARR stack:
|
||||
#!/bin/bash
|
||||
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/appdata-Failover/Arrs_Stack
|
||||
|
||||
Set the schedule to match your desired frequency:
|
||||
Profile Recommended Schedule
|
||||
Emby Every 1 hour
|
||||
Critical-Data Every 2 hours
|
||||
Important-Data Every 6 hours
|
||||
Arrs_Stack Every 12 hours
|
||||
|
||||
Important: Set each script to run as a Background Task — this ensures output streams correctly to the log rather than buffering in the browser.
|
||||
|
||||
Daily Sync Orchestrator
|
||||
|
||||
Create one more script entry for the daily media sync:
|
||||
|
||||
Name it daily media sync.
|
||||
|
||||
In the script body paste:
|
||||
bash
|
||||
|
||||
#!/bin/bash
|
||||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync.sh
|
||||
Schedule daily at 01:00
|
||||
Set scripts to run as Background Tasks
|
||||
Step 6 — Verify the Setup
|
||||
|
||||
Set the schedule to Daily at 01:00.
|
||||
Step 7 — Verify the Setup
|
||||
|
||||
Before letting the scheduled jobs run, do a manual test from the terminal on the primary:
|
||||
bash
|
||||
Manually test from the primary:
|
||||
|
||||
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --log
|
||||
|
||||
A healthy run will show:
|
||||
You should see a healthy run with:
|
||||
|
||||
━━━ ⚙️ Setup ━━━
|
||||
ℹ️ [INFO] 🖥️ Host: unRAID-Gmer4Lfe → unRAID-Jayred365
|
||||
ℹ️ [INFO] 🌐 Remote IP: 100.x.x.x
|
||||
|
||||
━━━ 🛡️ Pre-flight Checks ━━━
|
||||
ℹ️ [INFO] 📡 unRAID-Jayred365 is reachable
|
||||
ℹ️ [INFO] 🩺 Remote rootfs: 12% used (threshold: 75%)
|
||||
ℹ️ [INFO] 🩺 Remote share verified: ...
|
||||
ℹ️ [INFO] 💾 disk1 🟢 — share present
|
||||
...
|
||||
✅ [OK] All disks backing share are online
|
||||
Step 7 — Secondary Server Initial Setup
|
||||
|
||||
If any pre-flight check fails the script will abort with a clear error and hint before touching anything.
|
||||
Step 8 — Secondary Server Initial Setup
|
||||
|
||||
If setting up the secondary from scratch (no existing data):
|
||||
|
||||
Complete Steps 1–5 on the secondary
|
||||
Start the array and create your shares in the unRAID UI
|
||||
Run the share recreation tool to create disk directories from your cfg files:
|
||||
|
||||
bash
|
||||
If setting up secondary from scratch:
|
||||
|
||||
Complete Steps 1–5 on the secondary
|
||||
Start the array and create your shares in the unRAID UI
|
||||
Run the share recreation tool:
|
||||
bash /mnt/user/appdata/unraid_scripts/Tools/recreate_shares.sh
|
||||
|
||||
Temporarily remove --delete from DEFAULT_RSYNC_OPTS in Master.conf
|
||||
Run the initial push from the primary — the .recovery marker files allow rsync to populate empty shares without aborting
|
||||
Once complete, restore --delete to Master.conf
|
||||
The next nightly run will clean up the .recovery marker files automatically
|
||||
|
||||
Temporarily remove --delete from DEFAULT_RSYNC_OPTS for initial push
|
||||
Restore --delete after the first run; next nightly run will clean .recovery files automatically
|
||||
Troubleshooting
|
||||
|
||||
SSH connection refused
|
||||
|
||||
Verify SSH is enabled on the target server (Step 3)
|
||||
Check the correct key is referenced in Master.conf
|
||||
Confirm the Tailscale IP resolves: tailscale ip -4 HOSTNAME
|
||||
Verify SSH is enabled on target server (Step 3)
|
||||
Check correct key referenced in Master.conf
|
||||
Confirm Tailscale IP resolves
|
||||
|
||||
Pre-flight aborts on rootfs
|
||||
Pre-flight aborts
|
||||
|
||||
Remote rootfs is above ROOTFS_WARN threshold
|
||||
Check if the remote array is started and drives are mounted
|
||||
df / on the remote to see current usage
|
||||
|
||||
Pre-flight aborts on empty share
|
||||
|
||||
Share exists but has no content — drives may not be mounted
|
||||
Run recreate_shares.sh if setting up fresh
|
||||
Check array status on the remote server
|
||||
|
||||
Pre-flight aborts on disk check
|
||||
|
||||
One or more disks backing the share are not mounted
|
||||
Check Main → Array Devices on the remote for offline disks
|
||||
Verify disk assignments are correct after any hardware changes
|
||||
Rootfs above ROOTFS_WARN
|
||||
Share exists but empty
|
||||
Disk offline
|
||||
|
||||
Script not found
|
||||
|
||||
Verify the repo was cloned to /mnt/user/appdata/unraid_scripts/
|
||||
Check scripts are executable: chmod +x /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh
|
||||
|
||||
Containers not stopping/starting
|
||||
|
||||
Verify container names in Master.conf match exactly what Docker shows
|
||||
Check SSH key has access to run docker commands on the remote
|
||||
Test manually: ssh -i /root/.ssh/KEY root@REMOTE_IP "docker ps"
|
||||
|
||||
Verify scripts exist in /mnt/user/appdata/unraid_scripts/
|
||||
Make scripts executable:
|
||||
chmod +x /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh
|
||||
chmod +x /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync.sh
|
||||
chmod +x /mnt/user/appdata/unraid_scripts/Tools/recreate_shares.sh
|
||||
Available Flags
|
||||
|
||||
All scripts support the following flags:
|
||||
Flag Description
|
||||
--dry-run or -n Run without making any changes
|
||||
--log Enable verbose logging output
|
||||
--no-log Disable logging (overrides Master.conf)
|
||||
--dry-run or -n Run without making changes
|
||||
--log Enable verbose logging
|
||||
--no-log Disable logging
|
||||
--status Print resolved configuration and exit
|
||||
|
||||
Example:
|
||||
bash
|
||||
Examples
|
||||
|
||||
# Preview what would be synced without transferring anything
|
||||
# Preview what would be synced without transferring
|
||||
bash rsync.sh /mnt/user/Movies --dry-run --log
|
||||
|
||||
# Check what profile and settings resolved for a share
|
||||
# Check profile and settings resolved for a share
|
||||
bash rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --status
|
||||
|
||||
Repository Structure
|
||||
|
||||
Unraid_Scripts/
|
||||
├── Master.conf # All user configuration — edit this file only
|
||||
├── common.sh # Shared library — functions used by all scripts
|
||||
@@ -351,4 +218,4 @@ Unraid_Scripts/
|
||||
├── Orchestrators/
|
||||
│ └── daily_sync.sh # Daily media sync orchestrator
|
||||
└── Tools/
|
||||
└── recreate_shares.sh # Share directory recreation from cfg files
|
||||
└── recreate_shares.sh # Share directory recreation from cfg fil
|
||||
Reference in New Issue
Block a user