Files

T

FailedProxy a3f90ed175 created loop for both watchdogs and consolidated orch lists

2026-04-23 22:44:56 -04:00

21 KiB

Raw Blame History

Rsync Setup Guide

For the unRAID Script Ecosystem — Master.conf · common.sh · rsync.sh · daily_sync_maintenance.sh · weekly_sync_maintenance.sh

Overview

This guide walks through setting up the rsync ecosystem on both your primary and secondary unRAID 7.x servers. By the end you will have:

SSH keys configured for server-to-server communication
Tailscale running on both servers for secure networking
A Gitea repository cloned to both servers
All scripts scheduled via the User Scripts plugin
Automated daily sync of media shares driven by orchestrators
Automated weekly clean sync of critical appdata (Emby + auth stack)
Optional personal encrypted shares synced for offsite backup

Prerequisites

Both servers need the following before starting:

unRAID 7.x
Community Applications plugin installed
User Scripts plugin installed via Community Applications
Tailscale plugin installed via Community Applications
Access to a Gitea instance (self-hosted recommended — Gitea runs as a Docker container on HOST1)
Terminal access to both servers (unRAID UI → Tools → Terminal, or SSH)

Step 1 — Tailscale Setup

Tailscale provides the secure network tunnel between your two servers. Scripts resolve the remote server's IP via Tailscale at runtime — no hardcoded IPs needed.

On Both Servers

Open Apps in the unRAID UI
Search for Tailscale and install the plugin
Go to Settings → Tailscale
Click Connect and authenticate with your Tailscale account
Verify both servers appear in your Tailscale admin console

Verify Connectivity

Run this on HOST1 to confirm it can reach HOST2:

tailscale ip -4 unRAID-Jayred365

You should get back a 100.x.x.x IP. If not, check both machines are authenticated in the Tailscale admin console.

Important: The hostnames in Master.conf (HOST1 and HOST2) must match the Tailscale machine names exactly — case sensitive.

Step 2 — Enable SSH on unRAID

unRAID 7.x has SSH disabled by default. Enable it on both servers so scripts can connect between them.

Go to Settings → Management Access
Under Secure Shell set SSH to Enabled
Set SSH port to 22
Click Apply

SSH is only exposed on your local network and Tailscale interface. Scripts connect via Tailscale IP — traffic is encrypted end-to-end.

Step 3 — Generate SSH Keys

The scripts use SSH keys for two purposes:

Server-to-server rsync and failover — each server authenticates to the other
Gitea access — both servers pull from the git repository

3a — Server-to-Server Keys

On HOST1 (unRAID-Gmer4Lfe):

ssh-keygen -t ed25519 -f /root/.ssh/Gmer4Lfe-rsync-key -C "gmer4lfe-rsync" -N ""

On HOST2 (unRAID-Jayred365):

ssh-keygen -t ed25519 -f /root/.ssh/Jayred365-rsync-key -C "jayred365-rsync" -N ""

3b — Authorise Keys on Each Server

HOST1's public key must be authorised on HOST2, and vice versa.

Copy HOST1 key → HOST2:

# On HOST1 — print the public key
cat /root/.ssh/Gmer4Lfe-rsync-key.pub

# On HOST2 — paste and authorise
mkdir -p /root/.ssh
echo "PASTE_PUBLIC_KEY_HERE" >> /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys

Copy HOST2 key → HOST1:

# On HOST2 — print the public key
cat /root/.ssh/Jayred365-rsync-key.pub

# On HOST1 — paste and authorise
echo "PASTE_PUBLIC_KEY_HERE" >> /root/.ssh/authorized_keys

3c — Test the Connection

From HOST1, verify it can SSH to HOST2 without a password prompt:

ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@$(tailscale ip -4 unRAID-Jayred365) "echo connected"

You should see connected. If prompted for a password the key was not authorised correctly — recheck Step 3b.

3d — Gitea SSH Key

Generate a separate key for Gitea access on each server:

ssh-keygen -t ed25519 -f /root/.ssh/unraid_gitea -C "unraid-gitea" -N ""

Add the public key to your Gitea account:

cat /root/.ssh/unraid_gitea.pub

Copy the output and add it in Gitea under Settings → SSH / GPG Keys → Add Key.

Step 4 — Clone the Git Repository

Both servers clone from the same Gitea repository. Updates pushed to the repo propagate to both servers on the next daily git pull.

On Both Servers

# Create the target directory
mkdir -p /mnt/user/appdata/unraid_scripts

# Clone the repository
GIT_SSH_COMMAND="ssh -i /root/.ssh/unraid_gitea" \
    git clone git@YOUR_GITEA_HOST:FailedProxy/Unraid_Scripts.git \
    /mnt/user/appdata/unraid_scripts

Replace YOUR_GITEA_HOST with your Gitea server address and port.

Verify the Structure

ls /mnt/user/appdata/unraid_scripts

You should see:

Master.conf
common.sh
Orchestrators/
Rsync/
Failover/
Docker_Essentials/
unRAID_Essentials/
Media/
Transcodes/
Monitors/
Tools/

Make Scripts Executable

find /mnt/user/appdata/unraid_scripts -name "*.sh" -exec chmod +x {} \;

Step 5 — Configure Master.conf

All user configuration lives in Master.conf. Open it and fill in your values:

nano /mnt/user/appdata/unraid_scripts/Master.conf

Host Configuration

HOST1="unRAID-Gmer4Lfe"          # must match Tailscale machine name exactly
HOST2="unRAID-Jayred365"

HOST1_SSH_KEY="/root/.ssh/Gmer4Lfe-rsync-key"
HOST2_SSH_KEY="/root/.ssh/Jayred365-rsync-key"

HOST1_EMBY_CONTAINER="Emby"
HOST1_EMBY_URL="http://localhost:8096"
HOST1_EMBY_API_KEY="your-host1-emby-api-key"    # Emby Dashboard → API Keys → + New Key

HOST2_EMBY_CONTAINER="Emby-Jayred365"
HOST2_EMBY_URL="http://localhost:8096"
HOST2_EMBY_API_KEY="your-host2-emby-api-key"

Git / Repo

GITEA_CONTAINER="Gitea"                            # exact Docker container name
GITEA_REPO_PATH="FailedProxy/Unraid_Scripts.git"
GITEA_DOMAIN=""                                     # optional public domain fallback
TARGET_DIR="/mnt/user/appdata/unraid_scripts"
GITEA_SSH_KEY="/root/.ssh/unraid_gitea"
SSH_PORT=221

Orchestrators — Daily Sync Shares

Define which shares each server owns. Each server only pushes the shares it is source of truth for — the other server mirrors and treats them as read-only.

HOST1_DAILY_SYNC_SHARES=(
    /mnt/user/Movies
    /mnt/user/Tv_Shows
    /mnt/user/Music
    # add all HOST1-managed shares here
)

HOST2_DAILY_SYNC_SHARES=(
    /mnt/user/Anime_Shows
    /mnt/user/Anime_Movies
    # add all HOST2-managed shares here
)

Never put the same share in both lists. One server is always the truth holder for each share.

Orchestrators — Weekly Sync Jobs

Shares synced during the Sunday maintenance window with containers stopped both sides:

WEEKLY_SYNC_JOBS=(
    "/mnt/user/Media_Server/Emby"               # full clean Emby mirror
    "/mnt/user/appdata-Failover/Critical-Data"  # auth stack
)

Step 6 — Rsync Profiles

Profiles control per-share rsync behaviour for appdata syncs. The profile key is matched automatically by the basename of the directory passed to rsync.sh (lowercased). Override with --profile=name.

One array drives both local and remote container stops. Same container names on both servers — consistent naming is a requirement of this ecosystem.

Current Profiles

Profile	Purpose	Containers Stopped
`arrs_stack`	Arr databases	Sonarr, Radarr, Lidarr, Prowlarr, Bazarr, Pinchflat
`critical-data`	Auth stack	Mariadb, Redis, LLDAP, NPM, Authelia (delayed start)
`important-data`	NextCloud + Postgres	Postgres, NextCloud (delayed start)
`gmer4lfe`	Server-specific appdata	Organizr, UptimeKuma, VaultWarden
`emby`	Weekly clean sync	Emby both sides — WAL checkpointed
`emby-failover`	Frequent dirty sync	None — Emby stays running

Two Emby Profiles

emby-failover — every 30-60min, Emby stays running:
  WAL and SHM excluded — safe while Emby is active
  Critical failover data only: users.db, library.db, authentication.db, config/
  Fast, small dataset — users continue watching without interruption on failover
  Also used for failover writeback on handback

emby — weekly Sunday 2:30am, both Emby instances stopped:
  WAL checkpointed on shutdown — full consistent mirror
  Full mirror: metadata, plugins, config all included
  Minimal excludes: logs, transcodes, cache, crash files only
  Cache stays warm on HOST2 all week — only reset on Sunday
  emby-failover covers the critical state between weekly syncs

Step 7 — Personal Encrypted Shares

Personal shares can be synced to the remote server for offsite backup. ZFS encrypts at the dataset level — the remote server receives encrypted blocks and cannot read the content without your passphrase or keyfile.

ZFS Encryption Setup (unRAID 7)

Step 1 — Create an encrypted dataset:

In the unRAID UI go to Main → click your ZFS pool name
Click + Dataset to create a new dataset
Name it — e.g. Gmer4Lfe-Personal
Enable Encryption → set your passphrase

⚠️ Write your passphrase down — if lost, data is unrecoverable

Step 2 — Create the share:

Go to Settings → Shares → Add Share
Set the share path to your new encrypted dataset
Set Use cache: Only — keeps data on ZFS pool, not array

Step 3 — Verify encryption is active before syncing:

zfs get encryption poolname/Gmer4Lfe-Personal
# Should show: encryption  aes-256-gcm

Step 4 — Auto-unlock on boot (keyfile approach — optional):

# Create keyfile — on HOST1 only, never sync this file
dd if=/dev/urandom bs=32 count=1 | base64 > /root/.zfs-keys/personal.key
chmod 600 /root/.zfs-keys/personal.key

# Set dataset to use keyfile
zfs change-key -o keylocation=file:///root/.zfs-keys/personal.key \
    -o keyformat=raw poolname/Gmer4Lfe-Personal

# Add to array start (via array_start.sh or User Scripts):
zfs load-key poolname/Gmer4Lfe-Personal
zfs mount poolname/Gmer4Lfe-Personal

Manual unlock alternative (most secure):

zfs load-key poolname/Gmer4Lfe-Personal   # prompts for passphrase
zfs mount poolname/Gmer4Lfe-Personal

Step 5 — Add to Master.conf:

HOST1_PERSONAL_SHARES=(
    /mnt/user/Gmer4Lfe-Personal
)

Personal shares sync automatically with the daily media share sync in daily_sync_maintenance.sh. The remote server receives encrypted blocks — content is unreadable without your key.

Step 8 — Set Up User Scripts

The ecosystem uses a single orchestrator entry for array startup plus a small number of scheduled scripts.

At Startup of Array

Create one script entry named array start:

#!/bin/bash
bash /mnt/user/appdata/unraid_scripts/Orchestrators/array_start.sh

Set schedule to: At Startup of Array

This single entry launches everything configured in ARRAY_START_SCRIPTS in Master.conf:

ramdisk_setup.sh — creates ramdisk before Emby starts
docker_syslog_filter.sh — suppresses veth log noise
php_fpm_max_children.sh — WebGUI tuning
docker_network_connect.sh — connects containers to extra networks
system_watchdog.sh — continuous system health monitor
docker_watchdog.sh — continuous container health monitor
failover.sh — continuous mutual failover

Cron Schedules

Script	Schedule	Purpose
`transcode_management.sh`	`/3 * * *`	Transcode cleanup + manager
`arrs_failed_stalled_recovery.sh`	`0 /6 * *`	Blocklist + re-search failed imports
`rsync.sh ... --profile=emby-failover`	`/30 * * *`	Emby dirty sync
`daily_sync_maintenance.sh`	`0 1 * * *`	Full daily maintenance window
`weekly_sync_maintenance.sh`	`30 2 * * 0`	Weekly sync + updates + restarts
`weekly_health_digest.sh`	`0 8 * * 6`	Saturday morning health digest

Emby Failover Dirty Sync

Create a separate script entry named rsync emby failover:

#!/bin/bash
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
    /mnt/user/Media_Server/Emby --profile=emby-failover

Set schedule to: */30 * * * *

Appdata Profile Syncs

Create one entry per appdata profile you want on a schedule:

#!/bin/bash
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
    /mnt/user/appdata-Failover/Arrs_Stack

Profile	Recommended Schedule
`Arrs_Stack`	Every 12-24 hours
`Important-Data`	Every 6-12 hours
`Gmer4Lfe`	Daily or weekly
`emby`	Via `weekly_sync_maintenance.sh` only — do NOT schedule separately
`Critical-Data`	Via `weekly_sync_maintenance.sh` only — do NOT schedule separately

Set all scripts to run as Background Task — output streams correctly rather than buffering in the browser.

Step 9 — Verify the Setup

Before letting scheduled jobs run, test manually from the terminal on HOST1:

# Test a single appdata profile sync
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
    /mnt/user/appdata-Failover/Arrs_Stack --dry-run --log

# Test the daily sync orchestrator
bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh --dry-run

A healthy run will show:

━━━ ⚙️ Setup ━━━
ℹ️  Host: unRAID-Gmer4Lfe → unRAID-Jayred365
ℹ️  Remote IP: 100.x.x.x

━━━ 🛡️ Pre-flight Checks ━━━
✅ Remote reachable
✅ Remote rootfs: 12% (threshold: 75%)
✅ All pre-flight checks passed

If any pre-flight check fails the script aborts with a clear error before touching anything.

Step 10 — Secondary Server Initial Sync

If setting up HOST2 from scratch with empty shares:

Complete Steps 1–8 on HOST2
Start the array and create your shares in the unRAID UI
Run the share recreation tool to create disk directories from cfg files:

bash /mnt/user/appdata/unraid_scripts/Tools/recreate_shares.sh

Run the initial push from HOST1 — this populates HOST2's empty shares:

bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh --log

Once complete, scheduled runs take over automatically.

Naming Consistency — Required

The ecosystem is built on the assumption that containers and shares have identical names on both servers. This is not optional — it is what makes one codebase work on both servers without modification.

Container names must match exactly:
  Emby                ← HOST1 and HOST2
  NginxProxyManager   ← HOST1 and HOST2
  Mariadb-Authelia    ← HOST1 and HOST2

Share paths must match exactly:
  /mnt/user/Movies    ← HOST1 and HOST2
  /mnt/user/Tv_Shows  ← HOST1 and HOST2

If a container or share has a different name on one server the script skips it gracefully — but it will not do what you expect. Diverge from consistent naming and every script that touches containers or shares needs custom logic for each server. Keep naming consistent and one codebase covers both servers automatically.

Troubleshooting

SSH connection refused

Verify SSH is enabled (Step 2)
Confirm the correct key is referenced in Master.conf
Test Tailscale: tailscale ip -4 HOSTNAME

Pre-flight aborts on rootfs

Remote rootfs above ROOTFS_WARN threshold
Check remote array is started and drives are mounted
Run df / on the remote to see current usage

Share exists but has no content — drives may not be mounted
Run recreate_shares.sh if setting up fresh

Containers not stopping/starting

Verify container names in Master.conf match Docker exactly — case sensitive
Test manually: ssh -i /root/.ssh/KEY root@REMOTE_IP "docker ps"

Script not found

Verify repo was cloned to /mnt/user/appdata/unraid_scripts/
Make scripts executable: find /mnt/user/appdata/unraid_scripts -name "*.sh" -exec chmod +x {} \;

Profile not matching

Profile key is matched by directory basename lowercased
/mnt/user/appdata-Failover/Arrs_Stack → basename Arrs_Stack → key arrs_stack
Override with --profile=name if basename doesn't match

Available Flags

All scripts support:

Flag	Description
`--dry-run`	Run without making any changes
`--log`	Enable verbose logging output
`--no-log`	Disable logging
`--status`	Print resolved configuration and exit

# Preview what would be synced
bash rsync.sh /mnt/user/Movies --dry-run --log

# Check resolved profile settings
bash rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --status

# Test daily orchestrator without changes
bash daily_sync_maintenance.sh --dry-run

Repository Structure

Unraid_Scripts/
├── Master.conf                          # All user configuration — edit this file only
├── common.sh                            # Shared library — functions used by all scripts
│
├── Orchestrators/
│   ├── array_start.sh                   # Single array-start entry point
│   ├── daily_sync_maintenance.sh        # Daily maintenance window orchestrator
│   ├── weekly_sync_maintenance.sh       # Weekly maintenance window orchestrator
│   ├── media_management.sh              # Permissions + cleaners + arr cleanup
│   └── transcode_management.sh          # Transcode cleanup + manager
│
├── Rsync/
│   └── rsync.sh                         # Core rsync script — called per share
│
├── Failover/
│   ├── failover.sh                      # Mutual container failover — continuous loop
│   ├── failover_test.sh                 # Controlled failover simulation
│   └── failover_state_reset.sh          # Reset failover state manually
│
├── Docker_Essentials/
│   ├── docker_watchdog.sh               # Two-tier container monitor — continuous loop
│   ├── docker_daily_restart.sh          # Daily container restarts
│   ├── docker_weekly_restart.sh         # Weekly container restarts
│   └── docker_network_connect.sh        # Connect containers to extra networks
│
├── unRAID_Essentials/
│   ├── system_watchdog.sh               # System health monitor — continuous loop
│   ├── ramdisk_setup.sh                 # Creates ramdisk + symlink at array start
│   ├── docker_syslog_filter.sh          # Suppress veth log noise
│   ├── php_fpm_max_children.sh          # WebGUI performance tuning
│   ├── server_reboot.sh                 # Graceful scheduled reboot
│   ├── mover_stop.sh                    # Stop mover cleanly
│   ├── clear_logs.sh                    # Weekly log cleanup
│   ├── webgui_restart.sh                # nginx + emhttp restart escalation
│   └── git_pull_execute.sh              # Pull latest scripts from Gitea
│
├── Media/
│   ├── media_shares_permissions.sh      # Apply permissions to media shares
│   ├── media_cleaner.sh                 # Remove junk files from media shares
│   ├── lidarr_cleanup.sh                # Remove orphaned music files
│   ├── sonarr_cleanup.sh                # Remove orphaned TV files
│   ├── radarr_cleanup.sh                # Remove orphaned movie files
│   └── arrs_failed_stalled_recovery.sh  # Blocklist + re-search failed imports
│
├── Transcodes/
│   ├── transcode_manager.sh             # Symlink direction management
│   ├── transcode_cleanup.sh             # Remove old inactive transcode files
│   └── ramdisk_setup.sh                 # (also in unRAID_Essentials — symlinked)
│
├── Monitors/
│   ├── cert_monitor.sh                  # SSL certificate expiry monitoring
│   ├── backup_verify.sh                 # Checksum verification against remote
│   ├── smart_health.sh                  # Drive SMART attribute monitoring
│   ├── zfs_memory_snapshot.sh           # Weekly ZFS health + memory report
│   ├── bandwidth_monitor.sh             # Rsync transfer logging + weekly summary
│   ├── weekly_health_digest.sh          # Aggregated health digest email
│   ├── emby_session_report.sh           # Weekly Emby usage statistics
│   └── emby_database_repair.sh          # Emby SQLite database repair
│
└── Tools/
    ├── recreate_shares.sh               # Recreate share directories from cfg files
    ├── bulk_permissions_repair.sh       # One-shot permission repair
    ├── watchdog_skip_list_manager.sh    # Manage docker watchdog skip list
    ├── rsync_stop.sh                    # Stop active rsync jobs cleanly
    ├── user_scripts_stop.sh             # Stop running user scripts
    └── container_data_export.sh         # Export container configuration

21 KiB Raw Blame History Unescape Escape

Rsync Setup Guide

Overview

Prerequisites

Step 1 — Tailscale Setup

On Both Servers

Verify Connectivity

Step 2 — Enable SSH on unRAID

Step 3 — Generate SSH Keys

3a — Server-to-Server Keys

3b — Authorise Keys on Each Server

3c — Test the Connection

3d — Gitea SSH Key

Step 4 — Clone the Git Repository

On Both Servers

Verify the Structure

Make Scripts Executable

Step 5 — Configure Master.conf

Host Configuration

Git / Repo

Orchestrators — Daily Sync Shares

Orchestrators — Weekly Sync Jobs

Step 6 — Rsync Profiles

Current Profiles

Two Emby Profiles

Step 7 — Personal Encrypted Shares

ZFS Encryption Setup (unRAID 7)

Step 8 — Set Up User Scripts

At Startup of Array

Cron Schedules

Emby Failover Dirty Sync

Appdata Profile Syncs

Step 9 — Verify the Setup

Step 10 — Secondary Server Initial Sync

Naming Consistency — Required

Troubleshooting

SSH connection refused

Pre-flight aborts on rootfs

Pre-flight aborts on empty share

Containers not stopping/starting

Script not found

Profile not matching

Available Flags

Repository Structure

21 KiB

Raw Blame History