feat: slskd reconnect guard in downloaders_reset, mass v2 sync
- downloaders_reset: connection check block before slskd API sections; triggers PUT /api/v0/server reconnect if disconnected, polls 60s, gates Stuck Searches and Dead Transfer Records on SLSKD_CONNECTED - Sync all modified/new/deleted files from v2 refactor across Docker_Essentials, Media, Monitors, Partnership, Rsync, Tools, Transcodes, unRAID_Essentials, common.sh, master confs, and new Manual/README docs
This commit is contained in:
Gmer4Lfe
@@ -0,0 +1,721 @@
|
||||
# ━━━━━ RSYNC — Manual ━━━━━
|
||||
|
||||
Config reference, procedures, operational workflows.
|
||||
For overview see README-Rsync.md. For per-script detail see the rsync.sh header.
|
||||
|
||||
This document also serves as the **initial setup guide** for standing up the
|
||||
two-server ecosystem from scratch.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ WHAT YOU'RE BUILDING ━━━
|
||||
|
||||
```
|
||||
HOST1 (unRAID-Gmer4Lfe) HOST2 (unRAID-Jayred365)
|
||||
────────────────────── ──────────────────────
|
||||
Downloads to: Downloads to:
|
||||
/mnt/user/Movies /mnt/user/Anime_Movies
|
||||
/mnt/user/Tv_Shows /mnt/user/Anime_Shows
|
||||
/mnt/user/Music
|
||||
|
||||
Both servers' arrs track ALL content across both servers.
|
||||
It does not matter who downloaded what or when.
|
||||
|
||||
Daily 3-phase window (1am):
|
||||
|
||||
Phase 1 — arr_sync.sh (runs first):
|
||||
All arrs on all nodes reconcile libraries using external IDs
|
||||
(TMDB, TVDB, MusicBrainz). Union model — any node that tracks
|
||||
an item, all nodes get it. After this phase both servers' arrs
|
||||
know about all content regardless of who downloaded it.
|
||||
|
||||
Phase 2 — rsync (bidirectional, no --delete on media shares):
|
||||
HOST1 pushes its shares ──────► HOST2 receives files
|
||||
HOST1 receives files ◄────── HOST2 pushes its shares
|
||||
Files arrive already tracked by the remote arr (arr_sync ran first).
|
||||
No --delete — media files only spread. Arr cleanup handles deletions.
|
||||
|
||||
Phase 3 — arr cleanup:
|
||||
sonarr_cleanup / radarr_cleanup / lidarr_cleanup query the live arr
|
||||
API and remove any files no longer tracked. Emby notified after.
|
||||
|
||||
Note: profiled syncs (arrs_stack, emby, critical-data…) use their own
|
||||
PROFILE_RSYNC_OPTS which include --delete. Only the no-profile media
|
||||
shares use DEFAULT_RSYNC_OPTS (spread only).
|
||||
|
||||
Weekly clean sync (2:30am Sunday, containers stopped):
|
||||
Emby — full clean mirror ◄──────► Emby
|
||||
Critical-Data ──────► Auth stack (HOST1 → HOST2)
|
||||
|
||||
Every 30 minutes — dirty sync:
|
||||
Emby watch states ──────► HOST2 stays current on playback
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ PREREQUISITES ━━━
|
||||
|
||||
Required on both servers before starting:
|
||||
|
||||
```
|
||||
unRAID 7.x
|
||||
Community Applications plugin — search "Community Applications" in unRAID plugins
|
||||
User Scripts plugin — install via Community Applications
|
||||
Tailscale plugin — install via Community Applications
|
||||
Terminal access — unRAID UI → Tools → Terminal, or SSH
|
||||
```
|
||||
|
||||
Optional but recommended:
|
||||
```
|
||||
Gitea (Docker container on HOST1) — self-hosted git for the script repository
|
||||
Working Emby installation — for transcode management and failover
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 1 — TAILSCALE ━━━
|
||||
|
||||
Tailscale provides the encrypted mesh network between servers. Scripts resolve the
|
||||
remote server's IP via Tailscale at runtime — no hardcoded IPs, no VPN configuration,
|
||||
no open ports. All server-to-server communication goes through Tailscale.
|
||||
|
||||
### Install on Both Servers
|
||||
|
||||
```
|
||||
1. Open Apps in the unRAID UI
|
||||
2. Search "Tailscale" — install the plugin
|
||||
3. Settings → Tailscale → Connect
|
||||
4. Authenticate with your Tailscale account (browser opens on your machine)
|
||||
5. Verify both servers appear: https://login.tailscale.com/admin/machines
|
||||
```
|
||||
|
||||
### Verify Connectivity
|
||||
|
||||
```bash
|
||||
# From HOST1 — should return HOST2's 100.x.x.x Tailscale IP:
|
||||
tailscale ip -4 unRAID-Jayred365
|
||||
|
||||
# Test actual connectivity:
|
||||
tailscale ping unRAID-Jayred365
|
||||
```
|
||||
|
||||
> **Critical:** The hostnames in `master.conf` (`HOST1` and `HOST2`) must match the
|
||||
> Tailscale machine names **exactly** — case sensitive. All remote IP resolution goes
|
||||
> through `tailscale ip -4 HOSTNAME`. A name mismatch causes every remote operation to
|
||||
> fail at the IP resolution step.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 2 — ENABLE SSH ━━━
|
||||
|
||||
unRAID 7.x has SSH disabled by default. Enable it on both servers.
|
||||
|
||||
```
|
||||
Settings → Management Access → Secure Shell
|
||||
SSH: Enabled
|
||||
SSH port: 22
|
||||
Apply
|
||||
```
|
||||
|
||||
SSH is only exposed on your local network and Tailscale interface. No ports are
|
||||
opened to the public internet.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 3 — SSH KEYS ━━━
|
||||
|
||||
Two sets of keys needed: rsync automation keys (server-to-server) and a Gitea
|
||||
access key (for script repository pulls).
|
||||
|
||||
### 3a — Rsync Automation Keys (ssh_setup.sh)
|
||||
|
||||
Run on **each server**. `ssh_setup.sh` generates the key, copies it to the remote,
|
||||
and updates `master_host*.conf` with the key path automatically.
|
||||
|
||||
```bash
|
||||
# On HOST1 — after the repo is cloned:
|
||||
bash /mnt/user/appdata/unraid_scripts/Partnership/ssh_setup.sh
|
||||
|
||||
# On HOST2:
|
||||
bash /mnt/user/appdata/unraid_scripts/Partnership/ssh_setup.sh
|
||||
```
|
||||
|
||||
Key naming convention: hostname lowercased, `unraid-` prefix stripped.
|
||||
```
|
||||
unRAID-Gmer4Lfe → /root/.ssh/gmer4lfe_rsync_automation
|
||||
unRAID-Jayred365 → /root/.ssh/jayred365_rsync_automation
|
||||
```
|
||||
|
||||
`master_host*.conf` is updated automatically with `HOST*_SSH_KEY` pointing to the
|
||||
generated key. Run `--status` to verify:
|
||||
|
||||
```bash
|
||||
bash /mnt/user/appdata/unraid_scripts/Partnership/ssh_setup.sh --status
|
||||
```
|
||||
|
||||
> **`ssh_setup.sh` is idempotent** — safe to re-run. Use `--force` to regenerate
|
||||
> a key (e.g., after a security incident) and re-copy it to the remote.
|
||||
|
||||
### 3b — Gitea SSH Key (manual)
|
||||
|
||||
```bash
|
||||
# On BOTH servers:
|
||||
ssh-keygen -t ed25519 -f /root/.ssh/unraid_gitea -C "unraid-gitea" -N ""
|
||||
|
||||
# Print the public key to add to Gitea:
|
||||
cat /root/.ssh/unraid_gitea.pub
|
||||
|
||||
# In Gitea: Settings → SSH / GPG Keys → Add Key → paste above
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 4 — CLONE THE REPOSITORY ━━━
|
||||
|
||||
```bash
|
||||
# Create target directory:
|
||||
mkdir -p /mnt/user/appdata/unraid_scripts
|
||||
|
||||
# Clone:
|
||||
GIT_SSH_COMMAND="ssh -i /root/.ssh/unraid_gitea" \
|
||||
git clone git@YOUR_GITEA_HOST:FailedProxy/Unraid_Scripts.git \
|
||||
/mnt/user/appdata/unraid_scripts
|
||||
|
||||
# Make scripts executable:
|
||||
find /mnt/user/appdata/unraid_scripts -name "*.sh" -exec chmod +x {} \;
|
||||
```
|
||||
|
||||
Expected structure after clone:
|
||||
```
|
||||
master.conf ← all shared configuration
|
||||
master_host1.conf ← HOST1-specific configuration
|
||||
master_host2.conf ← HOST2-specific configuration
|
||||
common.sh ← shared library
|
||||
load_config.sh ← config loader
|
||||
Orchestrators/
|
||||
Rsync/
|
||||
Fallback/
|
||||
Docker_Essentials/
|
||||
...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 5 — CONFIGURE MASTER.CONF ━━━
|
||||
|
||||
```bash
|
||||
nano /mnt/user/appdata/unraid_scripts/master.conf
|
||||
```
|
||||
|
||||
### Host Identity
|
||||
|
||||
```bash
|
||||
# These must match Tailscale machine names exactly — case sensitive.
|
||||
HOST1="unRAID-Gmer4Lfe" # REQUIRED
|
||||
HOST2="unRAID-Jayred365" # REQUIRED
|
||||
|
||||
# SSH keys — each server uses its own key to authenticate to the other:
|
||||
HOST1_SSH_KEY="/root/.ssh/Gmer4Lfe-rsync-key"
|
||||
HOST2_SSH_KEY="/root/.ssh/Jayred365-rsync-key"
|
||||
```
|
||||
|
||||
### Git Repository
|
||||
|
||||
```bash
|
||||
GITEA_CONTAINER="Gitea"
|
||||
GITEA_REPO_PATH="FailedProxy/Unraid_Scripts.git"
|
||||
TARGET_DIR="/mnt/user/appdata/unraid_scripts"
|
||||
GITEA_SSH_KEY="/root/.ssh/unraid_gitea"
|
||||
SSH_PORT=221
|
||||
```
|
||||
|
||||
### Daily Sync Shares
|
||||
|
||||
```bash
|
||||
# master_host1.conf
|
||||
# Shares this server downloads to — pushed to the other server nightly.
|
||||
# arr_sync.sh runs before rsync in the daily window, so the receiving
|
||||
# server's arrs already track incoming files when they arrive.
|
||||
# Never put the same path in both lists.
|
||||
HOST1_DAILY_SYNC_SHARES=(
|
||||
"/mnt/user/Movies"
|
||||
"/mnt/user/Tv_Shows"
|
||||
"/mnt/user/Music"
|
||||
"/mnt/user/Kids_Movies"
|
||||
"/mnt/user/Kids_Tv_Shows"
|
||||
"/mnt/user/Sports"
|
||||
"/mnt/user/stand-up_comedy"
|
||||
)
|
||||
|
||||
# master_host2.conf
|
||||
HOST2_DAILY_SYNC_SHARES=(
|
||||
"/mnt/user/Anime_Shows"
|
||||
"/mnt/user/Anime_Movies"
|
||||
)
|
||||
```
|
||||
|
||||
### Weekly Sync Shares
|
||||
|
||||
```bash
|
||||
# master.conf
|
||||
# Synced during the Sunday 2:30am window — containers stopped both sides.
|
||||
WEEKLY_SYNC_SHARES=(
|
||||
"/mnt/user/Media_Server/Emby"
|
||||
"/mnt/user/appdata-Failover/Critical-Data"
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 6 — MASTER_HOST*.CONF ━━━
|
||||
|
||||
`detect_hosts()` reads which server is running and aliases `HOST*_` prefixed vars
|
||||
to their unprefixed names. Scripts only ever reference the unprefixed name — they
|
||||
work identically on both servers.
|
||||
|
||||
```bash
|
||||
nano /mnt/user/appdata/unraid_scripts/master_host1.conf # on HOST1
|
||||
nano /mnt/user/appdata/unraid_scripts/master_host2.conf # on HOST2
|
||||
```
|
||||
|
||||
Every variable is documented in the conf files. Key values to set:
|
||||
- SSH key paths
|
||||
- DAILY_SYNC_SHARES
|
||||
- EMBY_URL / EMBY_API_KEY
|
||||
- CERT_MONITOR_DOMAINS
|
||||
- SMART_IGNORE_DRIVES
|
||||
- ZFS_REPORT_IGNORE_POOLS
|
||||
|
||||
---
|
||||
|
||||
## ━━━ RSYNC PROFILES ━━━
|
||||
|
||||
Profiles control per-share behavior. Profile key = directory basename lowercased.
|
||||
Override with `--profile=name`.
|
||||
|
||||
### Profile Matching
|
||||
|
||||
```bash
|
||||
# rsync.sh /mnt/user/appdata-Failover/Arrs_Stack
|
||||
# basename: Arrs_Stack → lowercased: arrs_stack → matches [arrs_stack] profile
|
||||
#
|
||||
# rsync.sh /mnt/user/Movies
|
||||
# basename: Movies → no profile match → global defaults (no containers stopped)
|
||||
#
|
||||
# rsync.sh /mnt/user/appdata-Failover/Critical-Data --profile=critical-data
|
||||
# explicit override
|
||||
```
|
||||
|
||||
### Current Profile Definitions
|
||||
|
||||
```bash
|
||||
# master.conf
|
||||
|
||||
# ── arrs_stack ───────────────────────────────────────────────────────────────
|
||||
# Arr databases — stopped for clean SQLite snapshot
|
||||
PROFILES["arrs_stack_CRITICAL_CONTAINER_NAMES"]=(
|
||||
"Sonarr" "Radarr" "Lidarr" "Prowlarr" "Bazarr" "Pinchflat"
|
||||
)
|
||||
|
||||
# ── critical-data ─────────────────────────────────────────────────────────────
|
||||
# Auth stack — stopped for clean database snapshot, Authelia delayed restart
|
||||
PROFILES["critical-data_CRITICAL_CONTAINER_NAMES"]=(
|
||||
"Mariadb-Authelia" "Redis-Authelia"
|
||||
"NginxProxyManager" "Lldap-Gmer4Lfe"
|
||||
)
|
||||
PROFILES["critical-data_DELAYED_CONTAINERS"]=(
|
||||
"Authelia" "Authelia-Secondary"
|
||||
)
|
||||
PROFILES["critical-data_CONTAINER_DELAY"]=30
|
||||
|
||||
# ── important-data ────────────────────────────────────────────────────────────
|
||||
# NextCloud + Postgres — stopped for clean snapshot
|
||||
PROFILES["important-data_CRITICAL_CONTAINER_NAMES"]=(
|
||||
"Postgres-NextCloud"
|
||||
)
|
||||
PROFILES["important-data_DELAYED_CONTAINERS"]=("NextCloud")
|
||||
|
||||
# ── emby ──────────────────────────────────────────────────────────────────────
|
||||
# Weekly full clean sync — both Emby instances stopped
|
||||
PROFILES["emby_CRITICAL_CONTAINER_NAMES"]=("Emby")
|
||||
PROFILES["emby_EXCLUDE_DIRS"]=(
|
||||
"transcodes/" "logs/" "crash*" "cache/"
|
||||
)
|
||||
|
||||
# ── emby-failover ─────────────────────────────────────────────────────────────
|
||||
# Every 30 minutes, Emby STAYS RUNNING — dirty sync of critical state only
|
||||
PROFILES["emby-failover_CRITICAL_CONTAINER_NAMES"]=() # nothing stops
|
||||
PROFILES["emby-failover_EXCLUDE_DIRS"]=(
|
||||
"*.wal" "*.shm" # unsafe mid-write
|
||||
"transcodes/" "logs/" "crash*" "cache/"
|
||||
)
|
||||
PROFILES["emby-failover_REMOTE_RESTART_CONTAINERS"]=("Emby")
|
||||
```
|
||||
|
||||
### Two Emby Profiles — Why Both Exist
|
||||
|
||||
**emby-failover** (every 30 minutes, Emby stays running):
|
||||
- Syncs: users.db, library.db, authentication.db, config/
|
||||
- Skips: \*.wal, \*.shm, transcodes/, logs/, cache/
|
||||
- Why: WAL files are written while Emby runs — copying them would produce a corrupt database on HOST2
|
||||
- Result: HOST2 is always within 30 minutes of HOST1 on watch state and user activity
|
||||
|
||||
**emby** (Sunday 2:30am, both Emby instances stopped):
|
||||
- Syncs: everything except transcodes, logs, cache, crash files
|
||||
- Includes: metadata, plugins, full database state, all config
|
||||
- Why: WAL is checkpointed on clean shutdown — safe to copy everything
|
||||
- Result: HOST2 gets a gold-standard Emby state once per week
|
||||
|
||||
The two profiles work together. emby-failover keeps HOST2 current for immediate failover.
|
||||
emby gives HOST2 full fidelity once per week. Neither alone is sufficient.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ PERSONAL ENCRYPTED SHARES ━━━
|
||||
|
||||
Personal shares sync to the remote for offsite backup. ZFS encrypts at the dataset
|
||||
level — the remote receives encrypted blocks and cannot read the content without your
|
||||
passphrase or keyfile.
|
||||
|
||||
### Create an Encrypted ZFS Dataset
|
||||
|
||||
```bash
|
||||
# In the unRAID UI:
|
||||
# Main → click your ZFS pool name → + Dataset
|
||||
# Name: Gmer4Lfe-Personal
|
||||
# Encryption: Enabled
|
||||
# Passphrase: [your passphrase]
|
||||
# Write your passphrase down — if lost, data is completely unrecoverable
|
||||
|
||||
# Verify encryption is active before syncing:
|
||||
zfs get encryption poolname/Gmer4Lfe-Personal
|
||||
# Should show: encryption aes-256-gcm
|
||||
```
|
||||
|
||||
### Auto-Unlock on Boot (Optional)
|
||||
|
||||
```bash
|
||||
# Keyfile approach — more convenient, but the keyfile itself is a secret
|
||||
dd if=/dev/urandom bs=32 count=1 | base64 > /root/.zfs-keys/personal.key
|
||||
chmod 600 /root/.zfs-keys/personal.key
|
||||
|
||||
zfs change-key \
|
||||
-o keylocation=file:///root/.zfs-keys/personal.key \
|
||||
-o keyformat=raw \
|
||||
poolname/Gmer4Lfe-Personal
|
||||
|
||||
# Add to array_start.sh or ramdisk_setup.sh:
|
||||
zfs load-key poolname/Gmer4Lfe-Personal
|
||||
zfs mount poolname/Gmer4Lfe-Personal
|
||||
|
||||
# Manual unlock alternative (most secure):
|
||||
zfs load-key poolname/Gmer4Lfe-Personal # prompts for passphrase
|
||||
zfs mount poolname/Gmer4Lfe-Personal
|
||||
```
|
||||
|
||||
### Add to master_host1.conf
|
||||
|
||||
```bash
|
||||
HOST1_PERSONAL_SHARES=(
|
||||
"/mnt/user/Gmer4Lfe-Personal"
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 7 — USER SCRIPTS SETUP ━━━
|
||||
|
||||
Only a small number of User Scripts entries are needed — each one an orchestrator.
|
||||
Individual scripts are never scheduled directly except the 30-minute Emby sync.
|
||||
|
||||
### At Startup of Array
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/array_start.sh
|
||||
# Schedule: At Startup of Array
|
||||
# Run as: Background Task
|
||||
```
|
||||
|
||||
This single entry launches everything defined in ARRAY_START_SCRIPTS from master.conf:
|
||||
inotify_tuning, docker_syslog_filter, php_fpm_max_children, ramdisk_setup,
|
||||
docker_network_connect, system_watchdog, docker_watchdog, failover.
|
||||
|
||||
### Cron Schedule
|
||||
|
||||
```bash
|
||||
# Every 30 minutes — Emby dirty sync:
|
||||
*/30 * * * *
|
||||
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
|
||||
/mnt/user/Media_Server/Emby --profile=emby-failover
|
||||
|
||||
# Every 6 hours — failed import + stalled download recovery:
|
||||
0 */6 * * *
|
||||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/arrs_failed_stalled_recovery.sh
|
||||
|
||||
# 1am daily — full maintenance window:
|
||||
0 1 * * *
|
||||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh
|
||||
|
||||
# 2:30am Sunday — weekly maintenance window:
|
||||
30 2 * * 0
|
||||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/weekly_sync_maintenance.sh
|
||||
|
||||
# 8am daily — health digest:
|
||||
0 8 * * *
|
||||
bash /mnt/user/appdata/unraid_scripts/Monitors/weekly_health_digest.sh
|
||||
|
||||
# Every 6 hours — inotify + php-fpm snapshot:
|
||||
0 */6 * * *
|
||||
bash /mnt/user/appdata/unraid_scripts/Monitors/system_tuning_monitor.sh
|
||||
|
||||
# Sunday morning — weekly reports:
|
||||
0 6 * * 0 bash .../Monitors/zfs_memory_snapshot.sh
|
||||
0 7 * * 0 bash .../Monitors/smart_health.sh
|
||||
0 9 * * 0 bash .../Monitors/cert_monitor.sh
|
||||
0 10 * * 0 bash .../Monitors/backup_verify.sh
|
||||
0 11 * * 0 bash .../Monitors/emby_session_report.sh
|
||||
0 11 * * 0 bash .../Monitors/bandwidth_monitor.sh --report
|
||||
```
|
||||
|
||||
Set all entries to **Background Task** — non-background tasks can appear to hang
|
||||
on long-running scripts.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ VERIFICATION ━━━
|
||||
|
||||
Test with `--dry-run` first — all pre-flight checks run, no changes made.
|
||||
|
||||
### Test a Single Profile Sync
|
||||
|
||||
```bash
|
||||
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
|
||||
/mnt/user/appdata-Failover/Arrs_Stack --dry-run --log
|
||||
```
|
||||
|
||||
Expected output (healthy):
|
||||
```
|
||||
━━━ Setup ━━━
|
||||
Host: HOST1 (unRAID-Gmer4Lfe) → HOST2 (unRAID-Jayred365)
|
||||
Remote IP: 100.x.x.x
|
||||
Profile: arrs_stack
|
||||
|
||||
━━━ Pre-flight ━━━
|
||||
Remote reachable
|
||||
version parity — both on unRAID X.Y.Z
|
||||
Remote Docker daemon responding
|
||||
Remote rootfs: 12% (threshold: 75%)
|
||||
Remote share exists and not empty
|
||||
All pre-flight checks passed
|
||||
```
|
||||
|
||||
### Test the Daily Orchestrator
|
||||
|
||||
```bash
|
||||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh --dry-run
|
||||
```
|
||||
|
||||
### Check Configuration Resolution
|
||||
|
||||
```bash
|
||||
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
|
||||
/mnt/user/appdata-Failover/Arrs_Stack --status
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ INITIAL HOST2 SYNC ━━━
|
||||
|
||||
If HOST2 is being set up from scratch with empty shares:
|
||||
|
||||
```bash
|
||||
# Create share directories on HOST2:
|
||||
bash /mnt/user/appdata/unraid_scripts/Tools/recreate_shares.sh
|
||||
|
||||
# Initial push from HOST1 — first run may take several hours for large libraries:
|
||||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh --log
|
||||
```
|
||||
|
||||
The scheduled nightly sync will be incremental after the initial push.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ NAMING CONSISTENCY REQUIREMENT ━━━
|
||||
|
||||
The ecosystem uses one codebase on both servers. This only works if containers and
|
||||
shares have identical names on both servers. **This is not configurable — it is a
|
||||
design requirement.**
|
||||
|
||||
```
|
||||
Container names must match exactly on both servers:
|
||||
"Emby" ← both HOST1 and HOST2
|
||||
"NginxProxyManager" ← both HOST1 and HOST2
|
||||
"Mariadb-Authelia" ← both HOST1 and HOST2
|
||||
|
||||
Share paths must match exactly on both servers:
|
||||
/mnt/user/Movies ← both HOST1 and HOST2
|
||||
/mnt/user/Tv_Shows ← both HOST1 and HOST2
|
||||
```
|
||||
|
||||
If a container has a different name on one server: the script skips it silently.
|
||||
You only notice when the container is not stopped during a sync that requires it.
|
||||
|
||||
If a share has a different path: rsync.sh aborts with "remote share missing."
|
||||
Easier to catch — but still requires renaming the share to fix.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ TROUBLESHOOTING ━━━
|
||||
|
||||
### SSH Connection Refused / Timeout
|
||||
|
||||
```bash
|
||||
# Verify SSH is enabled on the remote:
|
||||
# Settings → Management Access → Secure Shell → Enabled
|
||||
|
||||
# Verify Tailscale is connected:
|
||||
tailscale ip -4 unRAID-Jayred365
|
||||
|
||||
# Test SSH manually (key path from --status output):
|
||||
ssh -i /root/.ssh/gmer4lfe_rsync_automation \
|
||||
root@$(tailscale ip -4 unRAID-Jayred365) "hostname"
|
||||
# If password prompted: key not authorised — re-run ssh_setup.sh
|
||||
|
||||
# Re-run setup (idempotent, re-copies key to remote):
|
||||
bash /mnt/user/appdata/unraid_scripts/Partnership/ssh_setup.sh
|
||||
```
|
||||
|
||||
### Pre-flight Aborts on Remote Rootfs
|
||||
|
||||
```bash
|
||||
# Check current usage on remote:
|
||||
ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@[HOST2-ip] "df /"
|
||||
|
||||
# Common cause: array not started, drives not mounted
|
||||
```
|
||||
|
||||
### Remote Share Missing
|
||||
|
||||
```bash
|
||||
# Verify the share exists on HOST2:
|
||||
ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@[HOST2-ip] "ls /mnt/user/"
|
||||
|
||||
# If missing: create the share on HOST2, then run initial sync
|
||||
```
|
||||
|
||||
### Containers Not Stopping / Starting
|
||||
|
||||
```bash
|
||||
# Verify container names match Docker exactly — case sensitive:
|
||||
docker ps --format "{{.Names}}"
|
||||
|
||||
# Test remote:
|
||||
ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@[HOST2-ip] "docker ps --format '{{.Names}}'"
|
||||
```
|
||||
|
||||
### Profile Not Matching
|
||||
|
||||
```bash
|
||||
# Profile key = directory basename lowercased
|
||||
# /mnt/user/appdata-Failover/Arrs_Stack → key: arrs_stack
|
||||
|
||||
# Override explicitly:
|
||||
bash rsync.sh /mnt/user/appdata-Failover/My_Stuff --profile=arrs_stack
|
||||
|
||||
# Verify what profile resolved:
|
||||
bash rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --status
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ FULL CONFIGURATION REFERENCE ━━━
|
||||
|
||||
### master.conf
|
||||
|
||||
```bash
|
||||
# Rsync engine
|
||||
RSYNC_ENABLED=true
|
||||
DEFAULT_RSYNC_OPTS="-az --no-perms --no-owner --no-group --inplace"
|
||||
# No --delete in DEFAULT_RSYNC_OPTS — bidirectional media shares spread files only.
|
||||
# Each server's arrs are source of truth for their content; arr cleanup scripts
|
||||
# handle deletions. Profiles set PROFILE_RSYNC_OPTS with --delete explicitly.
|
||||
BW_LIMIT=0 # KB/s, 0 = unlimited
|
||||
RETRY_COUNT=3
|
||||
SLEEP=60 # seconds between retries
|
||||
ROOTFS_WARN_PCT=75 # abort if remote rootfs above this %
|
||||
|
||||
# Shared with Monitors/
|
||||
BANDWIDTH_LOG="/boot/config/bandwidth_history.db"
|
||||
BANDWIDTH_LOG_RETENTION=90
|
||||
BANDWIDTH_WARN_GB=50
|
||||
|
||||
# Profile definitions (see RSYNC PROFILES section above)
|
||||
declare -A PROFILES
|
||||
declare -A PROFILE_BW_LIMIT
|
||||
declare -A PROFILE_RETRY_COUNT
|
||||
declare -A PROFILE_SLEEP
|
||||
declare -A PROFILE_CONTAINER_DELAY
|
||||
declare -A PROFILE_CRITICAL_CONTAINER_NAMES
|
||||
declare -A PROFILE_DELAYED_CONTAINERS
|
||||
declare -A PROFILE_EXCLUDE_DIRS
|
||||
declare -A PROFILE_REMOTE_RESTART_CONTAINERS
|
||||
```
|
||||
|
||||
### master_host*.conf
|
||||
|
||||
```bash
|
||||
# Daily sync shares — one list per server (mutually exclusive)
|
||||
HOST1_DAILY_SYNC_SHARES=(...)
|
||||
HOST2_DAILY_SYNC_SHARES=(...)
|
||||
|
||||
# Personal encrypted shares
|
||||
HOST1_PERSONAL_SHARES=(...)
|
||||
|
||||
# SSH key for this server to authenticate to the remote
|
||||
# Set automatically by ssh_setup.sh — do not edit manually
|
||||
HOST1_SSH_KEY="/root/.ssh/gmer4lfe_rsync_automation"
|
||||
HOST2_SSH_KEY="/root/.ssh/jayred365_rsync_automation"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ FLAG REFERENCE ━━━
|
||||
|
||||
All flags work on `rsync.sh` and all orchestrators.
|
||||
|
||||
### --dry-run
|
||||
|
||||
Runs all pre-flight checks. Shows what rsync would transfer. No transfer, no container
|
||||
stops, no bandwidth log entry. Safe to run at any time.
|
||||
|
||||
```bash
|
||||
rsync.sh /mnt/user/Movies --dry-run
|
||||
rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --dry-run --log
|
||||
daily_sync_maintenance.sh --dry-run
|
||||
```
|
||||
|
||||
### --status
|
||||
|
||||
Shows resolved configuration — profile, remote identity, all vars that would be
|
||||
used — then exits. No pre-flight checks, no rsync. Use to verify configuration
|
||||
loaded correctly.
|
||||
|
||||
```bash
|
||||
rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --status
|
||||
```
|
||||
|
||||
### --log
|
||||
|
||||
Verbose output throughout. Every decision, every container operation, every rsync
|
||||
progress line. Use for first-time runs or when investigating issues.
|
||||
|
||||
### --profile=name
|
||||
|
||||
Override profile selection. Bypasses basename inference. Use when the directory
|
||||
name doesn't match any profile key, or when testing a specific profile.
|
||||
|
||||
```bash
|
||||
rsync.sh /mnt/user/appdata-Failover/Critical-Data --profile=critical-data
|
||||
rsync.sh /mnt/user/Media_Server/Emby --profile=emby-failover
|
||||
```
|
||||
@@ -0,0 +1,133 @@
|
||||
# ━━━━━ RSYNC ━━━━━
|
||||
|
||||
The transfer engine for the two-server ecosystem. `rsync.sh` is the single script
|
||||
called by every orchestrator that moves data between servers — it handles profiles,
|
||||
pre-flight checks, container stops, the actual transfer, and bandwidth logging.
|
||||
Orchestrators decide what to sync and when. `rsync.sh` decides how to do it safely.
|
||||
|
||||
> **Never schedule rsync.sh directly for daily/weekly syncs.** Use the orchestrators
|
||||
> in `Orchestrators/`. rsync.sh is called directly only for manual runs and the
|
||||
> 30-minute Emby dirty sync, which needs its own cron entry.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ THE PROBLEM THAT BUILT THIS ━━━
|
||||
|
||||
**rsync Alone Isn't Safe Enough for Live Databases**
|
||||
Running rsync against a share while SQLite databases are being written produces
|
||||
corrupt snapshots on the remote. The arr databases, Emby library database, and
|
||||
Authelia session store all write continuously. A plain rsync copies them mid-write.
|
||||
The remote gets a file that opens cleanly but has internal inconsistencies.
|
||||
Fix: profiles stop specific containers before syncing and restart them after.
|
||||
The database is quiesced, rsync runs against a static snapshot, containers come back up.
|
||||
|
||||
**Each Share Needs Different Behavior**
|
||||
Media shares (Movies, TV) just spread files — nothing stops, no `--delete`
|
||||
(arr cleanup scripts own deletions, and arr_sync ensures both arrs already
|
||||
track incoming files before they arrive). Arr databases need containers stopped,
|
||||
clean SQLite snapshot, restart. Emby has two modes: weekly full-stop clean mirror
|
||||
and 30-minute dirty sync while Emby stays running (WAL files excluded). Critical-Data
|
||||
stops the auth stack, waits for Authelia to come back after restart delay.
|
||||
Fix: the profile system — one script, behavior defined entirely by the profile key.
|
||||
|
||||
**A Failed Remote Shouldn't Corrupt a Live Sync**
|
||||
If the remote's rootfs is nearly full, an rsync that starts will write partial files
|
||||
then fail mid-transfer, leaving the remote in a worse state than before. If the remote's
|
||||
backing disks are offline, rsync writes to an empty mount point and "succeeds."
|
||||
Fix: pre-flight checks abort before touching anything if remote conditions are wrong.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ WHAT THIS FOLDER DOES ━━━
|
||||
|
||||
One script. One job: move data from this server to the remote safely.
|
||||
|
||||
`rsync.sh` handles the full transfer lifecycle:
|
||||
1. Infer or accept a profile for the given directory
|
||||
2. Run pre-flight checks (connectivity, rootfs, disk temps, remote share exists)
|
||||
3. Stop containers specified by the profile (both sides)
|
||||
4. Run rsync with profile flags, excludes, and bandwidth limit
|
||||
5. Restart containers (with delay if configured)
|
||||
6. Restart remote containers if dirty-sync profile specifies it
|
||||
7. Log the transfer to bandwidth_monitor.sh
|
||||
|
||||
Everything else — deciding which shares to sync, in what order, on what schedule —
|
||||
lives in the orchestrators.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ RELATIONSHIP TO OTHER FOLDERS ━━━
|
||||
|
||||
```
|
||||
Media/
|
||||
arr_sync.sh ── runs before rsync in daily window ──────────► all arrs agree on library
|
||||
↓
|
||||
Orchestrators/ ← decides what to sync, when, and in what order
|
||||
daily_sync_maintenance.sh ──────────────────────────────────► rsync.sh (per share)
|
||||
weekly_sync_maintenance.sh ──────────────────────────────────► rsync.sh (emby, critical-data)
|
||||
critical_sync_maintenance.sh ────────────────────────────────► rsync.sh (partnership shares)
|
||||
|
||||
Cron (direct):
|
||||
*/30 * * * * ──────────────────────────────────► rsync.sh --profile=emby-failover
|
||||
|
||||
Monitors/
|
||||
bandwidth_monitor.sh ◄─── called by rsync.sh after each sync (--log-transfer)
|
||||
|
||||
Fallback/
|
||||
fallback.sh ──── rsync writeback during handback ──► rsync.sh
|
||||
```
|
||||
|
||||
rsync.sh never calls other scripts except `bandwidth_monitor.sh` at the end of a sync.
|
||||
All orchestration logic lives in the callers. arr_sync.sh (Media/) is a peer that runs
|
||||
before rsync in the daily window — it is not called by rsync.sh directly.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ THE PROFILE SYSTEM ━━━
|
||||
|
||||
Profile key = directory basename lowercased. `--profile=name` overrides.
|
||||
|
||||
| Profile | What It Syncs | Containers Stopped | Notes |
|
||||
|---------|--------------|-------------------|-------|
|
||||
| *(none)* | Media shares (Movies, TV, Music…) | None | Bidirectional spread — no `--delete` in DEFAULT_RSYNC_OPTS. Arr cleanup scripts own deletions. |
|
||||
| `arrs_stack` | Arr databases | Sonarr, Radarr, Lidarr, Prowlarr, Bazarr, Pinchflat | Clean SQLite snapshot |
|
||||
| `critical-data` | Auth stack | Mariadb-Authelia, Redis-Authelia, NPM, Lldap | Authelia has restart delay |
|
||||
| `important-data` | NextCloud + Postgres | Postgres-NextCloud | NextCloud has restart delay |
|
||||
| `emby` | Full Emby mirror | Emby (both sides) | Weekly — Sunday 2:30am |
|
||||
| `emby-failover` | Emby watch state delta | None | Dirty sync — Emby stays running |
|
||||
|
||||
For full profile definitions see `Manual-Rsync.md`.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ SCRIPTS IN THIS FOLDER ━━━
|
||||
|
||||
| Script | Role | When It Runs |
|
||||
|--------|------|-------------|
|
||||
| `rsync.sh` | Core transfer engine — profile resolution, pre-flights, container management, transfer, bandwidth logging | Called by orchestrators; directly for manual and 30-min Emby dirty sync |
|
||||
|
||||
---
|
||||
|
||||
## ━━━ HOW THE SCRIPTS RELATE ━━━
|
||||
|
||||
```
|
||||
Callers (Orchestrators/) ──────────────────────────────────────────────────────
|
||||
daily_sync_maintenance.sh │
|
||||
weekly_sync_maintenance.sh rsync.sh /path/to/share [--profile=name] │
|
||||
critical_sync_maintenance.sh ─────────────────────────────────────────────► │
|
||||
fallback.sh (writeback) │
|
||||
Direct cron (emby-failover) │
|
||||
▼
|
||||
┌─────────────────────────────────────┐
|
||||
│ 1. Infer/accept profile │
|
||||
│ 2. Pre-flight checks │
|
||||
│ - connectivity │
|
||||
│ - rootfs / disk temp / disks │
|
||||
│ - remote share exists │
|
||||
│ 3. Stop containers (profile) │
|
||||
│ 4. rsync transfer │
|
||||
│ 5. Restart containers │
|
||||
│ 6. Remote restart (dirty sync) │
|
||||
│ 7. Log to bandwidth_monitor.sh │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
@@ -1,993 +0,0 @@
|
||||
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
# 🔄 RSYNC SETUP GUIDE
|
||||
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
**Complete setup guide for the two-server rsync ecosystem.** By the end of this guide
|
||||
both servers will have SSH keys configured, Tailscale connected, the git repository
|
||||
cloned, and all scheduled operations running automatically.
|
||||
|
||||
> **This is a setup guide, not a script reference.** For how rsync.sh works internally,
|
||||
> profiles, safety checks, and operational details — those belong in the orchestrator
|
||||
> and rsync script documentation. This guide is about standing the ecosystem up from
|
||||
> scratch and verifying it works.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ WHAT YOU'RE BUILDING ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
```
|
||||
HOST1 (unRAID-Gmer4Lfe) HOST2 (unRAID-Jayred365)
|
||||
────────────────────── ──────────────────────
|
||||
Source of truth: Source of truth:
|
||||
Movies, Tv_Shows, Music Anime_Shows, Anime_Movies
|
||||
Critical-Data (auth stack)
|
||||
Emby userdata
|
||||
|
||||
Pushes to HOST2 daily: ──────→ Receives:
|
||||
All HOST1 shares Mirror of HOST1 shares
|
||||
Personal encrypted shares Personal (encrypted blocks)
|
||||
|
||||
Receives from HOST2 daily: ←────── Pushes:
|
||||
Anime_Shows, Anime_Movies All HOST2 shares
|
||||
|
||||
Weekly clean sync (both sides stopped):
|
||||
Emby — full clean mirror ←──────→ Emby
|
||||
Critical-Data ──────→ Auth stack (HOST1 → HOST2)
|
||||
|
||||
Every 30 minutes — dirty sync:
|
||||
Emby watch states ──────→ HOST2 stays current on playback
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ PREREQUISITES ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
Both servers need these before starting:
|
||||
|
||||
```bash
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Required on both servers:
|
||||
unRAID 7.x
|
||||
Community Applications plugin — search "Community Applications" in unRAID plugins
|
||||
User Scripts plugin — install via Community Applications
|
||||
Tailscale plugin — install via Community Applications
|
||||
Terminal access — unRAID UI → Tools → Terminal, or SSH
|
||||
|
||||
# Optional but recommended:
|
||||
Gitea (Docker container on HOST1) — self-hosted git for the script repository
|
||||
Working Emby installation — for transcode management and failover
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 1 — TAILSCALE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
Tailscale provides the encrypted mesh network between servers. Scripts resolve the
|
||||
remote server's IP via Tailscale at runtime — no hardcoded IPs, no VPN configuration,
|
||||
no open ports. All server-to-server communication goes through Tailscale.
|
||||
|
||||
---
|
||||
|
||||
### ── Install on Both Servers ────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# On each server:
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# 1. Open Apps in the unRAID UI
|
||||
# 2. Search for "Tailscale" — install the plugin
|
||||
# 3. Settings → Tailscale → Connect
|
||||
# 4. Authenticate with your Tailscale account (browser opens on your machine)
|
||||
# 5. Verify both servers appear: https://login.tailscale.com/admin/machines
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Verify Connectivity ─────────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# From HOST1 — should return HOST2's 100.x.x.x Tailscale IP:
|
||||
tailscale ip -4 unRAID-Jayred365
|
||||
|
||||
# From HOST2 — should return HOST1's 100.x.x.x Tailscale IP:
|
||||
tailscale ip -4 unRAID-Gmer4Lfe
|
||||
|
||||
# Test actual connectivity:
|
||||
tailscale ping unRAID-Jayred365 # run from HOST1
|
||||
```
|
||||
|
||||
> **Critical:** The hostnames in `master.conf` (`HOST1` and `HOST2`) must match the
|
||||
> Tailscale machine names **exactly** — case sensitive. The ecosystem resolves all
|
||||
> remote IPs via `tailscale ip -4 HOSTNAME` at runtime. A name mismatch means every
|
||||
> script that touches the remote will fail at the IP resolution step.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 2 — ENABLE SSH ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
unRAID 7.x has SSH disabled by default. Enable it on both servers.
|
||||
|
||||
```bash
|
||||
# On each server:
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Settings → Management Access → Secure Shell
|
||||
# SSH: Enabled
|
||||
# SSH port: 22
|
||||
# Apply
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
```
|
||||
|
||||
> SSH is only exposed on your local network and Tailscale interface. Scripts connect
|
||||
> via Tailscale IP — all traffic is encrypted end-to-end. No ports are opened to
|
||||
> the public internet.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 3 — SSH KEYS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
Two sets of keys needed: server-to-server for rsync and failover, and Gitea access
|
||||
for script repository pull. Generate all keys before configuring anything else.
|
||||
|
||||
---
|
||||
|
||||
### ── 3a — Server-to-Server Keys ─────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# On HOST1 — generate HOST1's key pair:
|
||||
ssh-keygen -t ed25519 -f /root/.ssh/Gmer4Lfe-rsync-key -C "gmer4lfe-rsync" -N ""
|
||||
|
||||
# On HOST2 — generate HOST2's key pair:
|
||||
ssh-keygen -t ed25519 -f /root/.ssh/Jayred365-rsync-key -C "jayred365-rsync" -N ""
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── 3b — Authorise Keys Bidirectionally ─────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# HOST1's public key must be authorised on HOST2 (so HOST1 can SSH into HOST2):
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# On HOST1 — print the public key:
|
||||
cat /root/.ssh/Gmer4Lfe-rsync-key.pub
|
||||
|
||||
# On HOST2 — create authorized_keys and paste HOST1's public key:
|
||||
mkdir -p /root/.ssh
|
||||
echo "PASTE_HOST1_PUBLIC_KEY_HERE" >> /root/.ssh/authorized_keys
|
||||
chmod 600 /root/.ssh/authorized_keys
|
||||
|
||||
# HOST2's public key must be authorised on HOST1 (so HOST2 can SSH into HOST1):
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# On HOST2 — print the public key:
|
||||
cat /root/.ssh/Jayred365-rsync-key.pub
|
||||
|
||||
# On HOST1 — append HOST2's public key:
|
||||
echo "PASTE_HOST2_PUBLIC_KEY_HERE" >> /root/.ssh/authorized_keys
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── 3c — Test Both Directions ────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# From HOST1 — should print "connected" without a password prompt:
|
||||
ssh -i /root/.ssh/Gmer4Lfe-rsync-key \
|
||||
root@$(tailscale ip -4 unRAID-Jayred365) \
|
||||
"echo connected"
|
||||
|
||||
# From HOST2 — should print "connected" without a password prompt:
|
||||
ssh -i /root/.ssh/Jayred365-rsync-key \
|
||||
root@$(tailscale ip -4 unRAID-Gmer4Lfe) \
|
||||
"echo connected"
|
||||
```
|
||||
|
||||
```
|
||||
If prompted for a password: the key was not authorised correctly.
|
||||
→ Recheck Step 3b — the public key content must be on one line
|
||||
→ Check permissions: chmod 600 /root/.ssh/authorized_keys
|
||||
→ Check the key file referenced in the SSH command matches what was generated
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── 3d — Gitea SSH Key ───────────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# On BOTH servers — generate a key for Gitea access:
|
||||
ssh-keygen -t ed25519 -f /root/.ssh/unraid_gitea -C "unraid-gitea" -N ""
|
||||
|
||||
# Print the public key to add to Gitea:
|
||||
cat /root/.ssh/unraid_gitea.pub
|
||||
|
||||
# In Gitea: Settings → SSH / GPG Keys → Add Key → paste the output above
|
||||
# Do this for both servers if they have separate Gitea accounts, or once if shared
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 4 — CLONE THE REPOSITORY ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
Both servers clone from the same Gitea repository. Updates pushed to the repo
|
||||
propagate to both servers automatically via `git_pull_execute.sh` at the start of
|
||||
each daily maintenance window.
|
||||
|
||||
---
|
||||
|
||||
### ── On Both Servers ─────────────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# Create the target directory:
|
||||
mkdir -p /mnt/user/appdata/unraid_scripts
|
||||
|
||||
# Clone the repository:
|
||||
GIT_SSH_COMMAND="ssh -i /root/.ssh/unraid_gitea" \
|
||||
git clone git@YOUR_GITEA_HOST:FailedProxy/Unraid_Scripts.git \
|
||||
/mnt/user/appdata/unraid_scripts
|
||||
|
||||
# Replace YOUR_GITEA_HOST with your Gitea server address and port
|
||||
# Example: git@192.168.50.2:221
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Verify the Structure ─────────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
ls /mnt/user/appdata/unraid_scripts/
|
||||
```
|
||||
|
||||
```
|
||||
Expected output:
|
||||
master.conf ← all user configuration — the only file you edit
|
||||
master_host1.conf ← HOST1-specific configuration
|
||||
master_host2.conf ← HOST2-specific configuration
|
||||
common.sh ← shared library — functions used by all scripts
|
||||
load_config.sh ← config loader
|
||||
Orchestrators/
|
||||
Rsync/
|
||||
Failover/
|
||||
Docker_Essentials/
|
||||
unRAID_Essentials/
|
||||
Media/
|
||||
Transcodes/
|
||||
Monitors/
|
||||
Tools/
|
||||
Partnership/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Make Scripts Executable ─────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# Execute permission on all scripts — required once after clone:
|
||||
find /mnt/user/appdata/unraid_scripts -name "*.sh" -exec chmod +x {} \;
|
||||
```
|
||||
|
||||
> `array_start.sh` auto-fixes permissions on scripts that lost the execute bit —
|
||||
> but this initial chmod ensures the first run works before that safeguard is active.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 5 — CONFIGURE MASTER.CONF ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
All user configuration lives in `master.conf`. Every value with a comment
|
||||
`# REQUIRED` must be set before the first run. Everything else has working defaults.
|
||||
|
||||
```bash
|
||||
nano /mnt/user/appdata/unraid_scripts/master.conf
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Host Identity ─────────────────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# master.conf
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# These must match Tailscale machine names exactly — case sensitive.
|
||||
# The ecosystem uses these to resolve remote IPs at runtime.
|
||||
#
|
||||
HOST1="unRAID-Gmer4Lfe" # REQUIRED — must match tailscale machine name
|
||||
HOST2="unRAID-Jayred365" # REQUIRED — same
|
||||
|
||||
# SSH key paths — each server's key for authenticating to the other:
|
||||
HOST1_SSH_KEY="/root/.ssh/Gmer4Lfe-rsync-key" # HOST1 uses this to SSH to HOST2
|
||||
HOST2_SSH_KEY="/root/.ssh/Jayred365-rsync-key" # HOST2 uses this to SSH to HOST1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Emby API Keys ─────────────────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# master_host1.conf
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Used by: emby_session_report.sh, sunday_morning_coffee_report.sh,
|
||||
# sonarr/radarr cleanup (notify_emby_scan after deletion)
|
||||
#
|
||||
# Get from: Emby Dashboard → Settings → API Keys → + New API Key
|
||||
#
|
||||
HOST1_EMBY_URL="http://192.168.50.2:8096"
|
||||
HOST1_EMBY_API_KEY="your-host1-emby-api-key" # REQUIRED for Emby features
|
||||
HOST1_EMBY_CONTAINER="Emby"
|
||||
|
||||
# master_host2.conf
|
||||
HOST2_EMBY_URL="http://localhost:8096"
|
||||
HOST2_EMBY_API_KEY="your-host2-emby-api-key"
|
||||
HOST2_EMBY_CONTAINER="Emby"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Git Repository ────────────────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# master.conf
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Used by git_pull_execute.sh — pulls latest scripts at start of each daily window.
|
||||
#
|
||||
GITEA_CONTAINER="Gitea" # exact Docker container name
|
||||
GITEA_REPO_PATH="FailedProxy/Unraid_Scripts.git"
|
||||
TARGET_DIR="/mnt/user/appdata/unraid_scripts"
|
||||
GITEA_SSH_KEY="/root/.ssh/unraid_gitea"
|
||||
SSH_PORT=221 # your Gitea SSH port
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Daily Sync Shares ────────────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# master_host1.conf
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Shares HOST1 is source of truth for — pushed to HOST2 every night at 1am.
|
||||
# HOST2 treats these as read-only mirrors. Never put the same share in both lists.
|
||||
#
|
||||
HOST1_DAILY_SYNC_SHARES=(
|
||||
"/mnt/user/Movies" # HOST1 manages this — Radarr runs here
|
||||
"/mnt/user/Tv_Shows" # HOST1 manages this — Sonarr runs here
|
||||
"/mnt/user/Music" # HOST1 manages this — Lidarr runs here
|
||||
"/mnt/user/Kids_Movies"
|
||||
"/mnt/user/Kids_Tv_Shows"
|
||||
"/mnt/user/Sports"
|
||||
"/mnt/user/stand-up_comedy"
|
||||
)
|
||||
|
||||
# master_host2.conf
|
||||
HOST2_DAILY_SYNC_SHARES=(
|
||||
"/mnt/user/Anime_Shows" # HOST2 manages this — his Sonarr runs here
|
||||
"/mnt/user/Anime_Movies" # HOST2 manages this — his Radarr runs here
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Weekly Sync Shares ───────────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# master.conf
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Synced during the Sunday 2:30am window — containers stopped both sides.
|
||||
# Do NOT add these to a separate cron schedule — they run via weekly_sync_maintenance.sh.
|
||||
#
|
||||
WEEKLY_SYNC_SHARES=(
|
||||
"/mnt/user/Media_Server/Emby" # full clean Emby mirror
|
||||
"/mnt/user/appdata-Failover/Critical-Data" # auth stack clean state
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 6 — MASTER_HOST*.CONF ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
Per-host configuration lives in `master_host1.conf` and `master_host2.conf`.
|
||||
`detect_hosts()` in `common.sh` reads which server is running and aliases the correct
|
||||
`HOST*_` prefixed variables to their unprefixed names. Scripts only ever reference the
|
||||
unprefixed name — they work identically on both servers.
|
||||
|
||||
```bash
|
||||
# master_host1.conf is only sourced on HOST1
|
||||
# master_host2.conf is only sourced on HOST2
|
||||
# Changes go in the right file for the right server
|
||||
|
||||
nano /mnt/user/appdata/unraid_scripts/master_host1.conf # on HOST1
|
||||
nano /mnt/user/appdata/unraid_scripts/master_host2.conf # on HOST2
|
||||
```
|
||||
|
||||
See each conf file's comments — every variable is documented with its purpose
|
||||
and the reasoning behind the value.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 7 — RSYNC PROFILES ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
Profiles control per-share behaviour — which containers to stop, rsync flags,
|
||||
bandwidth limits, what to exclude. Profile is matched by directory basename
|
||||
(lowercased). Override with `--profile=name`.
|
||||
|
||||
---
|
||||
|
||||
### ── How Profile Matching Works ──────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# rsync.sh /mnt/user/appdata-Failover/Arrs_Stack
|
||||
# basename: Arrs_Stack
|
||||
# lowercased: arrs_stack
|
||||
# matched profile: [arrs_stack]
|
||||
#
|
||||
# rsync.sh /mnt/user/Movies
|
||||
# basename: Movies
|
||||
# lowercased: movies
|
||||
# no matching profile → global defaults apply (no containers stopped)
|
||||
#
|
||||
# rsync.sh /mnt/user/appdata-Failover/Critical-Data --profile=critical-failover
|
||||
# explicit override → uses [critical-failover] profile regardless of path
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Current Profiles ─────────────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# master.conf — profile definitions
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Each profile defines which containers to stop, rsync flags, excludes, etc.
|
||||
# Containers in PROFILE_CRITICAL_CONTAINER_NAMES are stopped on BOTH servers.
|
||||
# PROFILE_DELAYED_CONTAINERS restart after PROFILE_CONTAINER_DELAY seconds.
|
||||
|
||||
# ── arrs_stack ─────────────────────────────────────────────────────────────
|
||||
# Arr databases — stopped for clean SQLite snapshot
|
||||
PROFILES["arrs_stack_CRITICAL_CONTAINER_NAMES"]=(
|
||||
"Sonarr" "Radarr" "Lidarr" "Prowlarr" "Bazarr" "Pinchflat"
|
||||
)
|
||||
|
||||
# ── critical-data ──────────────────────────────────────────────────────────
|
||||
# Auth stack — stopped for clean database snapshot, delayed restart
|
||||
PROFILES["critical-data_CRITICAL_CONTAINER_NAMES"]=(
|
||||
"Mariadb-Authelia" "Redis-Authelia"
|
||||
"NginxProxyManager" "Lldap-Gmer4Lfe"
|
||||
)
|
||||
PROFILES["critical-data_DELAYED_CONTAINERS"]=(
|
||||
"Authelia" "Authelia-Secondary" # auth services restart after delay
|
||||
)
|
||||
PROFILES["critical-data_CONTAINER_DELAY"]=30 # seconds before delayed containers start
|
||||
|
||||
# ── important-data ─────────────────────────────────────────────────────────
|
||||
# NextCloud + Postgres — stopped for clean snapshot
|
||||
PROFILES["important-data_CRITICAL_CONTAINER_NAMES"]=(
|
||||
"Postgres-NextCloud"
|
||||
)
|
||||
PROFILES["important-data_DELAYED_CONTAINERS"]=("NextCloud")
|
||||
|
||||
# ── emby ───────────────────────────────────────────────────────────────────
|
||||
# Weekly full clean sync — both Emby instances stopped
|
||||
PROFILES["emby_CRITICAL_CONTAINER_NAMES"]=("Emby")
|
||||
PROFILES["emby_EXCLUDE_DIRS"]=(
|
||||
"transcodes/" "logs/" "crash*" "cache/"
|
||||
)
|
||||
|
||||
# ── emby-failover ──────────────────────────────────────────────────────────
|
||||
# Every 30 minutes, Emby STAYS RUNNING — dirty sync of critical state only
|
||||
# WAL and SHM excluded — safe to copy while Emby is writing
|
||||
PROFILES["emby-failover_CRITICAL_CONTAINER_NAMES"]=() # empty — nothing stops
|
||||
PROFILES["emby-failover_EXCLUDE_DIRS"]=(
|
||||
"*.wal" "*.shm" # WAL files — unsafe mid-write
|
||||
"transcodes/" "logs/" "crash*" "cache/" # volatile data — skip
|
||||
)
|
||||
PROFILES["emby-failover_REMOTE_RESTART_CONTAINERS"]=("Emby")
|
||||
# Emby on HOST2 restarts after sync to pick up config changes
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Two Emby Profiles — Why Both Exist ──────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# emby-failover — every 30 minutes, Emby stays running:
|
||||
# What syncs: users.db, library.db, authentication.db, config/
|
||||
# What skips: *.wal *.shm transcodes/ logs/ cache/
|
||||
# Why: WAL files are being written while Emby runs — copying them
|
||||
# would produce a corrupt database on HOST2
|
||||
# Result: HOST2 is always within 30 minutes of HOST1 on watch state
|
||||
# and user activity. Failover is seamless — nobody notices.
|
||||
#
|
||||
# emby — Sunday 2:30am, both Emby instances stopped:
|
||||
# What syncs: everything except transcodes, logs, cache, crash files
|
||||
# What includes: metadata, plugins, full database state, all config
|
||||
# Why: WAL is checkpointed on clean shutdown — safe to copy everything
|
||||
# Full consistent mirror including metadata and plugin state
|
||||
# Result: HOST2 has a gold-standard Emby state once per week
|
||||
# Image cache warm for 6 days — only reset Sunday when users sleep
|
||||
#
|
||||
# The two profiles work together:
|
||||
# emby-failover: keeps HOST2 current on what matters for immediate failover
|
||||
# emby: gives HOST2 full fidelity once per week
|
||||
# Neither alone is sufficient — both are needed.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 8 — PERSONAL ENCRYPTED SHARES ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
Personal shares are synced to the remote server for offsite backup. ZFS encrypts at
|
||||
the dataset level — the remote server receives encrypted blocks and cannot read the
|
||||
content without your passphrase or keyfile.
|
||||
|
||||
---
|
||||
|
||||
### ── Create an Encrypted ZFS Dataset ────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# In the unRAID UI:
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Main → click your ZFS pool name → + Dataset
|
||||
# Name: Gmer4Lfe-Personal
|
||||
# Encryption: Enabled
|
||||
# Passphrase: [your passphrase]
|
||||
# ⚠️ Write your passphrase down — if lost, data is completely unrecoverable
|
||||
#
|
||||
# Settings → Shares → Add Share
|
||||
# Share path: point to the new encrypted dataset
|
||||
# Use cache: Only — keeps data on ZFS pool, not array
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# Verify encryption is active before syncing:
|
||||
zfs get encryption poolname/Gmer4Lfe-Personal
|
||||
# Should show: encryption aes-256-gcm
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Auto-Unlock on Boot (Optional) ─────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Keyfile approach — passphrase stored in a file, loaded at boot.
|
||||
# More convenient but the keyfile is a secret that must be protected.
|
||||
# Never sync the keyfile to the remote server.
|
||||
#
|
||||
# Create keyfile — on HOST1 only:
|
||||
dd if=/dev/urandom bs=32 count=1 | base64 > /root/.zfs-keys/personal.key
|
||||
chmod 600 /root/.zfs-keys/personal.key
|
||||
|
||||
# Set dataset to use keyfile instead of passphrase:
|
||||
zfs change-key \
|
||||
-o keylocation=file:///root/.zfs-keys/personal.key \
|
||||
-o keyformat=raw \
|
||||
poolname/Gmer4Lfe-Personal
|
||||
|
||||
# Add to ramdisk_setup.sh or array_start.sh custom scripts:
|
||||
zfs load-key poolname/Gmer4Lfe-Personal
|
||||
zfs mount poolname/Gmer4Lfe-Personal
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Manual unlock alternative (most secure — passphrase only in your head):
|
||||
zfs load-key poolname/Gmer4Lfe-Personal # prompts for passphrase
|
||||
zfs mount poolname/Gmer4Lfe-Personal
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Add to master_host1.conf ────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# master_host1.conf
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Personal shares append to the daily sync after DAILY_SYNC_SHARES.
|
||||
# Remote server receives encrypted blocks — cannot read content without your key.
|
||||
#
|
||||
HOST1_PERSONAL_SHARES=(
|
||||
"/mnt/user/Gmer4Lfe-Personal"
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 9 — USER SCRIPTS SETUP ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
The ecosystem is designed so the User Scripts plugin has only a small number of entries —
|
||||
each one an orchestrator. Individual scripts are never scheduled directly.
|
||||
|
||||
---
|
||||
|
||||
### ── At Startup of Array ─────────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# Create one script entry named "array start":
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
#!/bin/bash
|
||||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/array_start.sh
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Schedule: At Startup of Array
|
||||
# Run as: Background Task
|
||||
#
|
||||
# This is the ONLY "At Startup of Array" entry needed.
|
||||
# It launches everything in ARRAY_START_SCRIPTS from master.conf:
|
||||
# inotify_tuning.sh — raise inotify limits before containers start
|
||||
# docker_syslog_filter.sh — suppress veth log noise
|
||||
# php_fpm_max_children.sh — WebGUI tuning
|
||||
# ramdisk_setup.sh — create ramdisk before Emby starts
|
||||
# docker_network_connect.sh — connect containers to extra networks
|
||||
# system_watchdog.sh — continuous system health monitor
|
||||
# docker_watchdog.sh — continuous container health monitor
|
||||
# failover.sh — continuous mutual failover
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Cron Schedule ────────────────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# Create one script entry per cron schedule below.
|
||||
# All entries: Run as Background Task
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# Every 3 minutes — transcode cleanup + manager:
|
||||
*/3 * * * *
|
||||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/transcode_management.sh
|
||||
|
||||
# Every 30 minutes — Emby dirty sync (watch states, library delta):
|
||||
*/30 * * * *
|
||||
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
|
||||
/mnt/user/Media_Server/Emby --profile=emby-failover
|
||||
|
||||
# Every 6 hours — failed import + stalled download recovery:
|
||||
0 */6 * * *
|
||||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/arrs_failed_stalled_recovery.sh
|
||||
|
||||
# 1am daily — full maintenance window:
|
||||
# git pull → rsync all shares → permissions → cleaners → arr cleanup → docker restart
|
||||
0 1 * * *
|
||||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh
|
||||
|
||||
# 2:30am Sunday — weekly maintenance window:
|
||||
# stop containers → pull updates → clean sync → start containers → weekly restarts
|
||||
30 2 * * 0
|
||||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/weekly_sync_maintenance.sh
|
||||
|
||||
# 8am daily — health digest (DIGEST_PROFILE in master.conf controls when it notifies):
|
||||
0 8 * * *
|
||||
bash /mnt/user/appdata/unraid_scripts/Monitors/weekly_health_digest.sh
|
||||
|
||||
# Every 6 hours — inotify + php-fpm utilisation snapshot:
|
||||
0 */6 * * *
|
||||
bash /mnt/user/appdata/unraid_scripts/Monitors/system_tuning_monitor.sh
|
||||
|
||||
# Sunday morning — weekly reports:
|
||||
0 6 * * 0 bash .../Monitors/zfs_memory_snapshot.sh
|
||||
0 7 * * 0 bash .../Monitors/smart_health.sh
|
||||
0 9 * * 0 bash .../Monitors/cert_monitor.sh
|
||||
0 10 * * 0 bash .../Monitors/backup_verify.sh
|
||||
0 11 * * 0 bash .../Monitors/emby_session_report.sh
|
||||
0 11 * * 0 bash .../Monitors/bandwidth_monitor.sh --report
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
```
|
||||
|
||||
> **Set all entries to "Background Task"** — output streams correctly to the User
|
||||
> Scripts log rather than buffering in the browser tab. Non-background tasks can
|
||||
> appear to hang on long-running scripts.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 10 — VERIFY THE SETUP ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
Before relying on scheduled jobs, test manually from the terminal on HOST1.
|
||||
Test with `--dry-run` first — no changes made, but the full pre-flight and
|
||||
configuration resolution runs.
|
||||
|
||||
---
|
||||
|
||||
### ── Test a Single Profile Sync ─────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# Dry run with verbose output — shows every decision the script makes:
|
||||
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
|
||||
/mnt/user/appdata-Failover/Arrs_Stack --dry-run --log
|
||||
```
|
||||
|
||||
```
|
||||
Expected output (healthy):
|
||||
━━━ ⚙️ Setup ━━━
|
||||
Host: HOST1 (unRAID-Gmer4Lfe) → HOST2 (unRAID-Jayred365)
|
||||
Remote IP: 100.x.x.x
|
||||
Profile: arrs_stack
|
||||
|
||||
━━━ 🛡️ Pre-flight Checks ━━━
|
||||
✅ Remote reachable
|
||||
✅ version parity — both on unRAID X.Y.Z
|
||||
✅ Remote Docker daemon responding
|
||||
✅ Remote rootfs: 12% (threshold: 75%)
|
||||
✅ Remote share exists and not empty
|
||||
✅ All pre-flight checks passed
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Test the Daily Orchestrator ─────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# Dry run of the full daily window — shows every job that would run:
|
||||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh --dry-run
|
||||
```
|
||||
|
||||
```
|
||||
If any pre-flight check fails, the script aborts with a clear error message
|
||||
before touching anything. Fix the reported issue and re-run --dry-run.
|
||||
|
||||
Common pre-flight failures and their causes:
|
||||
"Remote not reachable" → Tailscale not connected on HOST2
|
||||
"Version mismatch" → different unRAID versions — update before syncing
|
||||
"Remote rootfs above X%" → HOST2's root filesystem nearly full
|
||||
"Remote share missing" → share doesn't exist on HOST2 yet (see Step 11)
|
||||
"Docker daemon not responding" → HOST2's Docker service not started
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Check the Configuration Resolved Correctly ──────────────────────────────
|
||||
|
||||
```bash
|
||||
# --status shows how master.conf resolved for this server and profile:
|
||||
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
|
||||
/mnt/user/appdata-Failover/Arrs_Stack --status
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ STEP 11 — INITIAL HOST2 SYNC ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
If HOST2 is being set up from scratch with empty shares:
|
||||
|
||||
---
|
||||
|
||||
### ── Create Share Structure on HOST2 ────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# On HOST2 — start the array and create shares via the unRAID UI.
|
||||
# Or use the share recreation tool to create disk directories from HOST1's cfg files:
|
||||
bash /mnt/user/appdata/unraid_scripts/Tools/recreate_shares.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ── Initial Push From HOST1 ─────────────────────────────────────────────────
|
||||
|
||||
```bash
|
||||
# On HOST1 — push all shares to HOST2 for the first time:
|
||||
# Use --log for verbose output on first run
|
||||
bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync_maintenance.sh --log
|
||||
```
|
||||
|
||||
```
|
||||
First run may take several hours for large libraries — this is normal.
|
||||
The scheduled nightly sync will be incremental after the initial push.
|
||||
Progress shows per-share throughout.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ NAMING CONSISTENCY — THIS IS REQUIRED ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
The ecosystem uses one codebase on both servers. This only works if containers and
|
||||
shares have identical names on both servers. This is not configurable — it is a
|
||||
design requirement.
|
||||
|
||||
```bash
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Container names must match exactly on both servers:
|
||||
"Emby" ← both HOST1 and HOST2
|
||||
"NginxProxyManager" ← both HOST1 and HOST2
|
||||
"Mariadb-Authelia" ← both HOST1 and HOST2
|
||||
|
||||
# Share paths must match exactly on both servers:
|
||||
/mnt/user/Movies ← both HOST1 and HOST2 (HOST2 has a mirror)
|
||||
/mnt/user/Tv_Shows ← both HOST1 and HOST2
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# If a container has a different name on one server: the script skips it
|
||||
# without error. It silently does the wrong thing. You only notice when
|
||||
# the container is not stopped during a sync that requires it to stop.
|
||||
#
|
||||
# If a share has a different path: rsync.sh aborts with "remote share missing".
|
||||
# Easier to catch — but still requires renaming the share to fix.
|
||||
#
|
||||
# Keep names consistent and one codebase covers both servers automatically.
|
||||
# Diverge and every script that touches containers or shares needs custom logic.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ REPOSITORY STRUCTURE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
```
|
||||
Unraid_Scripts/
|
||||
├── master.conf ← All shared configuration — edit this file
|
||||
├── master_host1.conf ← HOST1-specific configuration
|
||||
├── master_host2.conf ← HOST2-specific configuration
|
||||
├── common.sh ← Shared library — functions used by all scripts
|
||||
├── load_config.sh ← Config loader — sources all conf files
|
||||
│
|
||||
├── Orchestrators/
|
||||
│ ├── array_start.sh ← Single "At Startup of Array" entry point
|
||||
│ ├── daily_sync_maintenance.sh ← 1am daily window orchestrator
|
||||
│ ├── weekly_sync_maintenance.sh ← Sunday 2:30am window orchestrator
|
||||
│ ├── critical_sync_maintenance.sh ← Every 15 minutes — critical sync + partnership
|
||||
│ ├── media_management.sh ← Permissions + cleaners + arr cleanup
|
||||
│ ├── transcode_management.sh ← Transcode cleanup then manager
|
||||
│ └── arrs_failed_stalled_recovery.sh ← Failed import + stalled download recovery
|
||||
│
|
||||
├── Rsync/
|
||||
│ └── rsync.sh ← Core rsync script — called per share
|
||||
│
|
||||
├── Failover/
|
||||
│ ├── failover.sh ← Mutual container failover — continuous loop
|
||||
│ ├── failover_test.sh ← Controlled iptables failover simulation
|
||||
│ └── failover_state_reset.sh ← Reset failover state manually
|
||||
│
|
||||
├── Docker_Essentials/
|
||||
│ ├── docker_watchdog.sh ← Two-tier container monitor — continuous loop
|
||||
│ ├── docker_daily_restart.sh ← Nightly container restarts
|
||||
│ ├── docker_weekly_restart.sh ← Weekly container restarts
|
||||
│ ├── docker_network_connect.sh ← Ensure networks + connections at array start
|
||||
│ └── watchdog_skip_list_manager.sh ← Skip list inspection and recovery
|
||||
│
|
||||
├── unRAID_Essentials/
|
||||
│ ├── system_watchdog.sh ← Three-tier system health monitor — continuous
|
||||
│ ├── ramdisk_setup.sh ← Creates ramdisk + symlink at array start
|
||||
│ ├── inotify_tuning.sh ← Raise inotify limits at array start
|
||||
│ ├── docker_syslog_filter.sh ← Suppress veth log noise
|
||||
│ ├── php_fpm_max_children.sh ← WebGUI performance tuning
|
||||
│ ├── server_reboot.sh ← Graceful reboot with pre-flight warnings
|
||||
│ ├── mover_stop.sh ← Stop mover cleanly with wall warning
|
||||
│ ├── clear_logs.sh ← Size-threshold log cleanup
|
||||
│ ├── webgui_restart.sh ← nginx → php-fpm → emhttp escalation
|
||||
│ └── git_pull_execute.sh ← Pull latest scripts from Gitea
|
||||
│
|
||||
├── Media/
|
||||
│ ├── media_shares_permissions.sh ← Apply permissions to media shares
|
||||
│ ├── media_cleaner.sh ← Remove junk files from media shares
|
||||
│ ├── lidarr_cleanup.sh ← Remove orphaned music files (HOST1 only)
|
||||
│ ├── sonarr_cleanup.sh ← Remove orphaned TV files (host-aware)
|
||||
│ └── radarr_cleanup.sh ← Remove orphaned movie files (host-aware)
|
||||
│
|
||||
├── Transcodes/
|
||||
│ ├── transcode_manager.sh ← Ramdisk/SSD symlink management
|
||||
│ └── transcode_cleanup.sh ← Remove stale segment files
|
||||
│
|
||||
├── Monitors/
|
||||
│ ├── cert_monitor.sh ← SSL cert expiry via live TLS connection
|
||||
│ ├── backup_verify.sh ← rsync mirror MD5 checksum verification
|
||||
│ ├── smart_health.sh ← Drive SMART attribute monitoring
|
||||
│ ├── zfs_memory_snapshot.sh ← ZFS health + ARC + memory report
|
||||
│ ├── bandwidth_monitor.sh ← rsync transfer logging + weekly report
|
||||
│ ├── weekly_health_digest.sh ← Full ecosystem health aggregation
|
||||
│ ├── emby_session_report.sh ← Emby streaming usage statistics
|
||||
│ ├── system_tuning_monitor.sh ← inotify + php-fpm utilisation tracking
|
||||
│ └── continuous_scripts_status.sh ← Live dashboard for background processes
|
||||
│
|
||||
├── Partnership/
|
||||
│ └── partnership_manage.sh ← Two-server relationship lifecycle manager
|
||||
│
|
||||
└── Tools/
|
||||
├── recreate_shares.sh ← Create share directories from cfg files
|
||||
├── bulk_permissions_repair.sh ← One-shot permission repair
|
||||
├── rsync_stop.sh ← Stop active rsync jobs cleanly
|
||||
├── user_scripts_stop.sh ← Stop running user script processes
|
||||
├── server_reboot.sh ← Graceful scheduled reboot
|
||||
├── zfs_pool_scrub.sh ← Trigger ZFS pool scrub
|
||||
└── container_data_export.sh ← Export container configuration
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ TROUBLESHOOTING ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
---
|
||||
|
||||
### 🔴 SSH Connection Refused / Timeout
|
||||
|
||||
```bash
|
||||
# Verify SSH is enabled on the remote:
|
||||
# Settings → Management Access → Secure Shell → Enabled
|
||||
|
||||
# Verify Tailscale is connected:
|
||||
tailscale ip -4 unRAID-Jayred365 # should return 100.x.x.x
|
||||
|
||||
# Test SSH manually with the key:
|
||||
ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@$(tailscale ip -4 unRAID-Jayred365) "hostname"
|
||||
# Expected: unRAID-Jayred365
|
||||
# If password prompted: key not authorised — recheck Step 3b
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🔴 Pre-flight Aborts on Remote Rootfs
|
||||
|
||||
```bash
|
||||
# Remote rootfs above ROOTFS_WARN threshold
|
||||
# Check current usage on remote:
|
||||
ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@[HOST2-ip] "df /"
|
||||
|
||||
# Common cause: array not started, drives not mounted
|
||||
# Verify array is started on HOST2 before running syncs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🔴 Remote Share Missing
|
||||
|
||||
```bash
|
||||
# Share exists locally but not on remote
|
||||
# Verify the share exists on HOST2:
|
||||
ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@[HOST2-ip] "ls /mnt/user/"
|
||||
|
||||
# If missing — create the share on HOST2 first, then run initial sync (Step 11)
|
||||
# Or run recreate_shares.sh on HOST2 to create directories from HOST1's cfg files
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🔴 Containers Not Stopping / Starting
|
||||
|
||||
```bash
|
||||
# Verify container names in master_host*.conf match Docker exactly — case sensitive
|
||||
# Check what Docker actually calls the container:
|
||||
docker ps --format "{{.Names}}"
|
||||
|
||||
# Test Docker commands to remote manually:
|
||||
ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@[HOST2-ip] "docker ps --format '{{.Names}}'"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🔴 Profile Not Matching
|
||||
|
||||
```bash
|
||||
# Profile key = directory basename lowercased
|
||||
# /mnt/user/appdata-Failover/Arrs_Stack → basename: Arrs_Stack → key: arrs_stack
|
||||
|
||||
# Override explicitly if basename doesn't match a profile name:
|
||||
bash rsync.sh /mnt/user/appdata-Failover/My_Stuff --profile=arrs_stack
|
||||
|
||||
# Verify what profile resolved for a path:
|
||||
bash rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --status
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🔴 Script Not Found
|
||||
|
||||
```bash
|
||||
# Verify repo was cloned to the correct location:
|
||||
ls /mnt/user/appdata/unraid_scripts/master.conf
|
||||
|
||||
# Make scripts executable:
|
||||
find /mnt/user/appdata/unraid_scripts -name "*.sh" -exec chmod +x {} \;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ AVAILABLE FLAGS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
All scripts support these flags. Use `--dry-run` before any live operation.
|
||||
|
||||
```bash
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
--dry-run run without making any changes — pre-flight still runs ✅
|
||||
--log verbose output — show every decision made
|
||||
--status show resolved configuration and exit — no rsync, no sync
|
||||
--no-log suppress verbose output (some scripts)
|
||||
|
||||
# Examples:
|
||||
bash rsync.sh /mnt/user/Movies --dry-run --log # preview a media sync
|
||||
bash rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --status # check profile resolution
|
||||
bash daily_sync_maintenance.sh --dry-run # preview full daily window
|
||||
bash docker_watchdog.sh --status # check watchdog state
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
```
|
||||
+118
-49
@@ -1,64 +1,133 @@
|
||||
#!/bin/bash
|
||||
# ==============================================================================================
|
||||
# ================================= Rsync Core Script ==========================================
|
||||
# ============================= Rsync Core Script ==============================================
|
||||
# ==============================================================================================
|
||||
# Core rsync script — called per share or per appdata profile.
|
||||
# Called by orchestrators (daily/weekly/critical sync) and directly for manual syncs.
|
||||
#
|
||||
# ── PROFILE SYSTEM ────────────────────────────────────────────────────────────────────────────
|
||||
# Profile is inferred from the directory basename (lowercased).
|
||||
# Override with --profile=name for explicit profile selection.
|
||||
# If no profile match found → all settings fall back to global defaults in master.conf.
|
||||
# PURPOSE
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Core rsync engine for the two-server ecosystem. Called per share or per
|
||||
# appdata profile by orchestrators (daily_sync_maintenance, weekly_sync_maintenance,
|
||||
# critical_sync_maintenance) and directly for manual or scheduled dirty syncs.
|
||||
#
|
||||
# Profiles define:
|
||||
# PROFILE_RSYNC_OPTS — rsync flags (does NOT inherit DEFAULT_RSYNC_OPTS)
|
||||
# PROFILE_BW_LIMIT — bandwidth limit in KB/s
|
||||
# PROFILE_RETRY_COUNT — retry attempts before giving up
|
||||
# PROFILE_SLEEP — seconds between retry attempts
|
||||
# PROFILE_CRITICAL_CONTAINER_NAMES — containers stopped both sides before sync
|
||||
# PROFILE_DELAYED_CONTAINERS — containers needing delay before starting after sync
|
||||
# PROFILE_CONTAINER_DELAY — seconds before starting delayed containers
|
||||
# PROFILE_EXCLUDE_DIRS — paths excluded from transfer
|
||||
# Profile is inferred from the directory basename (lowercased). Override with
|
||||
# --profile=name for explicit selection. If no profile matches, global defaults
|
||||
# from master.conf apply and no containers are stopped.
|
||||
#
|
||||
# After each sync, logs transfer data to bandwidth_monitor.sh for the weekly
|
||||
# bandwidth report. Silent on success — only failures produce visible output.
|
||||
#
|
||||
# ==============================================================================================
|
||||
# OPERATIONAL MODEL
|
||||
# ==============================================================================================
|
||||
#
|
||||
# Profiles define per-share behavior:
|
||||
# PROFILE_CRITICAL_CONTAINER_NAMES — containers stopped on both servers before sync
|
||||
# PROFILE_DELAYED_CONTAINERS — containers with a delay before restart after sync
|
||||
# PROFILE_CONTAINER_DELAY — seconds before delayed containers start
|
||||
# PROFILE_RSYNC_OPTS — rsync flags (does not inherit DEFAULT_RSYNC_OPTS)
|
||||
# PROFILE_BW_LIMIT — bandwidth limit in KB/s
|
||||
# PROFILE_RETRY_COUNT — retry attempts on failure
|
||||
# PROFILE_SLEEP — seconds between retry attempts
|
||||
# PROFILE_EXCLUDE_DIRS — paths excluded from transfer
|
||||
# PROFILE_REMOTE_RESTART_CONTAINERS — containers restarted on remote after dirty sync
|
||||
# (critical-fallback, emby-fallback profiles)
|
||||
# Was running → restart. Was stopped → leave stopped.
|
||||
#
|
||||
# ── RSYNC ENABLE/DISABLE ──────────────────────────────────────────────────────────────────────
|
||||
# Two-tier toggle system — checked at entry:
|
||||
# Tier 1: RSYNC_ENABLED=false → all rsync stops
|
||||
# Tier 2: Per-orchestrator flag (DAILY_RSYNC_ENABLED etc.) — checked by caller
|
||||
# Direct calls to rsync.sh only check Tier 1
|
||||
# Two-tier rsync enable/disable:
|
||||
# Tier 1: RSYNC_ENABLED=false → all rsync stops immediately (checked by this script)
|
||||
# Tier 2: per-orchestrator flag (DAILY_RSYNC_ENABLED etc.) → checked by caller
|
||||
#
|
||||
# ── BANDWIDTH LOGGING ─────────────────────────────────────────────────────────────────────────
|
||||
# After each sync logs to bandwidth_monitor.sh --log-transfer:
|
||||
# profile | duration_seconds | status | bytes_transferred
|
||||
# Bytes captured from rsync --stats output — version-proof parsing.
|
||||
# bandwidth_monitor.sh flags syncs exceeding BANDWIDTH_WARN_GB.
|
||||
# Bandwidth logging: after each sync, logs profile/duration/status/bytes to
|
||||
# bandwidth_monitor.sh --log-transfer. Bytes captured from rsync --stats via awk
|
||||
# using version-stable field names.
|
||||
#
|
||||
# ── DIRTY SYNC REMOTE RESTART ─────────────────────────────────────────────────────────────────
|
||||
# Profiles using dirty sync (critical-fallback, emby-fallback) define
|
||||
# PROFILE_REMOTE_RESTART_CONTAINERS — containers restarted on remote after sync completes.
|
||||
# This ensures the remote picks up config changes synced during the dirty window.
|
||||
# Was running → restart. Was stopped → leave stopped.
|
||||
# ==============================================================================================
|
||||
# OPERATIONAL SAFEGUARDS
|
||||
# ==============================================================================================
|
||||
#
|
||||
# ── SAFEGUARDS ────────────────────────────────────────────────────────────────────────────────
|
||||
# check_rsync_enabled() — Tier 1 gate before any operation
|
||||
# check_unraid_version_parity — refuses if servers on incompatible unRAID versions
|
||||
# check_remote_docker_daemon — verifies remote daemon before container operations
|
||||
# check_local_disk_temps() — temp check before transfer (exit 1=skip, 2=abort all)
|
||||
# check_connectivity() — verifies remote reachable
|
||||
# check_remote_rootfs() — aborts if remote rootfs nearly full
|
||||
# check_remote_share() — aborts if target directory missing or empty
|
||||
# check_remote_disks() — verifies all backing disks online on remote
|
||||
# acquire_rsync_lock() — per-profile lock + global concurrent limit
|
||||
# validate_unraid_cmd — notify validated before use
|
||||
# Silent by default — only failures produce visible output
|
||||
# Global Rsync Gate
|
||||
# check_rsync_enabled() — RSYNC_ENABLED=false exits cleanly before any operation.
|
||||
#
|
||||
# Partnership Blocklist
|
||||
# Refuses to sync if REMOTE_SERVER_NAME appears in the partnership blocklist.
|
||||
# Written at offboard — prevents stale access after a partnership ends.
|
||||
#
|
||||
# Version Parity
|
||||
# check_unraid_version_parity — refuses sync if servers on incompatible unRAID versions.
|
||||
#
|
||||
# Remote Health Pre-flights
|
||||
# check_connectivity() — Tailscale IP reachable before any SSH
|
||||
# check_remote_rootfs() — aborts if remote rootfs exceeds ROOTFS_WARN_PCT
|
||||
# check_remote_share() — aborts if target directory missing or empty on remote
|
||||
# check_remote_disks() — verifies all backing disks online on remote
|
||||
#
|
||||
# Drive Temperature Check
|
||||
# check_local_disk_temps() — runs before any transfer. Exit 1 = skip this profile,
|
||||
# exit 2 = abort all remaining syncs (CRITICAL temperature).
|
||||
#
|
||||
# Remote Docker Daemon Check
|
||||
# check_remote_docker_daemon — verified before any container stop/start operations.
|
||||
# If daemon unresponsive: container operations skipped, rsync proceeds without stopping.
|
||||
#
|
||||
# Per-Profile Concurrency Lock
|
||||
# acquire_rsync_lock() — per-profile lock prevents parallel runs of the same profile.
|
||||
# Global concurrent limit prevents too many simultaneous rsync processes.
|
||||
#
|
||||
# Notification Validated
|
||||
# validate_unraid_cmd confirms the notify script is present before use.
|
||||
#
|
||||
# ==============================================================================================
|
||||
# CONFIGURATION
|
||||
# ==============================================================================================
|
||||
#
|
||||
# master.conf
|
||||
#
|
||||
# RSYNC_ENABLED
|
||||
# Global on/off toggle for all rsync operations. (default: true)
|
||||
#
|
||||
# DEFAULT_RSYNC_OPTS
|
||||
# Base rsync flags for unproiled shares. Note: --delete is intentionally absent —
|
||||
# media shares spread files only, arr cleanup scripts own deletions. Profile-specific
|
||||
# opts set --delete explicitly where needed.
|
||||
#
|
||||
# BW_LIMIT
|
||||
# Default bandwidth cap in KB/s when no PROFILE_BW_LIMIT is set. (default: 0 = unlimited)
|
||||
#
|
||||
# RETRY_COUNT
|
||||
# Default retry attempts on rsync failure. (default: 3)
|
||||
#
|
||||
# SLEEP
|
||||
# Default seconds between retry attempts. (default: 60)
|
||||
#
|
||||
# ROOTFS_WARN_PCT
|
||||
# Abort threshold for remote rootfs percentage full. (default: 75)
|
||||
#
|
||||
# PROFILES["profile_KEY"]
|
||||
# Profile definitions — one entry per PROFILE_* key per profile name.
|
||||
# See OPERATIONAL MODEL above for all supported keys.
|
||||
#
|
||||
# BANDWIDTH_LOG / BANDWIDTH_WARN_GB
|
||||
# Shared with bandwidth_monitor.sh — set once, used by both.
|
||||
#
|
||||
# ==============================================================================================
|
||||
# RUNTIME MODES
|
||||
# ==============================================================================================
|
||||
#
|
||||
# rsync.sh /path/to/share
|
||||
# Sync the given path to the remote. Profile inferred from directory basename.
|
||||
#
|
||||
# rsync.sh /path/to/share --profile=name
|
||||
# Sync with explicit profile override — bypasses basename inference.
|
||||
#
|
||||
# rsync.sh /path/to/share --dry-run
|
||||
# Run all pre-flight checks and show what rsync would transfer. No transfer,
|
||||
# no container stops.
|
||||
#
|
||||
# rsync.sh /path/to/share --status
|
||||
# Show resolved profile, remote identity, and configuration. Then exit.
|
||||
#
|
||||
# rsync.sh /path/to/share --log
|
||||
# Verbose output throughout — every decision logged.
|
||||
#
|
||||
# ── USAGE ─────────────────────────────────────────────────────────────────────────────────────
|
||||
# rsync.sh /mnt/user/Movies — media share, global defaults
|
||||
# rsync.sh /mnt/user/appdata-Fallback/Arrs_Stack — matched to [arrs_stack] profile
|
||||
# rsync.sh /mnt/user/appdata-Fallback/Arrs_Stack --dry-run --log
|
||||
# rsync.sh /mnt/user/appdata-Fallback/Critical-Data --profile=critical-fallback
|
||||
# ==============================================================================================
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
Reference in New Issue
Block a user