699 lines
21 KiB
Markdown
699 lines
21 KiB
Markdown
# ━━━━━ RSYNC — Manual ━━━━━
|
|
|
|
Config reference, procedures, operational workflows.
|
|
For overview see README-Rsync.md. For per-script detail see the rsync.sh header.
|
|
|
|
This document also serves as the **initial setup guide** for standing up the
|
|
two-server ecosystem from scratch.
|
|
|
|
---
|
|
|
|
## ━━━ WHAT YOU'RE BUILDING ━━━
|
|
|
|
```
|
|
HOST1 (unRAID-Gmer4Lfe) HOST2 (unRAID-Jayred365)
|
|
────────────────────── ──────────────────────
|
|
Downloads to: Downloads to:
|
|
/mnt/user/Movies /mnt/user/Anime_Movies
|
|
/mnt/user/Tv_Shows /mnt/user/Anime_Shows
|
|
/mnt/user/Music
|
|
|
|
Both servers' arrs track ALL content across both servers.
|
|
It does not matter who downloaded what or when.
|
|
|
|
Daily 3-phase window (1am):
|
|
|
|
Phase 1 — arr_sync.sh (runs first):
|
|
All arrs on all nodes reconcile libraries using external IDs
|
|
(TMDB, TVDB, MusicBrainz). Union model — any node that tracks
|
|
an item, all nodes get it. After this phase both servers' arrs
|
|
know about all content regardless of who downloaded it.
|
|
|
|
Phase 2 — rsync (bidirectional, no --delete on media shares):
|
|
HOST1 pushes its shares ──────► HOST2 receives files
|
|
HOST1 receives files ◄────── HOST2 pushes its shares
|
|
Files arrive already tracked by the remote arr (arr_sync ran first).
|
|
No --delete — media files only spread. Arr cleanup handles deletions.
|
|
|
|
Phase 3 — arr cleanup:
|
|
sonarr_cleanup / radarr_cleanup / lidarr_cleanup query the live arr
|
|
API and remove any files no longer tracked. Emby notified after.
|
|
|
|
Note: profiled syncs (arrs_stack, emby, critical-data…) use their own
|
|
PROFILE_RSYNC_OPTS which include --delete. Only the no-profile media
|
|
shares use DEFAULT_RSYNC_OPTS (spread only).
|
|
|
|
Weekly clean sync (2:30am Sunday, containers stopped):
|
|
Emby — full clean mirror ◄──────► Emby
|
|
Critical-Data ──────► Auth stack (HOST1 → HOST2)
|
|
|
|
Every 30 minutes — dirty sync:
|
|
Emby watch states ──────► HOST2 stays current on playback
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━ PREREQUISITES ━━━
|
|
|
|
Required on both servers before starting:
|
|
|
|
```
|
|
unRAID 7.x
|
|
Community Applications plugin — search "Community Applications" in unRAID plugins
|
|
User Scripts plugin — install via Community Applications
|
|
Tailscale plugin — install via Community Applications
|
|
Terminal access — unRAID UI → Tools → Terminal, or SSH
|
|
```
|
|
|
|
Optional but recommended:
|
|
```
|
|
Gitea (Docker container on HOST1) — self-hosted git for the script repository
|
|
Working Emby installation — for transcode management and fallback
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━ STEP 1 — TAILSCALE ━━━
|
|
|
|
Tailscale provides the encrypted mesh network between servers. Scripts resolve the
|
|
remote server's IP via Tailscale at runtime — no hardcoded IPs, no VPN configuration,
|
|
no open ports. All server-to-server communication goes through Tailscale.
|
|
|
|
### Install on Both Servers
|
|
|
|
```
|
|
1. Open Apps in the unRAID UI
|
|
2. Search "Tailscale" — install the plugin
|
|
3. Settings → Tailscale → Connect
|
|
4. Authenticate with your Tailscale account (browser opens on your machine)
|
|
5. Verify both servers appear: https://login.tailscale.com/admin/machines
|
|
```
|
|
|
|
### Verify Connectivity
|
|
|
|
```bash
|
|
# From HOST1 — should return HOST2's 100.x.x.x Tailscale IP:
|
|
tailscale ip -4 unRAID-Jayred365
|
|
|
|
# Test actual connectivity:
|
|
tailscale ping unRAID-Jayred365
|
|
```
|
|
|
|
> **Critical:** The hostnames in `master.conf` (`HOST1` and `HOST2`) must match the
|
|
> Tailscale machine names **exactly** — case sensitive. All remote IP resolution goes
|
|
> through `tailscale ip -4 HOSTNAME`. A name mismatch causes every remote operation to
|
|
> fail at the IP resolution step.
|
|
|
|
---
|
|
|
|
## ━━━ STEP 2 — ENABLE SSH ━━━
|
|
|
|
unRAID 7.x has SSH disabled by default. Enable it on both servers.
|
|
|
|
```
|
|
Settings → Management Access → Secure Shell
|
|
SSH: Enabled
|
|
SSH port: 22
|
|
Apply
|
|
```
|
|
|
|
SSH is only exposed on your local network and Tailscale interface. No ports are
|
|
opened to the public internet.
|
|
|
|
---
|
|
|
|
## ━━━ STEP 3 — SSH KEYS ━━━
|
|
|
|
Two sets of keys needed: rsync automation keys (server-to-server) and a Gitea
|
|
access key (for script repository pulls).
|
|
|
|
### 3a — Rsync Automation Keys (ssh_setup.sh)
|
|
|
|
Run on **each server**. `ssh_setup.sh` generates the key, copies it to the remote,
|
|
and updates `host*.conf` with the key path automatically.
|
|
|
|
```bash
|
|
# On HOST1 — after the repo is cloned:
|
|
bash /boot/config/plugins/varaverk/Partnership/ssh_setup.sh
|
|
|
|
# On HOST2:
|
|
bash /boot/config/plugins/varaverk/Partnership/ssh_setup.sh
|
|
```
|
|
|
|
Key naming convention: hostname lowercased, `unraid-` prefix stripped.
|
|
```
|
|
unRAID-Gmer4Lfe → /root/.ssh/gmer4lfe_rsync_automation
|
|
unRAID-Jayred365 → /root/.ssh/jayred365_rsync_automation
|
|
```
|
|
|
|
`host*.conf` is updated automatically with `HOST*_SSH_KEY` pointing to the
|
|
generated key. Run `--status` to verify:
|
|
|
|
```bash
|
|
bash /boot/config/plugins/varaverk/Partnership/ssh_setup.sh --status
|
|
```
|
|
|
|
> **`ssh_setup.sh` is idempotent** — safe to re-run. Use `--force` to regenerate
|
|
> a key (e.g., after a security incident) and re-copy it to the remote.
|
|
|
|
### 3b — Gitea SSH Key (manual)
|
|
|
|
```bash
|
|
# On BOTH servers:
|
|
ssh-keygen -t ed25519 -f /root/.ssh/unraid_gitea -C "unraid-gitea" -N ""
|
|
|
|
# Print the public key to add to Gitea:
|
|
cat /root/.ssh/unraid_gitea.pub
|
|
|
|
# In Gitea: Settings → SSH / GPG Keys → Add Key → paste above
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━ STEP 4 — CLONE THE REPOSITORY ━━━
|
|
|
|
```bash
|
|
# Create target directory:
|
|
mkdir -p /boot/config/plugins/varaverk
|
|
|
|
# Clone:
|
|
GIT_SSH_COMMAND="ssh -i /root/.ssh/unraid_gitea" \
|
|
git clone git@YOUR_GITEA_HOST:Varaverk/varaverk.git \
|
|
/boot/config/plugins/varaverk
|
|
|
|
# Make scripts executable:
|
|
find /boot/config/plugins/varaverk -name "*.sh" -exec chmod +x {} \;
|
|
```
|
|
|
|
Expected structure after clone:
|
|
```
|
|
master.conf ← all shared configuration
|
|
host1.conf ← HOST1-specific configuration
|
|
host2.conf ← HOST2-specific configuration
|
|
common.sh ← shared library
|
|
load_config.sh ← config loader
|
|
Orchestrators/
|
|
Rsync/
|
|
Fallback/
|
|
Docker_Essentials/
|
|
...
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━ STEP 5 — CONFIGURE MASTER.CONF ━━━
|
|
|
|
```bash
|
|
nano /boot/config/plugins/varaverk/master.conf
|
|
```
|
|
|
|
### Host Identity
|
|
|
|
```bash
|
|
# These must match Tailscale machine names exactly — case sensitive.
|
|
HOST1="unRAID-Gmer4Lfe" # REQUIRED
|
|
HOST2="unRAID-Jayred365" # REQUIRED
|
|
|
|
# SSH keys — each server uses its own key to authenticate to the other:
|
|
HOST1_SSH_KEY="/root/.ssh/Gmer4Lfe-rsync-key"
|
|
HOST2_SSH_KEY="/root/.ssh/Jayred365-rsync-key"
|
|
```
|
|
|
|
### Git Repository
|
|
|
|
```bash
|
|
GITEA_CONTAINER="Gitea"
|
|
GITEA_REPO_PATH="Varaverk/varaverk.git"
|
|
TARGET_DIR="/boot/config/plugins/varaverk"
|
|
GITEA_SSH_KEY="/root/.ssh/unraid_gitea"
|
|
SSH_PORT=221
|
|
```
|
|
|
|
### Daily Sync Shares
|
|
|
|
```bash
|
|
# host1.conf
|
|
# Shares this server downloads to — pushed to the other server nightly.
|
|
# arr_sync.sh runs before rsync in the daily window, so the receiving
|
|
# server's arrs already track incoming files when they arrive.
|
|
# Never put the same path in both lists.
|
|
HOST1_DAILY_SYNC_SHARES=(
|
|
"/mnt/user/Movies"
|
|
"/mnt/user/Tv_Shows"
|
|
"/mnt/user/Music"
|
|
"/mnt/user/Kids_Movies"
|
|
"/mnt/user/Kids_Tv_Shows"
|
|
"/mnt/user/Sports"
|
|
"/mnt/user/stand-up_comedy"
|
|
)
|
|
|
|
# host2.conf
|
|
HOST2_DAILY_SYNC_SHARES=(
|
|
"/mnt/user/Anime_Shows"
|
|
"/mnt/user/Anime_Movies"
|
|
)
|
|
```
|
|
|
|
### Weekly Sync Shares
|
|
|
|
```bash
|
|
# master.conf
|
|
# Synced during the Sunday 2:30am window — containers stopped both sides.
|
|
WEEKLY_SYNC_SHARES=(
|
|
"/mnt/user/Media_Server/Emby"
|
|
"/mnt/user/appdata-Fallback/Critical-Data"
|
|
)
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━ STEP 6 — MASTER_HOST*.CONF ━━━
|
|
|
|
`detect_hosts()` reads which server is running and aliases `HOST*_` prefixed vars
|
|
to their unprefixed names. Scripts only ever reference the unprefixed name — they
|
|
work identically on both servers.
|
|
|
|
```bash
|
|
nano /boot/config/plugins/varaverk/host1.conf # on HOST1
|
|
nano /boot/config/plugins/varaverk/host2.conf # on HOST2
|
|
```
|
|
|
|
Every variable is documented in the conf files. Key values to set:
|
|
- SSH key paths
|
|
- DAILY_SYNC_SHARES
|
|
- EMBY_URL / EMBY_API_KEY
|
|
- CERT_MONITOR_DOMAINS
|
|
- SMART_IGNORE_DRIVES
|
|
- ZFS_REPORT_IGNORE_POOLS
|
|
|
|
---
|
|
|
|
## ━━━ RSYNC PROFILES ━━━
|
|
|
|
Profiles control per-share behavior. Profile key = directory basename lowercased.
|
|
Override with `--profile=name`.
|
|
|
|
### Profile Matching
|
|
|
|
```bash
|
|
# rsync.sh /mnt/user/appdata-Fallback/Arrs_Stack
|
|
# basename: Arrs_Stack → lowercased: arrs_stack → matches [arrs_stack] profile
|
|
#
|
|
# rsync.sh /mnt/user/Movies
|
|
# basename: Movies → no profile match → global defaults (no containers stopped)
|
|
#
|
|
# rsync.sh /mnt/user/appdata-Fallback/Critical-Data --profile=critical-data
|
|
# explicit override
|
|
```
|
|
|
|
### Current Profile Definitions
|
|
|
|
```bash
|
|
# master.conf
|
|
|
|
# ── arrs_stack ───────────────────────────────────────────────────────────────
|
|
# Arr databases — stopped for clean SQLite snapshot
|
|
PROFILES["arrs_stack_CRITICAL_CONTAINER_NAMES"]=(
|
|
"Sonarr" "Radarr" "Lidarr" "Prowlarr" "Bazarr" "Pinchflat"
|
|
)
|
|
|
|
# ── critical-data ─────────────────────────────────────────────────────────────
|
|
# Auth stack — stopped for clean database snapshot, Authelia delayed restart
|
|
PROFILES["critical-data_CRITICAL_CONTAINER_NAMES"]=(
|
|
"Mariadb-Authelia" "Redis-Authelia"
|
|
"NginxProxyManager" "Lldap"
|
|
)
|
|
PROFILES["critical-data_DELAYED_CONTAINERS"]=(
|
|
"Authelia" "Authelia-Secondary"
|
|
)
|
|
PROFILES["critical-data_CONTAINER_DELAY"]=30
|
|
|
|
# ── important-data ────────────────────────────────────────────────────────────
|
|
# NextCloud + Postgres — stopped for clean snapshot
|
|
PROFILES["important-data_CRITICAL_CONTAINER_NAMES"]=(
|
|
"Postgres-NextCloud"
|
|
)
|
|
PROFILES["important-data_DELAYED_CONTAINERS"]=("NextCloud")
|
|
|
|
# ── emby ──────────────────────────────────────────────────────────────────────
|
|
# Weekly full clean sync — both Emby instances stopped
|
|
PROFILES["emby_CRITICAL_CONTAINER_NAMES"]=("Emby")
|
|
PROFILES["emby_EXCLUDE_DIRS"]=(
|
|
"transcodes/" "logs/" "crash*" "cache/"
|
|
)
|
|
```
|
|
|
|
### The `emby` Profile and Watch-State Sync
|
|
|
|
**emby** (Sunday 2:30am, both Emby instances stopped):
|
|
- Syncs: everything except transcodes, logs, cache, crash files
|
|
- Includes: metadata, plugins, full database state, all config
|
|
- Why: WAL is checkpointed on clean shutdown — safe to copy everything
|
|
- Result: HOST2 gets a gold-standard Emby state once per week
|
|
|
|
Between weekly mirrors, Emby **watch/resume state** is kept current by
|
|
`Media/play_state_sync.sh`, which reads and writes play state over the Emby API every
|
|
30 minutes (CRITICAL_MAINTENANCE_SCRIPTS) — so HOST2 never falls more than one cycle
|
|
behind on what's been watched. This replaced the old `emby-fallback` dirty-rsync profile:
|
|
copying Emby's live SQLite databases mid-write risked landing a half-written WAL file on
|
|
HOST2, and an API-level sync sidesteps the database-consistency problem entirely.
|
|
|
|
---
|
|
|
|
## ━━━ PERSONAL ENCRYPTED SHARES ━━━
|
|
|
|
Personal shares sync to the remote for offsite backup. ZFS encrypts at the dataset
|
|
level — the remote receives encrypted blocks and cannot read the content without your
|
|
passphrase or keyfile.
|
|
|
|
### Create an Encrypted ZFS Dataset
|
|
|
|
```bash
|
|
# In the unRAID UI:
|
|
# Main → click your ZFS pool name → + Dataset
|
|
# Name: Gmer4Lfe-Personal
|
|
# Encryption: Enabled
|
|
# Passphrase: [your passphrase]
|
|
# Write your passphrase down — if lost, data is completely unrecoverable
|
|
|
|
# Verify encryption is active before syncing:
|
|
zfs get encryption poolname/Gmer4Lfe-Personal
|
|
# Should show: encryption aes-256-gcm
|
|
```
|
|
|
|
### Auto-Unlock on Boot (Optional)
|
|
|
|
```bash
|
|
# Keyfile approach — more convenient, but the keyfile itself is a secret
|
|
dd if=/dev/urandom bs=32 count=1 | base64 > /root/.zfs-keys/personal.key
|
|
chmod 600 /root/.zfs-keys/personal.key
|
|
|
|
zfs change-key \
|
|
-o keylocation=file:///root/.zfs-keys/personal.key \
|
|
-o keyformat=raw \
|
|
poolname/Gmer4Lfe-Personal
|
|
|
|
# Add to array_started.sh or ramdisk_setup.sh:
|
|
zfs load-key poolname/Gmer4Lfe-Personal
|
|
zfs mount poolname/Gmer4Lfe-Personal
|
|
|
|
# Manual unlock alternative (most secure):
|
|
zfs load-key poolname/Gmer4Lfe-Personal # prompts for passphrase
|
|
zfs mount poolname/Gmer4Lfe-Personal
|
|
```
|
|
|
|
### Add to host1.conf
|
|
|
|
```bash
|
|
HOST1_PERSONAL_SHARES=(
|
|
"/mnt/user/Gmer4Lfe-Personal"
|
|
)
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━ STEP 7 — SCHEDULER SETUP ━━━
|
|
|
|
All scheduling is managed by the Varaverk plugin — no User Scripts entries needed.
|
|
|
|
### Array start (Varaverk disks_mounted event hook)
|
|
|
|
The Varaverk plugin fires `array_started.sh` automatically via its `disks_mounted` event
|
|
hook. It's registered in `schedule.json`:
|
|
|
|
```json
|
|
"Orchestrators/array_started.sh": { "cron": "array_start", "enabled": true }
|
|
```
|
|
|
|
Launches everything in ARRAY_START_SCRIPTS: api_key_renew, inotify_tuning,
|
|
docker_syslog_filter, php_fpm_max_children, ramdisk_setup, docker_network_connect, fallback.
|
|
Watchdogs run separately via watchdog_orchestrator.sh (cron — every 15 minutes).
|
|
|
|
### Cron Schedule (Varaverk Scheduler)
|
|
|
|
```bash
|
|
*/7 * * * * transcode_management.sh
|
|
*/15 * * * * watchdog_orchestrator.sh
|
|
*/30 * * * * critical_sync_maintenance.sh
|
|
0 */4 * * * intermediate_sync_maintenance.sh
|
|
0 1 * * * daily_sync_maintenance.sh
|
|
0 7 * * 0 sunday_morning_coffee_report.sh
|
|
30 2 * * 0 weekly_sync_maintenance.sh
|
|
0 0 15 * * monthly_maintenance.sh
|
|
```
|
|
|
|
All entries are configured via the Varaverk scheduler UI or `schedule.json`.
|
|
Scripts live at `/boot/config/plugins/varaverk/`.
|
|
|
|
---
|
|
|
|
## ━━━ VERIFICATION ━━━
|
|
|
|
Test with `--dry-run` first — all pre-flight checks run, no changes made.
|
|
|
|
### Test a Single Profile Sync
|
|
|
|
```bash
|
|
bash /boot/config/plugins/varaverk/Rsync/rsync.sh \
|
|
/mnt/user/appdata-Fallback/Arrs_Stack --dry-run --log
|
|
```
|
|
|
|
Expected output (healthy):
|
|
```
|
|
━━━ Setup ━━━
|
|
Host: HOST1 (unRAID-Gmer4Lfe) → HOST2 (unRAID-Jayred365)
|
|
Remote IP: 100.x.x.x
|
|
Profile: arrs_stack
|
|
|
|
━━━ Pre-flight ━━━
|
|
Remote reachable
|
|
version parity — both on unRAID X.Y.Z
|
|
Remote Docker daemon responding
|
|
Remote rootfs: 12% (threshold: 75%)
|
|
Remote share exists and not empty
|
|
All pre-flight checks passed
|
|
```
|
|
|
|
### Test the Daily Orchestrator
|
|
|
|
```bash
|
|
bash /boot/config/plugins/varaverk/Orchestrators/daily_sync_maintenance.sh --dry-run
|
|
```
|
|
|
|
### Check Configuration Resolution
|
|
|
|
```bash
|
|
bash /boot/config/plugins/varaverk/Rsync/rsync.sh \
|
|
/mnt/user/appdata-Fallback/Arrs_Stack --status
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━ INITIAL HOST2 SYNC ━━━
|
|
|
|
If HOST2 is being set up from scratch with empty shares:
|
|
|
|
```bash
|
|
# Create share directories on HOST2:
|
|
bash /boot/config/plugins/varaverk/Tools/recreate_shares.sh
|
|
|
|
# Initial push from HOST1 — first run may take several hours for large libraries:
|
|
bash /boot/config/plugins/varaverk/Orchestrators/daily_sync_maintenance.sh --log
|
|
```
|
|
|
|
The scheduled nightly sync will be incremental after the initial push.
|
|
|
|
---
|
|
|
|
## ━━━ NAMING CONSISTENCY REQUIREMENT ━━━
|
|
|
|
The ecosystem uses one codebase on both servers. This only works if containers and
|
|
shares have identical names on both servers. **This is not configurable — it is a
|
|
design requirement.**
|
|
|
|
```
|
|
Container names must match exactly on both servers:
|
|
"Emby" ← both HOST1 and HOST2
|
|
"NginxProxyManager" ← both HOST1 and HOST2
|
|
"Mariadb-Authelia" ← both HOST1 and HOST2
|
|
|
|
Share paths must match exactly on both servers:
|
|
/mnt/user/Movies ← both HOST1 and HOST2
|
|
/mnt/user/Tv_Shows ← both HOST1 and HOST2
|
|
```
|
|
|
|
If a container has a different name on one server: the script skips it silently.
|
|
You only notice when the container is not stopped during a sync that requires it.
|
|
|
|
If a share has a different path: rsync.sh aborts with "remote share missing."
|
|
Easier to catch — but still requires renaming the share to fix.
|
|
|
|
---
|
|
|
|
## ━━━ TROUBLESHOOTING ━━━
|
|
|
|
### SSH Connection Refused / Timeout
|
|
|
|
```bash
|
|
# Verify SSH is enabled on the remote:
|
|
# Settings → Management Access → Secure Shell → Enabled
|
|
|
|
# Verify Tailscale is connected:
|
|
tailscale ip -4 unRAID-Jayred365
|
|
|
|
# Test SSH manually (key path from --status output):
|
|
ssh -i /root/.ssh/gmer4lfe_rsync_automation \
|
|
root@$(tailscale ip -4 unRAID-Jayred365) "hostname"
|
|
# If password prompted: key not authorised — re-run ssh_setup.sh
|
|
|
|
# Re-run setup (idempotent, re-copies key to remote):
|
|
bash /boot/config/plugins/varaverk/Partnership/ssh_setup.sh
|
|
```
|
|
|
|
### Pre-flight Aborts on Remote Rootfs
|
|
|
|
```bash
|
|
# Check current usage on remote:
|
|
ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@[HOST2-ip] "df /"
|
|
|
|
# Common cause: array not started, drives not mounted
|
|
```
|
|
|
|
### Remote Share Missing
|
|
|
|
```bash
|
|
# Verify the share exists on HOST2:
|
|
ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@[HOST2-ip] "ls /mnt/user/"
|
|
|
|
# If missing: create the share on HOST2, then run initial sync
|
|
```
|
|
|
|
### Containers Not Stopping / Starting
|
|
|
|
```bash
|
|
# Verify container names match Docker exactly — case sensitive:
|
|
docker ps --format "{{.Names}}"
|
|
|
|
# Test remote:
|
|
ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@[HOST2-ip] "docker ps --format '{{.Names}}'"
|
|
```
|
|
|
|
### Profile Not Matching
|
|
|
|
```bash
|
|
# Profile key = directory basename lowercased
|
|
# /mnt/user/appdata-Fallback/Arrs_Stack → key: arrs_stack
|
|
|
|
# Override explicitly:
|
|
bash rsync.sh /mnt/user/appdata-Fallback/My_Stuff --profile=arrs_stack
|
|
|
|
# Verify what profile resolved:
|
|
bash rsync.sh /mnt/user/appdata-Fallback/Arrs_Stack --status
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━ FULL CONFIGURATION REFERENCE ━━━
|
|
|
|
### master.conf
|
|
|
|
```bash
|
|
# Rsync engine
|
|
RSYNC_ENABLED=true
|
|
DEFAULT_RSYNC_OPTS="-az --no-perms --no-owner --no-group --inplace"
|
|
# No --delete in DEFAULT_RSYNC_OPTS — media share rsync is additive only.
|
|
# arr_sync.sh keeps all arr databases in union — either server can download anything.
|
|
# arr cleanup handles deletions (orphans only). Profiles use --delete explicitly.
|
|
BW_LIMIT=0 # KB/s, 0 = unlimited
|
|
RETRY_COUNT=3
|
|
SLEEP=60 # seconds between retries
|
|
ROOTFS_WARN_PCT=75 # abort if remote rootfs above this %
|
|
|
|
# Shared with Monitors/
|
|
BANDWIDTH_LOG="$DATA_DIR/bandwidth_history.db"
|
|
BANDWIDTH_LOG_RETENTION=90
|
|
BANDWIDTH_WARN_GB=50
|
|
|
|
# Profile definitions (see RSYNC PROFILES section above)
|
|
declare -A PROFILES
|
|
declare -A PROFILE_BW_LIMIT
|
|
declare -A PROFILE_RETRY_COUNT
|
|
declare -A PROFILE_SLEEP
|
|
declare -A PROFILE_CONTAINER_DELAY
|
|
declare -A PROFILE_CRITICAL_CONTAINER_NAMES
|
|
declare -A PROFILE_DELAYED_CONTAINERS
|
|
declare -A PROFILE_EXCLUDE_DIRS
|
|
declare -A PROFILE_REMOTE_RESTART_CONTAINERS
|
|
```
|
|
|
|
### host*.conf
|
|
|
|
```bash
|
|
# Daily sync shares — one list per server (mutually exclusive)
|
|
HOST1_DAILY_SYNC_SHARES=(...)
|
|
HOST2_DAILY_SYNC_SHARES=(...)
|
|
|
|
# Personal encrypted shares
|
|
HOST1_PERSONAL_SHARES=(...)
|
|
|
|
# SSH key for this server to authenticate to the remote
|
|
# Set automatically by ssh_setup.sh — do not edit manually
|
|
HOST1_SSH_KEY="/root/.ssh/gmer4lfe_rsync_automation"
|
|
HOST2_SSH_KEY="/root/.ssh/jayred365_rsync_automation"
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━ OUTPUT TIERS ━━━
|
|
|
|
Without `--log`, each run shows: section headers (Pre-flight, Transfer, Stop/Start
|
|
Containers), the transfer identity block (source, remote, profile, identity), a
|
|
progress indicator while rsync is running, and a summary block with duration,
|
|
bytes transferred, and status. Warnings and errors are always visible.
|
|
|
|
With `--log`, every decision is shown — pre-flight check results, profile resolution,
|
|
per-container stop/start state, retry details, and bandwidth log confirmation.
|
|
Use for first-run validation or when investigating unexpected behavior.
|
|
|
|
---
|
|
|
|
## ━━━ FLAG REFERENCE ━━━
|
|
|
|
All flags work on `rsync.sh` and all orchestrators.
|
|
|
|
### --dry-run
|
|
|
|
Runs all pre-flight checks. Shows what rsync would transfer. No transfer, no container
|
|
stops, no bandwidth log entry. Safe to run at any time.
|
|
|
|
```bash
|
|
rsync.sh /mnt/user/Movies --dry-run
|
|
rsync.sh /mnt/user/appdata-Fallback/Arrs_Stack --dry-run --log
|
|
daily_sync_maintenance.sh --dry-run
|
|
```
|
|
|
|
### --status
|
|
|
|
Shows resolved configuration — profile, remote identity, all vars that would be
|
|
used — then exits. No pre-flight checks, no rsync. Use to verify configuration
|
|
loaded correctly.
|
|
|
|
```bash
|
|
rsync.sh /mnt/user/appdata-Fallback/Arrs_Stack --status
|
|
```
|
|
|
|
### --log
|
|
|
|
Verbose output throughout. Every decision, every container operation, every rsync
|
|
progress line. Use for first-time runs or when investigating issues.
|
|
|
|
### --profile=name
|
|
|
|
Override profile selection. Bypasses basename inference. Use when the directory
|
|
name doesn't match any profile key, or when testing a specific profile.
|
|
|
|
```bash
|
|
rsync.sh /mnt/user/appdata-Fallback/Critical-Data --profile=critical-data
|
|
rsync.sh /mnt/user/Media_Server/Emby --profile=emby
|
|
```
|