guide adjustment needs more in the futur

Rsync/README_Rsync_Setup.md

@@ -5,18 +5,14 @@ Rsync Setup Guide
 Status: Work in Progress
 For the unRAID Rsync Ecosystem — common.sh · Master.conf · rsync.sh · daily_sync.sh
 
-
-
 Overview
 
 This guide walks through setting up the rsync ecosystem on both your primary and secondary unRAID 7.x servers. By the end you will have:
 
-    A Gitea repository cloned to both servers
 SSH keys configured for server-to-server communication
 Tailscale running on both servers for secure networking
 Scripts scheduled and running via the User Scripts plugin
 Automated daily sync of media shares and appdata profiles
-
 Prerequisites
 
 Both servers need the following before starting:
@@ -24,175 +20,101 @@ Both servers need the following before starting:
 unRAID 7.x
 User Scripts plugin installed via Community Applications
 Tailscale plugin installed via Community Applications
-    Access to a Gitea instance (self-hosted or remote)
 Terminal access to both servers (via unRAID UI → Tools → Terminal, or SSH)
-
 Step 1 — Tailscale Setup
 
 Tailscale provides the secure network tunnel between your two servers. The scripts resolve the remote server's IP via Tailscale at runtime.
+
 On Both Servers
 
 Open Apps in the unRAID UI
 Search for Tailscale and install the plugin
-    Once installed go to Settings → Tailscale
+Once installed, go to Settings → Tailscale
 Click Connect and authenticate with your Tailscale account
 Verify both servers appear in your Tailscale admin console
 
 Verify Connectivity
 
 Run this on the primary to confirm it can see the secondary:
-bash
 
 tailscale ip -4 unRAID-Jayred365
 
-You should get back a 100.x.x.x IP. If not, check that both machines are authenticated and connected in the Tailscale admin console.
+You should get back a 100.x.x.x IP. If not, check both servers in the Tailscale admin console.
 
 Note: The hostnames used in Master.conf (HOST1 and HOST2) must match the Tailscale machine names exactly — these are case sensitive.
 
-Step 2 — Generate SSH Keys
+Step 2 — Generate SSH Keys (Server-to-Server)
 
-The scripts use SSH keys for two purposes:
+The scripts use SSH keys for server-to-server rsync.
-
-    Server-to-server rsync — primary authenticates to secondary (and vice versa)
-    Gitea access — both servers pull from the git repository
-
-2a — Server-to-Server Keys
-
-Run the following on each server to generate its rsync key. Replace the filename with the appropriate server name.
 
 On Primary (unRAID-Gmer4Lfe):
-bash
 
 ssh-keygen -t ed25519 -f /root/.ssh/Gmer4Lfe-rsync-key -C "gmer4lfe-rsync" -N ""
 
 On Secondary (unRAID-Jayred365):
-bash
 
 ssh-keygen -t ed25519 -f /root/.ssh/Jayred365-rsync-key -C "jayred365-rsync" -N ""
+Copy Public Keys to Each Server
 
-2b — Copy Public Keys to Each Server
+Primary → Secondary
 
-The primary's public key must be authorised on the secondary, and vice versa.
+# On primary
-
-Copy primary key → secondary:
-bash
-
-# Run on primary
 cat /root/.ssh/Gmer4Lfe-rsync-key.pub
 
-Copy the output. Then on the secondary:
+Copy the output. Then on secondary:
-bash
 
-# Run on secondary
 mkdir -p /root/.ssh
 echo "PASTE_PUBLIC_KEY_HERE" >> /root/.ssh/authorized_keys
 chmod 600 /root/.ssh/authorized_keys
 
-Copy secondary key → primary:
+Secondary → Primary
-bash
 
-# Run on secondary
+# On secondary
 cat /root/.ssh/Jayred365-rsync-key.pub
 
-Copy the output. Then on the primary:
+Copy the output. Then on primary:
-bash
 
-# Run on primary
 mkdir -p /root/.ssh
 echo "PASTE_PUBLIC_KEY_HERE" >> /root/.ssh/authorized_keys
 chmod 600 /root/.ssh/authorized_keys
+Test the Connection
 
-2c — Test the Connection
+From the primary, test SSH access to the secondary:
-
-From the primary, test that it can SSH to the secondary without a password prompt:
-bash
 
 ssh -i /root/.ssh/Gmer4Lfe-rsync-key root@$(tailscale ip -4 unRAID-Jayred365) "echo connected"
 
-You should see connected. If prompted for a password the key was not authorised correctly — recheck Step 2b.
+You should see connected. If prompted for a password, recheck the key authorization.
-2d — Gitea SSH Key
 
-Generate a separate key for Gitea access on each server:
-bash
-
-ssh-keygen -t ed25519 -f /root/.ssh/id_gitea_rsync -C "unraid-gitea" -N ""
-
-Add the public key to your Gitea account:
-bash
-
-cat /root/.ssh/id_gitea_rsync.pub
-
-Copy the output and add it in Gitea under Settings → SSH / GPG Keys → Add Key.
 Step 3 — Enable SSH on unRAID
 
-unRAID 7.x has SSH disabled by default. Enable it on both servers so the scripts can connect between them.
+unRAID 7.x has SSH disabled by default. Enable it on both servers so the scripts can connect:
 
 Go to Settings → Management Access
-    Under Secure Shell set SSH to Enabled
+Under Secure Shell, set SSH to Enabled
 Set SSH port to 22 (default)
 Click Apply
 
-    Security note: SSH is only exposed on your local network and Tailscale interface. The rsync scripts connect via the Tailscale IP so traffic is encrypted end-to-end.
+Security note: SSH is only exposed on your local network and Tailscale interface. The rsync scripts connect via Tailscale IP, so traffic is encrypted end-to-end.
 
-Step 4 — Clone the Git Repository
+Step 4 — Configure Master.conf
-
-The scripts live in a Gitea repository. Both servers clone from the same repo so updates propagate everywhere via a single git pull.
-On Both Servers
-bash
-
-# Create the target directory
-mkdir -p /mnt/user/appdata/unraid_scripts
-
-# Clone the repository
-GIT_SSH_COMMAND="ssh -i /root/.ssh/id_gitea_rsync" \
-    git clone git@YOUR_GITEA_HOST:YOUR_USER/Unraid_Scripts.git \
-    /mnt/user/appdata/unraid_scripts
-
-Replace YOUR_GITEA_HOST and YOUR_USER with your Gitea server address and username.
-Verify the Structure
-bash
-
-ls /mnt/user/appdata/unraid_scripts
-
-You should see:
-
-Master.conf
-common.sh
-Rsync/
-    rsync.sh
-Orchestrators/
-    daily_sync.sh
-Tools/
-    recreate_shares.sh
-
-Make Scripts Executable
-bash
-
-chmod +x /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh
-chmod +x /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync.sh
-chmod +x /mnt/user/appdata/unraid_scripts/Tools/recreate_shares.sh
-
-Step 5 — Configure Master.conf
 
 All user configuration lives in Master.conf. Open it and adjust the following to match your setup:
-bash
 
 nano /mnt/user/appdata/unraid_scripts/Master.conf
 
-Required Changes
+Required Changes:
+
 Variable	Description	Example
 HOST1	Hostname of your primary server	unRAID-Gmer4Lfe
 HOST2	Hostname of your secondary server	unRAID-Jayred365
 HOST1_SSH_KEY	Path to primary's rsync private key	/root/.ssh/Gmer4Lfe-rsync-key
 HOST2_SSH_KEY	Path to secondary's rsync private key	/root/.ssh/Jayred365-rsync-key
-REPO_SSH	SSH URL of your Gitea repository	git@192.168.50.2:User/Unraid_Scripts.git
-GITEA_SSH_KEY	Path to Gitea private key	/root/.ssh/id_gitea_rsync
 BW_LIMIT	Global bandwidth limit in KB/s	12500
 ROOTFS_WARN	Remote rootfs % threshold before aborting	75
+
 Daily Sync Shares
 
 Add the full paths of all media shares you want synced nightly:
-bash
 
 DAILY_SYNC_SHARES=(
     /mnt/user/Movies
@@ -203,8 +125,7 @@ DAILY_SYNC_SHARES=(
 
 Profiles
 
-Profiles control per-share rsync behaviour for your frequently synced appdata shares. Each profile is matched by the directory basename (lowercased). Add a key to each profile array for any share that needs custom settings:
+Profiles control per-share rsync behaviour for your frequently synced appdata shares.
-bash
 
 declare -A PROFILE_CRITICAL_CONTAINER_NAMES=(
     [arrs_stack]="Sonarr Radarr Lidarr Prowlarr"
@@ -212,137 +133,83 @@ declare -A PROFILE_CRITICAL_CONTAINER_NAMES=(
 )
 
 Any share with no matching profile falls through to the global DEFAULT_RSYNC_OPTS.
-Step 6 — Set Up User Scripts
 
-The User Scripts plugin is how unRAID schedules and runs the scripts. Each sync job is its own script entry in the plugin.
+Step 5 — Set Up User Scripts
-Frequent Sync Jobs (Scheduled Individually)
 
-Create one script entry per appdata profile. Go to Plugins → User Scripts → Add New Script.
+The User Scripts plugin schedules and runs the scripts. Each sync job is its own script entry.
 
-Name it descriptively — e.g. rsync appdata arrs_stack.
+Frequent Sync Jobs
-
-In the script body paste:
-bash
 
+Create one script entry per appdata profile (Plugins → User Scripts → Add New Script)
+Example for ARR stack:
 #!/bin/bash
 bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/appdata-Failover/Arrs_Stack
 
-Set the schedule to match your desired frequency:
-Profile	Recommended Schedule
-Emby	Every 1 hour
-Critical-Data	Every 2 hours
-Important-Data	Every 6 hours
-Arrs_Stack	Every 12 hours
-
-    Important: Set each script to run as a Background Task — this ensures output streams correctly to the log rather than buffering in the browser.
-
 Daily Sync Orchestrator
 
-Create one more script entry for the daily media sync:
-
-Name it daily media sync.
-
-In the script body paste:
-bash
-
 #!/bin/bash
 bash /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync.sh
+Schedule daily at 01:00
+Set scripts to run as Background Tasks
+Step 6 — Verify the Setup
 
-Set the schedule to Daily at 01:00.
+Manually test from the primary:
-Step 7 — Verify the Setup
-
-Before letting the scheduled jobs run, do a manual test from the terminal on the primary:
-bash
 
 bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --log
 
-A healthy run will show:
+You should see a healthy run with:
 
 ━━━ ⚙️ Setup ━━━
 ℹ️  [INFO]  🖥️ Host: unRAID-Gmer4Lfe → unRAID-Jayred365
 ℹ️  [INFO]  🌐 Remote IP: 100.x.x.x
-
+...
-━━━ 🛡️ Pre-flight Checks ━━━
-ℹ️  [INFO]  📡 unRAID-Jayred365 is reachable
-ℹ️  [INFO]  🩺 Remote rootfs: 12% used (threshold: 75%)
-ℹ️  [INFO]  🩺 Remote share verified: ...
-ℹ️  [INFO]  💾 disk1 🟢 — share present
 ✅ [OK]  All disks backing share are online
+Step 7 — Secondary Server Initial Setup
 
-If any pre-flight check fails the script will abort with a clear error and hint before touching anything.
+If setting up secondary from scratch:
-Step 8 — Secondary Server Initial Setup
-
-If setting up the secondary from scratch (no existing data):
 
 Complete Steps 1–5 on the secondary
 Start the array and create your shares in the unRAID UI
-    Run the share recreation tool to create disk directories from your cfg files:
+Run the share recreation tool:
-
-bash
-
 bash /mnt/user/appdata/unraid_scripts/Tools/recreate_shares.sh
-
+Temporarily remove --delete from DEFAULT_RSYNC_OPTS for initial push
-    Temporarily remove --delete from DEFAULT_RSYNC_OPTS in Master.conf
+Restore --delete after the first run; next nightly run will clean .recovery files automatically
-    Run the initial push from the primary — the .recovery marker files allow rsync to populate empty shares without aborting
-    Once complete, restore --delete to Master.conf
-    The next nightly run will clean up the .recovery marker files automatically
-
 Troubleshooting
+
 SSH connection refused
 
-    Verify SSH is enabled on the target server (Step 3)
+Verify SSH is enabled on target server (Step 3)
-    Check the correct key is referenced in Master.conf
+Check correct key referenced in Master.conf
-    Confirm the Tailscale IP resolves: tailscale ip -4 HOSTNAME
+Confirm Tailscale IP resolves
 
-Pre-flight aborts on rootfs
+Pre-flight aborts
 
-    Remote rootfs is above ROOTFS_WARN threshold
+Rootfs above ROOTFS_WARN
-    Check if the remote array is started and drives are mounted
+Share exists but empty
-    df / on the remote to see current usage
+Disk offline
-
-Pre-flight aborts on empty share
-
-    Share exists but has no content — drives may not be mounted
-    Run recreate_shares.sh if setting up fresh
-    Check array status on the remote server
-
-Pre-flight aborts on disk check
-
-    One or more disks backing the share are not mounted
-    Check Main → Array Devices on the remote for offline disks
-    Verify disk assignments are correct after any hardware changes
 
 Script not found
 
-    Verify the repo was cloned to /mnt/user/appdata/unraid_scripts/
+Verify scripts exist in /mnt/user/appdata/unraid_scripts/
-    Check scripts are executable: chmod +x /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh
+Make scripts executable:
-
+chmod +x /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh
-Containers not stopping/starting
+chmod +x /mnt/user/appdata/unraid_scripts/Orchestrators/daily_sync.sh
-
+chmod +x /mnt/user/appdata/unraid_scripts/Tools/recreate_shares.sh
-    Verify container names in Master.conf match exactly what Docker shows
-    Check SSH key has access to run docker commands on the remote
-    Test manually: ssh -i /root/.ssh/KEY root@REMOTE_IP "docker ps"
-
 Available Flags
-
-All scripts support the following flags:
 Flag	Description
---dry-run or -n	Run without making any changes
+--dry-run or -n	Run without making changes
---log	Enable verbose logging output
+--log	Enable verbose logging
---no-log	Disable logging (overrides Master.conf)
+--no-log	Disable logging
 --status	Print resolved configuration and exit
 
-Example:
+Examples
-bash
 
-# Preview what would be synced without transferring anything
+# Preview what would be synced without transferring
 bash rsync.sh /mnt/user/Movies --dry-run --log
 
-# Check what profile and settings resolved for a share
+# Check profile and settings resolved for a share
 bash rsync.sh /mnt/user/appdata-Failover/Arrs_Stack --status
-
 Repository Structure
-
 Unraid_Scripts/
 ├── Master.conf               # All user configuration — edit this file only
 ├── common.sh                 # Shared library — functions used by all scripts
@@ -351,4 +218,4 @@ Unraid_Scripts/
 ├── Orchestrators/
 │   └── daily_sync.sh         # Daily media sync orchestrator
 └── Tools/
-    └── recreate_shares.sh    # Share directory recreation from cfg files
+    └── recreate_shares.sh    # Share directory recreation from cfg fil