updated user scripts

Failover/README-Failover.md

@@ -12,16 +12,30 @@ Two fully independent unRAID servers connected via Tailscale:
 
 ```
 HOST1 — unRAID-Gmer4Lfe (Primary)
-  Location:   Local
+  Hardware:   Threadripper 1950X, 128GB RAM
   Owns:       Gmer4Lfe.com DDNS
   Runs:       Full service stack
-  
+
 HOST2 — unRAID-Jayred365 (Secondary / Buddy server)
+  Hardware:   Intel i5 10th gen — completely different hardware
   Location:   Remote — 50 miles away
   Owns:       Gmer4Lfe.us DDNS
-  Runs:       Its own service stack + mirrors HOST1 data
+  Runs:       Its own service stack + mirrors HOST1 critical data
 ```
 
+**Hardware doesn't need to match.** Everything is accessed through `/mnt/user/` — unRAID's fused share layer. HOST1 has a Threadripper with multiple ZFS pools. HOST2 has a completely different CPU, fewer drives, different layout. The failover containers on HOST2 mount `/mnt/user/Movies` and see the same data because the share names match and the data is mirrored. The hardware underneath is irrelevant.
+
+**What needs to match between servers:**
+- Share names (`/mnt/user/Movies` = `/mnt/user/Movies`)
+- Container names for shared failover services
+- Docker custom network names (so NPM can reach containers by name, not IP)
+
+**Normal operation — HOST2 is essentially passive:**
+
+HOST1 is the source of truth. It runs everything — all arrs, all downloads, all active services. HOST2 keeps its media mirror current via nightly rsync and waits. All media is mirrored so when Emby starts on HOST2 during failover it has the exact same library, same metadata, same watch history. No separate library, no separate database, no user-visible difference.
+
+Don't run arrs on both servers simultaneously — two instances writing to the same share causes conflicts and duplicate downloads. HOST2's arrs only start at Tier 4 (18hr+ outage) when genuine workflow continuity is needed.
+
 Both servers run `failover.sh` as a background task continuously. Neither server knows what the other is doing — they only know what they can ping from their own network perspective.
 
 ---

README.md

@@ -27,15 +27,80 @@ HOST1 — unRAID-Gmer4Lfe (Primary)
   Storage:    Multiple ZFS pools + cache
   Role:       Primary services, full media stack, all Docker containers
 
-HOST2 — unRAID-Jayred365 (Secondary)
+HOST2 — unRAID-Jayred365 (Secondary / Buddy server)
+  Hardware:   Intel i5 10th gen — completely different hardware
+  Storage:    Different disk count, different pool layout
   Location:   Remote — 50 miles from HOST1
   Role:       Mirror, failover coverage, independent services
-  
+
 Network:    Tailscale — encrypted tunnel between both servers
 Repo:       Self-hosted Gitea on HOST1
 Deployment: git pull on either server → both stay current
 ```
 
+**The servers do not need to be identical hardware.** This is one of the most important design decisions in the ecosystem. Because everything is accessed through `/mnt/user/` — unRAID's fused share layer — the underlying disk count, pool layout, and hardware generation don't matter. A share called `Movies` on HOST1 is `/mnt/user/Movies`. The same share name on HOST2 is also `/mnt/user/Movies`. rsync syncs between them. Failover containers on HOST2 mount `/mnt/user/Movies` and see the same data. The hardware underneath is completely irrelevant.
+
+```
+HOST1: Threadripper, 10 drives across 5 ZFS pools
+HOST2: i5, 4 drives, completely different layout
+Both:  /mnt/user/Movies — identical path, same data, different everything else
+```
+
+**What is actually mirrored:**
+
+**Share names** — `/mnt/user/Movies` is `/mnt/user/Movies` on both servers. rsync syncs the content nightly. Container volume mounts use the same path. No path translation anywhere.
+
+**Media shares** — all media is mirrored to HOST2. Since both servers mount the same share names, every container on HOST2 — Emby, the arr stack, everything — has access to the same library HOST1 serves. HOST1 is the source of truth. HOST2 is the mirror. They share the same data pool when failover kicks in — no separate libraries, no separate metadata, no separate databases for media.
+
+```
+HOST1 /mnt/user/Movies  →  rsync nightly  →  HOST2 /mnt/user/Movies
+HOST1 /mnt/user/Tv_Shows →  rsync nightly  →  HOST2 /mnt/user/Tv_Shows
+HOST1 /mnt/user/Music    →  rsync nightly  →  HOST2 /mnt/user/Music
+...all media shares mirrored...
+```
+
+**Normal operation — HOST2 piggybacks on HOST1:**
+
+In day-to-day operation HOST2 is essentially passive from HOST1's perspective. HOST1 handles everything — all arrs, all downloads, all transcoding, all active services. HOST2 keeps its mirror current and waits.
+
+```
+Normal operation:
+  HOST1 — source of truth, all active services running
+  HOST2 — mirror current, containers ready but stopped
+          no arrs running (would create conflicts with HOST1)
+          no downloaders running
+          just keeping data fresh and ready to take over
+```
+
+**Don't run arrs on both servers simultaneously.** The arr stack manages your library — two instances writing to the same share causes conflicts, duplicate downloads, and database corruption. Either designate one server as the arr master (HOST1 in this setup) or split arrs between servers by type. HOST2's arrs only start during Tier 4 failover — when HOST1 has been down 18+ hours and workflow continuity is genuinely needed.
+
+**Container names** — failover containers on HOST2 must use the same Docker container name as on HOST1. The failover script starts them by name via SSH. Server-specific containers get server-specific names:
+
+```
+Shared containers — same name on both servers:
+  Emby                 ← starts on HOST2 during HOST1 outage
+                         uses HOST2's mirrored media, same library
+  NginxProxyManager    ← same name, same config, same network
+  Authelia             ← same name
+
+Server-specific containers — unique names:
+  Emby-Gmer4Lfe        ← HOST1's own Emby instance, always on HOST1
+  Emby-Jayred365       ← HOST2's own Emby instance, always on HOST2
+  VaultWarden-Gmer4Lfe ← HOST1's password manager
+  VaultWarden-Jayred365← HOST2's password manager
+```
+
+**Docker custom networks** — containers communicate by container name within a custom network, not by IP address. When a container restarts and gets a new IP, NginxProxyManager still reaches it by name. Failover containers on HOST2 join the same named custom networks and are immediately reachable by NPM without any IP reconfiguration required.
+
+```
+Custom network: high-availability
+  NginxProxyManager → Authelia   (by name — IP changes are invisible)
+  NginxProxyManager → Emby       (by name)
+  NginxProxyManager → NextCloud  (by name)
+```
+
+This flexibility means any two unRAID servers can participate — regardless of hardware generation, CPU, drive count, or storage layout. The only requirements are matching share names, matching container names for shared services, and the same custom Docker network names.
+
 ---
 
 ## How It Works