{"parentUuid":null,"isSidechain":true,"promptId":"14997b41-0464-47e1-9df2-adbecd208513","agentId":"ad9a317b416360985","type":"user","message":{"role":"user","content":"Read the following files and report any discrepancies between the documentation and the actual project structure/script headers. The goal is to find places where the root README or Manual says something that no longer matches what the scripts actually do (wrong schedules, wrong config vars, missing/extra scripts referenced, wrong directory names, etc.).\n\nFiles to read:\n1. /boot/config/plugins/varaverk/README.md\n2. /boot/config/plugins/varaverk/Manual.md\n3. /boot/config/plugins/varaverk/README-User_Script_Plug-in.md\n4. For spot-checking claims, read headers of:\n - /boot/config/plugins/varaverk/load_config.sh (first 50 lines)\n - /boot/config/plugins/varaverk/common.sh (first 60 lines)\n - /boot/config/plugins/varaverk/git_pull_execute.sh (first 60 lines)\n\nReport: list each discrepancy as \"FILE: what the doc says vs what is actually true\". If everything matches, say so. Be specific — quote the conflicting text. Keep the report under 500 words."},"uuid":"2e6c0867-d816-4216-b23e-12e932937dae","timestamp":"2026-06-14T05:52:15.331Z","userType":"external","entrypoint":"cli","cwd":"/boot/config/plugins/varaverk","sessionId":"685e6c5b-62c1-40bd-9cfd-2c9f7e15c50f","version":"2.1.177","gitBranch":"HEAD","slug":"mutable-sniffing-naur"} {"parentUuid":"2e6c0867-d816-4216-b23e-12e932937dae","isSidechain":true,"agentId":"ad9a317b416360985","attachment":{"type":"deferred_tools_delta","addedNames":["EnterWorktree","ExitWorktree","Monitor","TaskStop","WebFetch","WebSearch","mcp__claude_ai_Google_Drive__authenticate","mcp__claude_ai_Google_Drive__complete_authentication"],"addedLines":["EnterWorktree","ExitWorktree","Monitor","TaskStop","WebFetch","WebSearch","mcp__claude_ai_Google_Drive__authenticate","mcp__claude_ai_Google_Drive__complete_authentication"],"removedNames":[],"readdedNames":[]},"type":"attachment","uuid":"e3f4a7c5-eccc-44e3-9903-87dfbfed1409","timestamp":"2026-06-14T05:52:15.334Z","userType":"external","entrypoint":"cli","cwd":"/boot/config/plugins/varaverk","sessionId":"685e6c5b-62c1-40bd-9cfd-2c9f7e15c50f","version":"2.1.177","gitBranch":"HEAD","slug":"mutable-sniffing-naur"} {"parentUuid":"e3f4a7c5-eccc-44e3-9903-87dfbfed1409","isSidechain":true,"agentId":"ad9a317b416360985","attachment":{"type":"skill_listing","content":"- update-config: Use this skill to configure the Claude Code harness via settings.json. Automated behaviors (\"from now on when X\", \"each time X\", \"whenever X\", \"before/after X\") require hooks configured in settings.json - the harness executes these, not Claude, so memory/preferences cannot fulfill them. Also use for: permissions (\"allow X\", \"add permission\", \"move permission to\"), env vars (\"set X=Y\"), hook troubleshooting, or any changes to settings.json/settings.local.json files. Examples: \"allow npm commands\", \"add bq permission to global settings\", \"move permission to user settings\", \"set DEBUG=true\", \"when claude stops show X\". For simple settings like theme/model, suggest the /config command.\n- keybindings-help: Use when the user wants to customize keyboard shortcuts, rebind keys, add chord bindings, or modify ~/.claude/keybindings.json. Examples: \"rebind ctrl+s\", \"add a chord shortcut\", \"change the submit key\", \"customize keybindings\".\n- verify: Verify that a code change actually does what it's supposed to by running the app and observing behavior. Use when asked to verify a PR, confirm a fix works, test a change manually, check that a feature works, or validate local changes before pushing.\n- code-review: Review the current diff for correctness bugs and reuse/simplification/efficiency cleanups at the given effort level (low/medium: fewer, high-confidence findings; high→max: broader coverage, may include uncertain findings; ultra: deep multi-agent review in the cloud). Pass --comment to post findings as inline PR comments, or --fix to apply the findings to the working tree after the review.\n- simplify: Review the changed code for reuse, simplification, efficiency, and altitude cleanups, then apply the fixes. Quality only — it does not hunt for bugs; use /code-review for that.\n- fewer-permission-prompts: Scan your transcripts for common read-only Bash and MCP tool calls, then add a prioritized allowlist to project .claude/settings.json to reduce permission prompts.\n- loop: Run a prompt or slash command on a recurring interval (e.g. /loop 5m /foo). Omit the interval to let the model self-pace. - When the user wants to set up a recurring task, poll for status, or run something repeatedly on an interval (e.g. \"check the deploy every 5 minutes\", \"keep running /babysit-prs\"). Do NOT invoke for one-off tasks.\n- schedule: Create, update, list, or run scheduled cloud agents (routines) that execute on a cron schedule. - When the user wants to schedule a recurring cloud agent, set up automated tasks, create a cron job for Claude Code, or manage their scheduled agents/routines. Also use when the user wants a one-time scheduled run (\"run this once at 3pm\", \"remind me to check X tomorrow\").\n- claude-api: Reference for the Claude API / Anthropic SDK — model ids, pricing, params, streaming, tool use, MCP, agents, caching, token counting, model migration.\nTRIGGER — read BEFORE opening the target file; don't skip because it \"looks like a one-liner\" — whenever: the prompt names Claude/Anthropic in any form (Claude, Anthropic, Fable, Opus, Sonnet, Haiku, `anthropic`, `@anthropic-ai`, `claude-*`, `us.anthropic.*`, `[1m]`); the user asks about an LLM (pricing/model choice/limits/caching) — never answer from memory; OR the task is LLM-shaped with provider unstated (agent/MCP/tool-definition/multi-agent/RAG/LLM-judge/computer-use; generate/summarize/extract/classify/rewrite/converse over NL; debugging refusals/cutoffs/streaming/tool-calls/tokens).\nSKIP only when another provider is being worked on (overrides all triggers): OpenAI/GPT/Gemini/Llama/Mistral/Cohere/Ollama named in the query; OR `grep -rE 'openai|langchain_openai|google.generativeai|genai|mistralai|cohere|ollama'` over the project hits (run this grep FIRST if no provider named — don't Read the file).\n- run: Launch and drive this project's app to see a change working. Use when asked to run, start, or screenshot the app, or to confirm a change works in the real app (not just tests). First looks for a project skill that already covers launching the app; otherwise falls back to built-in patterns per project type (CLI, server, TUI, Electron, browser-driven, library).\n- init: Initialize a new CLAUDE.md file with codebase documentation\n- review: Review a pull request\n- security-review: Complete a security review of the pending changes on the current branch","skillCount":13,"isInitial":true,"names":["update-config","keybindings-help","verify","code-review","simplify","fewer-permission-prompts","loop","schedule","claude-api","run","init","review","security-review"]},"type":"attachment","uuid":"79f4fa1c-35cf-48d7-a461-768d6920f3da","timestamp":"2026-06-14T05:52:15.335Z","userType":"external","entrypoint":"cli","cwd":"/boot/config/plugins/varaverk","sessionId":"685e6c5b-62c1-40bd-9cfd-2c9f7e15c50f","version":"2.1.177","gitBranch":"HEAD","slug":"mutable-sniffing-naur"} {"parentUuid":"79f4fa1c-35cf-48d7-a461-768d6920f3da","isSidechain":true,"agentId":"ad9a317b416360985","message":{"model":"claude-haiku-4-5-20251001","id":"msg_013XUUgMCTzC3LNxeckM5BUw","type":"message","role":"assistant","content":[{"type":"text","text":"I'll read these files to identify any discrepancies between documentation and actual implementation."}],"stop_reason":null,"stop_sequence":null,"stop_details":null,"usage":{"input_tokens":3,"cache_creation_input_tokens":1968,"cache_read_input_tokens":5697,"cache_creation":{"ephemeral_5m_input_tokens":1968,"ephemeral_1h_input_tokens":0},"output_tokens":1,"service_tier":"standard","inference_geo":"not_available"},"diagnostics":null},"requestId":"req_011Cc2daJFeiE5pgF2fQ7t3p","attributionAgent":"Explore","type":"assistant","uuid":"2cb81493-3d33-48a6-a332-e49f04c96cd8","timestamp":"2026-06-14T05:52:17.571Z","userType":"external","entrypoint":"cli","cwd":"/boot/config/plugins/varaverk","sessionId":"685e6c5b-62c1-40bd-9cfd-2c9f7e15c50f","version":"2.1.177","gitBranch":"HEAD","slug":"mutable-sniffing-naur"} {"parentUuid":"2cb81493-3d33-48a6-a332-e49f04c96cd8","isSidechain":true,"agentId":"ad9a317b416360985","message":{"model":"claude-haiku-4-5-20251001","id":"msg_013XUUgMCTzC3LNxeckM5BUw","type":"message","role":"assistant","content":[{"type":"tool_use","id":"toolu_011mjB8pAmxVQtXbDLcaP1Kb","name":"Read","input":{"file_path":"/boot/config/plugins/varaverk/README.md"},"caller":{"type":"direct"}}],"stop_reason":null,"stop_sequence":null,"stop_details":null,"usage":{"input_tokens":3,"cache_creation_input_tokens":1968,"cache_read_input_tokens":5697,"cache_creation":{"ephemeral_5m_input_tokens":1968,"ephemeral_1h_input_tokens":0},"output_tokens":1,"service_tier":"standard","inference_geo":"not_available"},"diagnostics":null},"requestId":"req_011Cc2daJFeiE5pgF2fQ7t3p","attributionAgent":"Explore","type":"assistant","uuid":"c35c17bd-e27f-4285-8469-975e994513c5","timestamp":"2026-06-14T05:52:17.573Z","userType":"external","entrypoint":"cli","cwd":"/boot/config/plugins/varaverk","sessionId":"685e6c5b-62c1-40bd-9cfd-2c9f7e15c50f","version":"2.1.177","gitBranch":"HEAD","slug":"mutable-sniffing-naur"} {"parentUuid":"c35c17bd-e27f-4285-8469-975e994513c5","isSidechain":true,"promptId":"14997b41-0464-47e1-9df2-adbecd208513","agentId":"ad9a317b416360985","type":"user","message":{"role":"user","content":[{"tool_use_id":"toolu_011mjB8pAmxVQtXbDLcaP1Kb","type":"tool_result","content":"1\t# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n2\t# 🏠 VARAVERK\n3\t# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n4\t\n5\t**A complete self-healing, self-maintaining, mutually-redundant two-server home server\n6\tecosystem.** One codebase runs on both servers. Both servers run their own lives\n7\tindependently. When one goes down the other covers it — not because of conventional\n8\tfallback infrastructure, but because both servers already have copies of everything\n9\tthat matters and both know how to run it.\n10\t\n11\t> **This is not conventional failover.** There is no primary and standby. There is no\n12\t> shared storage. There is no cluster manager. There are two fully independent servers,\n13\t> living in different locations on different power utilities, each running their own\n14\t> household's media stack — and a layer of scripts that keeps them synchronised,\n15\t> coordinated, and mutually covering. The goal is minimal disruption to a media stack,\n16\t> not enterprise high availability.\n17\t\n18\t**Started on Unraid, built for portability.** The ecosystem runs on Unraid today. The\n19\tplatform adapter layer (`Plugin/unraid/adapter.sh`) isolates every OS-specific call so\n20\tbash scripts stay clean. A `Plugin/truenas/adapter.sh` or `Plugin/ubuntu/adapter.sh`\n21\tprovides the same function names — `load_config.sh` detects the OS and sources the\n22\tcorrect adapter. The scripts themselves never branch on the OS.\n23\t\n24\t---\n25\t\n26\t## ━━━ THE TWO SERVERS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n27\t\n28\t```\n29\tHOST1 — unRAID-Gmer4Lfe\n30\t Hardware: Threadripper 1950X, 128GB RAM, ZFS cache pools\n31\t Location: Primary site\n32\t Domain: Gmer4Lfe.com\n33\t Arrs: Movies, Tv_Shows, Music — union with HOST2 via arr_sync\n34\t Auth stack — source of truth (HOST1 owned, syncs to HOST2)\n35\t Emby — primary media server\n36\t\n37\tHOST2 — unRAID-Jayred365\n38\t Hardware: Intel i5 10th gen, 64GB RAM\n39\t Location: Remote — different building, different power utility\n40\t Domain: Gmer4Lfe.us\n41\t Arrs: Anime_Shows, Anime_Movies — union with HOST1 via arr_sync\n42\t His own Emby — for his household\n43\t```\n44\t\n45\t**Hardware does not need to match.** Everything is accessed through `/mnt/user/` —\n46\tunRAID's unified share layer. HOST1 has a Threadripper with ZFS. HOST2 has a completely\n47\tdifferent CPU and fewer drives. A fallback container on HOST2 mounts `/mnt/user/Movies`\n48\tand sees mirrored data because the share names match. The hardware underneath is\n49\tirrelevant.\n50\t\n51\t---\n52\t\n53\t## ━━━ CONTAINER NAMING — YOU NEED TO KNOW THIS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n54\t\n55\tThis is the most important convention in the ecosystem. Get it wrong and the scripts\n56\tbreak silently.\n57\t\n58\t---\n59\t\n60\t### ── Shared Containers — No Suffix ───────────────────────────────────────────\n61\t\n62\t```bash\n63\t# ─────────────────────────────────────────────────────────────────────────────\n64\t# Shared containers — meaning both servers run this container and scripts treat\n65\t# them as the same logical service — use the plain name with no suffix.\n66\t#\n67\t# The convention: \"no suffix = this container participates in the shared stack\"\n68\t#\n69\t Emby # the shared Emby — runs on HOST1, picked up by HOST2 on fallback\n70\t NginxProxyManager # shared NPM — runs on both, same config via sync\n71\t Lldap-Gmer4Lfe # shared LLDAP — same user directory, both servers\n72\t Authelia # shared auth — both servers, same config\n73\t Mariadb-Authelia # Authelia's database — same on both\n74\t Redis-Authelia # Authelia's cache — same on both\n75\t VaultWarden # shared password manager\n76\t Dispatcharr # shared Live TV scheduler\n77\t```\n78\t\n79\tWhen `fallback.sh` on HOST2 reads `FALLBACK_HOST2_COVERS_HOST1_TIER1=(...\"Emby\"...)`,\n80\tit looks for a container called exactly `Emby` and starts it. HOST2 must have a\n81\tcontainer called `Emby` — not `Emby-Gmer4Lfe`, not `Emby-HOST1`, just `Emby` — with\n82\tits volume mounts pointing at the mirrored share paths. This is what makes one\n83\tcodebase work on both servers without modification.\n84\t\n85\t---\n86\t\n87\t### ── User-Specific Containers — Suffixed ────────────────────────────────────\n88\t\n89\t```bash\n90\t# ─────────────────────────────────────────────────────────────────────────────\n91\t# User-specific containers — things that belong to one person and only ever\n92\t# run on their server — use a suffix identifying the user/owner.\n93\t#\n94\t Emby-Gmer4Lfe # Gmer4Lfe's personal Emby instance (HOST1, never covered)\n95\t Emby-Jayred365 # Jayred365's personal Emby instance (HOST2, never covered)\n96\t Organizer-Gmer4Lfe # Gmer4Lfe's dashboard — personal, stays on HOST1\n97\t Sonarr-Gmer4Lfe # Gmer4Lfe's Sonarr — managing his Tv_Shows share\n98\t Radarr-Gmer4Lfe # Gmer4Lfe's Radarr — managing his Movies share\n99\t Immich-Gmer4Lfe # Gmer4Lfe's photo library — personal\n100\t```\n101\t\n102\tUser-specific containers are covered during fallback (see Tiered Fallback below) but\n103\tare distinct from the shared stack. They are never confused with each other because\n104\tthe suffix makes the owner unambiguous. Scripts targeting `Emby` and scripts targeting\n105\t`Emby-Gmer4Lfe` are targeting different containers for different purposes.\n106\t\n107\t---\n108\t\n109\t### ── Why This Matters ─────────────────────────────────────────────────────────\n110\t\n111\t```bash\n112\t# ─────────────────────────────────────────────────────────────────────────────\n113\t# The entire ecosystem uses container names as identifiers.\n114\t# docker_watchdog.sh: WATCHDOG_REQUIRED_CONTAINERS=(\"NginxProxyManager\" \"Authelia\")\n115\t# → looks for exactly \"NginxProxyManager\" on the running server\n116\t# fallback.sh: FALLBACK_HOST2_COVERS_HOST1_TIER1=(\"Emby\" \"VaultWarden\")\n117\t# → looks for exactly \"Emby\" and \"VaultWarden\" on HOST2\n118\t# docker_daily_restart.sh: DAILY_RESTART_CONTAINERS=(\"Dispatcharr\" \"Authelia\")\n119\t# → looks for exactly those names in docker ps\n120\t#\n121\t# A container named differently than expected is not found — no error, no restart,\n122\t# no fallback coverage. It silently does not happen.\n123\t# Keep names consistent on both servers for shared containers.\n124\t# ─────────────────────────────────────────────────────────────────────────────\n125\t```\n126\t\n127\t---\n128\t\n129\t## ━━━ SHARE NAMING — ALSO YOU NEED TO KNOW THIS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n130\t\n131\tShares must have **identical names on both servers**. The ecosystem uses share paths\n132\tas sync targets and mount points — a mismatch is a broken path.\n133\t\n134\t```bash\n135\t# ─────────────────────────────────────────────────────────────────────────────\n136\t# The rule: if a share exists on HOST1 and HOST2 needs a mirror of it,\n137\t# the share on HOST2 must have the exact same name.\n138\t#\n139\t# Same name on both servers:\n140\t /mnt/user/Movies # both servers — arr_sync.sh keeps Radarr in union\n141\t /mnt/user/Tv_Shows # both servers — arr_sync.sh keeps Sonarr in union\n142\t /mnt/user/Music # both servers — arr_sync.sh keeps Lidarr in union\n143\t /mnt/user/Anime_Shows # both servers — arr_sync.sh keeps Sonarr in union\n144\t /mnt/user/Anime_Movies # both servers — arr_sync.sh keeps Radarr in union\n145\t#\n146\t# rsync.sh syncs /mnt/user/Movies on HOST1 → /mnt/user/Movies on HOST2.\n147\t# If HOST2 has it at /mnt/user/Movies-Mirror → rsync aborts: path not found.\n148\t# ─────────────────────────────────────────────────────────────────────────────\n149\t```\n150\t\n151\t---\n152\t\n153\t## ━━━ THE MEDIA LIBRARY MODEL ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n154\t\n155\t### ── Common Libraries — Shared Pool ─────────────────────────────────────────\n156\t\n157\tCommon media like Movies, Music, and TV Shows are a **shared pool**. Every container\n158\tthat needs music — Emby, LidaTube, Navidrome, whatever — mounts the same `/mnt/user/Music`\n159\tshare. There is one share, one location, one source of truth. No duplication of content\n160\tacross separate shares per-user or per-application.\n161\t\n162\t```bash\n163\t# ─────────────────────────────────────────────────────────────────────────────\n164\t# One share for the library — all containers that need it mount the same path.\n165\t# Emby container: /mnt/user/Music → /media/Music (read)\n166\t# Lidarr: /mnt/user/Music → /music (read+write)\n167\t# LidaTube: /mnt/user/Music → /downloads (write)\n168\t# Navidrome: /mnt/user/Music → /music (read)\n169\t#\n170\t# All pointing at the same /mnt/user/Music. No per-user music copies.\n171\t# When Lidarr downloads and Emby scans — they're looking at the same files.\n172\t# ─────────────────────────────────────────────────────────────────────────────\n173\t```\n174\t\n175\t---\n176\t\n177\t### ── Both Arrs Manage All Shares — arr_sync.sh Is the Source of Truth ───────\n178\t\n179\t```bash\n180\t# ─────────────────────────────────────────────────────────────────────────────\n181\t# Both servers run Sonarr, Radarr, and Lidarr — all pointing at the same\n182\t# share names. Either server can download anything to any share at any time.\n183\t#\n184\t# How it stays coherent:\n185\t# arr_sync.sh runs first (before rsync) — bidirectional library union.\n186\t# Both Sonarr instances know about every TV episode on either server.\n187\t# Both Radarr instances know about every movie on either server.\n188\t# Both Lidarr instances know about every album on either server.\n189\t#\n190\t# Then rsync spreads the files additively (no --delete).\n191\t# Any file downloaded on HOST1 propagates to HOST2 on the next cycle.\n192\t# Any file downloaded on HOST2 propagates to HOST1 on the next cycle.\n193\t#\n194\t# HOST1 downloads Tv_Shows episode:\n195\t# → HOST1 Sonarr tracks it immediately\n196\t# → arr_sync.sh runs → HOST2 Sonarr now tracks it\n197\t# → daily rsync → file lands on HOST2\n198\t#\n199\t# HOST2 downloads the same show's next episode that same night:\n200\t# → HOST2 Sonarr tracks it immediately\n201\t# → arr_sync.sh runs → HOST1 Sonarr now tracks it\n202\t# → daily rsync → file lands on HOST1\n203\t#\n204\t# Both servers converge. Neither owns the share. The union is the truth.\n205\t# ─────────────────────────────────────────────────────────────────────────────\n206\t```\n207\t\n208\t---\n209\t\n210\t### ── How arr_sync.sh Keeps the Union Stable ────────────────────────────────\n211\t\n212\t```bash\n213\t# ─────────────────────────────────────────────────────────────────────────────\n214\t# arr_sync.sh runs as the fixed first step of intermediate_sync_maintenance.sh\n215\t# (every 4 hours). It syncs all arr libraries bidirectionally before any rsync.\n216\t#\n217\t# What it does:\n218\t# Reads each arr's full library via the local API\n219\t# SSHes to the remote, reads the remote arr library via its API\n220\t# Adds anything missing on either side — union, not overwrite\n221\t# Blocked items (blocklist) are excluded from sync\n222\t#\n223\t# What this means in practice:\n224\t# No share has an owner. No server is read-only.\n225\t# Either server can download content to any share at any time.\n226\t# arr_sync.sh guarantees both arrs agree on what exists before files move.\n227\t# rsync then spreads the files to make storage match what arrs already know.\n228\t#\n229\t# arr_cleanup.sh (daily) removes true orphans — files no arr tracks.\n230\t# arr_cleanup uses the union model too: a file is only an orphan if\n231\t# neither arr on either server has it indexed.\n232\t# ─────────────────────────────────────────────────────────────────────────────\n233\t```\n234\t\n235\t---\n236\t\n237\t## ━━━ WHAT SYNCS AND WHAT DOESN'T ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n238\t\n239\tNot everything needs to sync. Not everything should sync. Understanding what goes\n240\twhere determines whether you need encryption and whether sync failure matters.\n241\t\n242\t---\n243\t\n244\t### ── Shared Fallback Shares — Sync Required ─────────────────────────────────\n245\t\n246\t```bash\n247\t# ─────────────────────────────────────────────────────────────────────────────\n248\t# These shares enable fallback — HOST2 must have current copies for the covered\n249\t# services to work correctly when they start on HOST2.\n250\t#\n251\t /mnt/user/Media_Server/Emby # Emby userdata — watch history, playstates\n252\t /mnt/user/appdata-Fallback/Critical-Data # auth stack — NPM rules, certs, user accounts\n253\t /mnt/user/Movies # media library content\n254\t /mnt/user/Tv_Shows # media library content\n255\t /mnt/user/Music # media library content\n256\t#\n257\t# Sync failure here = fallback runs but with stale data (acceptable) or missing\n258\t# data (problematic). These sync every night via daily_sync_maintenance.sh.\n259\t# Critical-Data and Emby also sync every 30 minutes via critical_sync_maintenance.sh.\n260\t# ─────────────────────────────────────────────────────────────────────────────\n261\t```\n262\t\n263\t---\n264\t\n265\t### ── Non-Fallback Shares — Sync as Backup, No Encryption Required ───────────\n266\t\n267\t```bash\n268\t# ─────────────────────────────────────────────────────────────────────────────\n269\t# Some shares don't participate in fallback — their container only runs on one\n270\t# server and is covered during fallback from scratch (or not at all).\n271\t# These shares can sync as offsite backup without encryption.\n272\t# Content is not sensitive — anyone with access to HOST2 could read it.\n273\t#\n274\t# Examples:\n275\t /mnt/user/appdata-Fallback/Gmer4Lfe # Gmer4Lfe server appdata backup\n276\t /mnt/user/appdata-Fallback/Important-Data # NextCloud, Immich appdata\n277\t /mnt/user/Kids_Movies # kids content — not sensitive\n278\t /mnt/user/Sports # sports content — not sensitive\n279\t#\n280\t# These sync in DAILY_SYNC_SHARES — straight rsync, no special handling.\n281\t# If HOST2 is compromised, the content is readable. That is acceptable.\n282\t# ─────────────────────────────────────────────────────────────────────────────\n283\t```\n284\t\n285\t---\n286\t\n287\t### ── Personal Shares — Always Encrypted ─────────────────────────────────────\n288\t\n289\t```bash\n290\t# ─────────────────────────────────────────────────────────────────────────────\n291\t# Personal shares contain content only the owner should see — documents,\n292\t# photos, financial records, private files.\n293\t# ZFS encrypts at the dataset level — HOST2 receives encrypted blocks.\n294\t# Without your passphrase or keyfile, HOST2 cannot read the content.\n295\t#\n296\t HOST1_PERSONAL_SHARES=(\n297\t \"/mnt/user/Gmer4Lfe-Personal\" # encrypted ZFS dataset\n298\t )\n299\t HOST2_PERSONAL_SHARES=(\n300\t \"/mnt/user/Jayred365-Personal\" # encrypted ZFS dataset\n301\t )\n302\t#\n303\t# Append to HOST*_DAILY_SYNC_SHARES — synced nightly with everything else.\n304\t# Same rsync, same timing — but the content on HOST2 is ciphertext only you can unlock.\n305\t#\n306\t# Setup: README-Rsync_Setup.md → Step 8 — Personal Encrypted Shares\n307\t# ─────────────────────────────────────────────────────────────────────────────\n308\t```\n309\t\n310\t---\n311\t\n312\t## ━━━ THIS IS NOT CONVENTIONAL FAILOVER ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n313\t\n314\t> **Understanding this distinction prevents a lot of confusion.**\n315\t\n316\tConventional failover (Pacemaker, DRBD, Proxmox HA) operates on the premise of a\n317\tprimary server and a standby server with shared storage or synchronous replication.\n318\tThe standby exists only to take over when the primary fails. It runs nothing\n319\tindependently. Its purpose is 100% failover coverage with no service interruption.\n320\t\n321\t**This is not that.**\n322\t\n323\t---\n324\t\n325\t### ── What This Actually Is ─────────────────────────────────────────────────────\n326\t\n327\t```\n328\tTwo fully independent servers, each running their own household's services.\n329\tEach server has its own domain, its own users, its own media library management.\n330\tEach server is the primary for its own services every single day.\n331\t\n332\tThe \"fallback\" component is additive:\n333\t When HOST1 goes down → HOST2 starts HOST1's containers on top of its own\n334\t When HOST2 comes back → HOST1's containers hand back cleanly\n335\t HOST2 never stopped running its own services during this entire time\n336\t\n337\tThis is not standby. This is mutual coverage between peers.\n338\t```\n339\t\n340\t---\n341\t\n342\t### ── The Actual Goal: Minimal Disruption to a Media Stack ────────────────────\n343\t\n344\t```bash\n345\t# ─────────────────────────────────────────────────────────────────────────────\n346\t# \"Minimal disruption\" means:\n347\t# Live TV keeps playing → Dispatcharr is Tier 1 (starts immediately)\n348\t# Passwords are accessible → VaultWarden is Tier 1 (people lock themselves out)\n349\t# Emby keeps serving → Emby is Tier 1\n350\t# Auth works → NPM, Authelia, LLDAP already warm on both servers\n351\t#\n352\t# \"Minimal disruption\" does NOT mean:\n353\t# Zero downtime (DDNS TTL = 1 minute, brief gap expected)\n354\t# Perfect consistency (Emby userdata is 30min stale at most)\n355\t# All services covered immediately (tiered start over 24 hours)\n356\t#\n357\t# The system is designed around the real experience of a household:\n358\t# \"The TV stopped working\" → unacceptable, fixed in Tier 1 (< 2 minutes)\n359\t# \"NextCloud is down\" → annoying, fixed in Tier 2 (4 hours)\n360\t# \"Gitea is down\" → unacceptable, fixed in Tier 1 (< 2 minutes)\n361\t# \"Sonarr isn't downloading\" → fine, fixed in Tier 4 (24 hours)\n362\t# ─────────────────────────────────────────────────────────────────────────────\n363\t```\n364\t\n365\t---\n366\t\n367\t### ── What You Give Up With This Approach ─────────────────────────────────────\n368\t\n369\t```bash\n370\t# ─────────────────────────────────────────────────────────────────────────────\n371\t# Things conventional failover provides that this does NOT:\n372\t# Zero RPO (recovery point objective) — there is ~30min of potential staleness\n373\t# Synchronous state replication — changes don't reach HOST2 instantly\n374\t# Automatic failure detection in seconds — fallback.sh checks every 2 minutes\n375\t# Transparent IP migration — DNS TTL means up to 1 minute of routing to old IP\n376\t#\n377\t# What you get instead:\n378\t# Both servers run fully independently — no single point of failure in software\n379\t# Geographic redundancy — different buildings, different power, different ISP\n380\t# Both households stay operational during most disruption scenarios\n381\t# Simple, auditable scripts — no cluster manager black box\n382\t# ─────────────────────────────────────────────────────────────────────────────\n383\t```\n384\t\n385\t---\n386\t\n387\t## ━━━ HOW FALLBACK ACTUALLY WORKS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n388\t\n389\t`fallback.sh` runs on both servers as a continuous background process. Every 2 minutes\n390\tit pings two things: the other server's Tailscale IP, and `8.8.8.8`. The combination\n391\tof those two answers determines the current state.\n392\t\n393\t```\n394\tRemote ping PASS + Internet PASS → NORMAL — both servers running independently\n395\t\n396\tRemote ping FAIL + Internet PASS → FALLBACK — remote is down, start covering it\n397\t Tier 1 immediately: DDNS + Emby + Auth + VaultWarden + Live TV + Gitea\n398\t Tier 2 at 4 hours: NextCloud + Immich + Jellyseerr\n399\t Tier 3 at 12 hours: AdGuard + Uptime monitoring\n400\t Tier 4 at 24 hours: Arrs + download clients\n401\t\n402\tInternet FAIL (either) → NO_INTERNET — stop own DDNS, wait\n403\t Can't serve anyone without internet — stop claiming DNS ownership\n404\t\n405\tRemote FAIL + Internet FAIL → DARK — conservative, same as NO_INTERNET\n406\t```\n407\t\n408\tThe DDNS sequencing is critical — one domain, one DDNS container, at all times. The\n409\thandback sequence stops remote DDNS first, rsyncs, starts containers, starts local\n410\tDDNS last. Split-brain DNS is prevented by design.\n411\t\n412\t**→ Full detail: [README-Fallback.md](Fallback/README-Fallback.md)**\n413\t\n414\t---\n415\t\n416\t## ━━━ THE AUTH STACK — WARM ON BOTH SERVERS ALWAYS ━━━━━━━━━━━━━━━━━━━━━━━━━━\n417\t\n418\tNPM, LLDAP, Authelia, MariaDB, Redis — all running actively on both servers at all\n419\ttimes. Not cold standby. Not started at fallback. Running right now, serving each\n420\tserver's own domain.\n421\t\n422\t```bash\n423\t# ─────────────────────────────────────────────────────────────────────────────\n424\t# Why this eliminates the fallback dead zone:\n425\t#\n426\t# Cold standby approach: HOST1 goes down → DNS switches to HOST2 → HOST2 starts\n427\t# auth containers → 30-60 seconds before auth is ready → users hit the auth wall\n428\t# before the service behind it responds → failed logins, broken sessions\n429\t#\n430\t# Warm approach: HOST1 goes down → DNS switches to HOST2 → auth is ALREADY\n431\t# running → users are served immediately with no dead zone\n432\t#\n433\t# HOST1 is source of truth for the auth configuration. All changes to NPM proxy\n434\t# rules, LLDAP users, Authelia policies, and certs happen on HOST1 and propagate\n435\t# to HOST2 via Critical-Data sync every 30 minutes.\n436\t#\n437\t# HOST2 operator's experience: click NginxProxyManager in Docker UI →\n438\t# browser opens HOST1's NPM via Tailscale redirect → make changes there →\n439\t# 30 minutes later live on both servers. Zero auth management overhead on HOST2.\n440\t# ─────────────────────────────────────────────────────────────────────────────\n441\t```\n442\t\n443\t**→ Full detail: [README-Partnership.md](Partnership/README-Partnership.md)**\n444\t\n445\t---\n446\t\n447\t## ━━━ THE COMPLETE PICTURE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n448\t\n449\t```\n450\t HOST1 (Gmer4Lfe.com) HOST2 (Gmer4Lfe.us)\n451\t ────────────────── ──────────────────────\n452\t\n453\tAlways running:\n454\t Own stack Full service stack Full service stack (anime focus)\n455\t Auth NPM, Authelia, LLDAP NPM, Authelia, LLDAP (mirrored)\n456\t\n457\tSource of truth:\n458\t Media Movies, Tv_Shows, Music ←── Anime_Movies, Anime_Shows\n459\t ───────────────────────→ (bidirectional daily push)\n460\t Config auth stack config ──→ mirrored every 30 min\n461\t Emby state watch history, playstates──→ dirty sync every 30 min\n462\t\n463\tFallback coverage:\n464\t If HOST1 down: — starts Emby, VaultWarden,\n465\t Dispatcharr, all arr stack\n466\t serves Gmer4Lfe.com via DDNS\n467\t\n468\t If HOST2 down: starts his Emby, his arrs, —\n469\t serves Gmer4Lfe.us via DDNS\n470\t\n471\tMonitoring:\n472\t Both servers resource/docker/system/ resource/docker/system/\n473\t stability_watchdog.sh stability_watchdog.sh\n474\t fallback.sh fallback.sh\n475\t Sunday morning coffee report Sunday morning coffee report\n476\t```\n477\t\n478\t---\n479\t\n480\t## ━━━ FOLDER STRUCTURE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n481\t\n482\t```\n483\tvaraverk/\n484\t│\n485\t├── Configurations/ ← All configuration files — the only files you edit regularly\n486\t│ ├── master.conf ← Shared: thresholds, toggles, job lists, watchdog settings\n487\t│ ├── host1.conf ← HOST1-specific: share lists, container names, API keys\n488\t│ └── host2.conf ← HOST2-specific: same structure, different values\n489\t├── common.sh ← Shared library — all functions used by every script\n490\t├── load_config.sh ← Detects OS platform → sources Configurations/, common.sh, adapter\n491\t│\n492\t├── Plugin/ ← Platform adapters + Unraid web UI\n493\t│ └── unraid/\n494\t│ ├── adapter.sh ← Unraid adapter: platform_*() functions (rc.d, emhttp, dynamix)\n495\t│ ├── api/ ← PHP API endpoints\n496\t│ ├── include/ ← PHP business logic\n497\t│ ├── pages/ ← Per-tab page includes\n498\t│ └── ... ← Full web UI (scheduler, monitor, docker, arrs, fallback)\n499\t│\n500\t├── Orchestrators/ ← Scheduled job runners — what Varaverk scheduler executes\n501\t│ README: README-Orchestrators.md\n502\t│\n503\t├── Rsync/ ← Core rsync script — called per share with profile\n504\t│ README: README-Rsync_Setup.md\n505\t│\n506\t├── Fallback/ ← Mutual automatic fallback — continuous background process\n507\t│ README: README-Fallback.md\n508\t│\n509\t├── Watchdogs/ ← All watchdog scripts: resource, docker, system, stability\n510\t│ README: README-Watchdogs.md\n511\t│\n512\t├── Docker_Essentials/ ← Container lifecycle: restarts, updates, networks\n513\t│ README: README-Docker_Essentials.md\n514\t│\n515\t├── System_Essentials/ ← Server-level: WebGUI recovery, log hygiene, kernel tuning, reboot\n516\t│ README: README-System_Essentials.md\n517\t│\n518\t├── Media/ ← Library health + behavior-driven discovery: permissions, junk cleanup, orphan removal, weekly arr adds\n519\t│ README: README-Media.md\n520\t│\n521\t├── Transcodes/ ← Ramdisk transcode management with SSD fallback\n522\t│ README: README-Transcoding.md\n523\t│\n524\t├── Monitors/ ← Read-only observation: certs, SMART, bandwidth, digests\n525\t│ README: README-Monitors.md\n526\t│\n527\t├── Partnership/ ← Two-server relationship lifecycle: onboard, offboard, transfer\n528\t│ README: README-Partnership.md\n529\t│\n530\t├── Kernel/ ← Kernel module config — loaded at boot by go file\n531\t│ README: README-Kernel.md\n532\t│\n533\t└── Tools/ ← Situational utilities: repair, export, emergency tools\n534\t README: README-Tools.md\n535\t```\n536\t\n537\t---\n538\t\n539\t## ━━━ CONFIGURATION — Configurations/ FOLDER, THREE FILES ━━━━━━━━━━━━━━━━━━━\n540\t\n541\t```bash\n542\t# ─────────────────────────────────────────────────────────────────────────────\n543\t# All three conf files live in Configurations/. load_config.sh sources them.\n544\t# Sparse checkout ensures each server only receives its own host*.conf.\n545\t#\n546\t# load_config.sh does five things in order:\n547\t# 1. Detects OS platform (/etc/unraid-version → PLATFORM=unraid)\n548\t# 2. Sources master.conf\n549\t# 3. Auto-discovers and sources all host*.conf present\n550\t# 4. Sources common.sh\n551\t# 5. Sources Plugin/$PLATFORM/adapter.sh (platform_*() functions)\n552\t#\n553\t# master.conf — shared across both servers\n554\t# Everything that applies equally to both: thresholds, schedules, profile\n555\t# definitions, watchdog settings, arr cleanup config, DDNS timing, etc.\n556\t# Pushed to both servers via git. Never contains server-specific values.\n557\t#\n558\t# host1.conf — sourced only on HOST1\n559\t# HOST1_* prefixed variables: share lists, container names, API keys,\n560\t# ramdisk size, specific paths, per-server toggle overrides.\n561\t#\n562\t# host2.conf — sourced only on HOST2\n563\t# HOST2_* prefixed variables: same structure, different values.\n564\t#\n565\t# detect_hosts() in common.sh:\n566\t# Reads the current server's hostname. Matches it against HOST1 and HOST2.\n567\t# Aliases HOST1_DAILY_SYNC_SHARES → DAILY_SYNC_SHARES (if on HOST1)\n568\t# Aliases HOST2_DAILY_SYNC_SHARES → DAILY_SYNC_SHARES (if on HOST2)\n569\t# Every script uses the unprefixed name. The same script on either server\n570\t# automatically uses the correct values. No hostname comparisons in scripts.\n571\t#\n572\t# Plugin/$PLATFORM/adapter.sh — OS-specific function implementations\n573\t# Scripts call platform_restart_service(), platform_storage_healthy(), etc.\n574\t# The adapter maps those to rc.d scripts, emhttp commands, and dynamix files\n575\t# on Unraid — or to systemctl, mount checks, etc. on a future platform.\n576\t# ─────────────────────────────────────────────────────────────────────────────\n577\t```\n578\t\n579\t---\n580\t\n581\t## ━━━ WHAT RUNS WHEN ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n582\t\n583\tThe Varaverk scheduler runs only orchestrators — never individual scripts directly.\n584\t\n585\t```\n586\tArray start (Varaverk event hook):\n587\t array_started.sh launches all startup scripts in order\n588\t → inotify_tuning.sh raise kernel limits before containers start\n589\t → docker_syslog_filter.sh suppress veth noise before containers create interfaces\n590\t → php_fpm_max_children.sh WebGUI tuning before first request\n591\t → ramdisk_setup.sh create ramdisk before Emby starts\n592\t → docker_network_connect.sh connect containers to extra networks\n593\t → fallback.sh continuous — mutual fallback state machine\n594\t\n595\tEvery 15 minutes:\n596\t watchdog_orchestrator.sh fires each watchdog in sequence\n597\t → resource_watchdog.sh reduce pressure before healing attempts\n598\t → docker_watchdog.sh two-tier container healing\n599\t → system_watchdog.sh system components: storage + WebGUI health\n600\t → stability_watchdog.sh last resort — reboots when all else fails\n601\t\n602\tEvery 7 minutes:\n603\t transcode_management.sh cleanup → manager (order non-negotiable)\n604\t\n605\tEvery 30 minutes:\n606\t critical_sync_maintenance.sh Critical-Data + Emby dirty sync + partnership check\n607\t\n608\tEvery 30 minutes:\n609\t rsync.sh --profile=emby-fallback Emby watch state sync (separate cron entry)\n610\t\n611\tEvery 6 hours:\n612\t arrs_failed_stalled_recovery.sh blocklist + re-search failed imports\n613\t\n614\t1am daily:\n615\t daily_sync_maintenance.sh git pull → sync shares → media maintenance → restarts\n616\t\n617\t2:30am Sunday:\n618\t weekly_sync_maintenance.sh stop → update images → clean sync → start → weekly restarts\n619\t + playback_aware_lidarr/radarr/sonarr_discovery.sh (behavior-driven arr discovery)\n620\t\n621\t8am daily:\n622\t weekly_health_digest.sh DIGEST_PROFILE controls when it actually notifies\n623\t\n624\tEvery 6 hours:\n625\t system_tuning_monitor.sh inotify + php-fpm utilisation snapshot\n626\t\n627\tSunday morning block (6–11am):\n628\t zfs_memory_snapshot.sh, smart_health.sh, cert_monitor.sh,\n629\t backup_verify.sh, emby_session_report.sh, bandwidth_monitor.sh --report\n630\t```\n631\t\n632\t**→ Complete schedule: [README-Orchestrators.md](Orchestrators/README-Orchestrators.md)**\n633\t\n634\t---\n635\t\n636\t## ━━━ THE DAILY CYCLE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n637\t\n638\tWhat actually happens on a typical day, from the ecosystem's perspective:\n639\t\n640\t```\n641\tThroughout the day (every 15 minutes via watchdog_orchestrator.sh):\n642\t resource_watchdog.sh managing: system pressure (throttle/pause/stop)\n643\t docker_watchdog.sh healing: memory leaks, HTTP failures, required containers\n644\t system_watchdog.sh watching: system components (storage growth, WebGUI)\n645\t stability_watchdog.sh watching: RAM, CPU temp, rootfs, kernel, daemon\n646\t\n647\tThroughout the day:\n648\t fallback.sh watching: remote server, internet connectivity\n649\t transcode_management.sh managing: ramdisk ↔ SSD, session cleanup (every 7min)\n650\t critical_sync_maintenance.sh keeping: auth stack + Emby current (every 30min)\n651\t\n652\t1am:\n653\t git pull (latest scripts first)\n654\t rsync Movies, Tv_Shows, Music → HOST2 (HOST1 pushes its truth shares)\n655\t rsync Anime_Shows, Anime_Movies → HOST1 (HOST2 pushes its truth shares)\n656\t media_shares_permissions.sh (fix ownership before arr cleanup)\n657\t media_cleaner.sh anime + media (remove scene junk)\n658\t lidarr/sonarr/radarr_cleanup.sh (remove orphaned files)\n659\t docker_daily_restart.sh (NPM, Authelia, Dispatcharr fresh start)\n660\t\n661\t2:30am Sunday:\n662\t Stop Emby + auth stack both sides\n663\t Pull updated Docker images (zero extra downtime — already stopped)\n664\t rsync Emby + Critical-Data clean mirror\n665\t Start containers on new images\n666\t docker_weekly_restart.sh (NextCloud, AdGuard, Immich)\n667\t\n668\t7am Sunday:\n669\t sunday_morning_coffee_report.sh (everything that happened this week)\n670\t```\n671\t\n672\t---\n673\t\n674\t## ━━━ THE SELF-HEALING STACK ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n675\t\n676\tProblems are handled at the most appropriate level. Scripts escalate only when the\n677\tlevel below them has failed:\n678\t\n679\t```\n680\tProblem: Emby container using 22GB RAM (limit: 18GB)\n681\t → docker_watchdog.sh Tier 1: restart Emby immediately\n682\t → System continues, no reboot\n683\t\n684\tProblem: Authelia crashed and won't restart\n685\t → docker_watchdog.sh Tier 1: attempt restart (dependency check: DB + Redis first)\n686\t → Fails 3 times in 1 hour → skip list → critical notification\n687\t → Tools: watchdog_skip_list_manager.sh to clear after fixing\n688\t\n689\tProblem: RAM drops to 3GB\n690\t → stability_watchdog.sh: stop non-essential containers\n691\t → Wait for RAM to recover to 30GB\n692\t → docker_watchdog.sh defers (reads mem_shutdown_active=true)\n693\t → If RAM drops to 4GB AND 3+ OOM kills: bypass strikes → reboot\n694\t\n695\tProblem: Docker daemon hung\n696\t → docker_watchdog.sh: attempt platform_restart_service docker (rc.d on Unraid)\n697\t → If still hung: sets daemon_confirmed_down flag → stability_watchdog escalates → reboot\n698\t\n699\tProblem: HOST1 loses power\n700\t → fallback.sh on HOST2 detects at next 2-minute check\n701\t → Tier 1 starts immediately: DDNS, Emby, VaultWarden, Live TV\n702\t → Household barely notices. Live TV may buffer for < 2 minutes.\n703\t```\n704\t\n705\t**→ Container healing: [README-Docker_Essentials.md](Docker_Essentials/README-Docker_Essentials.md)**\n706\t**→ System stability: [README-System_Essentials.md](System_Essentials/README-System_Essentials.md)**\n707\t\n708\t---\n709\t\n710\t## ━━━ GPU CONTAINERS — IGNORE GOOGLE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n711\t\n712\t> **Every tutorial, forum post, and Reddit thread shows the wrong way to add GPU\n713\t> access to a Docker container on unRAID.** The old method breaks silently in ways\n714\t> that are very hard to diagnose. Read this before touching any GPU container.\n715\t\n716\t### The Wrong Way (What Google Shows)\n717\t\n718\t```bash\n719\t# DO NOT DO THIS\n720\tExtra Parameters: --runtime=nvidia\n721\tVariables: NVIDIA_VISIBLE_DEVICES=GPU-xxxxxxxx\n722\t NVIDIA_DRIVER_CAPABILITIES=compute,video,utility\n723\t```\n724\t\n725\tThis is `--runtime=nvidia` — the old NVIDIA Container Toolkit approach. It is what\n726\tevery guide written before Docker 19.03 recommends. It still shows up in official\n727\tunRAID forum posts, community applications templates, and most search results.\n728\t\n729\t**Why it breaks here:**\n730\t\n731\t1. `--runtime=nvidia` combined with `NVIDIA_VISIBLE_DEVICES` has caused conflicts\n732\t historically and breaks on container rebuilds. With Docker 29.x (Unraid 7.3+),\n733\t `bind-propagation=shared` crashes runc v1.3.5 entirely — with any container,\n734\t GPU or not. The container fails to start with:\n735\t ```\n736\t OCI runtime create failed: unable to start container process:\n737\t error jailing process inside rootfs:\n738\t open /proc/self/mountinfo: no such file or directory\n739\t ```\n740\t The ramdisk tmpfs is already MS\\_SHARED at the kernel level. Do not use\n741\t `bind-propagation=shared` in Extra Parameters on Unraid 7.3+.\n742\t\n743\t2. When unRAID rebuilds a container from template (update, reinstall from Community\n744\t Applications), the `--runtime=nvidia` field and the NVIDIA env vars can be dropped\n745\t or corrupted. Rebuilding a container is a normal maintenance operation — it should\n746\t not break GPU access. With the old method, it sometimes does.\n747\t\n748\t### The Right Way\n749\t\n750\t```bash\n751\t# DO THIS — everything in Extra Parameters, nothing in Variables\n752\t--gpus \"device=GPU-62e1659d-1ed4-935f-3df3-4bb4339438f1\" --pids-limit=0 --mount type=bind,source=/mnt/ram-transcode,target=/ext-ram-transcode\n753\t```\n754\t\n755\t`--gpus \"device=UUID\"` is Docker-native GPU support (Docker 19.03+, released 2019).\n756\tOne field in Extra Parameters. The UUID pins a specific device — no ambiguity on\n757\tsingle-GPU systems. No env vars needed. No `--runtime=nvidia`. Survives container\n758\trebuilds. This is the correct method.\n759\t\n760\tFind your GPU UUID: `nvidia-smi -L`\n761\t\n762\t**HOST1 GPU UUID (Quadro P2000):** `GPU-62e1659d-1ed4-935f-3df3-4bb4339438f1`\n763\t\n764\t### GPU Memory and Media Servers\n765\t\n766\tA failure mode that is extremely non-obvious and not documented anywhere:\n767\t\n768\tAny container that uses VRAM and does not release it will starve Emby and Jellyfin\n769\tof the GPU memory they need to start transcoding sessions. The two media servers\n770\trespond differently:\n771\t\n772\t- **Jellyfin:** hard fails — no session is created, playback stops entirely\n773\t- **Emby:** falls back to CPU transcoding silently and keeps going\n774\t\n775\tThe non-obvious source is OCR sidecars — credit detection plugins, subtitle\n776\textraction tools — that talk to a GPU-accelerated OCR container. PaddleOCR in\n777\tparticular loads a neural network into VRAM at first use and does not release it\n778\tbetween runs. One credit scan, and VRAM stays consumed until the container restarts.\n779\t\n780\t**Emby startup probe:** Emby runs a one-shot NVIDIA hardware detection at container\n781\tstart. If VRAM is exhausted when Emby starts, NVIDIA is marked unavailable for the\n782\tentire session — no retry. Restarting Emby while VRAM is still full just fails the\n783\tprobe again. Correct recovery: free VRAM first, confirm with `nvidia-smi`, then\n784\trestart Emby.\n785\t\n786\tFor the EmbyCredits plugin: use the `yock1/embycreditocr` Tesseract image (port 8884,\n787\tCPU-based) instead of PaddleOCR. Same plugin, no VRAM.\n788\t\n789\t**→ Full detail: [Manual-Transcoding.md](Transcodes/Manual-Transcoding.md)**\n790\t\n791\t---\n792\t\n793\t## ━━━ SETUP — WHERE TO START ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n794\t\n795\tIf you're setting this up from scratch on two servers:\n796\t\n797\t```\n798\t1. Follow Manual.md — end-to-end setup guide\n799\t Tailscale → SSH keys → git clone → master.conf → Varaverk scheduler schedule\n800\t\n801\t2. Set your container names correctly before anything else\n802\t Shared services: no suffix (Emby, NginxProxyManager, etc.)\n803\t Personal services: suffixed (Emby-Gmer4Lfe, Organizer-Gmer4Lfe)\n804\t\n805\t3. Set your share names identically on both servers\n806\t /mnt/user/Movies on both, /mnt/user/Tv_Shows on both, etc.\n807\t\n808\t4. Run initial sync to populate HOST2\n809\t bash daily_sync_maintenance.sh --log\n810\t\n811\t5. Verify fallback works before relying on it\n812\t bash fallback_test.sh --dry-run first\n813\t bash fallback_test.sh when ready — uses iptables simulation, no real downtime\n814\t```\n815\t\n816\t---\n817\t\n818\t## ━━━ FOLDER README REFERENCE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n819\t\n820\t| README | What It Covers |\n821\t|--------|---------------|\n822\t| [README-Fallback.md](Fallback/README-Fallback.md) | DDNS sequencing, tiered fallback, handback, split brain prevention, fallback_test.sh |\n823\t| [README-Docker_Essentials.md](Docker_Essentials/README-Docker_Essentials.md) | Two-tier container watchdog, memory limits, dependency ordering, skip list, daily/weekly restarts |\n824\t| [README-System_Essentials.md](System_Essentials/README-System_Essentials.md) | WebGUI recovery, inotify tuning, PHP-FPM, log hygiene, mover/reboot/rsync stop |\n825\t| [README-Orchestrators.md](Orchestrators/README-Orchestrators.md) | What runs when, execution order, daily/weekly windows, adding jobs |\n826\t| [README-Media.md](Media/README-Media.md) | Permissions model, junk cleanup, arr orphan removal, safety layers, behavior-driven discovery, testing procedure |\n827\t| [README-Transcoding.md](Transcodes/README-Transcoding.md) | Ramdisk design, symlink architecture, Docker mount requirement, SSD fallback |\n828\t| [README-Monitors.md](Monitors/README-Monitors.md) | Cert monitoring, SMART health, bandwidth tracking, health digest profiles |\n829\t| [README-Partnership.md](Partnership/README-Partnership.md) | Auth stack sharing, onboard/offboard/transfer lifecycle, deferred offboard |\n830\t| [README-Plugin.md](Plugin/README-Plugin.md) | Platform adapter API, Unraid web UI, scheduler/monitor/docker pages |\n831\t| [Manual.md](Manual.md) | Complete setup guide — Tailscale, SSH keys, git clone, master.conf, Varaverk scheduler |\n832\t| [README-Tools.md](Tools/README-Tools.md) | fallback_state_reset, skip list manager, emby_database_repair, container export |\n833\t\n834\t---\n835\t\n836\t## ━━━ A FEW THINGS THAT WILL SURPRISE YOU ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n837\t\n838\tThings that are different from what you might expect:\n839\t\n840\t```bash\n841\t# ─────────────────────────────────────────────────────────────────────────────\n842\t# 1. The Varaverk scheduler has very few entries.\n843\t# Everything runs through orchestrators. Individual scripts are never\n844\t# scheduled directly. array_started.sh is the only array_start event entry.\n845\t\n846\t# 2. Scripts are silent when healthy.\n847\t# stability_watchdog.sh runs 288 times per day. It produces zero output on clean\n848\t# cycles. If you see output, something needs attention. If you see nothing,\n849\t# the system is healthy. This is by design.\n850\t\n851\t# 3. master.conf is the only file you edit.\n852\t# Master_host*.conf for per-server values. Everything else is code — don't\n853\t# add configuration into scripts directly. Everything configurable has a var.\n854\t\n855\t# 4. --dry-run works on everything.\n856\t# Every single script supports --dry-run. Pre-flight checks still run.\n857\t# Review what would happen before anything changes. Always available.\n858\t\n859\t# 5. detect_hosts() does the heavy lifting.\n860\t# Scripts never compare hostnames directly. detect_hosts() aliases the correct\n861\t# HOST*_ vars based on the running server. The same script runs correctly on\n862\t# HOST1 and HOST2 — no branching on server identity inside scripts.\n863\t\n864\t# 6. GPU containers: ignore every Google result. Use --gpus, not --runtime=nvidia.\n865\t# --runtime=nvidia breaks on container rebuilds. See GPU CONTAINERS section above.\n866\t\n867\t# 7. The Emby/Jellyfin Docker mount is unusual and critical.\n868\t# --mount type=bind,source=/mnt/ram-transcode,target=/ext-ram-transcode\n869\t# in Extra Parameters — not a standard path mapping. Do NOT add bind-propagation=shared:\n870\t# runc v1.3.5+ (Docker 29.x, Unraid 7.3+) rejects it. The ramdisk is already\n871\t# MS_SHARED at the kernel level. README-Transcoding.md explains in detail.\n872\t\n873\t# 8. arr cleanup scripts can delete files.\n874\t# lidarr_cleanup.sh, sonarr_cleanup.sh, radarr_cleanup.sh permanently delete\n875\t# orphaned files. Safety layers prevent catastrophic runs but always test with\n876\t# --dry-run --log first on a new system. README-Media.md has the full procedure.\n877\t\n878\t# 9. Scripts never call OS commands directly.\n879\t# All platform-specific operations go through plugin_*() functions defined in\n880\t# Plugin/$PLATFORM/adapter.sh. rc.d scripts, emhttp, dynamix, disks.ini — all\n881\t# isolated in the adapter. This is what makes the codebase portable.\n882\t# Adding a new OS = writing one adapter file, not patching dozens of scripts.\n883\t\n884\t# 10. System_Essentials/ is platform-agnostic at the script level.\n885\t# Scripts like server_reboot.sh, mover_stop.sh, and webgui_watchdog.sh call\n886\t# adapter functions (platform_is_mover_running, platform_restart_service, etc.)\n887\t# The adapter handles what those mean on the actual OS. The scripts are clean.\n888\t# ─────────────────────────────────────────────────────────────────────────────\n889\t```\n890\t\n891\t---\n892\t\n893\t## ━━━ PROJECT NAME ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n894\t\n895\t```\n896\tName: Varaverk\n897\tRepository: git@192.168.50.2:Varaverk/varaverk.git\n898\tInstall path: /boot/config/plugins/varaverk\n899\t```"}]},"uuid":"ced3c8ea-0638-41aa-be7a-0971b96dee89","timestamp":"2026-06-14T05:52:17.717Z","sourceToolAssistantUUID":"c35c17bd-e27f-4285-8469-975e994513c5","userType":"external","entrypoint":"cli","cwd":"/boot/config/plugins/varaverk","sessionId":"685e6c5b-62c1-40bd-9cfd-2c9f7e15c50f","version":"2.1.177","gitBranch":"HEAD","slug":"mutable-sniffing-naur"} {"parentUuid":"ced3c8ea-0638-41aa-be7a-0971b96dee89","isSidechain":true,"agentId":"ad9a317b416360985","message":{"model":"claude-haiku-4-5-20251001","id":"msg_013XUUgMCTzC3LNxeckM5BUw","type":"message","role":"assistant","content":[{"type":"tool_use","id":"toolu_01BtgcQTeMGC9CsQE1Woma1w","name":"Read","input":{"file_path":"/boot/config/plugins/varaverk/Manual.md"},"caller":{"type":"direct"}}],"stop_reason":null,"stop_sequence":null,"stop_details":null,"usage":{"input_tokens":3,"cache_creation_input_tokens":1968,"cache_read_input_tokens":5697,"cache_creation":{"ephemeral_5m_input_tokens":1968,"ephemeral_1h_input_tokens":0},"output_tokens":1,"service_tier":"standard","inference_geo":"not_available"},"diagnostics":null},"requestId":"req_011Cc2daJFeiE5pgF2fQ7t3p","attributionAgent":"Explore","type":"assistant","uuid":"87fe6649-7a98-4f11-b9ed-a5167137d9f3","timestamp":"2026-06-14T05:52:17.790Z","userType":"external","entrypoint":"cli","cwd":"/boot/config/plugins/varaverk","sessionId":"685e6c5b-62c1-40bd-9cfd-2c9f7e15c50f","version":"2.1.177","gitBranch":"HEAD","slug":"mutable-sniffing-naur"} {"parentUuid":"87fe6649-7a98-4f11-b9ed-a5167137d9f3","isSidechain":true,"promptId":"14997b41-0464-47e1-9df2-adbecd208513","agentId":"ad9a317b416360985","type":"user","message":{"role":"user","content":[{"tool_use_id":"toolu_01BtgcQTeMGC9CsQE1Woma1w","type":"tool_result","content":"1\t# ━━━━━ VARAVERK — Setup Manual ━━━━━\n2\t\n3\tGetting a fresh two-server ecosystem running from scratch. \n4\tFor system overview see [README.md](README.md). For individual subsystem detail see folder READMEs and script headers.\n5\t\n6\t---\n7\t\n8\t## ━━━ BEFORE YOU START ━━━\n9\t\n10\tThings that must be in place before you touch any scripts.\n11\t\n12\t---\n13\t\n14\t### ── Two unRAID Servers ───────────────────────────────────────────────────────\n15\t\n16\tBoth servers need to be up and have their arrays started. Hardware does not need to\n17\tmatch. The scripts work via `/mnt/user/` — the same share name on both servers resolves\n18\tcorrectly regardless of the hardware underneath.\n19\t\n20\tDecide which server is **HOST1 (owner)** and which is **HOST2 (mirror)**. This is just a\n21\tlabel — both servers run their own full stack. The owner manages the shared auth\n22\tconfiguration. Pick the more capable or more reliably-online server as owner.\n23\t\n24\t---\n25\t\n26\t### ── Plugins — install on both servers ────────────────────────────────────────\n27\t\n28\tInstall these via **Apps** (Community Applications) on both servers:\n29\t\n30\t| Plugin | Why |\n31\t|--------|-----|\n32\t| **Tailscale** | Encrypted VPN between servers — all script traffic travels over it |\n33\t\n34\t---\n35\t\n36\t## ━━━ STEP 1: TAILSCALE ━━━\n37\t\n38\tBoth servers must be visible to each other on the same Tailscale network.\n39\t\n40\t---\n41\t\n42\t### ── Connect both servers ────────────────────────────────────────────────────\n43\t\n44\t1. Open the Tailscale plugin on HOST1. Log in with your Tailscale account.\n45\t2. Open the Tailscale plugin on HOST2. Log in with the **same account** — or have the\n46\t HOST2 owner share their machine into your tailnet (Tailscale → Share → invite by email).\n47\t3. Confirm both devices appear in your Tailscale admin panel.\n48\t\n49\t---\n50\t\n51\t### ── Verify reachability ──────────────────────────────────────────────────────\n52\t\n53\tOn HOST1:\n54\t```bash\n55\t# Get your Tailscale IP\n56\ttailscale ip -4\n57\t\n58\t# Ping HOST2 by its Tailscale hostname (the unRAID server name, e.g. unRAID-Jayred365)\n59\tping -c 3 unRAID-Jayred365\n60\t```\n61\t\n62\tOn HOST2 — same in reverse. Both pings must succeed before continuing.\n63\t\n64\t---\n65\t\n66\t### ── Note the Tailscale hostnames ──────────────────────────────────────────────\n67\t\n68\tThe scripts use Tailscale DNS to resolve IPs — no hardcoded addresses anywhere.\n69\tFor this to work, the Tailscale device name must match the unRAID server hostname exactly.\n70\t\n71\tCheck the unRAID hostname: **Settings → System Identification → Server Name** \n72\tCheck the Tailscale device name: Tailscale admin panel → Machines\n73\t\n74\tIf they differ, rename the Tailscale device to match the unRAID hostname. Case matters.\n75\t\n76\t```bash\n77\t# On each server — confirm the hostname the scripts will use\n78\thostname\n79\t# Should return e.g. \"unRAID-Gmer4Lfe\"\n80\t```\n81\t\n82\t---\n83\t\n84\t## ━━━ STEP 2: CLONE THE REPOSITORY ━━━\n85\t\n86\t---\n87\t\n88\t### ── Clone on HOST1 ───────────────────────────────────────────────────────────\n89\t\n90\t```bash\n91\tgit clone /boot/config/plugins/varaverk\n92\tcd /boot/config/plugins/varaverk\n93\t```\n94\t\n95\t---\n96\t\n97\t### ── Clone on HOST2 ───────────────────────────────────────────────────────────\n98\t\n99\t```bash\n100\tgit clone /boot/config/plugins/varaverk\n101\tcd /boot/config/plugins/varaverk\n102\t```\n103\t\n104\t---\n105\t\n106\t### ── Sparse checkout — keep credentials separate ────────────────────────────\n107\t\n108\t`host1.conf` contains HOST1's SSH keys, API keys, and passwords.\n109\t`host2.conf` contains HOST2's equivalents. Neither server should hold the other's\n110\tcredentials. Set up sparse checkout so each server only receives its own conf file.\n111\t\n112\t**On HOST1** — exclude host2.conf:\n113\t```bash\n114\tcd /boot/config/plugins/varaverk\n115\tgit sparse-checkout init --no-cone\n116\tgit sparse-checkout set '/*' '!/Configurations/host2.conf'\n117\tgit checkout\n118\t```\n119\t\n120\t**On HOST2** — exclude host1.conf:\n121\t```bash\n122\tcd /boot/config/plugins/varaverk\n123\tgit sparse-checkout init --no-cone\n124\tgit sparse-checkout set '/*' '!/Configurations/host1.conf'\n125\tgit checkout\n126\t```\n127\t\n128\tVerify: `ls Configurations/` — each server should see `master.conf` and only its own `host*.conf`.\n129\t\n130\t> If you are not using git (deploying manually) skip this and ensure you only copy\n131\t> each server's own host*.conf when deploying.\n132\t\n133\t---\n134\t\n135\t## ━━━ STEP 3: CONFIGURE ━━━\n136\t\n137\tConfiguration lives in `Configurations/`. Three files:\n138\t- `master.conf` — shared, both servers see it — thresholds, toggles, schedules\n139\t- `host1.conf` — HOST1 only — credentials, share lists, container names\n140\t- `host2.conf` — HOST2 only — same structure, different values\n141\t\n142\tEdit these with any text editor. Everything in the scripts is controlled through these\n143\tfiles — never edit scripts directly to change behaviour.\n144\t\n145\t---\n146\t\n147\t### ── 3a. master.conf — server hostnames (do this first) ─────────────────────\n148\t\n149\tOpen `Configurations/master.conf`. Find the HOST IDENTITIES section near the top:\n150\t\n151\t```bash\n152\tHOST1=\"unRAID-Gmer4Lfe\"\n153\tHOST2=\"unRAID-Jayred365\"\n154\t```\n155\t\n156\tReplace these with your actual unRAID server hostnames. These must match exactly —\n157\t`detect_hosts()` compares the running server's hostname against these two values to\n158\tknow which server it is on. Everything else in the ecosystem flows from this.\n159\t\n160\t---\n161\t\n162\t### ── 3b. host1.conf — HOST1 identity ────────────────────────────────────────\n163\t\n164\tOpen `Configurations/host1.conf`. Fill in the IDENTITY section:\n165\t\n166\t```bash\n167\tHOST1_SSH_KEY=\"/root/.ssh/gmer4lfe_rsync_automation\" # path — ssh_setup.sh will create this\n168\tHOST1_OWNER=\"gmer4lfe\" # short name, lowercase, no spaces\n169\tHOST1_OWNER_EMAIL=\"you@example.com\" # for partnership notifications\n170\t```\n171\t\n172\tThe SSH key does not exist yet — `ssh_setup.sh` in Step 4 will create it and fill in\n173\tthe path automatically. Leave the path as-is for now; it will be overwritten.\n174\t\n175\t---\n176\t\n177\t### ── 3c. host2.conf — HOST2 identity ────────────────────────────────────────\n178\t\n179\tOpen `Configurations/host2.conf`. Fill in the same fields with HOST2's values:\n180\t\n181\t```bash\n182\tHOST2_SSH_KEY=\"/root/.ssh/jayred365_rsync_automation\"\n183\tHOST2_OWNER=\"jayred365\"\n184\tHOST2_OWNER_EMAIL=\"them@example.com\"\n185\t```\n186\t\n187\t---\n188\t\n189\t### ── 3d. Remaining configuration ────────────────────────────────────────────\n190\t\n191\tThe scripts will not break if the remaining fields are empty — they self-guard\n192\twhen things are not configured. Fill in sections as you enable each feature. The\n193\tminimum set needed to get a working sync + fallback:\n194\t\n195\t**In host1.conf (and host2.conf mirror):**\n196\t```bash\n197\t# Shares HOST1 pushes to HOST2 nightly\n198\tHOST1_DAILY_SYNC_SHARES=(\n199\t \"/mnt/user/Movies\"\n200\t \"/mnt/user/Tv_Shows\"\n201\t \"/mnt/user/Music\"\n202\t)\n203\t\n204\t# Shares synced every 30 minutes (auth config + Emby watch state)\n205\tHOST1_CRITICAL_SYNC_SHARES=(\n206\t \"/mnt/user/appdata-Fallback/Critical-Data|critical-data\"\n207\t)\n208\t\n209\t# Containers fallback.sh starts on HOST2 when HOST1 goes down (Tier 1 = immediate)\n210\tHOST2_FALLBACK_HOST1_COVERS_HOST2_TIER1=() # HOST2's own covers for HOST1, and vice versa\n211\tHOST1_FALLBACK_HOST2_COVERS_HOST1_TIER1=(\n212\t \"Emby\"\n213\t \"VaultWarden\"\n214\t \"NginxProxyManager\"\n215\t)\n216\t```\n217\t\n218\tFor full configuration reference see `Manual-Fallback.md`, `Manual-Rsync.md`,\n219\tand `Manual-Watchdogs.md`.\n220\t\n221\t---\n222\t\n223\t## ━━━ STEP 4: SSH KEYS ━━━\n224\t\n225\tThe servers need passwordless SSH access to each other. `ssh_setup.sh` handles\n226\tkey generation and remote installation in one step.\n227\t\n228\t---\n229\t\n230\t### ── Run on HOST2 first ────────────────────────────────────────────────────\n231\t\n232\tThe mirror runs first — it generates its own key and copies it to HOST1. You will be\n233\tprompted for HOST1's root password once (for the initial key copy). After that,\n234\tall SSH is keyless.\n235\t\n236\t```bash\n237\tcd /boot/config/plugins/varaverk\n238\tbash Partnership/ssh_setup.sh\n239\t```\n240\t\n241\tWhen prompted, enter HOST1's root password. The script:\n242\t- Generates `/root/.ssh/jayred365_rsync_automation` (ed25519)\n243\t- Copies the public key to HOST1's `~/.ssh/authorized_keys`\n244\t- Updates `host2.conf` with the key path\n245\t\n246\t---\n247\t\n248\t### ── Run on HOST1 ──────────────────────────────────────────────────────────\n249\t\n250\t```bash\n251\tcd /boot/config/plugins/varaverk\n252\tbash Partnership/ssh_setup.sh\n253\t```\n254\t\n255\tEnter HOST2's root password when prompted. Same steps, opposite direction.\n256\t\n257\t---\n258\t\n259\t### ── Verify both directions ────────────────────────────────────────────────\n260\t\n261\t```bash\n262\t# On HOST1 — SSH to HOST2 should succeed without a password\n263\tssh -i /root/.ssh/gmer4lfe_rsync_automation root@unRAID-Jayred365 \"hostname\"\n264\t# Expected: unRAID-Jayred365\n265\t\n266\t# On HOST2 — SSH to HOST1 should succeed without a password\n267\tssh -i /root/.ssh/jayred365_rsync_automation root@unRAID-Gmer4Lfe \"hostname\"\n268\t# Expected: unRAID-Gmer4Lfe\n269\t```\n270\t\n271\tIf either fails:\n272\t```bash\n273\t# Check status and fingerprints\n274\tbash Partnership/ssh_setup.sh --status\n275\t\n276\t# Re-run with --force to regenerate and re-copy\n277\tbash Partnership/ssh_setup.sh --force\n278\t```\n279\t\n280\t---\n281\t\n282\t### ── Tailscale SSH note ──────────────────────────────────────────────────────\n283\t\n284\tThe scripts SSH via Tailscale hostnames, not local IPs. The first connection to a\n285\tnew host requires accepting the host key. Accept it now so scripts never block on an\n286\tinteractive prompt:\n287\t\n288\t```bash\n289\t# On HOST1\n290\tssh -o StrictHostKeyChecking=accept-new root@unRAID-Jayred365 \"echo ok\"\n291\t\n292\t# On HOST2\n293\tssh -o StrictHostKeyChecking=accept-new root@unRAID-Gmer4Lfe \"echo ok\"\n294\t```\n295\t\n296\t---\n297\t\n298\t## ━━━ STEP 5: SCHEDULER SETUP ━━━\n299\t\n300\t**Everything runs through the Varaverk plugin's built-in scheduler** — no User Scripts\n301\tentries needed. The plugin handles all triggers natively:\n302\t\n303\t- **Array start** → `Plugin/unraid/event/disks_mounted/array_start_jobs` fires `array_started.sh`\n304\t- **Array stop** → `Plugin/unraid/event/disks_unmounting/array_stop_jobs` fires `array_stopping.sh`\n305\t- **Cron** → `Plugin/unraid/event/disks_mounted/rebuild_cron` rebuilds the cron file from `schedule.json` on every boot\n306\t\n307\tConfigure via the Varaverk plugin Scheduler tab (or edit `schedule.json` directly).\n308\tIndividual scripts are never scheduled — only orchestrators.\n309\t\n310\t---\n311\t\n312\t### ── Schedule (Varaverk Scheduler) ───────────────────────────────────────────\n313\t\n314\t| Script | Event/Cron | Purpose |\n315\t|--------|-----------|---------|\n316\t| `array_started` | `array_start` | All array startup scripts in order |\n317\t| `array_stopping` | `array_stop` | Ordered graceful shutdown |\n318\t| `transcode_management` | `*/7 * * * *` | Cleanup then manager — order critical |\n319\t| `watchdog_orchestrator` | `*/15 * * * *` | resource → docker → system → stability |\n320\t| `critical_sync_maintenance` | `*/30 * * * *` | auth + Emby dirty sync + partnership |\n321\t| `intermediate_sync_maintenance` | `0 */4 * * *` | arr sync + failed recovery |\n322\t| `daily_sync_maintenance` | `0 1 * * *` | full daily maintenance window |\n323\t| `weekly_sync_maintenance` | `30 2 * * 0` | clean sync + image updates |\n324\t| `monthly_maintenance` | `0 0 15 * *` | ZFS scrub, SMART tests (uptime-gated) |\n325\t\n326\tConcurrent runs are prevented by `acquire_lock`. For the complete schedule see\n327\t[README-Orchestrators.md](Orchestrators/README-Orchestrators.md).\n328\t\n329\t---\n330\t\n331\t## ━━━ STEP 6: INITIAL SYNC ━━━\n332\t\n333\tBefore enabling fallback, HOST2 needs a copy of HOST1's data. This first sync\n334\twill take a while depending on library size.\n335\t\n336\t---\n337\t\n338\t### ── Dry run first ────────────────────────────────────────────────────────────\n339\t\n340\t```bash\n341\t# On HOST1 — preview what would sync, check paths and profiles\n342\tbash /boot/config/plugins/varaverk/Orchestrators/daily_sync_maintenance.sh --dry-run --log\n343\t```\n344\t\n345\tReview the output. Check that:\n346\t- Share paths resolve correctly on both sides\n347\t- No pre-flight errors (disk space, rootfs, connectivity)\n348\t- Profiles look correct for each share\n349\t\n350\t---\n351\t\n352\t### ── Run the initial sync ─────────────────────────────────────────────────────\n353\t\n354\t```bash\n355\t# On HOST1 — run the full daily sync (this includes rsync for all DAILY_SYNC_SHARES)\n356\tbash /boot/config/plugins/varaverk/Orchestrators/daily_sync_maintenance.sh --log\n357\t```\n358\t\n359\tThis will take longer than future nightly runs — it is transferring everything for\n360\tthe first time. Monitor progress via the Varaverk log viewer or:\n361\t\n362\t```bash\n363\t# Watch transfer in real time\n364\twatch -n5 'ls -lh /mnt/user/Movies/ | tail -5'\n365\t```\n366\t\n367\tAfter it finishes, also run the critical sync to populate auth and Emby state:\n368\t```bash\n369\tbash /boot/config/plugins/varaverk/Orchestrators/critical_sync_maintenance.sh --log\n370\t```\n371\t\n372\t---\n373\t\n374\t## ━━━ STEP 7: ENABLE FALLBACK ━━━\n375\t\n376\t`fallback.sh` is disabled by default — it should only be enabled when both servers\n377\tare correctly synced and the data on HOST2 is current.\n378\t\n379\t---\n380\t\n381\t### ── Enable in master.conf ────────────────────────────────────────────────────\n382\t\n383\t```bash\n384\tFALLBACK_ENABLED=true\n385\t```\n386\t\n387\tCommit and pull on both servers so both pick up the change.\n388\t\n389\t---\n390\t\n391\t### ── Fallback starts automatically on array start ────────────────────────────\n392\t\n393\t`array_started.sh` launches `fallback.sh` as a background process. Once\n394\t`FALLBACK_ENABLED=true` is in master.conf, restarting the array starts it.\n395\t\n396\tTo start it now without a reboot:\n397\t```bash\n398\tbash /boot/config/plugins/varaverk/Fallback/fallback.sh &\n399\t```\n400\t\n401\t---\n402\t\n403\t### ── Verify it is running ──────────────────────────────────────────────────────\n404\t\n405\t```bash\n406\tpgrep -a -f fallback.sh\n407\t# Should show the running process\n408\t```\n409\t\n410\tCheck current state:\n411\t```bash\n412\tcat /boot/config/fallback_state.db\n413\t# state=NORMAL — both servers up\n414\t```\n415\t\n416\t---\n417\t\n418\t## ━━━ STEP 8: TEST FALLBACK ━━━\n419\t\n420\tBefore relying on the system, confirm it actually triggers. `fallback_test.sh`\n421\tsimulates an outage using `iptables` — no real downtime, no real data changes.\n422\t\n423\t```bash\n424\t# On HOST2 — dry run first (sees the sequence, no container starts)\n425\tbash /boot/config/plugins/varaverk/Fallback/fallback_test.sh --dry-run --log\n426\t\n427\t# When ready — real test (uses iptables to simulate HOST1 unreachable)\n428\tbash /boot/config/plugins/varaverk/Fallback/fallback_test.sh --log\n429\t```\n430\t\n431\tThe test runs in phases — blocks HOST1's Tailscale IP, waits for `fallback.sh` to\n432\tdetect it and start Tier 1 containers, then unblocks and waits for handback.\n433\t\n434\tSee [Manual-Fallback.md](Fallback/Manual-Fallback.md) for what each phase does and\n435\thow to interpret the output.\n436\t\n437\t---\n438\t\n439\t## ━━━ STEP 9: PARTNERSHIP ONBOARD ━━━\n440\t\n441\tThe partnership is the optional layer that shares HOST1's auth stack with HOST2 —\n442\tNPM, Authelia, LLDAP, MariaDB, Redis all warm on HOST2, managed centrally from HOST1.\n443\t\n444\tSkip this step if HOST2 does not need HOST1's auth stack (it runs fully independent auth).\n445\t\n446\t---\n447\t\n448\t### ── Prerequisites ────────────────────────────────────────────────────────────\n449\t\n450\tBefore running partnership onboard:\n451\t\n452\t- Steps 1–8 complete on both servers\n453\t- HOST1's auth containers are running and healthy\n454\t- `PARTNERSHIP_AUTH_STACK` in `host1.conf` lists the auth container XML templates\n455\t- `PARTNERSHIP_ARR_STACK` in `host1.conf` lists arr container XML templates (if sharing arrs)\n456\t- FolderView3 plugin installed on both servers (if using `PARTNERSHIP_FOLDERVIEW3=true`)\n457\t\n458\t---\n459\t\n460\t### ── Run on HOST2 first (mirror) ────────────────────────────────────────────\n461\t\n462\tHOST2 runs first — it generates its SSH key and waits. The owner completes setup remotely.\n463\t\n464\t```bash\n465\t# On HOST2\n466\tbash /boot/config/plugins/varaverk/Partnership/partnership_onboard.sh --dry-run --log\n467\t# Review output, then:\n468\tbash /boot/config/plugins/varaverk/Partnership/partnership_onboard.sh --log\n469\t```\n470\t\n471\tHOST2's work is done after Step 1 (SSH key setup). The rest happens on HOST1.\n472\t\n473\t---\n474\t\n475\t### ── Run on HOST1 (owner) ────────────────────────────────────────────────────\n476\t\n477\t```bash\n478\t# On HOST1\n479\tbash /boot/config/plugins/varaverk/Partnership/partnership_onboard.sh --dry-run --log\n480\t# Review output — this will deploy auth + arr stacks to HOST2 remotely\n481\tbash /boot/config/plugins/varaverk/Partnership/partnership_onboard.sh --log\n482\t```\n483\t\n484\tThe owner path deploys containers to HOST2 via SSH, configures WebUI targets, runs\n485\tan initial arr library sync, and writes the partnership state files on both servers.\n486\t\n487\tFor full detail on what each step does see\n488\t[Manual-Partnership.md](Partnership/Manual-Partnership.md).\n489\t\n490\t---\n491\t\n492\t## ━━━ STEP 10: VERIFY THE ECOSYSTEM ━━━\n493\t\n494\t---\n495\t\n496\t### ── Check script output ────────────────────────────────────────────────────\n497\t\n498\tRun the main orchestrators manually with `--log` to see verbose output:\n499\t\n500\t```bash\n501\t# Watchdogs — should find everything healthy\n502\tbash Orchestrators/watchdog_orchestrator.sh --log\n503\t\n504\t# Daily sync — dry run to confirm share paths and profiles\n505\tbash Orchestrators/daily_sync_maintenance.sh --dry-run --log\n506\t```\n507\t\n508\t---\n509\t\n510\t### ── Check fallback state ────────────────────────────────────────────────────\n511\t\n512\t```bash\n513\tcat /boot/config/fallback_state.db\n514\t# Expected:\n515\t# state=NORMAL\n516\t# fallback_start=0\n517\t# handback_strikes=0\n518\t# tier2_started=false\n519\t# tier3_started=false\n520\t# tier4_started=false\n521\t```\n522\t\n523\t---\n524\t\n525\t### ── Check sync logs ─────────────────────────────────────────────────────────\n526\t\n527\tBandwidth monitor records every sync:\n528\t```bash\n529\t# Latest transfer log\n530\tls -lt /boot/config/plugins/varaverk/data/bandwidth_monitor/\n531\tcat /boot/config/plugins/varaverk/data/bandwidth_monitor/\n532\t```\n533\t\n534\t---\n535\t\n536\t### ── Run status on key scripts ─────────────────────────────────────────────\n537\t\n538\t```bash\n539\tbash Fallback/fallback.sh --status\n540\tbash Partnership/ssh_setup.sh --status\n541\tbash Monitors/cert_monitor.sh --status\n542\tbash Watchdogs/resource_watchdog.sh --status\n543\t```\n544\t\n545\t---\n546\t\n547\t## ━━━ COMMON FIRST-RUN ISSUES ━━━\n548\t\n549\t---\n550\t\n551\t### ── Script exits silently / does nothing ────────────────────────────────────\n552\t\n553\t```bash\n554\t# Run with --log to see verbose output\n555\tbash