Commit Graph
46 Commits
Author SHA1 Message Date
Gmer4Lfe 28d434b76c Propagate new fallback model to all remaining script references
All FALLBACK_${MY_ID}_COVERS_${REMOTE_ID}_TIER* references updated to
FALLBACK_${REMOTE_ID}_TIER* across fallback_test.sh, partnership_manager.sh,
docker_update.sh, mesh_monitor.sh, and monitor.php. mesh_monitor.sh drops
the inner covering-host loop — tier data now lives in the covered host's own
conf so no cross-host scan is needed. monitor.php reads from the covered
host's conf file rather than the local host's.
2026-06-19 17:49:26 -04:00
Gmer4Lfe f42ecc8464 Add docker actions, arr profile enforcer, monitor caching, and web file symlink
Web files now served via symlink to the git repo so git pull changes survive
reboots without rebuilding the txz. Also includes: docker pull/rebuild/restart
with live log streaming, arr_profile_enforcer for Sonarr/Radarr quality
profiles, monitor page cache fix (background writer now in cron), and
ARR_KIDS/SONARR/RADARR profile name vars in master.conf.
2026-06-19 11:09:40 -04:00
Gmer4Lfe 978a5865a7 Add webhook setup, media seed, and listener start to onboarding (steps 9b-9d) 2026-06-14 21:40:25 -04:00
Gmer4Lfe 5a7d5b5005 Add share_setup.sh — create missing mirror shares during onboarding (pool-aware, idempotent) 2026-06-14 14:53:51 -04:00
Gmer4Lfe 3964f6fb46 audit echo vs log across all scripts — outcomes always visible, verbose for per-item loops 2026-06-14 12:40:15 -04:00
Gmer4Lfe fdef61cc25 Fix container updates: rebuild on new image, consolidate update scripts
Weekly sync window was pulling images but using docker start after rsync,
which ignores the new digest. Containers in the emby/critical-data profiles
(Emby, Mariadb, Redis) never actually landed on updated images.

docker_update_remaining.sh merged into docker_update.sh --remainder, which
already had better exclusion logic. Added WEEKLY_REMAINING_UPDATES toggle
and WEEKLY_RESTART_CONTAINERS exclusion to remainder mode.

Onboard now runs docker_network_connect.sh on mirror before deploying stacks.
2026-06-14 10:58:19 -04:00
Gmer4Lfe ba3eed39e3 Sync READMEs and manuals to current script headers
Fix discrepancies found by auditing docs against headers:
- stability_watchdog.sh: wrong variable name (STRIKES → STRIKE_LIMIT) and wrong default (2hr → 12hr) in header
- Watchdogs manual: REBOOT_WINDOW_HRS example value was 2, should be 12
- Transcodes README: transcode_cleanup/manager table said "Every 3 min", should be "Every 7 minutes"
- Tools manual: fallback_state_reset.sh still marked "not yet built" — rewrite section to reflect current script
- Tools manual: claude_startup --setup flag doesn't exist; modes were inverted — fix and add --launch
- Tools README + manual: add docker_prune_images.sh (existed but undocumented)
- Tools manual: add play_state_sync.sh --full to flag reference
- Orchestrators README: arrs_failed_stalled_recovery runs via intermediate_sync (every 4hr), not standalone every 6hr
- Plugin README: add build.sh to scripts table
- Partnership README + manual: add partnership_transfer.sh and onboard_cancel.sh to script tables and flag reference
2026-06-14 02:01:27 -04:00
Gmer4Lfe 181114aed5 Header/code audit fixes across all core scripts
Behavior fixes:
- docker_daily_restart.sh: implement RESTART_VERIFY_WAIT sleep before
  verify_running() — variable was logged as config but never applied
- cert_monitor.sh: --log now shows expiry date and days remaining for
  healthy domains in the per-domain summary; problems always show details

Header corrections:
- downloaders_reset.sh: fix frequency contradiction (15 min → 30 min)
- lidarr_cleanup.sh: Emby scan triggers on actual deletions, not every run
- ssh_setup.sh: document --local-only flag in USAGE section
- zfs_memory_snapshot.sh: clarify dry-run skips log write in PURPOSE
- ramdisk_stop.sh / ramdisk_setup.sh: document TRANSCODE_STATE_FILE var
- smart_long_test.sh: remove SMART_TEMP_WARN/CRIT — not used in code
2026-06-14 01:40:35 -04:00
Gmer4Lfe b4d6866f40 Polish before first GitHub push: consistent adapter usage and naming
stability_watchdog.sh mover check now uses platform_is_mover_running()
instead of raw pgrep, matching server_reboot.sh.

check_unraid_version_parity() renamed to check_os_version_parity() —
the function internals were already platform-neutral after the adapter
refactor; the name was the last Unraid artifact.
2026-06-14 01:30:16 -04:00
Gmer4Lfe ee5a07be1c Remove remaining OS-specific hardcodes from core scripts
OS version reads now go through platform_get_os_version() and
platform_os_version_probe_cmd() instead of grepping /etc/unraid-version directly.

STATE_DIR fallbacks to /boot/config removed — STATE_DIR is always set by
load_config.sh and the fallback encoded a platform-specific path.

Setup DB path references use platform_setup_db_path() instead of the
VARAVERK_SETUP_FILE/-/boot/config compound fallback.

DOCKER_APPDATA_BASE default removed from arr_sync.sh — the adapter sets it.
2026-06-14 01:22:02 -04:00
Gmer4Lfe 3c900ae5c1 Resolve remote storage path from host*.conf for true cross-platform SSH support
detect_hosts() now sets REMOTE_STORAGE_PATH from HOST*_STORAGE_PATH in the
remote's conf instead of calling platform_storage_path() locally. SSH commands
that check the remote's storage (check_remote_array, check_remote_disks,
check_both_healthy) use $REMOTE_STORAGE_PATH so they work correctly when the
remote is a different platform with a different storage root.
2026-06-14 01:09:53 -04:00
Gmer4Lfe 27bfc21cb0 Platform-agnostic refactor: eliminate OS-specific hardcodes from core scripts
All bash scripts are now platform-neutral. Unraid-specific paths, commands,
and service checks moved to Plugin/unraid/adapter.sh. Core scripts call
platform_*() functions exclusively — no direct OS paths in runtime logic.

New adapter functions: platform_storage_path, platform_webui_install_path,
platform_scripts_dir_probe_cmd, platform_setup_db_path, platform_storage_healthy,
platform_is_service_enabled, platform_get_temp_thresholds, platform_disk_states_path,
platform_rebuild_container, platform_push_conf, platform_push_setup_state,
platform_get_templates_dir, platform_send_os_notification.

Partnership services stack (Emby/Jellyfin/Seerr/SeerrFin) added as third
onboarding stack alongside auth and arr stacks.
2026-06-14 00:59:19 -04:00
Gmer4Lfe 9c3ace95a7 Auth stack certs tab, arrs db fallbacks, cert monitor cache, conf parser fix
- Auth stack: fold cert monitor into Auth Stack page as fourth tab (Certs);
  remove standalone cert page and top-level tab
- cert_monitor.sh: write JSON status cache to State_Files/cert_status.json
  after each run; expose per-domain days/expiry via _CERT_DAYS/_CERT_EXPIRY globals
- api/cert.php: new — serves cached cert status; falls back to configured
  domains as UNKN when no cache exists; POST action=run triggers live check
- arrs db fallbacks: vv_arr_cleanup_stats/discovery_stats/recovery_stats now
  read from data/*.db files when log JSON files don't yet exist
- config.php vv_conf_vars(): unescape bash \$ → $ so passwords with dollar
  signs read correctly from conf files
- host1.conf: fill in HOST1_NPM_USER/PASS and HOST1_LLDAP_USER/PASS
- Partnership adapter pattern: Unraid-specific container logic extracted to
  Plugin/unraid/Partnership/; platform-agnostic structure stays in Partnership/
- First-run wizard: uniform multi-step flow for all hosts; HOST2 pull moved
  to checklist; auto SSH keygen and API key creation on save
- api/checklist.php: live setup checklist with pull_master action
- Fullscreen toggle: hide Unraid header/menu; state persists via localStorage
2026-06-05 23:17:30 -04:00
Gmer4Lfe 369a9e6c19 Platform adapter: rename System_Essentials, add Plugin/unraid/adapter.sh, wire call sites
- Rename unRAID_Essentials/ → System_Essentials/ (git detects as rename)
- Add Plugin/unraid/adapter.sh: 13 platform_*() functions providing OS-agnostic API
  for storage health, service management, mover, user scripts, notifications,
  disk temps, and platform command validation
- Update load_config.sh: detect PLATFORM (unraid/truenas/unknown), export SCRIPTS_DIR,
  auto-source Plugin/$PLATFORM/adapter.sh after common.sh
- Wire all call sites: replace direct rc.d, pgrep/pkill, var.ini, dynamix.cfg,
  disks.ini, and validate_unraid_cmd calls with platform_*() functions across
  watchdogs, orchestrators, and System_Essentials scripts
- Update all documentation: rename refs, update webgui escalation logic,
  add platform adapter section to Plugin README, update main README with
  portability vision and corrected self-healing stack description
2026-06-04 18:14:34 -04:00
Gmer4Lfe 0cb429222a Add storage mode migration: internal NVMe vs USB flash, Settings tab, fix hardcoded /boot/ paths 2026-06-03 17:40:35 -04:00
Gmer4Lfe cb7d7a688d rename: failover → fallback throughout codebase — this system is Fallback, state is FALLBACK; preserve conventional failover comparisons 2026-06-01 20:04:17 -04:00
Gmer4LfeandClaude Sonnet 4.6 fb0530deba Consolidate all paths to plugin flash dir, fix watchdog 7.3 triggers
- Move SCRIPTS_DIR/DATA_DIR/STATE_DIR from appdata to /boot/config/plugins/varaverk
- All state files now in STATE_DIR (no more /tmp or /boot/config root writes)
- Bootstrap: Gitea-first clone with GitHub fallback, no array dependency
- varaverk.cfg seeded with Gitea connection settings
- .gitignore: add State_Files/, varaverk.cfg, varaverk-*.txz
- Partnership/transcode/fallback scripts use STATE_DIR variables
- PHP config.php: DATA_DIR/STATE_DIR constants, VV_SETUP_STATE_FILE dynamic
- deploy.sh PROD_ROOT updated to plugin flash dir

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 13:30:20 -04:00
Gmer4Lfe 9191a54637 Fix ssh_setup.sh: HOST_CONF path missing Configurations/ subdirectory
Was looking for host1.conf at SCRIPTS_ROOT/host1.conf instead of
SCRIPTS_ROOT/Configurations/host1.conf — causing the 'host1.conf not found'
warning and leaving HOST1_SSH_KEY unwritten after key generation.
2026-05-30 21:31:46 -04:00
Gmer4Lfe 2989cefb79 Redesign Cancel/Delete Keys in Partnership Actions
onboard_cancel.sh:
- --direction=h1 (default): remove HOST1's key from HOST2 + delete local pair
- --direction=h2: remove HOST2's key from HOST1's authorized_keys (by hostname match)
- --direction=both: both directions (used by Cancel button at phase 1)

pages/partnership.php:
- Cancel button: phase 1 only, runs --direction=both (full undo)
- Delete Keys (🗑): available at all phases, expands inline panel showing
  HOST1→HOST2 and HOST2→HOST1 as separate removal buttons with descriptions
- _vvDeleteKeys toggle state persists across polls alongside _vvOnboarding
- vvPtShowDeleteKeys / vvPtHideDeleteKeys top-level toggle fns
- vvPtDeleteH1 / vvPtDeleteH2 targeted removal fns
2026-05-30 21:29:20 -04:00
Gmer4Lfe 1956b6b068 Fix: local keyword outside function in --local-only block
flag_key was declared with 'local' at script top level, which is only
valid inside bash functions. This left flag_key empty, writing '=true'
(malformed) to setup.db instead of 'HOST1_LOCAL_DONE=true'.
Changed to a plain variable assignment.
2026-05-30 21:20:56 -04:00
Gmer4Lfe 95fd38799d Phase 1: resilient SSH handling + Push Conf button
partnership_onboard.sh (--phase1-only):
- Test if SSH already works first (BatchMode) — skip ssh-copy-id if yes
- If ssh-copy-id fails (no TTY in background): soft-fail, generate key locally,
  print pubkey for manual install, write HOST2_KEY_READY=true, run local setup
- Only skip conf push if SSH truly unavailable; otherwise proceeds normally
- Exit 0 with clear manual-step instructions instead of hard failure

include/partnership.php:
- Read HOST2_KEY_READY from setup.db; expose as key_ready on remote nodes

pages/partnership.php:
- Node card: amber "Key generated · install on HOST2 then Push Conf" badge
  when key_ready=true and phase=0
- "Push Conf (key installed)" button appears when key_ready=true — runs
  --phase1-only --skip-ssh (assumes user manually did ssh-copy-id)
- vvPtPushConf() top-level function
2026-05-30 20:27:30 -04:00
Gmer4Lfe 7d4994f317 Local onboard: set PARTNERSHIP_ENABLED=true in master.conf + push
When HOST1 completes its local setup (--onboard --local-only), enable the
partnership in master.conf so sync scripts start functioning immediately.
Pushes the updated master.conf to all listed hosts via vv_push_master_conf().
2026-05-30 20:00:54 -04:00
Gmer4Lfe d1ee880586 HOST1 completes local onboard independently during Phase 1
partnership_manager.sh:
- --local-only flag for --onboard: skips remote pre-flight, WebUI reconfig,
  state push; runs FolderView3 folder creation + writes HOST1_LOCAL_DONE=true
  to setup.db + pushes it; exits without writing ACTIVE (partnership not yet
  established until Phase 2)

partnership_onboard.sh:
- Phase 1 exit now runs partnership_manager.sh --onboard --local-only after
  SSH + conf push, so HOST1 is fully configured on its own side before Phase 2

include/partnership.php:
- vv_pt_nodes() reads HOST1_LOCAL_DONE from setup.db; exposes as local_done
  on the self node (null for remote nodes)

pages/partnership.php:
- Self node card shows green "Local setup complete · waiting for partner"
  badge when local_done=true
- Actions: "Complete HOST1 Setup" button appears when HOST1 hasn't run local
  setup yet but HOST2 is already at phase 1 — lets user trigger it manually
- vvPtLocalSetup() top-level function calls partnership_manager.sh via run.php
2026-05-30 19:55:56 -04:00
Gmer4Lfe dfc24e37c0 Add Phase 1 cancel: remove SSH keys from both sides + reset state
onboard_cancel.sh:
- SSHes HOST2 while key still valid: removes HOST1 pubkey from authorized_keys
  and clears HOST2_PHASE* from setup.db on the remote side
- Deletes local private + public key pair
- Clears HOST2_PHASE* from local setup.db
- Graceful if HOST2 unreachable (local side still cleaned up)

partnership.php:
- Cancel button (warn style, slightly dimmed) at phase 0 and phase 1
- vvPtCancel() top-level function, calls onboard_cancel.sh via run.php
2026-05-30 18:33:56 -04:00
Gmer4Lfe a5755921ab Two-phase onboard + partner card enhancements
partnership_onboard.sh:
- --phase1-only (OWNER): SSH key exchange + conf push only; safe to run
  before HOST2 has Varaverk installed; writes HOST2_PHASE1_DONE to setup.db
- --phase2-only (OWNER): skips SSH, runs Steps 2-9 (containers, arr stack,
  onboard, arr sync, conf push); writes HOST2_PHASE2_DONE to setup.db
- Mirror path: after SSH setup, SSHes OWNER and fires --phase2-only in
  background (nohup); reads OWNER's SCRIPTS_DIR from varaverk.cfg first;
  falls back to manual instruction if SSH fails

partnership.php (include): vv_pt_nodes() reads setup.db and exposes
onboard_phase (0/1/2) per remote node

partnership.php (page): phase-aware Actions section per partner host;
node cards show phase badge (not provisioned / awaiting onboard / done);
Phase 1 and Phase 2 buttons call run.php with extra_args

monitor.php: partner card logo (top-right), CPU thread count, RAM total GB,
unRAID version row in remote stats grid
2026-05-30 18:12:49 -04:00
Gmer4Lfe d09ea11694 Onboard Step 9: push master.conf to all listed hosts on completion
After Step 8 (arr bootstrap), owner calls vv_push_master_conf() and
vv_push_setup_state() via php -r. SSH is established at this point so
the probe passes and all listed hosts get the authoritative master.conf
immediately — no manual save required.

Skipped if onboard failed, dry-run mode, or php unavailable (with
fallback instruction). Result shown in onboard summary as Step 9.
2026-05-30 16:14:07 -04:00
Gmer4Lfe da0c303f22 Push master.conf to all partners immediately on raw-editor save
vv_push_master_conf() in config.php: finds all remote HOST* entries,
resolves Tailscale IPs, SCPs master.conf to each.
rawconf.php calls it after a successful write when file=master.conf.
Scheduler save button shows sync status: "✓ Saved · synced to HOST2"
or "✓ Saved · push failed: HOST2" on error.

Also fix path bug in partnership_transfer.sh — was SCPing to
appdata root instead of Configurations/.
2026-05-30 14:21:02 -04:00
Gmer4Lfe 0fe99cf2a9 rename: appdata/varaverk → appdata/Varaverk (capital V throughout) 2026-05-30 10:22:51 -04:00
Gmer4Lfe 921900b23b rename appdata folder and repo from unraid_scripts to varaverk
- TARGET_DIR, DATA_DIR: /mnt/user/appdata/unraid_scripts → /mnt/user/appdata/varaverk
- GITEA_REPO_PATH: Varaverk/Unraid_Scripts.git → Varaverk/varaverk.git
- DEV_ROOT: .../Development/Unraid_Scripts → .../Development/varaverk
- deploy hook path updated in .claude/settings.json
- All script comments, docs, and manual paths updated throughout
2026-05-30 10:07:03 -04:00
Gmer4Lfe 5e4f510a42 Fix stale 15-minute critical sync references — corrected to 30 minutes
critical_sync_maintenance.sh runs every 30 minutes (*/30 cron).
Three references in README.md and Partnership/README-Partnership.md
still said 15 minutes from an earlier schedule.
2026-05-23 15:44:39 -04:00
Gmer4Lfe 96ff9b8f95 feat(partnership): extract transfer into partnership_transfer.sh
Follows the same extraction pattern as partnership_offboard.sh.
partnership_manager.sh --transfer now exec-forwards to the new script.
Legacy inline block preserved under `if false` for reference.

Fixes in partnership_transfer.sh vs the original inline code:
- WebUI reconfiguration corrected: SSH to new owner (mirror) to set localhost;
  reconfigure_local_webuis() on current server to point to new owner IP
- master.conf remote update uses scp push instead of self-SSH with wrong path
- SSH key uses $SSH_KEY directly (was using undefined {HOST*}_SSH_KEY expansion)
2026-05-23 09:42:39 -04:00
Gmer4Lfe fda853eed8 fix 9 bugs in partnership offboard flow
partnership_manager.sh:
- fix auto-offboard threshold: × 96 → × 48 (was triggering at 2× configured days)
- fix do_final_sync(): use CRITICAL_SYNC_SHARES if configured; fall back to hardcoded paths
- replace inline --check offboard blocks with background partnership_offboard.sh calls:
  • owner path was missing remote container cleanup (steps 8+9) and blocklist
  • mirror path was missing blocklist, SSH revocation, CRITICAL_RSYNC_ENABLED=false
  • local grace_seconds outside a function (bash error) eliminated with the block removal
  • 6hr Tailscale grace sleep was blocking the cron for the full grace window

partnership_offboard.sh:
- reorder owner path: state write moved from step 5 to step 10 (last) so re-running
  after a mid-offboard crash restarts from scratch instead of exiting as INACTIVE;
  state is now written before the Tailscale grace sleep so --check stops re-triggering
- add FolderView3 remote folder cleanup in step 7 (mirror's fallback folder persisted)
- track step 1+2 outcome with STEP_STOP_OK / STEP_SYNC_OK; summary now reflects actual
  result instead of hardcoding  regardless of dry-run or failure
- update header step list to match new step order
2026-05-23 09:24:34 -04:00
Gmer4Lfe ee086f309e rename appdata-Failover→appdata-Fallback in docs; critical sync 15→30min; transcode 3→7min
appdata-Failover → appdata-Fallback:
  All .md files updated — scripts and confs already used appdata-Fallback.
  Share was renamed; docs were the only stale references.

critical_sync_maintenance.sh: */15 → */30, 96/day → 48/day:
  critical_sync_maintenance.sh header, master.conf, host1/2.conf comments,
  Orchestrators README, User Script Plug-in README, Partnership README/manual/script,
  Docker_Essentials README/downloaders_reset.sh, Fallback README,
  weekly_sync_maintenance.sh, user_script_plug-in.sh all updated.

rsync_emby_failover User Script superseded:
  Emby dirty sync was already in HOST1_CRITICAL_SYNC_SHARES — it now runs inside
  critical_sync_maintenance.sh every 30min alongside auth sync and partnership check.
  Section in User Script Plug-in README marked for deletion from plugin.

transcode_management.sh: */3 → */7:
  transcode_management.sh, transcode_manager.sh, transcode_cleanup.sh headers,
  Transcodes README/manual, Orchestrators README, User Script Plug-in README,
  user_script_plug-in.sh all updated.
2026-05-22 22:22:21 -04:00
Gmer4Lfe 36190c965a Add Gitea SSH setup script and integrate Gitea into partnership/failover
Partnership/gitea_ssh_setup.sh: generates ed25519 keypair, registers public key
  with Gitea API (POST /api/v1/user/keys), tests SSH connection, writes ~/.ssh/config
  entry. Idempotent — skips steps already done. --force regenerates and re-registers.
  Resolves Gitea endpoint from container IP first, falls back to GITEA_DOMAIN.

common.sh: alias HOST*_GITEA_API_TOKEN → GITEA_API_TOKEN in detect_hosts().
  Add to detect_hosts() doc comment.

master.conf: add GITEA_HTTP_PORT=3000 for API endpoint construction.

host1.conf: add HOST1_GITEA_API_TOKEN (fill in from Gitea Settings → Applications).
  Add my-Gitea.xml to HOST1_PARTNERSHIP_AUTH_STACK — onboard pushes it to HOST2.

host2.conf: add Gitea to FALLBACK_HOST2_COVERS_HOST1_TIER1 — starts immediately
  when HOST1 goes down, making the source of truth reachable independently of
  HOST1's auth stack.
2026-05-22 18:15:25 -04:00
Gmer4Lfe 95151c2278 Watchdogs/ folder + host conf rename
Move all watchdog scripts to a dedicated Watchdogs/ folder:
  Docker_Essentials/docker_watchdog.sh   → Watchdogs/
  unRAID_Essentials/system_watchdog.sh   → Watchdogs/
  unRAID_Essentials/resource_watchdog.sh → Watchdogs/
  Orchestrators/watchdog_orchestrator.sh → Watchdogs/
  Tools/watchdog_skip_list_manager.sh    → Watchdogs/

Rename host config files:
  master_host1.conf → host1.conf
  master_host2.conf → host2.conf

Update all references across the ecosystem:
  master.conf: WATCHDOG_ORCHESTRATOR_SCRIPTS paths → Watchdogs/
  load_config.sh: host*.conf glob + all comments
  git_pull_execute.sh: sparse checkout glob + all comments
  Partnership/ssh_setup.sh: HOST_CONF path construction
  user_script_plug-in.sh: all script paths + per-host conf path
  common.sh, README.md, README-User_Script_Plug-in.md: comment refs
  All Partnership, Fallback, Monitors, Transcodes, Tools scripts: comment refs
2026-05-22 17:08:36 -04:00
Gmer4Lfe 86b31256ed Partnership echo/log audit pass + exit trap
Exit trap (_pm_trap_restart_stopped) in partnership_manager.sh: restarts locally
stopped containers if script crashes mid-cleanup; partnership_offboard.sh registers
same trap. echo/log audit: step result lines (stack deployed counts, state writes,
onboard/offboard complete) → echo; check mode ACTIVE/INACTIVE status → echo;
ssh_setup.sh key-exists and install/verify results → echo. Manual-Partnership.md:
added Output Tiers section.
2026-05-21 17:11:05 -04:00
Gmer4Lfe e932936acc Add docker availability check to all scripts that use docker commands
22 scripts used docker without verifying it's present. Consistent with the
pattern already established in Docker_Essentials scripts. Added after acquire_lock
and before detect_hosts in each script's setup section.
2026-05-20 18:35:34 -04:00
Gmer4Lfe 740710e0d0 Add missing acquire_lock to 8 scripts across Docker_Essentials, Rsync, Partnership, Old_Arch
docker_container_stop, docker_update, docker_update_remaining — concurrent Docker
operations on the same containers would conflict; now locked.

rsync.sh — two rsync processes running against the same share simultaneously
would produce incomplete or corrupted mirrors; now locked.

partnership_onboard, ssh_setup — one-shot setup scripts that mutate SSH config and
deploy containers; concurrent runs would produce undefined state; now locked.

Old_Arch_Still_Works: arr_cleanup, continuous_scripts_status — legacy scripts still
sourcing load_config.sh; added lock for consistency even in old-arch context.

partnership_manager.sh intentionally left unchanged — it uses a conditional lock
that excludes read-only "check" mode and "offboard" mode (which delegates to
partnership_offboard.sh, which has its own lock).
2026-05-20 18:03:03 -04:00
Gmer4Lfe e13f2fa14f feat: slskd reconnect guard in downloaders_reset, mass v2 sync
- downloaders_reset: connection check block before slskd API sections;
  triggers PUT /api/v0/server reconnect if disconnected, polls 60s,
  gates Stuck Searches and Dead Transfer Records on SLSKD_CONNECTED
- Sync all modified/new/deleted files from v2 refactor across Docker_Essentials,
  Media, Monitors, Partnership, Rsync, Tools, Transcodes, unRAID_Essentials,
  common.sh, master confs, and new Manual/README docs
2026-05-19 20:00:10 -04:00
Gmer4LfeandClaude Sonnet 4.6 0ae31b5fa6 feat: Tailscale resolution hardening, partnership offboard completion, Emby provisioning
common.sh:
- Add resolve_tailscale_ip() helper — tries `tailscale ip -4` first, falls back to
  parsing `tailscale status` output; handles hosts where MagicDNS short-name resolution
  is not active
- Add PARTNERSHIP_OWN_CONTAINERS alias in detect_hosts()
- Add aliasing for 4 Emby provisioning vars (PARTNERSHIP_PROVISION_EMBY_ADMIN,
  PARTNERSHIP_EMBY_ADMIN_USER, PARTNERSHIP_EMBY_ADMIN_PASS, PARTNERSHIP_EMBY_PORT)

Partnership/partnership_manager.sh:
- Replace 9 bare `tailscale ip -4` calls with resolve_tailscale_ip()
- Add read_remote_conf_var() and read_remote_conf_array() — SSH to mirror, source its
  own load_config.sh + detect_hosts(), return aliased variable; solves sparse-checkout
  problem where HOST1 cannot read master_host2.conf directly
- Add derive_short_name() — strips unraid- prefix, capitalises first char
- Add cleanup_partner_containers() — removes partner containers via FolderView3 folder
  if enabled, else falls back to FALLBACK_*_COVERS_*_TIER* arrays
- Add cleanup_owner_containers_on_mirror() — SSH to mirror, stops and removes containers
  matching *-${OWNER_SHORT} naming convention
- Add start_own_stack() and start_mirror_own_stack() — restart own containers locally
  or on mirror via SSH using PARTNERSHIP_OWN_CONTAINERS
- Add provision_emby_admin() — reads mirror credentials via read_remote_conf_var, checks
  for username collision, creates user + sets password + grants admin policy via Emby API
- Add revoke_emby_admin() — looks up mirror username on local Emby, deletes via REST API
- Wire offboard paths (both mirror-initiated and owner-initiated) to call container
  cleanup and stack restart; update --check finalisation paths accordingly
- Fix write_state_file in --onboard not gated on DRY_RUN (was writing ACTIVE state on
  dry runs)

master_host1.conf:
- Add HOST1_PARTNERSHIP_OWN_CONTAINERS array
- Add partnership Emby provisioning config (toggle + port + per-host credentials)

master_host2.conf:
- Add HOST2_PARTNERSHIP_OWN_CONTAINERS array
- Add HOST2_PARTNERSHIP_EMBY_ADMIN_USER and HOST2_PARTNERSHIP_EMBY_ADMIN_PASS

Tailscale fix applied to:
- Initial_run/ssh_setup.sh (2 callsites)
- unRAID_Essentials/rsync_stop.sh (1 callsite)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 20:09:13 -04:00
Gmer4LfeandClaude Sonnet 4.6 6674dd4c17 Add SSH key revocation to offboard flow
On offboard, neither server should retain SSH access to the other.
Keys are now revoked as the final step before Tailscale removal — after
all state pushes and syncs are complete so no SSH operation is cut short.

Revocation runs in both owner-initiated offboard and when --check
finalises a mirror-requested offboard. Both directions are handled:
  - Our pubkey removed from remote's authorized_keys via SSH (identified
    by key comment keyname@hostname set by ssh_setup.sh at keygen time)
  - Remote's pubkey removed from local authorized_keys (matched by
    @REMOTE_SERVER_NAME suffix — unique across the authorized_keys file)

Non-fatal: SSH failure logs a manual revocation instruction; local
cleanup still runs. Summary shows per-direction revocation status.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 10:21:01 -04:00
Gmer4LfeandClaude Sonnet 4.6 009820e981 refactor: rename failover/HA → fallback across entire codebase
Removes all references to "failover" and "HA" (high availability)
terminology from variable names, config keys, state values, rsync
profile names, directory paths, and user-visible strings.

Mapping:
  FAILOVER_*              → FALLBACK_*
  FAILOVER_HOST*_RUNS_FOR → FALLBACK_HOST*_COVERS
  critical-failover       → critical-fallback
  emby-failover           → emby-fallback
  appdata-Failover/       → appdata-Fallback/
  "FAILOVER" state value  → "FALLBACK"
  failover_start key      → fallback_start
  Failover/ directory     → Fallback/
  failover.sh             → fallback.sh
  failover_state.db       → fallback_state.db
  -Failover folder suffix → -Fallback

State machine: NORMAL | FALLBACK | DARK (unchanged)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 19:28:21 -04:00
Gmer4LfeandClaude Sonnet 4.6 69464ba385 feat(partnership): add blocklist to block access after offboard
After offboard, the former partner's hostname is written to
/boot/config/partnership_blocklist.db. This blocks passive reconnection:
  - --check skips the remote entirely (no auto-reconnect noise)
  - rsync.sh refuses to sync with a blocklisted host
  - --onboard warns about the previous offboard but proceeds,
    and clears the block on success (onboard = deliberate intent)

New --unblock <hostname> mode removes an entry to permit re-onboarding
without running a full --onboard first. --status shows the full blocklist.

Blocklist file survives reboots (/boot/config) and Tailscale reconnections —
application-layer guard that complements Tailscale device removal.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 18:59:29 -04:00
Gmer4LfeandClaude Sonnet 4.6 5526bc1eca v2 post-migration bug fixes, lidarr_missing_art port, and Partnership SSH/FolderView3 enhancements
Bug fixes across the ecosystem after v1→v2 architecture migration and Unraid 7.2.5 upgrade:

- common.sh: fix _alias_array() phantom empty element (removed [@]:-} pattern), fix
  resolve_remote_ip() with Tailscale FQDN lowercase + awk fallback, add FANART/LASTFM
  key aliases in detect_hosts(), global [@]:-} sweep across 10+ scripts
- webgui_restart.sh: fix emhttp detection (pgrep emhttpd) and restart command for 7.2.5
  (/usr/local/sbin/emhttp stop && start — rc.emhttp removed in 7.2.5)
- bandwidth_monitor.sh, continuous_scripts_status.sh: fix 'local' keyword outside function
- backup_verify.sh: fix resolve_remote_ip() called before detect_hosts()
- Orchestrators: fix script display duplication bug in status output (${entry##*/})
- rsync_stop.sh, git_pull_execute.sh, partnership_manager.sh, coffee_report: lowercase all
  tailscale ip -4 call sites to match Tailscale's lowercase device names
- master_host1.conf: fix SSH key path (gmer4lfe_rsync_automation), add FANART/LASTFM keys
- master_host2.conf: add FANART/LASTFM API keys

New: Media/lidarr_missing_art.sh
- Full ecosystem port of standalone Lidarr artwork fetcher
- Fetches missing album art via fanart.tv + Last.fm APIs
- @tsv batch extraction: 1 jq call per API response vs N*albums (8050 albums in 24s)
- HOST guard (HOST1 only), --status, --dry-run, acquire_lock

New: Initial_run/ssh_setup.sh
- Generates {hostname}_rsync_automation ed25519 keypair (skip if exists, --force to regen)
- ssh-copy-id to remote via Tailscale IP, auto-updates master_host*.conf
- --validate mode: strike tracking (SSH_MAX_STRIKES, SSH_STRIKE_RESET_HRS),
  notify at limit — Tailscale-unreachable remote does NOT count as SSH strike

New: Initial_run/partnership_onboard.sh
- Orchestrator: ssh_setup.sh then partnership_manager.sh --onboard in one command

Partnership/partnership_manager.sh: FolderView3 integration
- Derive partner folder name at runtime (strip unraid- prefix case-insensitively)
- --onboard: create {Mirror}-Failover folder with failover tier containers
- --offboard (both paths): stop + rm containers in folder, remove JSON entry
- --check: calls ssh_setup.sh --validate when IP resolves but SSH state empty
- --status: shows FolderView3 folder state and containers inline

master.conf: SSH_MAX_STRIKES, SSH_STRIKE_RESET_HRS, PARTNERSHIP_FOLDERVIEW3,
PARTNERSHIP_FOLDERVIEW3_URL added to Partnership section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 18:20:23 -04:00
FailedProxy ec7de648dc massive update. Master conf split, now modular with a load sceriprt to drive all configs to scripts. with unraid scpecific safeguard tests , and improved standardized ux. including dynamic host detect, who am i who else it there. EVERY SINGLE SCRIPT UPDATED. DEBATING THAT THIS IS ACUALLY V2 2026-05-03 17:16:49 -04:00
FailedProxy 94f90a4579 padded partnership set up 2026-04-29 20:09:28 -04:00