Commit Graph
43 Commits
Author SHA1 Message Date
Gmer4Lfe 1540c62d3a scheduler: smart auto-scroll replaces separate scroll lock checkbox; lidarr: allow single-seed candidates
Auto Scroll now pauses automatically when user scrolls up and resumes
when they reach the bottom — no separate Scroll Lock needed.

Lidarr discovery no longer hard-skips single-seed candidates; they
score low on breadth but can still reach threshold on merit.
2026-05-24 21:40:03 -04:00
Gmer4Lfe 4c559f7791 fix(log/echo): log → info for section-level status across 4 scripts
lidarr_cleanup, sonarr_cleanup, radarr_cleanup:
  container health result, safety layer 1 pass, import scan queued/complete,
  API query, found N items, in-memory map built, item|tracked count,
  tracked% safety check, first-run baseline, root/orphan-age config,
  empty folder cleanup start/done

cert_monitor:
  domains to check count, warn/crit threshold display

Per-item loop lines (Fetching N/M, Still scanning..., OK per-domain)
remain as log() — verbose only.
2026-05-23 09:58:19 -04:00
Gmer4Lfe b76072a318 fix(lidarr_missing_art): log → info for section-level status lines
Section headers (reachable, counts, summaries) were using log() and
invisible without --log. Per-item detail ([n/N] artist/album, fetched
file names) stays log.
2026-05-23 09:50:58 -04:00
Gmer4Lfe 8d8a4fd637 update stale media ownership language across all docs
arr_sync.sh bidirectional union replaced the old split-ownership model.
Old: HOST1 owns Movies/TV/Music, HOST2 owns Anime — each arr manages
different shares to avoid conflicts.
New: arr_sync.sh keeps all arr databases in union. Either server can
download to any share at any time. The union is the source of truth.

Auth stack references left intact — HOST1 still owns auth config
(one-way sync to HOST2). Only media library ownership language updated.

Files changed:
  README.md                          — server block + share annotations
  Fallback/README-Fallback.md        — role line + Split Source of Truth section
  Orchestrators/README-Orchestrators — daily sync share comments
  Rsync/Manual-Rsync.md              — DEFAULT_RSYNC_OPTS comment block
  Media/README-Media.md              — HOST AWARENESS section
2026-05-22 21:35:45 -04:00
Gmer4Lfe 95151c2278 Watchdogs/ folder + host conf rename
Move all watchdog scripts to a dedicated Watchdogs/ folder:
  Docker_Essentials/docker_watchdog.sh   → Watchdogs/
  unRAID_Essentials/system_watchdog.sh   → Watchdogs/
  unRAID_Essentials/resource_watchdog.sh → Watchdogs/
  Orchestrators/watchdog_orchestrator.sh → Watchdogs/
  Tools/watchdog_skip_list_manager.sh    → Watchdogs/

Rename host config files:
  master_host1.conf → host1.conf
  master_host2.conf → host2.conf

Update all references across the ecosystem:
  master.conf: WATCHDOG_ORCHESTRATOR_SCRIPTS paths → Watchdogs/
  load_config.sh: host*.conf glob + all comments
  git_pull_execute.sh: sparse checkout glob + all comments
  Partnership/ssh_setup.sh: HOST_CONF path construction
  user_script_plug-in.sh: all script paths + per-host conf path
  common.sh, README.md, README-User_Script_Plug-in.md: comment refs
  All Partnership, Fallback, Monitors, Transcodes, Tools scripts: comment refs
2026-05-22 17:08:36 -04:00
Gmer4Lfe a4edeab5eb Media echo/log audit pass — 11 scripts + Manual
Remove setup banners from all 8 scripts that had them (discovery scripts
use comment-only setup sections). Move per-item/detail echoes to log in
arr_sync, lidarr_missing_art, lidarr/radarr/sonarr_cleanup (scan progress,
API query status, root/orphan config lines), and per-title lines in
radarr_tmdb_removed / sonarr_tvdb_removed. Fix log→echo on clean-status
summary lines in arrs_failed_stalled_recovery, media_cleaner, and both
removed-title scripts. Add output tiers section to Manual-Media.md.
2026-05-21 16:48:08 -04:00
Gmer4Lfe e932936acc Add docker availability check to all scripts that use docker commands
22 scripts used docker without verifying it's present. Consistent with the
pattern already established in Docker_Essentials scripts. Added after acquire_lock
and before detect_hosts in each script's setup section.
2026-05-20 18:35:34 -04:00
Gmer4Lfe 5c13495a6c Add missing acquire_lock calls to arr_sync, bulk_permissions_repair, container_data_export
arr_sync.sh had acquire_lock documented in OPERATIONAL SAFEGUARDS header but the
call was never present in the code — two sync instances could run simultaneously.
bulk_permissions_repair.sh and container_data_export.sh had no lock at all.

All three are now consistent with every other script in Media/ and Tools/.
2026-05-20 17:55:18 -04:00
Gmer4Lfe 649a3b7b62 Docs pass: update headers and READMEs for all new scripts
Script headers (6 files):
- Add OPERATIONAL SAFEGUARDS section to all discovery + emby sync scripts
- Add STATE FILES section to all 3 discovery scripts
- Add CONFIGURATION section to emby_to_*_sync.sh
- Rename USAGE → RUNTIME MODES in emby_to_*_sync.sh (doc standard)
- Fix SONARR_DISCOVERY_MONITOR_MODE default doc: "future" → "all"

Media/README-Media.md:
- Remove "WIP prototype" label; describe all 3 discovery scripts
- Add discovery scripts to scripts table and HOW THE SCRIPTS RELATE diagram
- Add weekly discovery schedule block to EXECUTION ORDER section
- Update intro to mention behavior-driven discovery

Media/Manual-Media.md:
- Add configuration reference sections for all 3 discovery scripts
- Add flag reference sections for all 3 discovery scripts

Tools/README-Tools.md:
- Add "Library Sync Bootstrap" category and emby_to_*_sync.sh to scripts table
- Add emby sync scripts to HOW THE SCRIPTS RELATE diagram

Tools/Manual-Tools.md:
- Update emby sync notes: search commands now fire immediately after add

README.md:
- Update Media folder description to include discovery
- Update weekly schedule to show discovery scripts
- Update README table entry for Media
2026-05-20 17:48:30 -04:00
Gmer4Lfe 644225c4dc Fix Sonarr discovery monitor mode: future → all
future only monitors seasons not yet aired — shows with all existing
seasons would add as unmonitored with nothing to search. all monitors
everything so SeriesSearch actually finds episodes to grab.
2026-05-20 17:36:27 -04:00
Gmer4Lfe 6ffe5bea70 Add Sonarr discovery; wire search commands to all arr adds
- New: Media/playback_aware_sonarr_discovery.sh — two-stage TV show
  discovery using Emby episode play history → TMDB TV recommendations.
  Multi-user aware: diversity score (0-50) is primary seed driver so one
  user binge-watching alone cannot dominate. Per-user episode cap (default 8)
  limits single-user volume contribution. TMDB→TVDB via external_ids for
  reliable Sonarr lookup. monitor=future by default.

- Add to WEEKLY_MAINTENANCE_SCRIPTS; Sonarr discovery config block in
  master.conf (threshold=52, lookback=14d, max_adds=3, user_ep_cap=8).

- Fix: all arr add scripts were not triggering searches after adding.
  Added _radarr_command/_lidarr_command/_sonarr_command helpers and fire
  MoviesSearch/ArtistSearch/SeriesSearch immediately after each successful
  add. Affects: playback_aware_radarr_discovery.sh, emby_to_lidarr_sync.sh,
  emby_to_radarr_sync.sh, emby_to_sonarr_sync.sh.
2026-05-20 17:27:07 -04:00
Gmer4Lfe 9d371f6e27 Tune Radarr discovery threshold to 52 2026-05-20 17:07:26 -04:00
Gmer4Lfe 548c662709 Fix Radarr discovery: activity log seeding + calibrate threshold
- Stage 1 rewritten to use Emby activity log (playback.stop events)
  with ItemId batch-fetching instead of per-library UserData sort;
  SortBy=DatePlayed requires UserId context and errored server-wide

- Scoring simplified: recency (0-50) + play frequency across all
  users (0-50); CommunityRating is not exposed by Emby Items API
  so TMDB rating scoring moved entirely to Stage 2

- Batch IDs in groups of 100 — 1500+ unique item IDs in a 30-day
  window exceeded GET URL limits on a single request

- Threshold lowered to 50 (from 60); with diverse-genre seeds,
  recommendations rarely appear across multiple seeds so breadth
  score stays at 10/40 — threshold 50 yields ~5 adds per run at
  good quality (rating 7.0+, 1k+ votes)

- HOST1_TMDB_API_KEY moved to sit under Radarr section in
  master_host1.conf where it logically belongs
2026-05-20 17:05:35 -04:00
Gmer4Lfe 4ab3fa26d1 Add Radarr discovery + tighten arr sync filters
- playback_aware_radarr_discovery.sh: new two-stage movie discovery
  using recently watched Emby movies as seeds and TMDB recommendations
  as the similarity engine; scores on recency + rating + vote count;
  adds HOST*_TMDB_API_KEY alias to detect_hosts() in common.sh

- emby_to_sonarr_sync.sh: restrict to SONARR_EMBY_LIBRARIES allowlist
  (Anime, Kids Shows, Stand-Up Comedy, TV shows) — excludes Youtube
  and Recorded Sports libraries; add garbage title filter for multi-
  season folder names (S01-S03) and bare bracket tags ([Prof])

- emby_to_radarr_sync.sh: add _is_episode_title filter to block anime
  episode files stored in Movies library (underscore/dot patterns,
  fansub bracket format, episode markers, codec metadata strings)

- emby_to_lidarr_sync.sh: add U+FFFD Unicode replacement character
  filter alongside the existing ASCII ? filter for encoding corruption

- master.conf: RADARR_DISCOVERY_* settings, SONARR/RADARR_EMBY_LIBRARIES
  allowlists, Radarr discovery added to WEEKLY_MAINTENANCE_SCRIPTS
2026-05-20 16:51:49 -04:00
Gmer4Lfe 7ccb7e0e67 feat: two-stage Lidarr discovery + Emby→arr sync tools
Lidarr Discovery (playback_aware_lidarr_discovery.sh):
- Complete rewrite to two-stage pipeline: Stage 1 scores weekly Emby
  plays → top seeds; Stage 2 runs Last.fm getSimilar on seeds → scores
  candidates → adds top 0-5 to Lidarr
- Per-user influence cap (35%) prevents single listener dominating discovery
- Requires 2+ seeds to accept a candidate (single-seed skipped)
- Last.fm similarity limit 10 for tighter, higher-quality candidates
- 30-day reject cooldown history; MAX_ADDS=5 cap enforced
- Root folder fetched from Lidarr API at runtime (no config path)
- Added to WEEKLY_MAINTENANCE_SCRIPTS (uncommented)

Emby→arr sync tools (Tools/):
- emby_to_lidarr_sync.sh: finds Emby music artists not in Lidarr, adds them
  - Dirty tag filter: comma-list, feat./ft., &, vs, " - " patterns skipped
  - Root folder fetched from Lidarr API at runtime
- emby_to_sonarr_sync.sh: finds Emby series not in Sonarr, adds them
  - TVDB ID matching with title fallback
- emby_to_radarr_sync.sh: finds Emby movies not in Radarr, adds them
  - TMDB ID matching with title fallback
- All three: searchForMissing*: false (monitoring only, no searches triggered)
- All three documented in Tools/Manual-Tools.md
2026-05-19 23:34:32 -04:00
Gmer4Lfe e13f2fa14f feat: slskd reconnect guard in downloaders_reset, mass v2 sync
- downloaders_reset: connection check block before slskd API sections;
  triggers PUT /api/v0/server reconnect if disconnected, polls 60s,
  gates Stuck Searches and Dead Transfer Records on SLSKD_CONNECTED
- Sync all modified/new/deleted files from v2 refactor across Docker_Essentials,
  Media, Monitors, Partnership, Rsync, Tools, Transcodes, unRAID_Essentials,
  common.sh, master confs, and new Manual/README docs
2026-05-19 20:00:10 -04:00
FailedProxy eaade9a9d4 moved kernel folder 2026-05-10 20:12:36 -04:00
Gmer4LfeandClaude Sonnet 4.6 0ae31b5fa6 feat: Tailscale resolution hardening, partnership offboard completion, Emby provisioning
common.sh:
- Add resolve_tailscale_ip() helper — tries `tailscale ip -4` first, falls back to
  parsing `tailscale status` output; handles hosts where MagicDNS short-name resolution
  is not active
- Add PARTNERSHIP_OWN_CONTAINERS alias in detect_hosts()
- Add aliasing for 4 Emby provisioning vars (PARTNERSHIP_PROVISION_EMBY_ADMIN,
  PARTNERSHIP_EMBY_ADMIN_USER, PARTNERSHIP_EMBY_ADMIN_PASS, PARTNERSHIP_EMBY_PORT)

Partnership/partnership_manager.sh:
- Replace 9 bare `tailscale ip -4` calls with resolve_tailscale_ip()
- Add read_remote_conf_var() and read_remote_conf_array() — SSH to mirror, source its
  own load_config.sh + detect_hosts(), return aliased variable; solves sparse-checkout
  problem where HOST1 cannot read master_host2.conf directly
- Add derive_short_name() — strips unraid- prefix, capitalises first char
- Add cleanup_partner_containers() — removes partner containers via FolderView3 folder
  if enabled, else falls back to FALLBACK_*_COVERS_*_TIER* arrays
- Add cleanup_owner_containers_on_mirror() — SSH to mirror, stops and removes containers
  matching *-${OWNER_SHORT} naming convention
- Add start_own_stack() and start_mirror_own_stack() — restart own containers locally
  or on mirror via SSH using PARTNERSHIP_OWN_CONTAINERS
- Add provision_emby_admin() — reads mirror credentials via read_remote_conf_var, checks
  for username collision, creates user + sets password + grants admin policy via Emby API
- Add revoke_emby_admin() — looks up mirror username on local Emby, deletes via REST API
- Wire offboard paths (both mirror-initiated and owner-initiated) to call container
  cleanup and stack restart; update --check finalisation paths accordingly
- Fix write_state_file in --onboard not gated on DRY_RUN (was writing ACTIVE state on
  dry runs)

master_host1.conf:
- Add HOST1_PARTNERSHIP_OWN_CONTAINERS array
- Add partnership Emby provisioning config (toggle + port + per-host credentials)

master_host2.conf:
- Add HOST2_PARTNERSHIP_OWN_CONTAINERS array
- Add HOST2_PARTNERSHIP_EMBY_ADMIN_USER and HOST2_PARTNERSHIP_EMBY_ADMIN_PASS

Tailscale fix applied to:
- Initial_run/ssh_setup.sh (2 callsites)
- unRAID_Essentials/rsync_stop.sh (1 callsite)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 20:09:13 -04:00
FailedProxy 6948755c86 added decision engine 2026-05-10 20:07:19 -04:00
FailedProxy 8ec97162e7 updated playback aware 2026-05-10 18:04:05 -04:00
FailedProxy dd55ec0761 added. playback aware lidarr discovery tool. still stand alone 2026-05-10 15:26:01 -04:00
Gmer4LfeandClaude Sonnet 4.6 f277ce92d5 feat: intermediate orchestrator + arr_sync blocklist end-to-end + mesh sync
Add intermediate_sync_maintenance.sh (0 */4 * * *) — arr library sync as
fixed first step, optional mid-day rsync (INTERMEDIATE_SYNC_SHARES, empty
by default), then INTERMEDIATE_MAINTENANCE_SCRIPTS. Wired with full rsync
infrastructure (temperature abort, check_rsync_enabled, INTERMEDIATE_RSYNC_ENABLED
toggle) matching daily/weekly pattern. lidarr_missing_art.sh scheduled here.

arr_sync.sh: blocklist --add now end-to-end — looks up real display name
from local arr API, writes TSV, deletes from local arr API (deleteFiles=false),
SSHes each remote node and deletes from their arr API. Files become orphans
for arr_cleanup safety pass.

master.conf: DEFAULT_RSYNC_OPTS updated (--partial, --timeout=60,
--numeric-ids; --no-whole-file removed). Media share comments updated to
mesh model.

master_host1.conf + master_host2.conf: both hosts now push all media shares
bidirectionally (true mesh — no ownership per share, arr_cleanup enforces truth).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 15:24:35 -04:00
Gmer4LfeandClaude Sonnet 4.6 b65f572367 feat: arr-native distributed media sync + config architecture fixes
Arr sync (new):
- Media/arr_sync.sh — full mesh bidirectional sync across all HOST* nodes
  - Lidarr (MusicBrainz), Sonarr (TVDB), Radarr (TMDB) all handled in one script
  - Remote API keys read live from config.xml via SSH — never stored in conf files
  - Shared blocklist (DATA_DIR/arr_sync_blocklist.tsv) merged from all nodes at runtime
  - Graceful skip if arr not configured locally or not reachable on a remote node
  - --blocklist-add / --blocklist-remove / --blocklist-list management flags
- daily_sync_maintenance.sh — arr sync runs as explicit phase before rsync
- partnership_onboard.sh — Step 3 bootstraps merged library on both sides at onboard
- master.conf — ARR_SYNC_* config block, DOCKER_APPDATA_BASE

Rsync / cleanup:
- DEFAULT_RSYNC_OPTS — removed --delete; arr_cleanup.sh owns orphan enforcement
- lidarr_cleanup.sh — removed HOST1-only guard; runs on any node with Lidarr configured

Config architecture:
- HOST1/HOST2 hostnames moved from master_host*.conf → master.conf (not credentials)
- Sparse checkout now works correctly: each server only needs its own host conf
- detect_hosts() still resolves MY_ID + REMOTE_ID via master.conf hostname values

Bug fix:
- common.sh line 493 — watchdog toggle eval had broken quoting; all SYS_WATCHDOG_CHECK_*
  globals were silently set to empty instead of their configured values

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 09:53:40 -04:00
Gmer4LfeandClaude Sonnet 4.6 5526bc1eca v2 post-migration bug fixes, lidarr_missing_art port, and Partnership SSH/FolderView3 enhancements
Bug fixes across the ecosystem after v1→v2 architecture migration and Unraid 7.2.5 upgrade:

- common.sh: fix _alias_array() phantom empty element (removed [@]:-} pattern), fix
  resolve_remote_ip() with Tailscale FQDN lowercase + awk fallback, add FANART/LASTFM
  key aliases in detect_hosts(), global [@]:-} sweep across 10+ scripts
- webgui_restart.sh: fix emhttp detection (pgrep emhttpd) and restart command for 7.2.5
  (/usr/local/sbin/emhttp stop && start — rc.emhttp removed in 7.2.5)
- bandwidth_monitor.sh, continuous_scripts_status.sh: fix 'local' keyword outside function
- backup_verify.sh: fix resolve_remote_ip() called before detect_hosts()
- Orchestrators: fix script display duplication bug in status output (${entry##*/})
- rsync_stop.sh, git_pull_execute.sh, partnership_manager.sh, coffee_report: lowercase all
  tailscale ip -4 call sites to match Tailscale's lowercase device names
- master_host1.conf: fix SSH key path (gmer4lfe_rsync_automation), add FANART/LASTFM keys
- master_host2.conf: add FANART/LASTFM API keys

New: Media/lidarr_missing_art.sh
- Full ecosystem port of standalone Lidarr artwork fetcher
- Fetches missing album art via fanart.tv + Last.fm APIs
- @tsv batch extraction: 1 jq call per API response vs N*albums (8050 albums in 24s)
- HOST guard (HOST1 only), --status, --dry-run, acquire_lock

New: Initial_run/ssh_setup.sh
- Generates {hostname}_rsync_automation ed25519 keypair (skip if exists, --force to regen)
- ssh-copy-id to remote via Tailscale IP, auto-updates master_host*.conf
- --validate mode: strike tracking (SSH_MAX_STRIKES, SSH_STRIKE_RESET_HRS),
  notify at limit — Tailscale-unreachable remote does NOT count as SSH strike

New: Initial_run/partnership_onboard.sh
- Orchestrator: ssh_setup.sh then partnership_manager.sh --onboard in one command

Partnership/partnership_manager.sh: FolderView3 integration
- Derive partner folder name at runtime (strip unraid- prefix case-insensitively)
- --onboard: create {Mirror}-Failover folder with failover tier containers
- --offboard (both paths): stop + rm containers in folder, remove JSON entry
- --check: calls ssh_setup.sh --validate when IP resolves but SSH state empty
- --status: shows FolderView3 folder state and containers inline

master.conf: SSH_MAX_STRIKES, SSH_STRIKE_RESET_HRS, PARTNERSHIP_FOLDERVIEW3,
PARTNERSHIP_FOLDERVIEW3_URL added to Partnership section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 18:20:23 -04:00
FailedProxy 7cecda6b90 added lidar missing art checker 2026-05-08 16:26:22 -04:00
FailedProxy ec7de648dc massive update. Master conf split, now modular with a load sceriprt to drive all configs to scripts. with unraid scpecific safeguard tests , and improved standardized ux. including dynamic host detect, who am i who else it there. EVERY SINGLE SCRIPT UPDATED. DEBATING THAT THIS IS ACUALLY V2 2026-05-03 17:16:49 -04:00
FailedProxy e6a6bfd5e9 update radar and sonarr enpoints 2026-04-26 01:19:32 -04:00
FailedProxy 3df4542351 added manual neuclear option to arrs cleanup 2026-04-26 01:05:22 -04:00
FailedProxy 5f940a7181 now i fixed arrs ckean, maybe? 2026-04-24 17:44:11 -04:00
FailedProxy b3f72417a4 fixed arrs clean up showing only orphans 2026-04-24 17:22:34 -04:00
FailedProxy 8f60e5dde4 still working on lidarr endoints 2026-04-23 16:17:59 -04:00
FailedProxy 36c7bb66dc updated lidarr endpoints 2026-04-23 16:12:54 -04:00
FailedProxy 208fcb1f86 added other variables to check in the arrs. if it isnt importing for what ever reason blocklist and search 2026-04-22 18:45:56 -04:00
FailedProxy 8aa7aa7236 same 2026-04-22 18:33:56 -04:00
FailedProxy 5424ba8ca7 same 2026-04-22 18:32:37 -04:00
FailedProxy ebcad24e7d fixed script name 2026-04-22 18:24:06 -04:00
FailedProxy 39f2525d0f added a arrs atalled failled handler 2026-04-22 18:22:29 -04:00
FailedProxy e83473ab8e added safe guards to all scripts that need it. all reference comm.sh like it should be 2026-04-20 17:18:25 -04:00
FailedProxy cbfdc488c3 incorp, host profiles into scripts that needed it 2026-04-19 17:39:11 -04:00
FailedProxy 8c5667b1df added all ai generated Readme files, added the last of the tools scripts 2026-04-18 15:40:41 -04:00
FailedProxy c37f7adac6 small fixes 2026-04-15 17:58:25 -04:00
FailedProxy f1529db3a0 did way to much,,,,,, mostly added monitors, but almost every file was edited in some way 2026-04-14 17:11:49 -04:00
FailedProxy 771b89ae67 updated structure 2026-04-11 18:03:47 -04:00