# ━━━━━ ARRS STACK — Manual ━━━━━ Config reference, procedures, operational workflows. For overview see README-Arrs_Stack.md. For per-script detail see script headers. --- ## ━━━ ARR CLEANUP — FILE CLASSIFICATION ━━━ Every file found on disk during an arr cleanup run falls into exactly one category: ``` TRACKED → arr API returned this exact path → leave it alone PROTECTED → matches ARR_PROTECTED_PATTERNS → never delete ORPHAN → media extension, not tracked, old enough → delete JUNK → not a media extension, not protected → delete (any age) RECENT → not tracked, under ARR_ORPHAN_AGE days → skip (may be mid-import) ``` **Why protected patterns are critical:** arrs generate artwork (`*.jpg`), metadata (`*.nfo`), and subtitles/lyrics that do NOT appear in the tracked file API response. Without protection, these would be classified as orphans and deleted — removing cover art from every album, every movie poster, every TV show thumbnail. Requires a full rescan to recover. Never remove artwork extensions from protected patterns. --- ## ━━━ ARR CLEANUP — SAFETY LAYERS ━━━ All 7 layers must pass before any file is touched. There is no way to push through a failed safety check without the explicit override flag. ``` 1. Container running + healthy — a stopped container has an empty API 2. API reachable — no API = no tracked file list = everything looks orphaned 3. API version matches — major version must match tested version in master.conf 4. Item count > 0 — no artists/series/movies = something is wrong with DB 5. Tracked file count > 0 — empty response = everything would be deleted 6. Tracked count >= MIN_TRACKED_PCT — dramatic drop from last run = abort and alert 7. Deletion size < MAX_DELETE_GB — last line of defense against misconfigured root path ``` Layer 7 is the catastrophic failure prevention. A misconfigured root path — pointing cleanup at the wrong directory — means the API returns zero tracked files for a root that actually contains thousands. Everything walks as an orphan. Everything gets deleted. `LIDARR/SONARR/RADARR_MAX_DELETE_GB` requires `--i-know-what-im-doing` to proceed past it. --- ## ━━━ CONFIGURATION — master.conf ━━━ ### Lidarr Cleanup Thresholds ```bash LIDARR_ORPHAN_AGE=7 # days — files newer than this are RECENT (mid-import window) LIDARR_MIN_TRACKED_PCT=80 # abort if API returns < 80% of last known count LIDARR_MAX_DELETE_GB=5 # require --i-know-what-im-doing above this LIDARR_IMPORT_SCAN_TIMEOUT=600 # seconds to wait for pre-flight import scan LIDARR_VERSION_MAJOR=3 # expected Lidarr major version (API safety check) LIDARR_LOCK_WARN_AGE=3600 # 1hr — large libraries take time, not stuck LIDARR_EXTENSIONS=("flac" "mp3" "m4a" "wav" "aac" "ogg" "opus" "wma") LIDARR_PROTECTED_PATTERNS=("*.jpg" "*.jpeg" "*.png" "*.nfo" "*.lrc") LIDARR_TRACKED_COUNT_FILE="$DATA_DIR/lidarr_tracked.count" # persistent baseline ARR_CLEANUP_STATS="$DATA_DIR/arr_cleanup_stats.db" # read by coffee report ``` --- ### Lidarr Release Fixer ```bash LIDARR_RELEASE_FIXER_ENABLED=true # set false to disable without removing from job list ``` No additional thresholds — uses existing `LIDARR_URL`, `LIDARR_API_KEY`, `LIDARR_MUSIC_ROOT`, `LIDARR_PATH_MAP`, and `LIDARR_VERSION_MAJOR` from host*.conf and master.conf. Reads FLAC (vorbis comment block type 4) and MP3 (ID3v2 TXXX frame) tags. --- ### Sonarr Cleanup Thresholds ```bash SONARR_ORPHAN_AGE=7 SONARR_MAX_DELETE_GB=50 SONARR_IMPORT_SCAN_TIMEOUT=600 SONARR_VERSION_MAJOR=4 SONARR_EXTENSIONS=("mkv" "mp4" "avi" "m4v" "ts" "wmv" "mov") SONARR_PROTECTED_PATTERNS=("*.jpg" "*.jpeg" "*.png" "*.nfo" "*.srt" "*.sub" "*.ass" "*.ssa") ``` Note: `*.ts` IS in extensions — transport stream is used for Live TV recordings tracked by Sonarr. Orphaned `.ts` recordings should be cleaned like any other orphaned episode. --- ### Radarr Cleanup Thresholds ```bash RADARR_ORPHAN_AGE=7 RADARR_MAX_DELETE_GB=50 RADARR_IMPORT_SCAN_TIMEOUT=600 RADARR_VERSION_MAJOR=6 RADARR_EXTENSIONS=("mkv" "mp4" "avi" "m4v" "wmv" "mov") RADARR_PROTECTED_PATTERNS=("*.jpg" "*.jpeg" "*.png" "*.nfo" "*.srt" "*.sub" "*.ass" "*.ssa") ``` --- ### Arr Sync ```bash ARR_SYNC_ENABLED=true ARR_SYNC_BLOCKLIST="$DATA_DIR/arr_sync_blocklist.tsv" # tombstone file ARR_SYNC_CONNECT_TIMEOUT=10 # SSH connect timeout in seconds ARR_SYNC_API_TIMEOUT=60 # curl API call timeout in seconds DOCKER_APPDATA_BASE=/mnt/user/appdata ARR_SYNC_LIDARR_PORT=8686 ARR_SYNC_SONARR_PORT=8989 ARR_SYNC_RADARR_PORT=7878 ``` --- ### Arr Recovery ```bash ARR_IMPORT_RECOVERY_AGE=6 # hours — items newer than this are skipped SONARR_VERSION_MAJOR=4 RADARR_VERSION_MAJOR=6 LIDARR_VERSION_MAJOR=3 ARR_RECOVERY_STATS="$DATA_DIR/arr_recovery_stats.db" # read by coffee report ``` --- ### TMDb / TVDB Removed ```bash RADARR_DROPPED_ADD_EXCLUSION=true # add removed movies to Radarr import exclusion SONARR_DROPPED_ADD_EXCLUSION=true # add removed series to Sonarr import exclusion ``` --- ### Lidarr Missing Art ```bash FANART_API_KEY="your-fanart-tv-api-key" LASTFM_API_KEY="your-lastfm-api-key" LIDARR_ART_MIN_SIZE=5000 # minimum valid download size in bytes LIDARR_ART_MAX_PARALLEL=4 # concurrent background download jobs LIDARR_ART_RETRIES=2 # download retry attempts per image LIDARR_ART_SLEEP_BETWEEN=1 # seconds between fanart.tv API calls (rate limit) ``` --- ### Lidarr Discovery ```bash LIDARR_DISCOVERY_THRESHOLD=70 # minimum score for Stage 1 seeds and Stage 2 adds LIDARR_DISCOVERY_LOOKBACK_DAYS=7 # Emby play history window in days LIDARR_DISCOVERY_MIN_PLAYS=3 # min plays before an artist is evaluated as a seed LIDARR_DISCOVERY_MAX_ADDS=5 # max seeds (Stage 1) and max adds (Stage 2) per run LIDARR_DISCOVERY_USER_CAP_PCT=35 # max % any one user contributes to play weight LIDARR_DISCOVERY_REJECT_COOLDOWN=30 # days before re-evaluating a Stage 2 reject LIDARR_DISCOVERY_HISTORY="$DATA_DIR/lidarr_discovery_history.db" ``` Requires `HOST*_LASTFM_API_KEY` in `host*.conf`. --- ### Radarr Discovery ```bash RADARR_DISCOVERY_THRESHOLD=52 # minimum score to add a candidate RADARR_DISCOVERY_LOOKBACK_DAYS=30 # Emby watch history window in days RADARR_DISCOVERY_MAX_SEEDS=5 # max seed movies from Stage 1 RADARR_DISCOVERY_MAX_ADDS=5 # max movies to add per run RADARR_DISCOVERY_MIN_VOTE_COUNT=100 # min TMDB votes for a candidate RADARR_DISCOVERY_MIN_RATING=60 # min TMDB vote_average × 10 (60 = 6.0/10) RADARR_DISCOVERY_REJECT_COOLDOWN=60 # days before re-evaluating a rejected movie RADARR_DISCOVERY_SEED_LIBRARIES=("Movies") RADARR_DISCOVERY_HISTORY="$DATA_DIR/radarr_discovery_history.db" ``` Requires `HOST*_TMDB_API_KEY` in `host*.conf`. --- ### Sonarr Discovery ```bash SONARR_DISCOVERY_THRESHOLD=52 # minimum score to add a candidate SONARR_DISCOVERY_LOOKBACK_DAYS=14 # Emby episode history window in days SONARR_DISCOVERY_MAX_SEEDS=5 # max seed series from Stage 1 SONARR_DISCOVERY_MAX_ADDS=3 # max shows to add per run (TV is a larger commitment) SONARR_DISCOVERY_MIN_VOTE_COUNT=50 # min TMDB votes for a candidate SONARR_DISCOVERY_MIN_RATING=65 # min TMDB vote_average × 10 (65 = 6.5/10) SONARR_DISCOVERY_REJECT_COOLDOWN=60 # days before re-evaluating a rejected show SONARR_DISCOVERY_USER_EPISODE_CAP=8 # max episodes per user in seed scoring SONARR_DISCOVERY_MONITOR_MODE="all" # "all" = all seasons monitored; "future" = upcoming only SONARR_DISCOVERY_HISTORY="$DATA_DIR/sonarr_discovery_history.db" ``` Requires `HOST*_TMDB_API_KEY` in `host*.conf`. > **MONITOR_MODE note:** Use `"all"` (default) to have Sonarr search all existing seasons > after adding a show. `"future"` only marks upcoming seasons as monitored — shows where > all seasons have already aired will appear unmonitored and Sonarr will not search for them. --- ### Upgrade Webhook ```bash WEBHOOK_PORT=7821 # 0 = disable listener WEBHOOK_SECRET="" # auto-generated on first start if empty ``` When `WEBHOOK_PORT=0`, `start_webhook_listener.sh` exits cleanly and no listener starts. When `WEBHOOK_SECRET` is empty, a 32-byte hex secret is generated on first start and written back to `master.conf`. Run `Tools/webhook_setup.sh` to register the URL in arrs. Webhook log: `/var/log/varaverk/upgrade_webhook.log` --- ## ━━━ CONFIGURATION — host*.conf ━━━ ```bash # Arr connection details — must match arr settings exactly HOST1_LIDARR_URL="http://192.168.50.2:8686" HOST1_LIDARR_API_KEY="..." HOST1_LIDARR_MUSIC_ROOT="/mnt/user/Music" HOST1_LIDARR_PATH_MAP="" # container→host path translation if needed HOST1_SONARR_URL="http://192.168.50.2:8989" HOST1_SONARR_API_KEY="..." HOST1_SONARR_TV_ROOT="/mnt/user/Tv_Shows" HOST1_SONARR_PATH_MAP="" HOST1_RADARR_URL="http://192.168.50.2:7878" HOST1_RADARR_API_KEY="..." HOST1_RADARR_MOVIES_ROOT="/mnt/user/Movies" HOST1_RADARR_PATH_MAP="" HOST1_EMBY_URL="http://192.168.50.2:8096" HOST1_EMBY_API_KEY="..." ``` > **LIDARR/SONARR/RADARR_MUSIC/TV/MOVIES_ROOT must exactly match the Root Folder path in > the arr's own settings.** Arr UI → Settings → Media Management → Root Folders. > A mismatch means every file on disk looks untracked — all appear as orphans. > MAX_DELETE_GB is the only thing standing between a path mismatch and losing your library. --- ## ━━━ SAFE TESTING PROCEDURE ━━━ > **The arr cleanup scripts permanently delete files.** There is no recycle bin, no undo. > Follow this procedure on first use, after any root path change, after any API key change, > and after any significant arr library change. ### Step 1 — Dry Run With Full Logging ```bash lidarr_release_fixer.sh --dry-run --log lidarr_cleanup.sh --dry-run --log sonarr_cleanup.sh --dry-run --log radarr_cleanup.sh --dry-run --log ``` ### Step 2 — Review the Output ``` Release fixer: Are the "would fix" albums expected? → Wrong release UUIDs switching to correct ones is expected behaviour → If everything is already correct, nothing to do Cleanup: Are TRACKED files the ones you expect? → Known arr-managed files should show as TRACKED → If they show as ORPHAN, the root path is wrong — STOP Is the ORPHAN count reasonable? → Healthy cleanup removes dozens to hundreds, not tens of thousands → Large count = stop, investigate root path before proceeding Are PROTECTED patterns working? → Artwork (*.jpg) and subtitles (*.srt) must show as PROTECTED → If they show as ORPHAN, check PROTECTED_PATTERNS config Are RECENT files being correctly skipped? → Files downloaded in the last 7 days should show as RECENT, not ORPHAN ``` ### Step 3 — Check Numbers if Something Looks Wrong ```bash # Root path mismatch? Compare these: # Lidarr UI: Settings → Media Management → Root Folders # Sonarr UI: Settings → Media Management → Root Folders # Radarr UI: Settings → Media Management → Root Folders # Must exactly match LIDARR_MUSIC_ROOT / SONARR_TV_ROOT / RADARR_MOVIES_ROOT # Is the arr running? docker ps | grep -E "Lidarr|Sonarr|Radarr" ``` ### Step 4 — Run Live ```bash # Only after dry run review passes. lidarr_release_fixer.sh lidarr_cleanup.sh sonarr_cleanup.sh radarr_cleanup.sh ``` ### Step 5 — Verify in Arr UI ``` Library count — should not have dropped significantly Missing files — check if any monitored content shows as missing Emby library — should show no ghost entries (notify_emby_scan handles this automatically) ``` --- ## ━━━ PROCEDURES ━━━ ### Adding a New Arr ```bash # 1. Copy radarr_cleanup.sh as template cp Arrs_Stack/radarr_cleanup.sh Arrs_Stack/readarr_cleanup.sh # 2. Replace RADARR_ prefix with READARR_ throughout # Update API endpoint, tracked file API path, extension list, protected patterns # 3. Add to host*.conf HOST1_READARR_URL="http://192.168.50.2:8787" HOST1_READARR_API_KEY="your-api-key" HOST1_READARR_BOOKS_ROOT="/mnt/user/Books" # 4. Add thresholds to master.conf READARR_ORPHAN_AGE=7 READARR_MAX_DELETE_GB=50 READARR_EXTENSIONS=("epub" "pdf" "mobi" "azw3" "cbz" "cbr") READARR_PROTECTED_PATTERNS=("*.jpg" "*.jpeg" "*.png" "*.nfo") # 5. Add to DAILY_MAINTENANCE_SCRIPTS in master.conf "Arrs_Stack/readarr_cleanup.sh" ``` Run `--dry-run --log` before scheduling. --- ### Managing the Arr Sync Blocklist ```bash # Add item to blocklist (removes from all arrs + tombstones the ID) arr_sync.sh --blocklist-add lidarr "reason" arr_sync.sh --blocklist-add sonarr "reason" arr_sync.sh --blocklist-add radarr "reason" # Remove from blocklist (un-tombstones the ID — does NOT re-add to arrs) arr_sync.sh --blocklist-remove lidarr # View all blocklisted IDs arr_sync.sh --blocklist-list ``` `--blocklist-add` is the only destructive operation — it simultaneously: 1. Writes the tombstone entry to the blocklist TSV file 2. Deletes the item from the local arr API (no file deletion) 3. SSHes each remote node and deletes from their arr API Files become orphans on all nodes — arr_cleanup removes them on the next run. --- ## ━━━ TROUBLESHOOTING ━━━ ### Arr Cleanup Deleting Files It Shouldn't ``` 1. Check the protected patterns — artwork and subtitles must be listed LIDARR_PROTECTED_PATTERNS / SONARR_PROTECTED_PATTERNS / RADARR_PROTECTED_PATTERNS 2. Check the root path matches arr settings exactly Run: lidarr_cleanup.sh --status (shows configured root path) Compare: Lidarr UI → Settings → Media Management → Root Folders 3. Check if files are truly orphaned Run: lidarr_cleanup.sh --dry-run --log Look for the specific file — verify it shows ORPHAN, not PROTECTED or TRACKED ``` ### Lidarr Files Present But Not Importing ``` Symptom: album folder exists with properly tagged files, Lidarr shows 0 tracks imported. Cause: Lidarr selected the wrong MusicBrainz release edition. The file's MBID doesn't match Lidarr's foreignReleaseId, so track ID lookup fails. Fix: lidarr_release_fixer.sh --dry-run --log → Shows which albums would be corrected and which release UUIDs would change lidarr_release_fixer.sh → Applies corrections and queues RefreshArtist for affected artists If the file MBID doesn't match any Lidarr release (shows "MBID not in list"): → File was tagged from a source Lidarr doesn't know about (unofficial, re-tagged) → Manual import or re-tag with correct MBID ``` ### Arr Cleanup Aborting at Safety Layer 6 (Tracked Count Drop) ``` API returned far fewer tracked files than last run. Possible causes: - Arr database was recently rebuilt from scratch - Large manual library removal - Path map mismatch after arr migration If intentional (library intentionally reduced): Delete LIDARR_TRACKED_COUNT_FILE to reset the baseline Run cleanup once — it will establish a new baseline If unintentional: Investigate before proceeding — the arr may have a problem ``` ### Arr Sync Not Picking Up New Content ``` Is ARR_SYNC_ENABLED=true in master.conf? Can this host SSH to the remote without password? → ssh -i [SSH_KEY] root@[remote-tailscale-ip] "hostname" Is the arr accessible on the remote? → arr_sync.sh --status (shows each node's arr reachability) → arr_sync.sh --log (verbose output per-node, per-arr) Is the item in the blocklist? → arr_sync.sh --blocklist-list ``` ### Emby Still Showing Ghost Entries After Cleanup ``` notify_emby_scan() is called automatically after every arr cleanup deletion. If ghosts persist: 1. Is Emby's API responding? curl -s "http://[emby-ip]:8096/System/Info/Public" 2. Is EMBY_URL / EMBY_API_KEY correct in host*.conf? Run: sonarr_cleanup.sh --status (shows Emby config) 3. Trigger manually in Emby: Library → Manage Library → Clean Missing Files ``` --- ## ━━━ OUTPUT TIERS ━━━ All scripts have two output levels controlled by `--log`. Without `--log`, each script processes silently and always concludes with a summary block: identity, duration, counts (files removed, items added, arrs cleaned), and a status line. Warnings and errors are always visible. With `--log`, per-item detail appears: individual titles being processed, API query progress, per-node sync results, and per-file examination output. Use this when debugging unexpected results or validating configuration before the first scheduled run. Dry-run output follows the same tiers — `--dry-run` alone shows the summary of what would happen; `--dry-run --log` shows the full per-item preview list. --- ## ━━━ FLAG REFERENCE ━━━ ### lidarr_release_fixer.sh `lidarr_release_fixer.sh` Read MUSICBRAINZ_ALBUMID from FLAC/MP3 files, correct Lidarr release selection, queue RefreshArtist. `lidarr_release_fixer.sh --dry-run` Show which albums would be corrected and the before/after release UUIDs. No API writes. `lidarr_release_fixer.sh --status` Show configuration, Lidarr URL, enabled state. `lidarr_release_fixer.sh --log` Verbose per-album output — shows every album checked, not just those corrected. --- ### lidarr_cleanup.sh / sonarr_cleanup.sh / radarr_cleanup.sh `[script] --dry-run --log` Preview every classification decision. **Always run this first.** See Safe Testing Procedure. `[script]` Live run — deletes confirmed orphans and junk, triggers Emby clean. `[script] --log` Live run with verbose per-file output. `[script] --status` Show configuration, API status, tracked file count, and last run stats. `[script] --i-know-what-im-doing` Bypass the MAX_DELETE_GB size threshold. Required when deletion exceeds the configured limit. Long flag name is intentional — cannot be added accidentally. `[script] --skip-age-check` Bypass the ORPHAN_AGE age check. Deletes RECENT files too — files that are under the age threshold. Use when you know recent downloads are actually orphans. `[script] --i-know-what-im-doing --skip-age-check` **NUCLEAR MODE** — age check and size threshold both bypassed. Deletes on first pass. Use when you want a clean one-pass wipe of everything the arr doesn't track. No recovery possible after deletion. --- ### arrs_failed_stalled_recovery.sh `arrs_failed_stalled_recovery.sh` Check all configured arrs for failed imports and stalled downloads. Blocklist + remove + re-search for each problem item. `arrs_failed_stalled_recovery.sh --dry-run` Show what would be actioned per arr without making any changes. `arrs_failed_stalled_recovery.sh --status` Show configuration, arr reachability, and last recovery stats. `arrs_failed_stalled_recovery.sh --log` Verbose output per item per arr. --- ### arr_sync.sh `arr_sync.sh` Sync all arr types across all configured nodes. `arr_sync.sh --dry-run` Show what would be added/removed on each node without making changes. `arr_sync.sh --status` Show node configuration, arr reachability, and blocklist count. `arr_sync.sh --log` Verbose per-node, per-arr output. `arr_sync.sh --blocklist-add [arr] [id] "[reason]"` Remove item from all arrs and tombstone the ID. See Procedures above. `arr_sync.sh --blocklist-remove [arr] [id]` Remove tombstone — does NOT re-add item to arrs. `arr_sync.sh --blocklist-list` Show all tombstoned IDs. --- ### radarr_tmdb_removed.sh / sonarr_tvdb_removed.sh `[script]` Remove records for entries with status="deleted" (dropped from upstream database). Files are kept. Import exclusion is added. `[script] --delete-files` Also delete associated files from disk. Most dropped entries have no files — they were announced movies/series that were never downloaded. `[script] --dry-run` Preview what would be removed without making changes. `[script] --status` Show arr connection status and current count of dropped entries. `[script] --log` Verbose per-entry output. --- ### lidarr_missing_art.sh `lidarr_missing_art.sh` Fetch all missing album and artist artwork from fanart.tv and fallback sources. Never overwrites existing files. `lidarr_missing_art.sh --dry-run` Show what would be downloaded without writing any files. `lidarr_missing_art.sh --status` Show configuration and API key status. `lidarr_missing_art.sh --log` Verbose per-album, per-artist output. --- ### playback_aware_lidarr_discovery.sh `playback_aware_lidarr_discovery.sh` Score Emby play history, run Last.fm getSimilar on top artists, add candidates above threshold to Lidarr. Triggers ArtistSearch immediately after each successful add. `playback_aware_lidarr_discovery.sh --dry-run` Score and rank all Stage 1 seeds and Stage 2 candidates. No Lidarr API calls. No writes to history file. Shows exactly what would be added and at what score. `playback_aware_lidarr_discovery.sh --status` Show config values, history file path and size, and API key status. `playback_aware_lidarr_discovery.sh --log` Verbose per-artist scoring output for both stages. --- ### playback_aware_radarr_discovery.sh `playback_aware_radarr_discovery.sh` Score recently watched Emby movies, run TMDB recommendations on seeds, add candidates above threshold to Radarr. Triggers MoviesSearch immediately after each successful add. `playback_aware_radarr_discovery.sh --dry-run` Score and rank all Stage 1 seeds and Stage 2 candidates. No Radarr API calls. No writes to history file. `playback_aware_radarr_discovery.sh --status` Show config values, history file path and size, and API key status. `playback_aware_radarr_discovery.sh --log` Verbose per-movie scoring output for both stages. --- ### playback_aware_sonarr_discovery.sh `playback_aware_sonarr_discovery.sh` Score recently watched Emby series (weighted by user diversity), run TMDB TV recommendations on seeds, add candidates above threshold to Sonarr. Triggers SeriesSearch immediately after each successful add. `playback_aware_sonarr_discovery.sh --dry-run` Score and rank all Stage 1 seeds and Stage 2 candidates. No Sonarr API calls. No writes to history file. `playback_aware_sonarr_discovery.sh --status` Show config values, history file path and size, and API key status. `playback_aware_sonarr_discovery.sh --log` Verbose per-series scoring output — shows user diversity, recency, and volume scores per seed; breadth, rating, and votes scores per candidate.