Split Media/ docs into Media/ and Arrs_Stack/ to match folder reorganization
Media README and Manual now cover only the 3 remaining scripts (permissions, cleaner, play_state_sync). Arrs_Stack README and Manual cover all arr stack scripts including lidarr_release_fixer. Fixed stale --skip-strike-list reference in flag docs.
This commit is contained in:
Gmer4Lfe
@@ -0,0 +1,670 @@
|
||||
# ━━━━━ ARRS STACK — Manual ━━━━━
|
||||
|
||||
Config reference, procedures, operational workflows.
|
||||
For overview see README-Arrs_Stack.md. For per-script detail see script headers.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ ARR CLEANUP — FILE CLASSIFICATION ━━━
|
||||
|
||||
Every file found on disk during an arr cleanup run falls into exactly one category:
|
||||
|
||||
```
|
||||
TRACKED → arr API returned this exact path → leave it alone
|
||||
PROTECTED → matches ARR_PROTECTED_PATTERNS → never delete
|
||||
ORPHAN → media extension, not tracked, old enough → delete
|
||||
JUNK → not a media extension, not protected → delete (any age)
|
||||
RECENT → not tracked, under ARR_ORPHAN_AGE days → skip (may be mid-import)
|
||||
```
|
||||
|
||||
**Why protected patterns are critical:** arrs generate artwork (`*.jpg`), metadata
|
||||
(`*.nfo`), and subtitles/lyrics that do NOT appear in the tracked file API response.
|
||||
Without protection, these would be classified as orphans and deleted — removing cover art
|
||||
from every album, every movie poster, every TV show thumbnail. Requires a full rescan
|
||||
to recover. Never remove artwork extensions from protected patterns.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ ARR CLEANUP — SAFETY LAYERS ━━━
|
||||
|
||||
All 7 layers must pass before any file is touched. There is no way to push through a
|
||||
failed safety check without the explicit override flag.
|
||||
|
||||
```
|
||||
1. Container running + healthy — a stopped container has an empty API
|
||||
2. API reachable — no API = no tracked file list = everything looks orphaned
|
||||
3. API version matches — major version must match tested version in master.conf
|
||||
4. Item count > 0 — no artists/series/movies = something is wrong with DB
|
||||
5. Tracked file count > 0 — empty response = everything would be deleted
|
||||
6. Tracked count >= MIN_TRACKED_PCT — dramatic drop from last run = abort and alert
|
||||
7. Deletion size < MAX_DELETE_GB — last line of defense against misconfigured root path
|
||||
```
|
||||
|
||||
Layer 7 is the catastrophic failure prevention. A misconfigured root path — pointing
|
||||
cleanup at the wrong directory — means the API returns zero tracked files for a root
|
||||
that actually contains thousands. Everything walks as an orphan. Everything gets deleted.
|
||||
`LIDARR/SONARR/RADARR_MAX_DELETE_GB` requires `--i-know-what-im-doing` to proceed past it.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ CONFIGURATION — master.conf ━━━
|
||||
|
||||
### Lidarr Cleanup Thresholds
|
||||
|
||||
```bash
|
||||
LIDARR_ORPHAN_AGE=7 # days — files newer than this are RECENT (mid-import window)
|
||||
LIDARR_MIN_TRACKED_PCT=80 # abort if API returns < 80% of last known count
|
||||
LIDARR_MAX_DELETE_GB=5 # require --i-know-what-im-doing above this
|
||||
LIDARR_IMPORT_SCAN_TIMEOUT=600 # seconds to wait for pre-flight import scan
|
||||
LIDARR_VERSION_MAJOR=3 # expected Lidarr major version (API safety check)
|
||||
LIDARR_LOCK_WARN_AGE=3600 # 1hr — large libraries take time, not stuck
|
||||
LIDARR_EXTENSIONS=("flac" "mp3" "m4a" "wav" "aac" "ogg" "opus" "wma")
|
||||
LIDARR_PROTECTED_PATTERNS=("*.jpg" "*.jpeg" "*.png" "*.nfo" "*.lrc")
|
||||
LIDARR_TRACKED_COUNT_FILE="$DATA_DIR/lidarr_tracked.count" # persistent baseline
|
||||
ARR_CLEANUP_STATS="$DATA_DIR/arr_cleanup_stats.db" # read by coffee report
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Lidarr Release Fixer
|
||||
|
||||
```bash
|
||||
LIDARR_RELEASE_FIXER_ENABLED=true # set false to disable without removing from job list
|
||||
```
|
||||
|
||||
No additional thresholds — uses existing `LIDARR_URL`, `LIDARR_API_KEY`,
|
||||
`LIDARR_MUSIC_ROOT`, `LIDARR_PATH_MAP`, and `LIDARR_VERSION_MAJOR` from host*.conf and
|
||||
master.conf. Reads FLAC (vorbis comment block type 4) and MP3 (ID3v2 TXXX frame) tags.
|
||||
|
||||
---
|
||||
|
||||
### Sonarr Cleanup Thresholds
|
||||
|
||||
```bash
|
||||
SONARR_ORPHAN_AGE=7
|
||||
SONARR_MAX_DELETE_GB=50
|
||||
SONARR_IMPORT_SCAN_TIMEOUT=600
|
||||
SONARR_VERSION_MAJOR=4
|
||||
SONARR_EXTENSIONS=("mkv" "mp4" "avi" "m4v" "ts" "wmv" "mov")
|
||||
SONARR_PROTECTED_PATTERNS=("*.jpg" "*.jpeg" "*.png" "*.nfo" "*.srt" "*.sub" "*.ass" "*.ssa")
|
||||
```
|
||||
|
||||
Note: `*.ts` IS in extensions — transport stream is used for Live TV recordings tracked
|
||||
by Sonarr. Orphaned `.ts` recordings should be cleaned like any other orphaned episode.
|
||||
|
||||
---
|
||||
|
||||
### Radarr Cleanup Thresholds
|
||||
|
||||
```bash
|
||||
RADARR_ORPHAN_AGE=7
|
||||
RADARR_MAX_DELETE_GB=50
|
||||
RADARR_IMPORT_SCAN_TIMEOUT=600
|
||||
RADARR_VERSION_MAJOR=6
|
||||
RADARR_EXTENSIONS=("mkv" "mp4" "avi" "m4v" "wmv" "mov")
|
||||
RADARR_PROTECTED_PATTERNS=("*.jpg" "*.jpeg" "*.png" "*.nfo" "*.srt" "*.sub" "*.ass" "*.ssa")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Arr Sync
|
||||
|
||||
```bash
|
||||
ARR_SYNC_ENABLED=true
|
||||
ARR_SYNC_BLOCKLIST="$DATA_DIR/arr_sync_blocklist.tsv" # tombstone file
|
||||
ARR_SYNC_CONNECT_TIMEOUT=10 # SSH connect timeout in seconds
|
||||
ARR_SYNC_API_TIMEOUT=60 # curl API call timeout in seconds
|
||||
DOCKER_APPDATA_BASE=/mnt/user/appdata
|
||||
ARR_SYNC_LIDARR_PORT=8686
|
||||
ARR_SYNC_SONARR_PORT=8989
|
||||
ARR_SYNC_RADARR_PORT=7878
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Arr Recovery
|
||||
|
||||
```bash
|
||||
ARR_IMPORT_RECOVERY_AGE=6 # hours — items newer than this are skipped
|
||||
SONARR_VERSION_MAJOR=4
|
||||
RADARR_VERSION_MAJOR=6
|
||||
LIDARR_VERSION_MAJOR=3
|
||||
ARR_RECOVERY_STATS="$DATA_DIR/arr_recovery_stats.db" # read by coffee report
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### TMDb / TVDB Removed
|
||||
|
||||
```bash
|
||||
RADARR_DROPPED_ADD_EXCLUSION=true # add removed movies to Radarr import exclusion
|
||||
SONARR_DROPPED_ADD_EXCLUSION=true # add removed series to Sonarr import exclusion
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Lidarr Missing Art
|
||||
|
||||
```bash
|
||||
FANART_API_KEY="your-fanart-tv-api-key"
|
||||
LASTFM_API_KEY="your-lastfm-api-key"
|
||||
LIDARR_ART_MIN_SIZE=5000 # minimum valid download size in bytes
|
||||
LIDARR_ART_MAX_PARALLEL=4 # concurrent background download jobs
|
||||
LIDARR_ART_RETRIES=2 # download retry attempts per image
|
||||
LIDARR_ART_SLEEP_BETWEEN=1 # seconds between fanart.tv API calls (rate limit)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Lidarr Discovery
|
||||
|
||||
```bash
|
||||
LIDARR_DISCOVERY_THRESHOLD=70 # minimum score for Stage 1 seeds and Stage 2 adds
|
||||
LIDARR_DISCOVERY_LOOKBACK_DAYS=7 # Emby play history window in days
|
||||
LIDARR_DISCOVERY_MIN_PLAYS=3 # min plays before an artist is evaluated as a seed
|
||||
LIDARR_DISCOVERY_MAX_ADDS=5 # max seeds (Stage 1) and max adds (Stage 2) per run
|
||||
LIDARR_DISCOVERY_USER_CAP_PCT=35 # max % any one user contributes to play weight
|
||||
LIDARR_DISCOVERY_REJECT_COOLDOWN=30 # days before re-evaluating a Stage 2 reject
|
||||
LIDARR_DISCOVERY_HISTORY="$DATA_DIR/lidarr_discovery_history.db"
|
||||
```
|
||||
|
||||
Requires `HOST*_LASTFM_API_KEY` in `host*.conf`.
|
||||
|
||||
---
|
||||
|
||||
### Radarr Discovery
|
||||
|
||||
```bash
|
||||
RADARR_DISCOVERY_THRESHOLD=52 # minimum score to add a candidate
|
||||
RADARR_DISCOVERY_LOOKBACK_DAYS=30 # Emby watch history window in days
|
||||
RADARR_DISCOVERY_MAX_SEEDS=5 # max seed movies from Stage 1
|
||||
RADARR_DISCOVERY_MAX_ADDS=5 # max movies to add per run
|
||||
RADARR_DISCOVERY_MIN_VOTE_COUNT=100 # min TMDB votes for a candidate
|
||||
RADARR_DISCOVERY_MIN_RATING=60 # min TMDB vote_average × 10 (60 = 6.0/10)
|
||||
RADARR_DISCOVERY_REJECT_COOLDOWN=60 # days before re-evaluating a rejected movie
|
||||
RADARR_DISCOVERY_SEED_LIBRARIES=("Movies")
|
||||
RADARR_DISCOVERY_HISTORY="$DATA_DIR/radarr_discovery_history.db"
|
||||
```
|
||||
|
||||
Requires `HOST*_TMDB_API_KEY` in `host*.conf`.
|
||||
|
||||
---
|
||||
|
||||
### Sonarr Discovery
|
||||
|
||||
```bash
|
||||
SONARR_DISCOVERY_THRESHOLD=52 # minimum score to add a candidate
|
||||
SONARR_DISCOVERY_LOOKBACK_DAYS=14 # Emby episode history window in days
|
||||
SONARR_DISCOVERY_MAX_SEEDS=5 # max seed series from Stage 1
|
||||
SONARR_DISCOVERY_MAX_ADDS=3 # max shows to add per run (TV is a larger commitment)
|
||||
SONARR_DISCOVERY_MIN_VOTE_COUNT=50 # min TMDB votes for a candidate
|
||||
SONARR_DISCOVERY_MIN_RATING=65 # min TMDB vote_average × 10 (65 = 6.5/10)
|
||||
SONARR_DISCOVERY_REJECT_COOLDOWN=60 # days before re-evaluating a rejected show
|
||||
SONARR_DISCOVERY_USER_EPISODE_CAP=8 # max episodes per user in seed scoring
|
||||
SONARR_DISCOVERY_MONITOR_MODE="all" # "all" = all seasons monitored; "future" = upcoming only
|
||||
SONARR_DISCOVERY_HISTORY="$DATA_DIR/sonarr_discovery_history.db"
|
||||
```
|
||||
|
||||
Requires `HOST*_TMDB_API_KEY` in `host*.conf`.
|
||||
|
||||
> **MONITOR_MODE note:** Use `"all"` (default) to have Sonarr search all existing seasons
|
||||
> after adding a show. `"future"` only marks upcoming seasons as monitored — shows where
|
||||
> all seasons have already aired will appear unmonitored and Sonarr will not search for them.
|
||||
|
||||
---
|
||||
|
||||
### Upgrade Webhook
|
||||
|
||||
```bash
|
||||
WEBHOOK_PORT=7821 # 0 = disable listener
|
||||
WEBHOOK_SECRET="" # auto-generated on first start if empty
|
||||
```
|
||||
|
||||
When `WEBHOOK_PORT=0`, `start_webhook_listener.sh` exits cleanly and no listener starts.
|
||||
When `WEBHOOK_SECRET` is empty, a 32-byte hex secret is generated on first start and
|
||||
written back to `master.conf`. Run `Tools/webhook_setup.sh` to register the URL in arrs.
|
||||
|
||||
Webhook log: `/var/log/varaverk/upgrade_webhook.log`
|
||||
|
||||
---
|
||||
|
||||
## ━━━ CONFIGURATION — host*.conf ━━━
|
||||
|
||||
```bash
|
||||
# Arr connection details — must match arr settings exactly
|
||||
HOST1_LIDARR_URL="http://192.168.50.2:8686"
|
||||
HOST1_LIDARR_API_KEY="..."
|
||||
HOST1_LIDARR_MUSIC_ROOT="/mnt/user/Music"
|
||||
HOST1_LIDARR_PATH_MAP="" # container→host path translation if needed
|
||||
|
||||
HOST1_SONARR_URL="http://192.168.50.2:8989"
|
||||
HOST1_SONARR_API_KEY="..."
|
||||
HOST1_SONARR_TV_ROOT="/mnt/user/Tv_Shows"
|
||||
HOST1_SONARR_PATH_MAP=""
|
||||
|
||||
HOST1_RADARR_URL="http://192.168.50.2:7878"
|
||||
HOST1_RADARR_API_KEY="..."
|
||||
HOST1_RADARR_MOVIES_ROOT="/mnt/user/Movies"
|
||||
HOST1_RADARR_PATH_MAP=""
|
||||
|
||||
HOST1_EMBY_URL="http://192.168.50.2:8096"
|
||||
HOST1_EMBY_API_KEY="..."
|
||||
```
|
||||
|
||||
> **LIDARR/SONARR/RADARR_MUSIC/TV/MOVIES_ROOT must exactly match the Root Folder path in
|
||||
> the arr's own settings.** Arr UI → Settings → Media Management → Root Folders.
|
||||
> A mismatch means every file on disk looks untracked — all appear as orphans.
|
||||
> MAX_DELETE_GB is the only thing standing between a path mismatch and losing your library.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ SAFE TESTING PROCEDURE ━━━
|
||||
|
||||
> **The arr cleanup scripts permanently delete files.** There is no recycle bin, no undo.
|
||||
> Follow this procedure on first use, after any root path change, after any API key change,
|
||||
> and after any significant arr library change.
|
||||
|
||||
### Step 1 — Dry Run With Full Logging
|
||||
|
||||
```bash
|
||||
lidarr_release_fixer.sh --dry-run --log
|
||||
lidarr_cleanup.sh --dry-run --log
|
||||
sonarr_cleanup.sh --dry-run --log
|
||||
radarr_cleanup.sh --dry-run --log
|
||||
```
|
||||
|
||||
### Step 2 — Review the Output
|
||||
|
||||
```
|
||||
Release fixer:
|
||||
Are the "would fix" albums expected?
|
||||
→ Wrong release UUIDs switching to correct ones is expected behaviour
|
||||
→ If everything is already correct, nothing to do
|
||||
|
||||
Cleanup:
|
||||
Are TRACKED files the ones you expect?
|
||||
→ Known arr-managed files should show as TRACKED
|
||||
→ If they show as ORPHAN, the root path is wrong — STOP
|
||||
|
||||
Is the ORPHAN count reasonable?
|
||||
→ Healthy cleanup removes dozens to hundreds, not tens of thousands
|
||||
→ Large count = stop, investigate root path before proceeding
|
||||
|
||||
Are PROTECTED patterns working?
|
||||
→ Artwork (*.jpg) and subtitles (*.srt) must show as PROTECTED
|
||||
→ If they show as ORPHAN, check PROTECTED_PATTERNS config
|
||||
|
||||
Are RECENT files being correctly skipped?
|
||||
→ Files downloaded in the last 7 days should show as RECENT, not ORPHAN
|
||||
```
|
||||
|
||||
### Step 3 — Check Numbers if Something Looks Wrong
|
||||
|
||||
```bash
|
||||
# Root path mismatch? Compare these:
|
||||
# Lidarr UI: Settings → Media Management → Root Folders
|
||||
# Sonarr UI: Settings → Media Management → Root Folders
|
||||
# Radarr UI: Settings → Media Management → Root Folders
|
||||
# Must exactly match LIDARR_MUSIC_ROOT / SONARR_TV_ROOT / RADARR_MOVIES_ROOT
|
||||
|
||||
# Is the arr running?
|
||||
docker ps | grep -E "Lidarr|Sonarr|Radarr"
|
||||
```
|
||||
|
||||
### Step 4 — Run Live
|
||||
|
||||
```bash
|
||||
# Only after dry run review passes.
|
||||
lidarr_release_fixer.sh
|
||||
lidarr_cleanup.sh
|
||||
sonarr_cleanup.sh
|
||||
radarr_cleanup.sh
|
||||
```
|
||||
|
||||
### Step 5 — Verify in Arr UI
|
||||
|
||||
```
|
||||
Library count — should not have dropped significantly
|
||||
Missing files — check if any monitored content shows as missing
|
||||
Emby library — should show no ghost entries (notify_emby_scan handles this automatically)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ PROCEDURES ━━━
|
||||
|
||||
### Adding a New Arr
|
||||
|
||||
```bash
|
||||
# 1. Copy radarr_cleanup.sh as template
|
||||
cp Arrs_Stack/radarr_cleanup.sh Arrs_Stack/readarr_cleanup.sh
|
||||
|
||||
# 2. Replace RADARR_ prefix with READARR_ throughout
|
||||
# Update API endpoint, tracked file API path, extension list, protected patterns
|
||||
|
||||
# 3. Add to host*.conf
|
||||
HOST1_READARR_URL="http://192.168.50.2:8787"
|
||||
HOST1_READARR_API_KEY="your-api-key"
|
||||
HOST1_READARR_BOOKS_ROOT="/mnt/user/Books"
|
||||
|
||||
# 4. Add thresholds to master.conf
|
||||
READARR_ORPHAN_AGE=7
|
||||
READARR_MAX_DELETE_GB=50
|
||||
READARR_EXTENSIONS=("epub" "pdf" "mobi" "azw3" "cbz" "cbr")
|
||||
READARR_PROTECTED_PATTERNS=("*.jpg" "*.jpeg" "*.png" "*.nfo")
|
||||
|
||||
# 5. Add to DAILY_MAINTENANCE_SCRIPTS in master.conf
|
||||
"Arrs_Stack/readarr_cleanup.sh"
|
||||
```
|
||||
|
||||
Run `--dry-run --log` before scheduling.
|
||||
|
||||
---
|
||||
|
||||
### Managing the Arr Sync Blocklist
|
||||
|
||||
```bash
|
||||
# Add item to blocklist (removes from all arrs + tombstones the ID)
|
||||
arr_sync.sh --blocklist-add lidarr <musicbrainz-artist-id> "reason"
|
||||
arr_sync.sh --blocklist-add sonarr <tvdb-series-id> "reason"
|
||||
arr_sync.sh --blocklist-add radarr <tmdb-movie-id> "reason"
|
||||
|
||||
# Remove from blocklist (un-tombstones the ID — does NOT re-add to arrs)
|
||||
arr_sync.sh --blocklist-remove lidarr <id>
|
||||
|
||||
# View all blocklisted IDs
|
||||
arr_sync.sh --blocklist-list
|
||||
```
|
||||
|
||||
`--blocklist-add` is the only destructive operation — it simultaneously:
|
||||
1. Writes the tombstone entry to the blocklist TSV file
|
||||
2. Deletes the item from the local arr API (no file deletion)
|
||||
3. SSHes each remote node and deletes from their arr API
|
||||
|
||||
Files become orphans on all nodes — arr_cleanup removes them on the next run.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ TROUBLESHOOTING ━━━
|
||||
|
||||
### Arr Cleanup Deleting Files It Shouldn't
|
||||
|
||||
```
|
||||
1. Check the protected patterns — artwork and subtitles must be listed
|
||||
LIDARR_PROTECTED_PATTERNS / SONARR_PROTECTED_PATTERNS / RADARR_PROTECTED_PATTERNS
|
||||
|
||||
2. Check the root path matches arr settings exactly
|
||||
Run: lidarr_cleanup.sh --status (shows configured root path)
|
||||
Compare: Lidarr UI → Settings → Media Management → Root Folders
|
||||
|
||||
3. Check if files are truly orphaned
|
||||
Run: lidarr_cleanup.sh --dry-run --log
|
||||
Look for the specific file — verify it shows ORPHAN, not PROTECTED or TRACKED
|
||||
```
|
||||
|
||||
### Lidarr Files Present But Not Importing
|
||||
|
||||
```
|
||||
Symptom: album folder exists with properly tagged files, Lidarr shows 0 tracks imported.
|
||||
Cause: Lidarr selected the wrong MusicBrainz release edition. The file's MBID doesn't
|
||||
match Lidarr's foreignReleaseId, so track ID lookup fails.
|
||||
|
||||
Fix:
|
||||
lidarr_release_fixer.sh --dry-run --log
|
||||
→ Shows which albums would be corrected and which release UUIDs would change
|
||||
|
||||
lidarr_release_fixer.sh
|
||||
→ Applies corrections and queues RefreshArtist for affected artists
|
||||
|
||||
If the file MBID doesn't match any Lidarr release (shows "MBID not in list"):
|
||||
→ File was tagged from a source Lidarr doesn't know about (unofficial, re-tagged)
|
||||
→ Manual import or re-tag with correct MBID
|
||||
```
|
||||
|
||||
### Arr Cleanup Aborting at Safety Layer 6 (Tracked Count Drop)
|
||||
|
||||
```
|
||||
API returned far fewer tracked files than last run.
|
||||
Possible causes:
|
||||
- Arr database was recently rebuilt from scratch
|
||||
- Large manual library removal
|
||||
- Path map mismatch after arr migration
|
||||
|
||||
If intentional (library intentionally reduced):
|
||||
Delete LIDARR_TRACKED_COUNT_FILE to reset the baseline
|
||||
Run cleanup once — it will establish a new baseline
|
||||
|
||||
If unintentional:
|
||||
Investigate before proceeding — the arr may have a problem
|
||||
```
|
||||
|
||||
### Arr Sync Not Picking Up New Content
|
||||
|
||||
```
|
||||
Is ARR_SYNC_ENABLED=true in master.conf?
|
||||
|
||||
Can this host SSH to the remote without password?
|
||||
→ ssh -i [SSH_KEY] root@[remote-tailscale-ip] "hostname"
|
||||
|
||||
Is the arr accessible on the remote?
|
||||
→ arr_sync.sh --status (shows each node's arr reachability)
|
||||
→ arr_sync.sh --log (verbose output per-node, per-arr)
|
||||
|
||||
Is the item in the blocklist?
|
||||
→ arr_sync.sh --blocklist-list
|
||||
```
|
||||
|
||||
### Emby Still Showing Ghost Entries After Cleanup
|
||||
|
||||
```
|
||||
notify_emby_scan() is called automatically after every arr cleanup deletion.
|
||||
If ghosts persist:
|
||||
1. Is Emby's API responding?
|
||||
curl -s "http://[emby-ip]:8096/System/Info/Public"
|
||||
|
||||
2. Is EMBY_URL / EMBY_API_KEY correct in host*.conf?
|
||||
Run: sonarr_cleanup.sh --status (shows Emby config)
|
||||
|
||||
3. Trigger manually in Emby:
|
||||
Library → Manage Library → Clean Missing Files
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ OUTPUT TIERS ━━━
|
||||
|
||||
All scripts have two output levels controlled by `--log`.
|
||||
|
||||
Without `--log`, each script processes silently and always concludes with a summary
|
||||
block: identity, duration, counts (files removed, items added, arrs cleaned), and a
|
||||
status line. Warnings and errors are always visible.
|
||||
|
||||
With `--log`, per-item detail appears: individual titles being processed, API query
|
||||
progress, per-node sync results, and per-file examination output. Use this when
|
||||
debugging unexpected results or validating configuration before the first scheduled run.
|
||||
|
||||
Dry-run output follows the same tiers — `--dry-run` alone shows the summary of what
|
||||
would happen; `--dry-run --log` shows the full per-item preview list.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ FLAG REFERENCE ━━━
|
||||
|
||||
### lidarr_release_fixer.sh
|
||||
|
||||
`lidarr_release_fixer.sh`
|
||||
Read MUSICBRAINZ_ALBUMID from FLAC/MP3 files, correct Lidarr release selection, queue RefreshArtist.
|
||||
|
||||
`lidarr_release_fixer.sh --dry-run`
|
||||
Show which albums would be corrected and the before/after release UUIDs. No API writes.
|
||||
|
||||
`lidarr_release_fixer.sh --status`
|
||||
Show configuration, Lidarr URL, enabled state.
|
||||
|
||||
`lidarr_release_fixer.sh --log`
|
||||
Verbose per-album output — shows every album checked, not just those corrected.
|
||||
|
||||
---
|
||||
|
||||
### lidarr_cleanup.sh / sonarr_cleanup.sh / radarr_cleanup.sh
|
||||
|
||||
`[script] --dry-run --log`
|
||||
Preview every classification decision. **Always run this first.** See Safe Testing Procedure.
|
||||
|
||||
`[script]`
|
||||
Live run — deletes confirmed orphans and junk, triggers Emby clean.
|
||||
|
||||
`[script] --log`
|
||||
Live run with verbose per-file output.
|
||||
|
||||
`[script] --status`
|
||||
Show configuration, API status, tracked file count, and last run stats.
|
||||
|
||||
`[script] --i-know-what-im-doing`
|
||||
Bypass the MAX_DELETE_GB size threshold. Required when deletion exceeds the configured
|
||||
limit. Long flag name is intentional — cannot be added accidentally.
|
||||
|
||||
`[script] --skip-age-check`
|
||||
Bypass the ORPHAN_AGE age check. Deletes RECENT files too — files that are under the
|
||||
age threshold. Use when you know recent downloads are actually orphans.
|
||||
|
||||
`[script] --i-know-what-im-doing --skip-age-check`
|
||||
**NUCLEAR MODE** — age check and size threshold both bypassed. Deletes on first pass.
|
||||
Use when you want a clean one-pass wipe of everything the arr doesn't track.
|
||||
No recovery possible after deletion.
|
||||
|
||||
---
|
||||
|
||||
### arrs_failed_stalled_recovery.sh
|
||||
|
||||
`arrs_failed_stalled_recovery.sh`
|
||||
Check all configured arrs for failed imports and stalled downloads. Blocklist + remove +
|
||||
re-search for each problem item.
|
||||
|
||||
`arrs_failed_stalled_recovery.sh --dry-run`
|
||||
Show what would be actioned per arr without making any changes.
|
||||
|
||||
`arrs_failed_stalled_recovery.sh --status`
|
||||
Show configuration, arr reachability, and last recovery stats.
|
||||
|
||||
`arrs_failed_stalled_recovery.sh --log`
|
||||
Verbose output per item per arr.
|
||||
|
||||
---
|
||||
|
||||
### arr_sync.sh
|
||||
|
||||
`arr_sync.sh`
|
||||
Sync all arr types across all configured nodes.
|
||||
|
||||
`arr_sync.sh --dry-run`
|
||||
Show what would be added/removed on each node without making changes.
|
||||
|
||||
`arr_sync.sh --status`
|
||||
Show node configuration, arr reachability, and blocklist count.
|
||||
|
||||
`arr_sync.sh --log`
|
||||
Verbose per-node, per-arr output.
|
||||
|
||||
`arr_sync.sh --blocklist-add [arr] [id] "[reason]"`
|
||||
Remove item from all arrs and tombstone the ID. See Procedures above.
|
||||
|
||||
`arr_sync.sh --blocklist-remove [arr] [id]`
|
||||
Remove tombstone — does NOT re-add item to arrs.
|
||||
|
||||
`arr_sync.sh --blocklist-list`
|
||||
Show all tombstoned IDs.
|
||||
|
||||
---
|
||||
|
||||
### radarr_tmdb_removed.sh / sonarr_tvdb_removed.sh
|
||||
|
||||
`[script]`
|
||||
Remove records for entries with status="deleted" (dropped from upstream database).
|
||||
Files are kept. Import exclusion is added.
|
||||
|
||||
`[script] --delete-files`
|
||||
Also delete associated files from disk. Most dropped entries have no files — they were
|
||||
announced movies/series that were never downloaded.
|
||||
|
||||
`[script] --dry-run`
|
||||
Preview what would be removed without making changes.
|
||||
|
||||
`[script] --status`
|
||||
Show arr connection status and current count of dropped entries.
|
||||
|
||||
`[script] --log`
|
||||
Verbose per-entry output.
|
||||
|
||||
---
|
||||
|
||||
### lidarr_missing_art.sh
|
||||
|
||||
`lidarr_missing_art.sh`
|
||||
Fetch all missing album and artist artwork from fanart.tv and fallback sources.
|
||||
Never overwrites existing files.
|
||||
|
||||
`lidarr_missing_art.sh --dry-run`
|
||||
Show what would be downloaded without writing any files.
|
||||
|
||||
`lidarr_missing_art.sh --status`
|
||||
Show configuration and API key status.
|
||||
|
||||
`lidarr_missing_art.sh --log`
|
||||
Verbose per-album, per-artist output.
|
||||
|
||||
---
|
||||
|
||||
### playback_aware_lidarr_discovery.sh
|
||||
|
||||
`playback_aware_lidarr_discovery.sh`
|
||||
Score Emby play history, run Last.fm getSimilar on top artists, add candidates above
|
||||
threshold to Lidarr. Triggers ArtistSearch immediately after each successful add.
|
||||
|
||||
`playback_aware_lidarr_discovery.sh --dry-run`
|
||||
Score and rank all Stage 1 seeds and Stage 2 candidates. No Lidarr API calls. No writes
|
||||
to history file. Shows exactly what would be added and at what score.
|
||||
|
||||
`playback_aware_lidarr_discovery.sh --status`
|
||||
Show config values, history file path and size, and API key status.
|
||||
|
||||
`playback_aware_lidarr_discovery.sh --log`
|
||||
Verbose per-artist scoring output for both stages.
|
||||
|
||||
---
|
||||
|
||||
### playback_aware_radarr_discovery.sh
|
||||
|
||||
`playback_aware_radarr_discovery.sh`
|
||||
Score recently watched Emby movies, run TMDB recommendations on seeds, add candidates
|
||||
above threshold to Radarr. Triggers MoviesSearch immediately after each successful add.
|
||||
|
||||
`playback_aware_radarr_discovery.sh --dry-run`
|
||||
Score and rank all Stage 1 seeds and Stage 2 candidates. No Radarr API calls. No writes
|
||||
to history file.
|
||||
|
||||
`playback_aware_radarr_discovery.sh --status`
|
||||
Show config values, history file path and size, and API key status.
|
||||
|
||||
`playback_aware_radarr_discovery.sh --log`
|
||||
Verbose per-movie scoring output for both stages.
|
||||
|
||||
---
|
||||
|
||||
### playback_aware_sonarr_discovery.sh
|
||||
|
||||
`playback_aware_sonarr_discovery.sh`
|
||||
Score recently watched Emby series (weighted by user diversity), run TMDB TV
|
||||
recommendations on seeds, add candidates above threshold to Sonarr. Triggers SeriesSearch
|
||||
immediately after each successful add.
|
||||
|
||||
`playback_aware_sonarr_discovery.sh --dry-run`
|
||||
Score and rank all Stage 1 seeds and Stage 2 candidates. No Sonarr API calls. No writes
|
||||
to history file.
|
||||
|
||||
`playback_aware_sonarr_discovery.sh --status`
|
||||
Show config values, history file path and size, and API key status.
|
||||
|
||||
`playback_aware_sonarr_discovery.sh --log`
|
||||
Verbose per-series scoring output — shows user diversity, recency, and volume scores per
|
||||
seed; breadth, rating, and votes scores per candidate.
|
||||
@@ -0,0 +1,250 @@
|
||||
# ━━━━━ ARRS STACK ━━━━━
|
||||
|
||||
Lifecycle management for the arr suite (Lidarr, Sonarr, Radarr) across a multi-server
|
||||
ecosystem. Library sync so every node tracks the same content. Orphan cleanup against live
|
||||
arr APIs so deleted content actually leaves disk. Release correction so wrong MusicBrainz
|
||||
editions don't silently block Lidarr imports. Emby notified automatically after every
|
||||
deletion. Failed downloads recovered overnight. Artwork fetched continuously. Database
|
||||
hygiene for dropped upstream entries. Quality upgrades propagate to all nodes immediately
|
||||
via webhook. Weekly discovery adds new content based on what you actually play and watch.
|
||||
|
||||
> **These scripts permanently delete files and modify arr databases.** Orphan cleanup is
|
||||
> protected by multiple safety layers that must all pass before anything is touched — but
|
||||
> dry runs and log review are still the right first step on any new system or after any
|
||||
> configuration change. See Manual-Arrs_Stack.md for the safe testing procedure.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ THE PROBLEMS THAT BUILT THIS ━━━
|
||||
|
||||
**Deleted Shows and Removed Albums Still on Disk**
|
||||
When you remove a series from Sonarr and the delete command fails — permission issue,
|
||||
container wasn't running, path mismatch — the files stay permanently. Over years on an
|
||||
active library this accumulates significantly.
|
||||
The fix: arr cleanup scripts query the live API for every tracked file path, walk the
|
||||
disk, and delete anything absent from the API response that's old enough to be past the
|
||||
import window.
|
||||
|
||||
**Wrong MusicBrainz Release Edition Silently Blocking Lidarr Imports**
|
||||
Lidarr selects one specific release edition per album using a MusicBrainz release ID.
|
||||
When it picks the wrong edition (Brazil CD instead of US CD, Japan Digital instead of
|
||||
standard), the track IDs don't match what's embedded in the files. RescanFolders reports
|
||||
"Importing 0 tracks" even with perfectly tagged, complete files present. No error — just
|
||||
silence.
|
||||
The fix: `lidarr_release_fixer.sh` reads the MUSICBRAINZ_ALBUMID tag from each file,
|
||||
finds the matching release in Lidarr's known releases, switches the selection, and queues
|
||||
a RefreshArtist. Runs before cleanup so corrected albums are imported before the orphan
|
||||
scan ever sees them.
|
||||
|
||||
**Emby Showing Ghost Entries After Cleanup**
|
||||
After arr cleanup deletes files, Emby still shows them until its next scheduled scan —
|
||||
potentially hours later. Users see broken entries that produce "file not found" errors.
|
||||
The fix: `notify_emby_scan()` is called automatically after every deletion. Triggers
|
||||
Emby's "Clean Missing Files" task immediately.
|
||||
|
||||
**No Safety Net on Deletion Size**
|
||||
A misconfigured root path — pointing cleanup at the wrong directory — means the API
|
||||
returns zero tracked files for a root that actually contains thousands. Every file walks
|
||||
as an orphan. Everything gets deleted. This is the catastrophic failure mode.
|
||||
The fix: `LIDARR/SONARR/RADARR_MAX_DELETE_GB` — if total deletion size exceeds the
|
||||
limit, the script stops and requires `--i-know-what-im-doing` to proceed. The flag name
|
||||
is long and annoying by design. It cannot be added by accident.
|
||||
|
||||
**Failed Downloads Accumulating Silently**
|
||||
Import failures and stalled downloads sit in arr queues indefinitely. Without
|
||||
intervention they occupy queue slots, block new searches, and the item never gets
|
||||
downloaded. Checking queues manually across three arrs is tedious.
|
||||
The fix: `arrs_failed_stalled_recovery.sh` inspects all arr queues, blocklists the bad
|
||||
release, removes it, and triggers a re-search. Runs daily.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ WHAT THIS FOLDER DOES ━━━
|
||||
|
||||
**Library Sync**
|
||||
`arr_sync.sh` — full-mesh arr library sync across all nodes. Every node syncs with every
|
||||
other — union model, no hierarchy. Once arrs agree on what to track, rsync spreads the
|
||||
actual files.
|
||||
|
||||
**Release Correction**
|
||||
`lidarr_release_fixer.sh` — reads MUSICBRAINZ_ALBUMID from FLAC and MP3 files, matches
|
||||
against Lidarr's known releases per album, switches monitored=true to the correct
|
||||
edition, queues RefreshArtist. Runs before lidarr_cleanup.sh daily. Handles both FLAC
|
||||
(vorbis comment block) and MP3 (ID3v2 TXXX frame).
|
||||
|
||||
**Orphan Cleanup**
|
||||
`lidarr_cleanup.sh`, `sonarr_cleanup.sh`, `radarr_cleanup.sh` — API-verified orphan
|
||||
removal. Five classification categories (TRACKED/PROTECTED/ORPHAN/JUNK/RECENT), seven
|
||||
safety layers, automatic Emby notification after deletion.
|
||||
|
||||
**Database Hygiene**
|
||||
`radarr_tmdb_removed.sh`, `sonarr_tvdb_removed.sh` — remove entries that upstream
|
||||
databases have dropped (TMDb/TVDB status="deleted"). These generate health warnings in
|
||||
arrs and can never be monitored or downloaded. Most are announced-but-never-released
|
||||
entries. Files are kept by default — most have none.
|
||||
|
||||
**Library Recovery**
|
||||
`arrs_failed_stalled_recovery.sh` — detect and recover failed imports and stalled
|
||||
downloads across all arrs. Blocklists the bad release and triggers a re-search — hands-
|
||||
free overnight recovery.
|
||||
|
||||
**Library Enrichment**
|
||||
`lidarr_missing_art.sh` — fetch missing album and artist artwork from fanart.tv and
|
||||
fallback sources. Never overwrites existing files.
|
||||
|
||||
**Upgrade Propagation**
|
||||
`start_webhook_listener.sh` — Node.js HTTP server that receives Sonarr/Radarr/Lidarr
|
||||
OnUpgrade webhooks. Continuous; started at array start. Writes to
|
||||
`/var/log/varaverk/upgrade_webhook.log`.
|
||||
`upgrade_webhook_handler.sh` — triggered by the webhook listener. Pushes the upgraded
|
||||
item folder to every remote node immediately, then triggers an arr library rescan on
|
||||
each remote so the upgraded file is accepted without triggering a redundant quality search.
|
||||
|
||||
**Discovery**
|
||||
`playback_aware_lidarr_discovery.sh` — behavior-driven music discovery. Scores your
|
||||
Emby play history, runs Last.fm getSimilar on top artists, adds the best matches to
|
||||
Lidarr. 0–5 meaningful adds per week.
|
||||
`playback_aware_radarr_discovery.sh` — behavior-driven movie discovery. Scores recently
|
||||
watched movies, runs TMDB recommendations on seeds, adds top candidates to Radarr.
|
||||
`playback_aware_sonarr_discovery.sh` — behavior-driven TV discovery. Scores recently
|
||||
watched series weighted by user diversity, runs TMDB TV recommendations on seeds, adds
|
||||
top shows to Sonarr. Multi-user design: one person binge-watching does not dominate seeds.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ EXECUTION ORDER ━━━
|
||||
|
||||
**Daily via `daily_sync_maintenance.sh` (DAILY_MAINTENANCE_SCRIPTS):**
|
||||
|
||||
```
|
||||
1. lidarr_release_fixer.sh — correct wrong release editions before cleanup sees them
|
||||
2. lidarr_cleanup.sh — orphan removal (music)
|
||||
3. sonarr_cleanup.sh — orphan removal (TV)
|
||||
4. radarr_cleanup.sh — orphan removal (movies)
|
||||
5. lidarr_missing_art.sh — fetch missing artwork (HOST1 only)
|
||||
6. radarr_tmdb_removed.sh — remove TMDb-dropped movies
|
||||
7. sonarr_tvdb_removed.sh — remove TVDB-dropped series
|
||||
```
|
||||
|
||||
Note: `media_shares_permissions.sh` and `media_cleaner.sh` run before these from
|
||||
`Media/` — permissions and junk removal must complete first.
|
||||
|
||||
**Every 30 min + 4hr via orchestrators (CRITICAL/INTERMEDIATE_MAINTENANCE_SCRIPTS):**
|
||||
|
||||
```
|
||||
arrs_failed_stalled_recovery.sh — failed/stalled queue recovery
|
||||
arr_sync.sh — library sync across all nodes
|
||||
```
|
||||
|
||||
**Weekly via `weekly_sync_maintenance.sh` (WEEKLY_MAINTENANCE_SCRIPTS):**
|
||||
|
||||
```
|
||||
playback_aware_lidarr_discovery.sh — score play history → Last.fm similar → Lidarr
|
||||
playback_aware_radarr_discovery.sh — score watch history → TMDB recommendations → Radarr
|
||||
playback_aware_sonarr_discovery.sh — score episode history → TMDB TV → Sonarr
|
||||
```
|
||||
|
||||
**Continuous (started by `array_started.sh`):**
|
||||
|
||||
```
|
||||
start_webhook_listener.sh — Node.js webhook server; dispatches upgrade_webhook_handler.sh
|
||||
```
|
||||
|
||||
**On every arr upgrade (triggered by webhook):**
|
||||
|
||||
```
|
||||
upgrade_webhook_handler.sh — push upgraded folder to all remote nodes + trigger arr rescan
|
||||
```
|
||||
|
||||
**Why release fixer before cleanup:** the fixer corrects Lidarr's release selection so
|
||||
files get imported. If cleanup ran first, a correctable album could accumulate age toward
|
||||
the orphan threshold before the fixer had a chance to fix it.
|
||||
|
||||
**Why arr_sync before rsync:** once arrs agree on what to track, rsync spreads the actual
|
||||
files. An upgrade on one node — new tracked path, old path no longer in API — gets
|
||||
cleaned by arr_cleanup on all nodes after the next sync cycle.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ HOST AWARENESS ━━━
|
||||
|
||||
Scripts run on both servers via `detect_hosts()`, which aliases all `HOST*_` prefixed vars
|
||||
to their unprefixed names at runtime. No manual `HOST1`/`HOST2` comparisons exist in any
|
||||
script.
|
||||
|
||||
`arr_sync.sh` keeps all arr databases in bidirectional union — either server can download
|
||||
to any share. Arr cleanup uses the union model: a file is only an orphan if the arr on
|
||||
this host doesn't have it indexed. Arr scripts check the aliased URL — if empty (arr not
|
||||
configured on this host), they exit cleanly with no action.
|
||||
|
||||
`lidarr_release_fixer.sh` and `lidarr_missing_art.sh` exit cleanly on hosts without
|
||||
Lidarr configured — no HOST1_LIDARR_URL means nothing runs.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ SCRIPTS IN THIS FOLDER ━━━
|
||||
|
||||
| Script | Role | When It Runs |
|
||||
|--------|------|--------------|
|
||||
| `arr_sync.sh` | Full-mesh arr library sync — all nodes track the same content | Every 4hr + weekly before rsync |
|
||||
| `lidarr_release_fixer.sh` | Fix wrong MusicBrainz release editions so files get imported | Daily before lidarr_cleanup |
|
||||
| `lidarr_cleanup.sh` | Delete orphaned music files not tracked by Lidarr | Daily |
|
||||
| `sonarr_cleanup.sh` | Delete orphaned TV files not tracked by Sonarr | Daily |
|
||||
| `radarr_cleanup.sh` | Delete orphaned movie files not tracked by Radarr | Daily |
|
||||
| `arrs_failed_stalled_recovery.sh` | Auto-recover failed imports and stalled downloads | Every 30 min / daily |
|
||||
| `lidarr_missing_art.sh` | Fetch missing album and artist artwork | Daily (HOST1 only) |
|
||||
| `radarr_tmdb_removed.sh` | Remove movies dropped from TMDb | Daily |
|
||||
| `sonarr_tvdb_removed.sh` | Remove series dropped from TVDB | Daily |
|
||||
| `start_webhook_listener.sh` | Node.js webhook server — receive arr OnUpgrade and dispatch handler | Continuous |
|
||||
| `upgrade_webhook_handler.sh` | Push upgraded item folder to remote nodes + trigger arr rescan | On each arr upgrade |
|
||||
| `playback_aware_lidarr_discovery.sh` | Behavior-driven music discovery — Emby plays → Last.fm similar → Lidarr | Weekly |
|
||||
| `playback_aware_radarr_discovery.sh` | Behavior-driven movie discovery — Emby watches → TMDB recommendations → Radarr | Weekly |
|
||||
| `playback_aware_sonarr_discovery.sh` | Behavior-driven TV discovery — Emby episodes → TMDB TV recommendations → Sonarr | Weekly |
|
||||
|
||||
---
|
||||
|
||||
## ━━━ HOW THE SCRIPTS RELATE ━━━
|
||||
|
||||
```
|
||||
Every 4hr / weekly (arr sync before rsync):
|
||||
arr_sync.sh ──────────────── syncs tracked IDs across all nodes
|
||||
│ union model: any node adds → all nodes get it
|
||||
│
|
||||
└── then rsync spreads the actual files to all nodes
|
||||
└── then arr_cleanup removes orphans on all nodes (old paths, removed content)
|
||||
|
||||
Daily maintenance window:
|
||||
[Media/media_shares_permissions.sh + media_cleaner.sh run first — from Media/]
|
||||
│
|
||||
▼
|
||||
lidarr_release_fixer.sh ────── reads MBID tag → switches release in Lidarr → RefreshArtist
|
||||
│ (corrected albums get imported before cleanup scans for orphans)
|
||||
▼
|
||||
lidarr_cleanup.sh ──────────── queries Lidarr API → walks /Music → deletes orphans
|
||||
sonarr_cleanup.sh ──────────── queries Sonarr API → walks /Tv_Shows → deletes orphans
|
||||
radarr_cleanup.sh ──────────── queries Radarr API → walks /Movies → deletes orphans
|
||||
│
|
||||
└── each cleanup → notify_emby_scan() → Emby removes ghost entries
|
||||
|
||||
Daily recovery:
|
||||
arrs_failed_stalled_recovery.sh ── importFailed/stalled → blocklist → re-search
|
||||
|
||||
Weekly discovery (WEEKLY_MAINTENANCE_SCRIPTS):
|
||||
playback_aware_lidarr_discovery.sh ─ Emby plays → Last.fm similar → top candidates → Lidarr
|
||||
playback_aware_radarr_discovery.sh ─ Emby watches → TMDB recommendations → top candidates → Radarr
|
||||
playback_aware_sonarr_discovery.sh ─ Emby episodes → TMDB TV recommendations → top candidates → Sonarr
|
||||
│
|
||||
└── each discovery script fires arr search immediately after successful add
|
||||
|
||||
Continuous (started by array_started.sh):
|
||||
start_webhook_listener.sh ── Node.js HTTP server listens on WEBHOOK_PORT
|
||||
│ arr OnUpgrade fires webhook → POST to http://HOST_LAN_IP:WEBHOOK_PORT/webhook?key=SECRET
|
||||
└── upgrade_webhook_handler.sh
|
||||
├── rsync upgraded folder → all remote nodes immediately
|
||||
└── trigger arr library rescan on each remote (accept new file, no quality search)
|
||||
|
||||
Ad-hoc enrichment:
|
||||
lidarr_missing_art.sh ─────── discovers missing artwork → fetches from fanart.tv
|
||||
radarr_tmdb_removed.sh ────── status="deleted" → remove from Radarr + add exclusion
|
||||
sonarr_tvdb_removed.sh ────── status="deleted" → remove from Sonarr + add exclusion
|
||||
```
|
||||
Reference in New Issue
Block a user