970 lines
49 KiB
Markdown
970 lines
49 KiB
Markdown
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
# 🎬 MEDIA
|
|
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
|
|
**Library health, cleanliness, and consistency for a multi-server arr stack.**
|
|
Correct permissions so arrs can do their job. Junk file removal so orphan detection
|
|
is not confused by scene debris. Orphaned file cleanup against live arr APIs so
|
|
deleted shows and removed albums actually leave the disk. Emby notified automatically
|
|
so users never see ghost library entries.
|
|
|
|
> **These scripts permanently delete files.** The arr cleanup scripts are protected
|
|
> by multiple safety layers that must all pass before anything is touched — but dry
|
|
> runs and log review are still the right first step on any new system or after any
|
|
> configuration change. The testing procedure at the end of this document exists
|
|
> for a reason.
|
|
|
|
---
|
|
|
|
## ━━━ THE PROBLEM THAT BUILT THIS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
|
|
---
|
|
|
|
### 🔴 Files Owned by Root That Arrs Can't Touch
|
|
|
|
Download clients running without explicit PUID/PGID configuration write files owned
|
|
by root. Arrs running as `nobody:users` cannot rename, move, or delete those files.
|
|
Import fails. Upgrade attempts fail. Deletion requests from arr UI do nothing.
|
|
The file sits on disk permanently in a state where the arr thinks it's managed but
|
|
can't actually manage it.
|
|
|
|
The failure mode is subtle — arr shows the file as imported, everything looks correct
|
|
in the UI, but the file is untouchable by the processes that need to touch it. You
|
|
only discover this when an upgrade is requested and the old version refuses to delete.
|
|
|
|
The fix: `media_shares_permissions.sh` applies `nobody:users` ownership and correct
|
|
permissions to every file across every configured share, daily. Even if a container
|
|
is misconfigured, even if a manual admin copy created root-owned files, even after an
|
|
rsync that didn't preserve ownership — the next daily run normalises everything.
|
|
|
|
---
|
|
|
|
### 🔴 Scene Junk Files Confusing Orphan Detection
|
|
|
|
Scene releases — and many download groups for anime in particular — include a
|
|
constellation of junk files alongside the actual media: `.sfv` checksum files, `.nfo`
|
|
scene info, `.rar` source archives, `.sample` preview files, `.url` group websites.
|
|
After extraction and import these are worthless. They sit in the media folder forever.
|
|
|
|
The problem for arr cleanup: these files are not tracked by any arr. They are not
|
|
music files, TV files, or movie files. They look like orphans — but classifying them
|
|
as orphans means the orphan detection logic is constantly processing noise. Worse,
|
|
some cleanup implementations would try to delete them and log every single one as
|
|
a "cleaned orphan," making the output unreadable for finding actual orphaned media.
|
|
|
|
The fix: `media_cleaner.sh` runs before any arr cleanup script and removes all
|
|
known junk patterns first. By the time arr cleanup runs, every file it finds that
|
|
isn't tracked by the arr is actually a media file that shouldn't be there — not
|
|
a `.sfv` that was never tracked by anyone.
|
|
|
|
---
|
|
|
|
### 🔴 Deleted Shows and Removed Albums Still on Disk
|
|
|
|
When you remove a series from Sonarr, Sonarr deletes its database entries and — if
|
|
configured — sends delete commands for the files. When that fails (permission issue,
|
|
container wasn't running, file was on a different path than Sonarr expected), the
|
|
files stay on disk permanently. Sonarr has forgotten about them. Radarr has forgotten
|
|
about the movie. Lidarr has forgotten about the album. The files just sit there,
|
|
consuming space, invisible to the arr that should manage them.
|
|
|
|
Over years on an active library this accumulates. A drive that should have 2TB free
|
|
has 400GB of content that no arr knows about.
|
|
|
|
The fix: arr cleanup scripts that query the live API for every tracked file path, walk
|
|
the disk, and identify files that exist on disk but are absent from the API response.
|
|
These are genuine orphans — content the arr has definitively moved on from. With a
|
|
configurable age threshold to avoid deleting files that are mid-import, and a maximum
|
|
deletion size gate to catch configuration errors before they become disasters.
|
|
|
|
---
|
|
|
|
### 🔴 Emby Showing Ghost Entries After Cleanup
|
|
|
|
After arr cleanup deletes files, Emby's library still shows them — ghost entries that
|
|
produce "file not found" errors when clicked. Emby's library scan runs on its own
|
|
schedule, which might not happen for hours. Users see broken entries in the library
|
|
until the next scheduled scan.
|
|
|
|
The fix: `notify_emby_scan()` — called automatically after every arr cleanup deletion.
|
|
Triggers Emby's "Clean Missing Files" task immediately. By the time the user refreshes
|
|
the library, the ghost entries are already gone.
|
|
|
|
---
|
|
|
|
### 🔴 No Safety Net on Deletion Size
|
|
|
|
A misconfigured root path — pointing arr cleanup at the wrong directory, or a root
|
|
path mismatch between master.conf and the arr's own settings — means the API returns
|
|
zero tracked files for a root that actually contains thousands. Every file walks as
|
|
an orphan. Everything gets deleted. A library is gone.
|
|
|
|
This is the catastrophic failure mode. It happened during development — once, with a
|
|
test library. It cannot happen again.
|
|
|
|
The fix: `LIDARR_MAX_DELETE_GB`, `SONARR_MAX_DELETE_GB`, `RADARR_MAX_DELETE_GB`.
|
|
If the total size of files queued for deletion exceeds the configured limit, the script
|
|
stops and requires `--i-know-what-im-doing` to proceed. The flag name is long and
|
|
annoying by design — it cannot be added by accident and it cannot be forgotten
|
|
what it means.
|
|
|
|
---
|
|
|
|
## ━━━ WHY ORDER MATTERS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
|
|
These scripts are called in sequence by `Orchestrators/media_management.sh`.
|
|
The order is not arbitrary — each script depends on the previous ones having run.
|
|
|
|
```bash
|
|
# master.conf — MEDIA_MAINTENANCE_JOBS defines the order
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
MEDIA_MAINTENANCE_JOBS=(
|
|
"Media/media_shares_permissions.sh" # 1. Permissions — always first
|
|
"Media/media_cleaner.sh anime" # 2. Junk removal — before orphan scan
|
|
"Media/media_cleaner.sh media" # 3. Same for media shares
|
|
"Media/lidarr_cleanup.sh" # 4. Arr cleanup — after permissions + clean
|
|
"Media/sonarr_cleanup.sh" # 5.
|
|
"Media/radarr_cleanup.sh" # 6.
|
|
)
|
|
```
|
|
|
|
**Permissions before everything else:**
|
|
Arr cleanup needs `nobody:users` ownership to delete files. If a file is owned by
|
|
`root:root` and the script runs as `nobody`, the deletion fails silently — the file
|
|
looks like it was processed but is still on disk. Permissions first ensures this
|
|
can't happen.
|
|
|
|
**Junk removal before arr cleanup:**
|
|
The arr cleanup scripts walk the disk and compare against the arr API. Junk files
|
|
(`.sfv`, `.rar`, `.nfo`) are not tracked by any arr — they look like orphans. Removing
|
|
them first means the orphan detection only encounters actual media files. Cleaner
|
|
output, more accurate detection, less noise.
|
|
|
|
**Arr cleanup last:**
|
|
Depends on clean folders and correct ownership. Both are guaranteed by the time
|
|
arr cleanup runs.
|
|
|
|
---
|
|
|
|
## ━━━ HOST AWARENESS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
|
|
Each script runs on both servers but only operates on the shares and arrs that belong
|
|
to that server. `detect_hosts()` aliases all `HOST*_` prefixed vars to their
|
|
unprefixed names.
|
|
|
|
```
|
|
HOST1 — source of truth for: HOST2 — source of truth for:
|
|
Movies (Radarr) Anime_Movies (his Radarr)
|
|
Tv_Shows (Sonarr) Anime_Shows (his Sonarr)
|
|
Music (Lidarr)
|
|
|
|
Permissions and cleaner scripts run on both servers against their own shares.
|
|
Arr cleanup scripts check MY_ID and only run against the arr they own.
|
|
Lidarr only runs on HOST1 — exits cleanly on HOST2 with no action.
|
|
```
|
|
|
|
No manual `HOST1`/`HOST2` comparisons exist in any of these scripts. Everything
|
|
routes through `MY_ID`.
|
|
|
|
---
|
|
|
|
## ━━━ SCRIPTS AT A GLANCE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
|
|
| Script | Purpose | When |
|
|
|--------|---------|------|
|
|
| `media_shares_permissions.sh` | Apply correct ownership + permissions to all media shares | Daily via `media_management.sh` |
|
|
| `media_cleaner.sh anime` | Remove junk files from anime share folders | Daily via `media_management.sh` |
|
|
| `media_cleaner.sh media` | Remove junk files from media share folders | Daily via `media_management.sh` |
|
|
| `lidarr_cleanup.sh` | Delete orphaned music files not tracked by Lidarr | Daily via `media_management.sh` |
|
|
| `sonarr_cleanup.sh` | Delete orphaned TV files not tracked by Sonarr | Daily via `media_management.sh` |
|
|
| `radarr_cleanup.sh` | Delete orphaned movie files not tracked by Radarr | Daily via `media_management.sh` |
|
|
|
|
---
|
|
|
|
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
## 🔐 media_shares_permissions.sh
|
|
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
|
|
Applies correct ownership and permissions recursively to all configured media shares.
|
|
The first job in the daily maintenance sequence — arr cleanup depends on this running
|
|
first. A failsafe that runs daily whether or not anything appears to need fixing.
|
|
|
|
---
|
|
|
|
### ── The Permissions Model ───────────────────────────────────────────────────
|
|
|
|
```bash
|
|
# master.conf
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# unRAID's standard for Docker-accessible media shares.
|
|
# All arr containers, Emby, Tdarr, and download clients use nobody:users.
|
|
# These values match the PUID=99 PGID=100 container environment variables.
|
|
#
|
|
# Directories: 755 nobody:users
|
|
# Owner (nobody) — rwx enter, list, create files
|
|
# Group (users) — r-x enter and list
|
|
# Others — r-x Samba guests can browse
|
|
# No world-write — prevents accidental deletion by unauthenticated access
|
|
#
|
|
# Files: 664 nobody:users
|
|
# Owner (nobody) — rw read + write
|
|
# Group (users) — rw arrs can import, rename, delete
|
|
# Others — r Samba guests can read
|
|
# No execute bit — media files are never executable
|
|
#
|
|
PERMISSIONS_DIR_MODE="755"
|
|
PERMISSIONS_FILE_MODE="664"
|
|
PERMISSIONS_OWNER="nobody:users"
|
|
```
|
|
|
|
**Two separate passes — not a single recursive chmod:**
|
|
Directories need the execute bit (to enter them). Files must never have the execute
|
|
bit. A single `chmod -R 664` would remove execute from directories, making them
|
|
inaccessible. The script runs `find -type d` and `find -type f` separately with the
|
|
correct mode for each.
|
|
|
|
---
|
|
|
|
### ── Why It Runs Daily Even When Things Look Fine ────────────────────────────
|
|
|
|
```bash
|
|
# master_host1.conf
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# Ownership drift happens from multiple sources, not just misconfigured containers:
|
|
#
|
|
# rsync without --chown → brings source server's nobody:users correctly
|
|
# but a misconfigured rsync brings root:root
|
|
# Manual admin copies → scp and cp default to root:root
|
|
# New containers → default to root before PUID/PGID is set
|
|
# unRAID updates → can reset container environment variables
|
|
# Download client restarts → if PUID/PGID lost on restart, next download is root
|
|
#
|
|
# Running daily means the window between "wrong ownership created" and
|
|
# "arr fails to manage the file" is at most 24 hours. Usually it is overnight.
|
|
#
|
|
# When this script corrects zero files per run, everything is configured correctly.
|
|
# When it corrects many files per run, a container has wrong PUID/PGID.
|
|
# Check these first: SABnzbd, qBittorrent, slskd — most common culprits.
|
|
# Correct values: PUID=99 PGID=100 in Docker template environment variables.
|
|
#
|
|
HOST1_MEDIA_PERMISSION_SHARES=(
|
|
"/mnt/user/Movies"
|
|
"/mnt/user/Tv_Shows"
|
|
"/mnt/user/Music"
|
|
"/mnt/user/Kids_Movies"
|
|
"/mnt/user/Kids_Tv_Shows"
|
|
"/mnt/user/Sports"
|
|
"/mnt/user/stand-up_comedy"
|
|
)
|
|
```
|
|
|
|
---
|
|
|
|
### ── Usage ───────────────────────────────────────────────────────────────────
|
|
|
|
```bash
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# Normal — called by media_management.sh, rarely run directly.
|
|
# Safe to run manually at any time — idempotent, only changes what's wrong.
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
media_shares_permissions.sh
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# Dry run — show how many files would be corrected per share.
|
|
# If the number is unexpectedly large, check container PUID/PGID settings first.
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
media_shares_permissions.sh --dry-run
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# Status — show configured share list and current ownership state.
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
media_shares_permissions.sh --status
|
|
|
|
# Verbose — show ownership correction count per share
|
|
media_shares_permissions.sh --log
|
|
```
|
|
|
|
> **On large libraries this runs for 20-30 minutes.** Millions of files with a
|
|
> recursive walk and chown/chmod on each takes time. This is expected and normal —
|
|
> it is why it runs overnight in the maintenance window, not during peak hours.
|
|
|
|
---
|
|
|
|
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
## 🧹 media_cleaner.sh
|
|
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
|
|
Removes junk files from media share folders using configurable pattern lists.
|
|
Two profiles — `anime` and `media` — each with their own folder list and file patterns.
|
|
Runs after `media_shares_permissions.sh` and before any arr cleanup script.
|
|
|
|
---
|
|
|
|
### ── What Gets Removed ────────────────────────────────────────────────────────
|
|
|
|
```bash
|
|
# master.conf
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# These patterns appear in anime releases and general media downloads.
|
|
# None of these are content you want in your library after extraction and import.
|
|
#
|
|
ANIME_FILE_PATTERNS=(
|
|
"*.sfv" # checksum verification — useless after download verified
|
|
"*.md5" "*.sha1" # other checksum formats — same reason
|
|
"*.nfo" # scene info file — group name, release notes, not metadata
|
|
"*.url" "*.lnk" # website shortcuts — never needed
|
|
"*.rar" "*.zip" # source archives — kept by some clients after extraction
|
|
"*.info" # tool output files
|
|
"*.torrent" # torrent descriptor left by some clients
|
|
"*.sample*" # scene preview clip — never needed after import
|
|
"*.proof*" # screenshot proving encode quality — never needed
|
|
"*sync-conflict*" # Syncthing conflict copies — should not be in media folders
|
|
"*.scr" "*.exe" # executables — should NEVER be in a media folder
|
|
"*.srr" # scene recovery record — useless post-download
|
|
"*.log" # tool/client logs
|
|
"*.json" # metadata or tool output
|
|
)
|
|
|
|
# Media profile has additional patterns not relevant for anime:
|
|
MEDIA_FILE_PATTERNS=(
|
|
"${ANIME_FILE_PATTERNS[@]}" # all anime patterns plus:
|
|
"*.iso" # disc images after ripping — keep the ripped version
|
|
"*.lrc" # lyric files in media folders — not needed here
|
|
)
|
|
```
|
|
|
|
---
|
|
|
|
### ── Two Profiles — Different Folders ───────────────────────────────────────
|
|
|
|
```bash
|
|
# master_host1.conf
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# Profiles map to different share sets on each server.
|
|
# HOST1 owns Movies/Shows/Music. HOST2 owns Anime_Movies/Anime_Shows.
|
|
# detect_hosts() ensures each server only cleans the shares it owns.
|
|
#
|
|
HOST1_ANIME_CLEAN_FOLDERS=(
|
|
# HOST1 mirrors anime FROM HOST2 — clean the mirrored copies
|
|
"/mnt/user/Anime_Movies"
|
|
"/mnt/user/Anime_Movies-Old"
|
|
"/mnt/user/Anime_Shows"
|
|
"/mnt/user/Anime_Shows-Old"
|
|
)
|
|
|
|
HOST1_MEDIA_CLEAN_FOLDERS=(
|
|
# HOST1's own shares — source of truth for these
|
|
"/mnt/user/Kids_Movies"
|
|
"/mnt/user/Kids_Tv_Shows"
|
|
"/mnt/user/Movies"
|
|
"/mnt/user/Music"
|
|
"/mnt/user/Sports"
|
|
"/mnt/user/stand-up_comedy"
|
|
"/mnt/user/Tv_Shows"
|
|
)
|
|
```
|
|
|
|
---
|
|
|
|
### ── Safety Note ─────────────────────────────────────────────────────────────
|
|
|
|
```
|
|
The cleaner deletes by pattern — it does not check what any arr thinks about
|
|
the file. It runs before arr cleanup specifically so arr cleanup sees clean folders.
|
|
|
|
DO NOT add patterns that match media you want to keep:
|
|
*.mkv *.mp4 *.avi *.m4v — video files
|
|
*.flac *.mp3 *.m4a — audio files
|
|
*.srt *.sub *.ass — subtitle files (managed by Bazarr)
|
|
*.jpg *.png — artwork (generated by arrs and Emby)
|
|
|
|
Always use --dry-run when adding new patterns — review before committing.
|
|
```
|
|
|
|
---
|
|
|
|
### ── Usage ───────────────────────────────────────────────────────────────────
|
|
|
|
```bash
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# Profile is required — no default.
|
|
# Always run --dry-run first when adding new patterns or folders.
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
media_cleaner.sh anime # clean anime share folders
|
|
media_cleaner.sh media # clean media share folders
|
|
|
|
media_cleaner.sh anime --dry-run # preview — show what would be deleted
|
|
media_cleaner.sh media --dry-run # same for media profile
|
|
|
|
media_cleaner.sh anime --log # verbose — show every file examined
|
|
media_cleaner.sh anime --status # show folder list and patterns
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
## 🎵 lidarr_cleanup.sh
|
|
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
|
|
Removes orphaned music files from the library that Lidarr no longer tracks.
|
|
Queries the live Lidarr API for every tracked file path, walks the music root on disk,
|
|
and deletes anything that is not tracked, not protected, and old enough to be
|
|
definitively past the import window.
|
|
|
|
> **Lidarr runs on HOST1 only.** This script exits cleanly on HOST2 with no action.
|
|
> HOST1 is source of truth for Music.
|
|
|
|
---
|
|
|
|
### ── File Classification ──────────────────────────────────────────────────────
|
|
|
|
```
|
|
Every file found on disk falls into exactly one category:
|
|
|
|
TRACKED → Lidarr API returned this exact path → leave it alone
|
|
PROTECTED → matches LIDARR_PROTECTED_PATTERNS → never delete
|
|
ORPHAN → music extension, not tracked, old enough → delete
|
|
JUNK → not a music extension, not protected → delete (any age)
|
|
RECENT → not tracked, under LIDARR_ORPHAN_AGE days → skip for now
|
|
```
|
|
|
|
---
|
|
|
|
### ── Why Protected Patterns Are Critical ────────────────────────────────────
|
|
|
|
```bash
|
|
# master.conf
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# Lidarr generates these files alongside your music — they are NOT in the
|
|
# tracked file API response. Without this list they would be deleted as orphans.
|
|
#
|
|
# Deleting *.jpg removes cover art from every album in the library.
|
|
# Emby and Lidarr both lose artwork display. Requires a full rescan to recover.
|
|
#
|
|
# Deleting *.nfo removes metadata Lidarr generated for media center compatibility.
|
|
#
|
|
# Deleting *.lrc removes synced lyrics used by music players that support them.
|
|
#
|
|
LIDARR_PROTECTED_PATTERNS=(
|
|
"*.jpg" "*.jpeg" "*.png" # cover art — generated by Lidarr per album
|
|
"*.nfo" # metadata — generated by Lidarr for media centers
|
|
"*.lrc" # synced lyrics — generated/downloaded by Lidarr
|
|
)
|
|
#
|
|
# Never remove from this list without understanding what Lidarr generates in
|
|
# your specific setup. The consequences are visible and annoying to fix.
|
|
```
|
|
|
|
---
|
|
|
|
### ── Safety Layers — All Must Pass Before Any Deletion ──────────────────────
|
|
|
|
```bash
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# Every check is a gate. If any gate fails, the script exits without touching
|
|
# a single file. There is no way to "push through" a failed safety check
|
|
# without the explicit override flag.
|
|
#
|
|
# 1. Container running + healthy
|
|
# Lidarr must be running and not in starting/unhealthy state.
|
|
# A stopped container has an empty (or inaccessible) API.
|
|
#
|
|
# 2. API reachable
|
|
# curl to the Lidarr endpoint must succeed. No API = no tracked file list.
|
|
# Without the tracked list, everything on disk looks like an orphan.
|
|
#
|
|
# 3. API version matches
|
|
# Major version must match the tested version in master.conf.
|
|
# API breaking changes between major versions would corrupt classification.
|
|
#
|
|
# 4. Artist count > 0
|
|
# If Lidarr has no artists, something is wrong with the database or config.
|
|
# Proceeding would delete everything.
|
|
#
|
|
# 5. Tracked file count > 0
|
|
# If Lidarr reports zero tracked files, the API returned an empty response.
|
|
# Proceeding would delete everything.
|
|
#
|
|
# 6. Tracked count >= LIDARR_MIN_TRACKED_PCT % of last known count
|
|
# If the API returns far fewer tracked files than the last run, something
|
|
# changed dramatically. Could be a Lidarr database corruption or a root
|
|
# path change. Stop and alert rather than delete thousands of files.
|
|
#
|
|
# 7. Deletion size < LIDARR_MAX_DELETE_GB
|
|
# If the total size of queued deletions exceeds this limit, stop.
|
|
# Require --i-know-what-im-doing to proceed.
|
|
# This is the last line of defense against a misconfigured root path.
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
```
|
|
|
|
---
|
|
|
|
### ── Age Threshold ────────────────────────────────────────────────────────────
|
|
|
|
```bash
|
|
# master.conf
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# When Lidarr downloads a file it exists on disk before it is fully processed.
|
|
# The import queue can take hours for large batches. A newly downloaded file
|
|
# that isn't tracked yet is not an orphan — it is mid-import.
|
|
#
|
|
# LIDARR_ORPHAN_AGE defines the safe window. Files under this age are
|
|
# classified as RECENT and skipped entirely, regardless of tracked status.
|
|
#
|
|
# 7 days is conservative — adjust if your import workflow is slower than this.
|
|
# Be careful reducing this below 3 days — Lidarr's import retry logic can
|
|
# hold files in queue for 24-48 hours after initial download failure.
|
|
#
|
|
LIDARR_ORPHAN_AGE=7 # days — files newer than this are never classified as orphans
|
|
```
|
|
|
|
---
|
|
|
|
### ── Override Flags ────────────────────────────────────────────────────────────
|
|
|
|
```bash
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# --i-know-what-im-doing
|
|
# Required when deletion would exceed LIDARR_MAX_DELETE_GB.
|
|
# The flag name is long and specific by design — you must type it deliberately.
|
|
# It cannot be added by accident. It cannot be forgotten what it means.
|
|
# Use when you have intentionally removed a large portion of your library
|
|
# and want cleanup to follow through.
|
|
#
|
|
# --skip-strike-list
|
|
# Bypasses the LIDARR_ORPHAN_AGE age check.
|
|
# Deletes RECENT files too — files that are under the age threshold.
|
|
# Use when you know a batch of recently downloaded files are actually orphans
|
|
# and you want them cleaned without waiting for the age window.
|
|
#
|
|
# ⚠️ NUCLEAR MODE — both flags active simultaneously:
|
|
# Age check bypassed. Size threshold bypassed. Deletes everything on first pass.
|
|
# Use when Soularr has filled the gaps across your whole library and you want
|
|
# a clean one-pass wipe of everything Lidarr doesn't track.
|
|
# The script author takes no responsibility for data loss with both flags active.
|
|
# The user accepts full responsibility. This is intentional by design.
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
```
|
|
|
|
---
|
|
|
|
### ── Configuration ────────────────────────────────────────────────────────────
|
|
|
|
```bash
|
|
# master_host1.conf
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# HOST1 only — Lidarr runs on HOST1, Music is HOST1's share.
|
|
#
|
|
HOST1_LIDARR_URL="http://192.168.50.2:8686"
|
|
HOST1_LIDARR_API_KEY="b2977e71ef074bc0a0529d9fcce3b2dc"
|
|
HOST1_LIDARR_MUSIC_ROOT="/mnt/user/Music" # must match Lidarr root path EXACTLY
|
|
#
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# LIDARR_MUSIC_ROOT must match the path configured in Lidarr:
|
|
# Lidarr UI → Settings → Media Management → Root Folders
|
|
#
|
|
# A mismatch means every file on disk looks untracked.
|
|
# All of them appear as orphans. LIDARR_MAX_DELETE_GB is the only thing
|
|
# between a path mismatch and losing your entire music library.
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
|
|
# master.conf
|
|
LIDARR_ORPHAN_AGE=7
|
|
LIDARR_MIN_TRACKED_PCT=80 # alert if API returns < 80% of last known count
|
|
LIDARR_MAX_DELETE_GB=50 # stop if total deletion size exceeds 50GB
|
|
LIDARR_EXTENSIONS=("flac" "mp3" "m4a" "wav" "aac" "ogg" "opus" "wma")
|
|
LIDARR_PROTECTED_PATTERNS=("*.jpg" "*.jpeg" "*.png" "*.nfo" "*.lrc")
|
|
```
|
|
|
|
---
|
|
|
|
### ── Usage ───────────────────────────────────────────────────────────────────
|
|
|
|
```bash
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# ALWAYS run --dry-run --log first. Review every classification decision.
|
|
# See the safe testing procedure at the end of this document.
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
lidarr_cleanup.sh --dry-run --log # full review before committing
|
|
|
|
lidarr_cleanup.sh # live run after dry run passes review
|
|
lidarr_cleanup.sh --log # live run with verbose output
|
|
lidarr_cleanup.sh --status # show configuration and API status
|
|
|
|
# Override flags — use with deliberate intention
|
|
lidarr_cleanup.sh --i-know-what-im-doing
|
|
lidarr_cleanup.sh --skip-strike-list
|
|
lidarr_cleanup.sh --i-know-what-im-doing --skip-strike-list # nuclear
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
## 📺 sonarr_cleanup.sh
|
|
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
|
|
Removes orphaned TV episode files from the library that Sonarr no longer tracks.
|
|
Same classification logic as `lidarr_cleanup.sh` applied to TV files.
|
|
|
|
> **Sonarr is host-aware.** HOST1 Sonarr manages `Tv_Shows`. HOST2 Sonarr manages
|
|
> `Anime_Shows`. `detect_hosts()` aliases the correct URL, API key, and root path.
|
|
> The same script works correctly on both servers with no manual routing.
|
|
|
|
---
|
|
|
|
### ── Protected Patterns — Sonarr ────────────────────────────────────────────
|
|
|
|
```bash
|
|
# master.conf
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# Sonarr generates artwork and metadata that do NOT appear in the tracked file
|
|
# API response. Additionally, Bazarr manages subtitles through Sonarr —
|
|
# subtitle files are not tracked by Sonarr directly.
|
|
#
|
|
SONARR_PROTECTED_PATTERNS=(
|
|
"*.jpg" "*.jpeg" "*.png" # show + episode artwork (per series and per episode)
|
|
"*.nfo" # metadata — NFO files for media center compatibility
|
|
"*.srt" "*.sub" # subtitles — managed by Bazarr via Sonarr
|
|
"*.ass" "*.ssa" # advanced subtitle formats — same
|
|
)
|
|
#
|
|
# *.ts (transport stream) IS in SONARR_EXTENSIONS — not protected.
|
|
# Live TV recordings are .ts files tracked by Sonarr for recorded episodes.
|
|
# Orphaned .ts recordings should be cleaned up like any other orphaned episode.
|
|
```
|
|
|
|
---
|
|
|
|
### ── Post-Deletion Emby Notification ─────────────────────────────────────────
|
|
|
|
```bash
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# After deleting any files, notify_emby_scan() is called automatically.
|
|
# Triggers Emby's "Clean Missing Files" task immediately.
|
|
# Users see clean library entries within seconds of deletion — not hours later
|
|
# when Emby's own scheduled scan would eventually catch up.
|
|
#
|
|
# Configured via:
|
|
HOST1_EMBY_URL="http://192.168.50.2:8096"
|
|
HOST1_EMBY_API_KEY="0c27448d93a7431f9ac63569f7655829"
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
```
|
|
|
|
---
|
|
|
|
### ── Configuration ────────────────────────────────────────────────────────────
|
|
|
|
```bash
|
|
# master_host1.conf (HOST2 has matching HOST2_ vars for Anime_Shows)
|
|
HOST1_SONARR_URL="http://192.168.50.2:8989"
|
|
HOST1_SONARR_API_KEY="d43a3ec6cf1549edb4af0cc63f98b2a9"
|
|
HOST1_SONARR_TV_ROOT="/mnt/user/Tv_Shows" # must match Sonarr root path EXACTLY
|
|
|
|
# master.conf
|
|
SONARR_ORPHAN_AGE=7
|
|
SONARR_MAX_DELETE_GB=50
|
|
SONARR_EXTENSIONS=("mkv" "mp4" "avi" "m4v" "ts" "wmv" "mov")
|
|
SONARR_PROTECTED_PATTERNS=("*.jpg" "*.jpeg" "*.png" "*.nfo" "*.srt" "*.sub" "*.ass" "*.ssa")
|
|
```
|
|
|
|
---
|
|
|
|
### ── Usage ───────────────────────────────────────────────────────────────────
|
|
|
|
```bash
|
|
sonarr_cleanup.sh --dry-run --log # always first
|
|
sonarr_cleanup.sh # live run
|
|
sonarr_cleanup.sh --log # verbose
|
|
sonarr_cleanup.sh --status # configuration and API status
|
|
sonarr_cleanup.sh --i-know-what-im-doing # size override
|
|
sonarr_cleanup.sh --skip-strike-list # age override
|
|
sonarr_cleanup.sh --i-know-what-im-doing --skip-strike-list # nuclear
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
## 🎞️ radarr_cleanup.sh
|
|
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
|
|
Removes orphaned movie files from the library that Radarr no longer tracks.
|
|
Same classification logic applied to movie files.
|
|
|
|
> **Radarr is host-aware.** HOST1 Radarr manages `Movies`. HOST2 Radarr manages
|
|
> `Anime_Movies`. `detect_hosts()` aliases the correct URL, API key, and root path.
|
|
|
|
---
|
|
|
|
### ── Protected Patterns — Radarr ────────────────────────────────────────────
|
|
|
|
```bash
|
|
# master.conf
|
|
RADARR_PROTECTED_PATTERNS=(
|
|
"*.jpg" "*.jpeg" "*.png" # movie artwork — fanart, posters, thumbnails
|
|
"*.nfo" # metadata — NFO for media center compatibility
|
|
"*.srt" "*.sub" # subtitles — managed by Bazarr via Radarr
|
|
"*.ass" "*.ssa" # advanced subtitle formats — same
|
|
)
|
|
```
|
|
|
|
---
|
|
|
|
### ── Configuration ────────────────────────────────────────────────────────────
|
|
|
|
```bash
|
|
# master_host1.conf (HOST2 has matching HOST2_ vars for Anime_Movies)
|
|
HOST1_RADARR_URL="http://192.168.50.2:7878"
|
|
HOST1_RADARR_API_KEY="d43a3ec6cf1549edb4af0cc63f98b2a9"
|
|
HOST1_RADARR_MOVIES_ROOT="/mnt/user/Movies" # must match Radarr root path EXACTLY
|
|
|
|
# master.conf
|
|
RADARR_ORPHAN_AGE=7
|
|
RADARR_MAX_DELETE_GB=50
|
|
RADARR_EXTENSIONS=("mkv" "mp4" "avi" "m4v" "wmv" "mov")
|
|
RADARR_PROTECTED_PATTERNS=("*.jpg" "*.jpeg" "*.png" "*.nfo" "*.srt" "*.sub" "*.ass" "*.ssa")
|
|
```
|
|
|
|
---
|
|
|
|
### ── Usage ───────────────────────────────────────────────────────────────────
|
|
|
|
```bash
|
|
radarr_cleanup.sh --dry-run --log # always first
|
|
radarr_cleanup.sh # live run
|
|
radarr_cleanup.sh --log # verbose
|
|
radarr_cleanup.sh --status # configuration and API status
|
|
radarr_cleanup.sh --i-know-what-im-doing # size override
|
|
radarr_cleanup.sh --skip-strike-list # age override
|
|
radarr_cleanup.sh --i-know-what-im-doing --skip-strike-list # nuclear
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━ SAFE TESTING PROCEDURE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
|
|
> **The arr cleanup scripts permanently delete files.** There is no recycle bin.
|
|
> There is no undo. Follow this procedure on first use, after any root path
|
|
> change, after any API key change, and after any significant arr library change.
|
|
|
|
---
|
|
|
|
### Step 1 — Dry Run With Full Logging
|
|
|
|
```bash
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# --dry-run: no files are touched
|
|
# --log: every classification decision is printed
|
|
# Review the complete output before proceeding.
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
lidarr_cleanup.sh --dry-run --log
|
|
sonarr_cleanup.sh --dry-run --log
|
|
radarr_cleanup.sh --dry-run --log
|
|
```
|
|
|
|
---
|
|
|
|
### Step 2 — Review the Output Carefully
|
|
|
|
```
|
|
Ask these questions about the dry run output:
|
|
|
|
Are TRACKED files the ones you expect?
|
|
→ Files you know are in the arr should show as TRACKED.
|
|
→ If known files show as ORPHAN, the root path may be wrong.
|
|
|
|
Is the ORPHAN count reasonable?
|
|
→ A healthy library cleanup removes dozens to hundreds of files, not tens of thousands.
|
|
→ If the number is unexpectedly large, stop. Do not proceed.
|
|
|
|
Are PROTECTED patterns working?
|
|
→ You should see artwork (*.jpg) and subtitles (*.srt) classified as PROTECTED.
|
|
→ If they show as ORPHAN, the protected patterns are wrong.
|
|
|
|
Are RECENT files being correctly skipped?
|
|
→ Files downloaded in the last 7 days should show as RECENT, not ORPHAN.
|
|
→ If recent downloads show as ORPHAN, the age threshold may be too aggressive.
|
|
|
|
Is the root path correct?
|
|
→ The scan should start at the right directory.
|
|
→ Check the first few lines of output — the scan root is logged.
|
|
```
|
|
|
|
---
|
|
|
|
### Step 3 — Check the Numbers
|
|
|
|
```bash
|
|
# If dry run reports an unexpectedly large number of orphans, check:
|
|
|
|
# 1. Root path mismatch — what does arr think the root is?
|
|
# Lidarr: Settings → Media Management → Root Folders
|
|
# Sonarr: Settings → Media Management → Root Folders
|
|
# Radarr: Settings → Media Management → Root Folders
|
|
# Must exactly match LIDARR_MUSIC_ROOT / SONARR_TV_ROOT / RADARR_MOVIES_ROOT
|
|
|
|
# 2. API returning empty — is the arr running?
|
|
docker ps | grep -E "Lidarr|Sonarr|Radarr"
|
|
|
|
# 3. Library scan not complete — recently added content not yet indexed
|
|
# Trigger a manual library scan in the arr UI and wait for completion
|
|
```
|
|
|
|
---
|
|
|
|
### Step 4 — Run Live
|
|
|
|
```bash
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# Only after dry run review passes. Start with the arr you're most confident about.
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
lidarr_cleanup.sh
|
|
sonarr_cleanup.sh
|
|
radarr_cleanup.sh
|
|
```
|
|
|
|
---
|
|
|
|
### Step 5 — Verify in Arr UI
|
|
|
|
```
|
|
After running, check in the arr's UI:
|
|
|
|
Library count: should not have dropped significantly
|
|
a healthy cleanup removes a small number of files,
|
|
not a large percentage of the library
|
|
|
|
Missing files: check if any monitored content shows as missing
|
|
this would indicate a tracked file was incorrectly deleted
|
|
|
|
Emby library: should show no ghost entries (notify_emby_scan handles this automatically)
|
|
if ghost entries appear, trigger a manual "Clean Missing Files" task in Emby
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━ CONFIGURATION REFERENCE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
|
|
---
|
|
|
|
### 📋 master_host*.conf — Per-Host Configuration
|
|
|
|
```bash
|
|
# master_host1.conf
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
|
|
# Shares this server applies permissions to
|
|
HOST1_MEDIA_PERMISSION_SHARES=(
|
|
"/mnt/user/Movies"
|
|
"/mnt/user/Tv_Shows"
|
|
"/mnt/user/Music"
|
|
)
|
|
|
|
# Folders cleaned by each profile
|
|
HOST1_ANIME_CLEAN_FOLDERS=(
|
|
"/mnt/user/Anime_Movies"
|
|
"/mnt/user/Anime_Shows"
|
|
)
|
|
HOST1_MEDIA_CLEAN_FOLDERS=(
|
|
"/mnt/user/Movies"
|
|
"/mnt/user/Tv_Shows"
|
|
"/mnt/user/Music"
|
|
)
|
|
|
|
# Arr connection details — must match arr settings exactly
|
|
HOST1_LIDARR_URL="http://192.168.50.2:8686"
|
|
HOST1_LIDARR_API_KEY="b2977e71ef074bc0a0529d9fcce3b2dc"
|
|
HOST1_LIDARR_MUSIC_ROOT="/mnt/user/Music"
|
|
|
|
HOST1_SONARR_URL="http://192.168.50.2:8989"
|
|
HOST1_SONARR_API_KEY="d43a3ec6cf1549edb4af0cc63f98b2a9"
|
|
HOST1_SONARR_TV_ROOT="/mnt/user/Tv_Shows"
|
|
|
|
HOST1_RADARR_URL="http://192.168.50.2:7878"
|
|
HOST1_RADARR_API_KEY="d43a3ec6cf1549edb4af0cc63f98b2a9"
|
|
HOST1_RADARR_MOVIES_ROOT="/mnt/user/Movies"
|
|
|
|
HOST1_EMBY_URL="http://192.168.50.2:8096"
|
|
HOST1_EMBY_API_KEY="0c27448d93a7431f9ac63569f7655829"
|
|
```
|
|
|
|
---
|
|
|
|
### 📋 master.conf — Shared Configuration
|
|
|
|
```bash
|
|
# master.conf
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
|
|
# ── Permissions ────────────────────────────────────────────────────────────
|
|
PERMISSIONS_DIR_MODE="755"
|
|
PERMISSIONS_FILE_MODE="664"
|
|
PERMISSIONS_OWNER="nobody:users"
|
|
|
|
# ── File Patterns ──────────────────────────────────────────────────────────
|
|
ANIME_FILE_PATTERNS=("*.sfv" "*.md5" "*.sha1" ...)
|
|
MEDIA_FILE_PATTERNS=("${ANIME_FILE_PATTERNS[@]}" "*.iso" "*.lrc")
|
|
|
|
# ── Arr Cleanup Thresholds ─────────────────────────────────────────────────
|
|
LIDARR_ORPHAN_AGE=7 # days — files newer than this are RECENT
|
|
LIDARR_MIN_TRACKED_PCT=80 # alert if API returns < 80% of last known count
|
|
LIDARR_MAX_DELETE_GB=50 # stop if total deletion exceeds this
|
|
LIDARR_EXTENSIONS=(...)
|
|
LIDARR_PROTECTED_PATTERNS=(...)
|
|
|
|
SONARR_ORPHAN_AGE=7
|
|
SONARR_MAX_DELETE_GB=50
|
|
SONARR_EXTENSIONS=(...)
|
|
SONARR_PROTECTED_PATTERNS=(...)
|
|
|
|
RADARR_ORPHAN_AGE=7
|
|
RADARR_MAX_DELETE_GB=50
|
|
RADARR_EXTENSIONS=(...)
|
|
RADARR_PROTECTED_PATTERNS=(...)
|
|
|
|
# ── Orchestrator Job Order ─────────────────────────────────────────────────
|
|
MEDIA_MAINTENANCE_JOBS=(
|
|
"Media/media_shares_permissions.sh"
|
|
"Media/media_cleaner.sh anime"
|
|
"Media/media_cleaner.sh media"
|
|
"Media/lidarr_cleanup.sh"
|
|
"Media/sonarr_cleanup.sh"
|
|
"Media/radarr_cleanup.sh"
|
|
)
|
|
```
|
|
|
|
---
|
|
|
|
## ━━━ ADDING A NEW ARR ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
|
|
To add Readarr, Whisparr, or any other arr cleanup to the ecosystem:
|
|
|
|
```bash
|
|
# 1. Copy radarr_cleanup.sh as the template — same classification logic applies
|
|
cp radarr_cleanup.sh readarr_cleanup.sh
|
|
|
|
# 2. Update the API endpoint variable names and root path variable names
|
|
# Replace RADARR_ with READARR_ throughout
|
|
|
|
# 3. Add configuration to master_host*.conf
|
|
HOST1_READARR_URL="http://192.168.50.2:8787"
|
|
HOST1_READARR_API_KEY="your-api-key"
|
|
HOST1_READARR_BOOKS_ROOT="/mnt/user/Books"
|
|
|
|
# 4. Add thresholds to master.conf
|
|
READARR_ORPHAN_AGE=7
|
|
READARR_MAX_DELETE_GB=50
|
|
READARR_EXTENSIONS=("epub" "pdf" "mobi" "azw3" "cbz" "cbr")
|
|
READARR_PROTECTED_PATTERNS=("*.jpg" "*.jpeg" "*.png" "*.nfo")
|
|
|
|
# 5. Add to MEDIA_MAINTENANCE_JOBS in master.conf
|
|
MEDIA_MAINTENANCE_JOBS=(
|
|
...existing jobs...
|
|
"Media/readarr_cleanup.sh" # add at the end — after permissions and cleaner
|
|
)
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# media_management.sh picks it up automatically.
|
|
# No changes to the orchestrator needed.
|
|
# Run --dry-run --log before scheduling.
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
``` |