Commit Graph
59 Commits
Author SHA1 Message Date
Gmer4Lfe 96d8a5e3f0 Make a file earn its deletion over consecutive runs, so a partial classification failure too small to trip the tracked-count gate cannot remove anything 2026-08-26 17:58:42 -04:00
Gmer4Lfe 0431e720de A cap that aborts cannot drain a backlog bigger than itself, so make it a per-run budget and let the queue clear over consecutive nights 2026-08-26 17:55:21 -04:00
Gmer4Lfe d42b1e2dda ffprobe exit 0 with a chapter-track warning is not corruption; guard the delete path against a detector that is wrong at scale 2026-08-25 18:29:43 -04:00
Gmer4Lfe b58282b3d6 Read a filesystem once where two probes were answering the same question 2026-08-24 18:46:26 -04:00
Gmer4Lfe 879109e55d Drain the orphan backlog safest-first instead of aborting, so a run over budget still makes progress 2026-08-22 16:37:31 -04:00
Gmer4Lfe 35e59d2510 Let the plugin read partner vars from the same RAM cache bash reads, so PHP and bash cannot disagree about a partner 2026-08-16 19:40:49 -04:00
Gmer4Lfe 00dee37e22 Triage the classification leaks the metadata rule says it cannot judge
The scan's own header calls reverse-anime leaks "genuine judgment calls" and
reports them as a count. Seventeen of them concealed two live-action crime
dramas filed under anime for as long as nobody read the list. The scan now
persists that bucket as its own verdict, and the sweep asks the model to sort
only those eleven — not the eleven hundred it already classifies correctly.
Files findings for misfiled and uncertain only; donghua and anime-adjacent are
answers, not problems, and filing them would rebuild the undifferentiated list
this exists to break up. Moves nothing — --move stays a flag a human types.
2026-08-14 22:03:55 -04:00
Gmer4Lfe 1b46033584 grep -c prints zero and exits one, so every echo-0 fallback produced a two-line count 2026-08-06 22:31:50 -04:00
Gmer4Lfe 8a2707ee37 Correct and extend folder docs for Docker Essentials, Watchdogs, Media and Arrs Stack
The docs had drifted from the scripts — a script that no longer exists, three wrong variable
names, a reversed run order, and seven scheduled scripts that were never documented at all.
2026-08-01 22:59:07 -04:00
Gmer4Lfe e8b114094a Bring script headers onto the template and close safeguard gaps
Headers claimed protections the code never had, and several destructive paths had no
guard against a collapsed config value.
2026-08-01 20:37:59 -04:00
Gmer4Lfe 19a9f6410d Cache fanart.tv misses so the nightly art run stops re-querying artwork upstream has never had 2026-07-28 18:51:06 -04:00
Gmer4Lfe a1db8f7b2d Restore executable bit on the three arr cleanup scripts 2026-07-27 18:38:46 -04:00
Gmer4Lfe d680bd0549 Gate arr orphan deletion on ctime and stop the nightly permissions pass from restamping it — imports preserve the release's original mtime, so the age gate never actually fired for real content 2026-07-27 18:38:18 -04:00
Gmer4Lfe 3084546b32 Delete download orphans the arr cannot match instead of holding them forever — past the age gate an unnameable entry will never import, and clearing it lets a monitored title search for a copy the arr can parse 2026-07-27 17:38:49 -04:00
Gmer4Lfe 74ea1a5218 Run the download orphan cleaner daily — a weekly batch of newly-eligible orphans can exceed the delete cap, which aborts the whole pass and rolls the backlog into an even larger next run 2026-07-27 17:24:40 -04:00
Gmer4Lfe 895347f20f Add weekly download orphan cleaner — nothing covered the SAB Completed folders and 755G of orphans accumulated since 2022 2026-07-26 09:55:17 -04:00
Gmer4Lfe 5e92d9920a Add Radarr/Movies coverage to arr_corruption_scan.sh
Extends the Sonarr-only corruption scan into a generic per-arr loop (same
pattern as arr_full_rescan.sh) instead of a second script, since the
scan/strike/remediate logic is identical and only the API shape differs.
Radarr's moviefile list is fetched batched to include secondary/alternate-
edition files, not just each movie's primary file.

Also fixes two bugs found while testing: build_arr_path_map()'s internal
non-local `for key in ...` loop was clobbering the per-arr API key
variable, and arr_api()'s error output (stdout, not stderr) was getting
appended into the batch fetch file on any single failed call, corrupting
jq's parse of the whole batch and silently zeroing out that arr's results.
2026-07-21 18:05:26 -04:00
Gmer4Lfe ec518d4758 Prune stale chronic failure counts once an item stops being a problem
FAILURE_COUNTS persisted across runs but was never reset on success —
confirmed live 2026-07-19 that Sekirei S06E04 sat permanently flagged
chronic at count 4 despite already having hasFile=true. "Consecutive
failures" was really being measured as cumulative-for-all-time. Now
prunes any media_id's count at the end of each arr's pass if it no
longer appears in that run's problem-item set, so a resolved item's
history doesn't linger and falsely trip the circuit breaker later.
2026-07-19 17:00:15 -04:00
Gmer4Lfe 7252d4aad3 Fix try_smart_import() silently failing every ManualImport it fires
The candidate objects from /manualimport only nest ids under .series.id
and .movie.id, but Sonarr/Radarr's ManualImport command requires them
flattened to top-level seriesId/episodeIds or movieId — every call was
failing instantly with "Series/Movie with ID 0 does not exist" while the
caller only checked for HTTP 201 (accepted), so every "smart-imported"
this script has ever logged was actually a silent no-op. Now flattens
the ids before submitting and briefly polls the command afterward so a
fast failure falls through to the normal blocklist+research path instead
of being reported as success.
2026-07-19 10:36:37 -04:00
Gmer4Lfe 2ce368abba Fix radarr_cleanup.sh missing secondary edition files as false-positive orphans
Radarr 6+ supports multiple tracked files per movie (alternate editions/extras),
but the movie list's embedded movieFile.path only ever reflects the primary one.
Relying on it alone flagged legitimately-tracked secondary edition files as
orphans. Now batch-fetches /moviefile?movieId=X across all tracked movies to
catch every file, not just the primary.
2026-07-19 10:04:47 -04:00
Gmer4Lfe 004bde2c62 Add strike system to arr_corruption_scan.sh — require repeat corrupt detections before remediating 2026-07-18 22:50:19 -04:00
Gmer4Lfe 77845829ad Add --filter to arr_corruption_scan.sh for targeted testing
Full-library scans are slow to reach any one show by chance — this lets a test
target a specific known-corrupt file/show without waiting on scan order.
2026-07-18 16:50:33 -04:00
Gmer4Lfe ab35fcd84d Add arr_corruption_scan.sh — sequential ffprobe-based corruption scan for Sonarr
Healarr does the same job but crashes on a Go concurrency bug (unsynchronized
map access) whenever multiple corruption events land close together — confirmed
via its own crash log, not fixable from our side. Processing one file at a time
here sidesteps the whole bug class instead of trying to work around it.

Delete + explicit EpisodeSearch rather than relying on Sonarr's own background
missing-search cycle, since that skips unmonitored episodes and this shouldn't.
2026-07-18 16:48:32 -04:00
Gmer4Lfe 7bf47be106 Poll MoveMovie command to completion instead of trusting DB fields
Mirrors the Sonarr fix — same MoveMovieService one-at-a-time drain architecture,
never confirmed live on the Radarr side but the DB-instant/move-async split
is identical, so the same batch-verification race applies.
2026-07-17 23:00:08 -04:00
Gmer4Lfe 4f0cd1083a Poll MoveSeries command to completion instead of trusting DB fields
Confirmed live: DB fields flip instantly on the moveFiles=true PUT but the
physical move drains async, one command at a time — a batch could report
every series moved while most were still sitting at the old path.
2026-07-17 22:54:44 -04:00
Gmer4Lfe 79059c20fa Add --move mode to sonarr_classification_scan.sh, matching Radarr
Acts on forward misplacements and reverse-kids-leak (adult content in
the kids root — not a judgment call, same reasoning as the Radarr
side), moving to the new SONARR_GENERAL_ROOT config var. Reverse-anime
-leak stays report-only — deliberate style placements like Castlevania
are common and valid there. episodeFileCount is Sonarr's hasFile
equivalent: series with 0 files get relocated + an immediate
SeriesSearch instead of a file move. One series at a time, verified
after each — matches the race condition found doing this by hand for
the Fate/Zero and Fate/Stay Night moves earlier in the session.
2026-07-17 13:02:22 -04:00
Gmer4Lfe 9c05692a24 Include reverse-kids-leak in --move — it's not actually a judgment call
Unlike reverse-anime-leak (deliberate style placements like Castlevania
are common and valid there), reverse-kids-leak's signal is specifically
"adult certification with zero Family/Animation genre" — confirmed live
the 4 titles it catches (Addams Family, Saving Mr. Banks, Dark Shadows,
The DUFF) are all genuinely non-kids content. Moves them to the new
RADARR_GENERAL_ROOT config var (container-path literal for the general
Movies root, needed since RADARR_MOVIES_ROOT is a filesystem path).
2026-07-17 12:20:23 -04:00
Gmer4Lfe e688500cf5 Add --remove-junk mode: delete + import-exclude bad-metadata entries
Junk entries have no release to blocklist and no file to delete —
only a bad monitored record with a thin/wrong TMDb match. Removes the
record and adds it to Radarr's import exclusion list, same mechanism
radarr_tmdb_removed.sh already uses, so the same bad match can't get
re-added by a future Overseerr request or list sync.
2026-07-17 12:15:55 -04:00
Gmer4Lfe 6f7bb8a6aa Relocate + trigger search for hasFile=false forward-misplacements
Instead of skipping monitored-but-undownloaded titles entirely,
correct their DB pointer (rootFolderPath/path) now so Radarr saves
under the right root whenever it grabs a release, and fire an
immediate MoviesSearch rather than waiting for the next scheduled
one. Junk entries stay excluded — nothing to search for there.
2026-07-17 12:11:30 -04:00
Gmer4Lfe 1777c56efa Refresh shared tracked-data cache after --move writes to avoid a stale-cache window for other scripts 2026-07-17 12:08:10 -04:00
Gmer4Lfe ca6f16bafb Exclude hasFile=false entries from move targets — nothing to physically move 2026-07-17 12:02:57 -04:00
Gmer4Lfe 46ddc55345 Add --move mode to radarr_classification_scan.sh
Acts only on forward misplacements (clear-cut anime/kids classified
content sitting outside its dedicated root) — reverse leaks and junk
entries stay report-only, since those need human judgment or a
different action (removal, not a move). One movie at a time with a
real re-fetch-and-verify after each, matching the race condition found
doing this by hand for Sonarr earlier in the session.
2026-07-17 11:58:28 -04:00
Gmer4Lfe 10d6f372b3 Widen Sonarr reverse-kids-leak to non-US adult certification scales 2026-07-17 11:53:39 -04:00
Gmer4Lfe de57f87b82 Fix reverse_kids_leak losing genre data after object field reduction in jq 2026-07-17 11:51:30 -04:00
Gmer4Lfe 9dbcdc0bd7 Add anime/kids content misclassification detection for Radarr and Sonarr
Overseerr lets users request content into the wrong root folder; these
new report-only scans classify every tracked movie/series (anime, kids,
regular) from metadata alone and flag mismatches against the actual
root folder, in both directions. Rules were validated against real
library data before being adopted — see the header comments in each
script and the master.conf notes above the curated lists.
2026-07-17 11:49:23 -04:00
Gmer4Lfe d2e071dece Use find -printf instead of per-file stat fork in cleanup scripts
lidarr_cleanup.sh/sonarr_cleanup.sh/radarr_cleanup.sh each forked a
separate stat call per file during classification. find already has to
stat() every entry to know it's -type f, so -printf '%s %T@ %p' gets
size+mtime for free during the walk itself. Measured: 5.77s for all
175,954 files in the Lidarr music root (walk + stat combined) vs 85.98s
for stat alone on a 20K-file subset of the same library -- roughly 130x
faster per file, and collapses two passes into one. Verified path
parsing preserves spaces/parens/unicode exactly via read's trailing-
field capture before switching.
2026-07-17 01:52:30 -04:00
Gmer4Lfe 072df6b153 Replace external dirname/basename with parameter expansion in hot loops
lidarr_missing_art.sh's album-directory-map loop (both the cache-hit and
live-fallback branches) and arr_profile_enforcer.sh's _is_kids_path()
called dirname/basename once per item -- 127K+ tracks and ~4000
series/movies respectively, each call forking a subprocess. Measured:
0.39s vs 72.2s for 20K calls, ~185x. Verified identical output against
real paths (including unicode/space/paren edge cases) before switching.
2026-07-17 01:39:21 -04:00
Gmer4Lfe 254b400caf Add missing RUNTIME MODES section, document --log
arr_cache_prefill.sh had no RUNTIME MODES section at all; arr_full_rescan.sh
had one but didn't mention --log despite supporting it via parse_args.
Comment-only.
2026-07-17 01:29:06 -04:00
Gmer4Lfe 976d8d84d7 Add OPERATIONAL SAFEGUARDS headers, skip prefill during active rescans
arr_full_rescan.sh, arr_cache_prefill.sh, and arr_rescan_monitor.sh were
missing the standard SAFEGUARDS header section other Arrs_Stack/Tools
scripts have. Also: arr_cache_prefill.sh now checks for an active rescan
before fetching, instead of doing a live fetch that arr_cache_write()
would just refuse to persist anyway -- avoids wasted API calls every
30min during a long rescan. arr_rescan_monitor.sh was also missing an
actual root check despite writing cache files; added it to match
convention rather than just document a safeguard that wasn't there.
2026-07-17 01:23:31 -04:00
Gmer4Lfe bac1ef1c17 Update headers on today's arr-caching changes
Comment-only. Headers on the scripts touched during today's caching work
(cache-first fetches, write-through per-item cache, single-walk
consolidation, movieFile-embedded fix) still described pre-change
behavior. Also brought common.sh's top-level cache doc block current --
it was written for the single-consumer 2026-07-16 state and didn't
mention the tmpfs move, the write guard, or the 15+ consumers that now
go through it.
2026-07-17 01:08:46 -04:00
Gmer4Lfe de2879bdee Add write-through cache for per-item track/episode data
lidarr_cleanup.sh and sonarr_cleanup.sh already walk every artist/series
individually (trackFile/episodefile) for their own cleanup decisions --
that walk now also writes the raw per-item data through to a short-lived
tmpfs cache (arr_item_cache_write, 4h freshness, no persistent backup
since every consumer already has its own live fallback). lidarr_missing_art.sh
runs later in the same nightly window and now reads that cache first,
skipping its own redundant per-artist walk entirely on a hit. Sonarr side
is write-through only for now -- no second consumer exists yet, but the
data's there for whenever one does. Future consumers: arr_get_cached_items()
first, live per-item fetch as fallback, same pattern as these two.
2026-07-17 00:55:14 -04:00
Gmer4Lfe a018245f40 Consolidate cleanup scripts' double filesystem walk into one
lidarr_cleanup.sh/sonarr_cleanup.sh/radarr_cleanup.sh each walked their
full media root twice per run: once to classify files and total the
deletion size for the safety-threshold check, then again to actually
delete. The size check needs to know the total before deleting anything,
not before knowing what to delete -- the classification pass now records
orphan/junk paths as it finds them, and the deletion pass just reads that
list instead of re-walking and re-classifying the whole tree again. Only
affects real (non-dry-run) executions, where the second walk used to
happen. Also merges two separate stat calls per file into one.
2026-07-17 00:37:31 -04:00
Gmer4Lfe a22433967a Eliminate radarr_cleanup.sh's per-movie API calls
Radarr's movie list already embeds movieFile.path on every hasFile=true
entry -- confirmed live, zero exceptions across the full library. The
separate moviefile?movieId=X call per movie (2896 of them) was fetching
data already sitting in the list this script fetches anyway. One live
list fetch replaces up to 2896 per-movie calls, every time this function
runs including rescan-aware retries.
2026-07-17 00:25:18 -04:00
Gmer4Lfe 5866097d6a Make all arr library-list fetches cache-first with live fallback
Every script that fetches the full Lidarr/Sonarr/Radarr tracked-library
list now goes through arr_get_tracked_data() instead of hitting the API
directly -- cache-first when fresh, live fetch as fallback when stale,
waits out an active rescan before either. Per-item file data (trackFile/
episodefile/moviefile) stays live-only everywhere, since that's the
actual disk-truth these scripts' decisions depend on and was never part
of what's cached.

Also adds arr_cache_prefill.sh to CRITICAL_MAINTENANCE_SCRIPTS (30min
tier) with a short 1min wait ceiling, so the cache stays consistently
fresh instead of only refreshing whenever some other script happens to
write through. A full cache refresh for all three arrs measured at ~12s
total live -- nothing like the multi-hour cost of an actual rescan.
2026-07-16 23:27:23 -04:00
Gmer4Lfe 2c3f0b9cb1 Guard arr cache writes against in-flight rescans, add rescan monitor tool
A direct arr_cache_write() call mid-rescan wrote a partial snapshot that
looked like real data loss to every consumer of the cache. The guard now
lives in arr_cache_write() itself so every caller is protected, not just
arr_get_tracked_data(). arr_rescan_monitor.sh closes the resulting gap for
rescans triggered outside arr_full_rescan.sh's own trigger-and-wait path.
2026-07-16 22:57:54 -04:00
Gmer4Lfe fb13958881 Add weekly full-library rescan job for Lidarr/Sonarr/Radarr
Organic scans only touch files actually involved in an import — an
artist/series/movie that already has files sitting untouched on disk
never gets its tracked-file stats refreshed on its own. Confirmed
2026-07-16: Lidarr reported ~23% of its true trackFileCount with no
scan running, for artists whose files were verified present and
readable the whole time. Downstream scripts trust these stats as
source of truth for the share, so drift needs to be caught before
someone notices a suspiciously low number.
2026-07-16 15:57:12 -04:00
Gmer4Lfe c7ecf99c3f Generalize tracked-data cache from Lidarr-only to all three arrs
Shared cache/rescan-duration logic in common.sh now takes an arr_type
param instead of being Lidarr-specific, so Sonarr and Radarr cleanup
scripts get the same cache-first fetch + rescan-aware retry Lidarr had.
Avoids redundant full-library API calls across scripts run back to back,
and stops false failures when a fetch lands mid-rescan.
2026-07-16 15:34:56 -04:00
Gmer4Lfe 59cee06f45 Add Lidarr tracked-data cache + duplicate artist cleanup
Shared cache (lidarr_get_tracked_data() in common.sh) so scripts stop
hitting Lidarr's live API for tracked counts every run, and stop
treating a mid-rescan dip as a genuine problem — a whole-library
RescanFolders legitimately makes trackFileCount read far below normal
while it re-verifies every file (confirmed 2026-07-16: 22% of normal
mid-scan). Cache reads fresh-if-recent, waits out an active rescan
(calibrated to that command's own historical duration, tracked per
command name since RescanFolders and DownloadedAlbumsScan take wildly
different amounts of time), then falls back to a stale cache rather
than hard-failing after a few strikes.

lidarr_cleanup.sh: no longer stacks a fresh DownloadedAlbumsScan on
top of one already running, and the tracked-count floor check now
waits out a genuine rescan instead of aborting on every overlap.

lidarr_duplicate_artist_cleanup.sh (new): finds case-insensitive
duplicate artist entries — same display name, different MusicBrainz
ID, added when a search/list-sync matches the wrong same-named artist.
Deletes the empty phantom side and blocks it from Import List
Exclusions, leaves genuinely-different-real-artists alone (checked by
album title overlap, deduped per-artist first so a legitimate reissue
under an artist's own catalog doesn't false-flag as cross-artist
overlap), and only notifies for the rare case where both sides have
real, overlapping content.

lidarr_cache_prefill.sh (new): warms the cache at array start so nothing
reads it cold after boot.

lidarr_missing_art.sh, lidarr_release_fixer.sh: write-through the cache
as a side effect of fetches they already needed for their own purposes.
2026-07-16 14:02:35 -04:00
Gmer4Lfe 8bdf7eeb9c Add smart-import decision for importBlocked items before blocklisting
Most importBlocked downloads are junk/duplicates and still fall straight
through to blocklist+research unchanged. But some are releases arr already
correctly parsed (episode/movie, quality, language all known) that just
trip the title-vs-grab-history safety net — those get imported directly
if the target has no file yet, or the candidate is a same-language
resolution upgrade over what's already there. Sonarr/Radarr only; Lidarr's
manual-import matching doesn't reliably resolve album/track context.

Gated by ARR_SMART_IMPORT_ENABLED (default true) and
ARR_SMART_IMPORT_PREFERRED_LANGUAGE (default English).
2026-07-15 17:30:18 -04:00
Gmer4Lfe f30863f452 Paginate get_queue_data() so the recovery script actually sees the whole queue
page=1&pageSize=200 silently truncated anything past record 200. Sonarr's
queue currently runs 1700+ during a large search campaign, which pushed
every importBlocked/warning item past page 1 — the script logged 'clean'
every run while 52 stuck imports sat completely unseen despite yesterday's
importBlocked fix matching them correctly once actually queried.
2026-07-15 16:48:53 -04:00