feat: slskd reconnect guard in downloaders_reset, mass v2 sync
- downloaders_reset: connection check block before slskd API sections; triggers PUT /api/v0/server reconnect if disconnected, polls 60s, gates Stuck Searches and Dead Transfer Records on SLSKD_CONNECTED - Sync all modified/new/deleted files from v2 refactor across Docker_Essentials, Media, Monitors, Partnership, Rsync, Tools, Transcodes, unRAID_Essentials, common.sh, master confs, and new Manual/README docs
This commit is contained in:
Gmer4Lfe
@@ -2,66 +2,98 @@
|
||||
# ==============================================================================================
|
||||
# ================================= Transcode Cleanup ==========================================
|
||||
# ==============================================================================================
|
||||
# Removes old inactive transcode files from both ramdisk and SSD fallback locations.
|
||||
# Called every 5 minutes by transcode_manager.sh — must be fast and non-blocking.
|
||||
# Never deletes files that are currently open by any process.
|
||||
#
|
||||
# ── SAFETY RULES ──────────────────────────────────────────────────────────────────────────────
|
||||
# PURPOSE
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Removes stale transcode files from both ramdisk and SSD fallback locations.
|
||||
# Called by transcode_management.sh (Orchestrators/) before transcode_manager.sh —
|
||||
# cleanup must run first so the manager sees real active-session usage, not
|
||||
# inflated usage from stale files. Must be fast and non-blocking.
|
||||
#
|
||||
# A file is eligible for deletion only if ALL conditions are true:
|
||||
# 1. Older than TRANSCODE_MAX_AGE minutes (mtime — last modified time)
|
||||
# 1. Older than TRANSCODE_MAX_AGE minutes (mtime — last write time)
|
||||
# 2. Not currently open by any process (checked via lsof pre-built map)
|
||||
#
|
||||
# ── WHY NOT SESSION-AWARE CLEANUP ─────────────────────────────────────────────────────────────
|
||||
# ffmpeg generates folder names independently of the media server API session IDs.
|
||||
# There is no reliable correlation between API session IDs and transcoding-temp subfolder
|
||||
# names — matching them would falsely treat active sessions as ended.
|
||||
# lsof is the correct and reliable active file check — if ffmpeg has a file open,
|
||||
# lsof sees it regardless of folder naming or session state.
|
||||
# ==============================================================================================
|
||||
# DESIGN PRINCIPLES
|
||||
# ==============================================================================================
|
||||
#
|
||||
# ── TRANSCODING-TEMP PROTECTION ───────────────────────────────────────────────────────────────
|
||||
# The transcoding-temp directory is excluded from deletion even when empty.
|
||||
# If cleanup removes the empty transcoding-temp folder from the ramdisk, Emby finds
|
||||
# the SSD version instead and all new sessions land on SSD until Emby restarts.
|
||||
# ! -name "transcoding-temp" exclusion in find prevents this permanently.
|
||||
# lsof Called Once, Not Per File
|
||||
# On a busy Live TV system the ramdisk contains thousands of HLS segment files.
|
||||
# Calling lsof once per file creates thousands of subprocess calls every 3 minutes.
|
||||
# lsof is called once per location to build a complete open-file map. All subsequent
|
||||
# checks are O(1) lookups against that map — thousands of files, one lsof call.
|
||||
#
|
||||
# ── PERFORMANCE ───────────────────────────────────────────────────────────────────────────────
|
||||
# lsof is called ONCE per location — never once per file.
|
||||
# Per-file lsof stalls on busy systems with live TV buffering hundreds of segments.
|
||||
# No Session-Aware Cleanup
|
||||
# ffmpeg generates folder names independently of the media server API session IDs.
|
||||
# There is no reliable correlation between API session IDs and transcoding-temp
|
||||
# subfolder names. Attempting to correlate them would falsely treat active sessions
|
||||
# as ended. lsof is the correct check — if ffmpeg has a file open, it is active
|
||||
# regardless of folder naming or session state.
|
||||
#
|
||||
# Open file check uses in-memory associative array (OPEN_FILES_MAP):
|
||||
# Was: echo "$OPEN_FILES" | grep -qF "$file" — O(n) per file → O(n²) total
|
||||
# Now: [[ -n "${OPEN_FILES_MAP[$file]:-}" ]] — O(1) per file → O(n) total
|
||||
# Same lesson as TRACKED_MAP in arr cleanup scripts.
|
||||
# transcoding-temp Is Never Deleted
|
||||
# If cleanup removes the empty transcoding-temp folder from the ramdisk, Emby
|
||||
# searches all accessible paths for an existing one, finds the SSD fallback version,
|
||||
# and routes all new sessions there until Emby restarts. The directory is excluded
|
||||
# from find by name — protected even when completely empty.
|
||||
#
|
||||
# ── POST-CLEANUP SYMLINK FLIP ─────────────────────────────────────────────────────────────────
|
||||
# After cleanup, if ramdisk has recovered below RAMDISK_LOW_GB and symlink currently
|
||||
# points at SSD → triggers transcode_manager.sh to flip back to ramdisk.
|
||||
# Post-Cleanup Flip-Back
|
||||
# After removing stale files, checks whether ramdisk usage dropped below
|
||||
# RAMDISK_LOW_GB. If so — and symlink currently points at SSD — triggers a
|
||||
# flip back to ramdisk. This is the recovery path; the manager handles
|
||||
# the fill-up path.
|
||||
#
|
||||
# ── HOST AWARENESS ────────────────────────────────────────────────────────────────────────────
|
||||
# detect_hosts() sets MY_ID and aliases RAMDISK_PATH, TRANSCODE_SSD, RAMDISK_LOW_GB.
|
||||
# Each server cleans its own transcode locations at the correct thresholds.
|
||||
# ==============================================================================================
|
||||
# OPERATIONAL SAFEGUARDS
|
||||
# ==============================================================================================
|
||||
#
|
||||
# ── SAFEGUARDS ────────────────────────────────────────────────────────────────────────────────
|
||||
# acquire_lock "wait" — wait if previous cleanup still running
|
||||
# detect_hosts() — correct paths and thresholds per host
|
||||
# lsof timeout — lsof call capped at 15 seconds per location
|
||||
# OPEN_FILES_MAP — in-memory O(1) active file lookup
|
||||
# transcoding-temp guard — never deletes this directory
|
||||
# Silent by default — runs every 5 minutes, must not produce noise when healthy
|
||||
# Wait Lock
|
||||
# acquire_lock "wait" — waits if a previous cleanup run is still active rather
|
||||
# than exiting. The caller's 3-minute interval can overlap on a slow system.
|
||||
#
|
||||
# lsof Timeout
|
||||
# lsof call capped at 15 seconds per location — prevents blocking indefinitely
|
||||
# on a system with many open files.
|
||||
#
|
||||
# transcoding-temp Guard
|
||||
# `! -name "transcoding-temp"` in the find command — protected unconditionally.
|
||||
#
|
||||
# Silent by Default
|
||||
# Runs every 3 minutes — must not produce noise when healthy.
|
||||
#
|
||||
# ==============================================================================================
|
||||
# CONFIGURATION
|
||||
# ==============================================================================================
|
||||
#
|
||||
# master_host*.conf
|
||||
#
|
||||
# ── CONFIGURATION (master_host*.conf) ─────────────────────────────────────────────────────────
|
||||
# HOST*_RAMDISK_PATH / HOST*_TRANSCODE_SSD / HOST*_RAMDISK_LOW_GB
|
||||
# Aliased by detect_hosts()
|
||||
# Aliased by detect_hosts() → RAMDISK_PATH / TRANSCODE_SSD / RAMDISK_LOW_GB.
|
||||
#
|
||||
# ── CONFIGURATION (master.conf) ───────────────────────────────────────────────────────────────
|
||||
# TRANSCODE_MAX_AGE — minutes before an inactive transcode file is eligible
|
||||
# TRANSCODE_ORPHAN_AGE — minutes for orphan detection (informational — future use)
|
||||
# master.conf
|
||||
#
|
||||
# TRANSCODE_MAX_AGE
|
||||
# Minutes before an inactive transcode file is eligible for deletion. (default: 20)
|
||||
#
|
||||
# TRANSCODE_ORPHAN_AGE
|
||||
# Minutes for orphan folder detection. (default: 30)
|
||||
#
|
||||
# ==============================================================================================
|
||||
# RUNTIME MODES
|
||||
# ==============================================================================================
|
||||
#
|
||||
# transcode_cleanup.sh
|
||||
# Remove stale files from ramdisk and SSD. Check for flip-back opportunity.
|
||||
#
|
||||
# transcode_cleanup.sh --dry-run
|
||||
# Show which files would be deleted. No deletions, no flip.
|
||||
#
|
||||
# transcode_cleanup.sh --status
|
||||
# Show current file counts, ages, and open-file status per location.
|
||||
#
|
||||
# transcode_cleanup.sh --log
|
||||
# Verbose per-file output including age, open status, and deletion result.
|
||||
#
|
||||
# ── USAGE ─────────────────────────────────────────────────────────────────────────────────────
|
||||
# transcode_cleanup.sh — normal cleanup run
|
||||
# transcode_cleanup.sh --dry-run — show what would be deleted
|
||||
# transcode_cleanup.sh --status — show current state
|
||||
# transcode_cleanup.sh --log — verbose per-file output
|
||||
# ==============================================================================================
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
Reference in New Issue
Block a user