Varaverk/Transcodes/README-Transcoding.md

━━━━━ TRANSCODES ━━━━━

Ramdisk-based transcode storage with automatic SSD fallback. Emby transcodes to RAM at full speed. When the ramdisk fills, new sessions shift to SSD automatically — without interrupting anything already playing. When pressure drops, new sessions shift back to RAM.

Two configuration requirements that are not obvious and were both discovered the hard way in production. The Docker mount must use bind-propagation=shared or symlink flips are silently ignored after the first flip. The transcoding-temp directory must be pre-created on the ramdisk or Emby finds the SSD version and routes all sessions there until restarted. Both are documented in Manual-Transcoding.md.

---

━━━ THE PROBLEM THAT BUILT THIS ━━━

Three Storage Options, None Perfect on Their Own
Hard drives: seek times cause buffering on multi-stream transcoding. SSD: fast enough, but constant small file writes at Emby volume accelerate wear over months. RAM: fastest, no wear, files vanish instantly on session end — but limited by available memory.
Fix: RAM by default, SSD as a safety net. The system manages the transition automatically.

Changing Transcode Location Requires Restarting Emby
Configuring Emby to switch between ramdisk and SSD requires a restart. Restarting during active streams drops everyone. A 7-person household with 5 Live TV streams at 9pm is not a good moment to restart Emby.
Fix: symlink indirection. Emby points at a fixed path. The symlink target changes. ffmpeg resolves the symlink once at session start — existing sessions are completely unaffected by flips. Only new sessions follow the new target.

Docker Bind Mount Silently Ignored After First Flip
Symlink flip from ramdisk → SSD worked. Flip back: nothing. All new sessions still land on SSD. The symlink on the host is correct. Emby doesn't see it.
Cause: Docker's default rprivate propagation resolves the symlink target at mount time and locks that inode. Subsequent flips are invisible to the container.
Fix: bind-propagation=shared in Extra Parameters. Host mount changes propagate into the container in real time. Requires --mount syntax — the path mapping UI doesn't support propagation.

Sessions Drifting to SSD After a Day of Operation
System working correctly for hours, then sessions gradually drift to SSD despite the ramdisk having plenty of space.
Cause: cleanup was removing the empty transcoding-temp directory from the ramdisk. Emby then found the SSD fallback version and routed all sessions there.
Fix: transcoding-temp is excluded from cleanup by name. ramdisk_setup.sh pre-creates it at mount time. Both protections together prevent this permanently.

lsof Per File on a Live TV System
Early cleanup called lsof filename per file to check if anything had it open. On a busy Live TV night with 5 simultaneous streams, the ramdisk contains thousands of HLS segment files — thousands of subprocess calls every 3 minutes.
Fix: lsof called once per location to build a complete open-file map. All subsequent checks are O(1) lookups against that map.

---

━━━ WHAT THIS FOLDER DOES ━━━

Three scripts, one goal: keep transcodes on RAM, fall back to SSD when needed.

ramdisk_setup.sh runs at array start — creates the tmpfs, SSD fallback directory, symlink, and pre-creates transcoding-temp. Everything that must exist before Emby starts.

transcode_cleanup.sh runs first in every 3-minute cycle — removes stale files from both ramdisk and SSD. Cleans up before usage is measured, so the manager sees real load.

transcode_manager.sh runs second — measures ramdisk usage, flips the symlink if thresholds are crossed, runs safety checks, displays active sessions, writes the daily log.

The symlink is the mechanism that makes this seamless. Emby writes to a fixed path. That path is a symlink whose target is managed at runtime. Sessions in progress never notice.

---

━━━ RELATIONSHIP TO OTHER FOLDERS ━━━

unRAID_Essentials/
  array_start.sh ──────────────────────────────► ramdisk_setup.sh (at array start)

Orchestrators/
  transcode_management.sh ──── cleanup first ──► transcode_cleanup.sh
                           ──── then manager ──► transcode_manager.sh
                           (every 3 minutes — order non-negotiable)

Monitors/
  weekly_health_digest.sh ◄─── reads ──────────── TRANSCODE_DAILY_LOG

Do not schedule transcode_cleanup.sh or transcode_manager.sh directly. Both are called by transcode_management.sh in the correct order.

---

━━━ SCRIPTS IN THIS FOLDER ━━━

Script	Role	When It Runs
ramdisk_setup.sh	Create tmpfs, SSD fallback dir, symlink, transcoding-temp	At array start (via array_start.sh)
transcode_cleanup.sh	Remove stale files, check for flip-back opportunity	Every 3 min via transcode_management.sh — runs first
transcode_manager.sh	Check usage, flip symlink, safety checks, session display, daily log	Every 3 min via transcode_management.sh — runs second

---

━━━ HOW THE SCRIPTS RELATE ━━━

Array starts
    │
    ▼
ramdisk_setup.sh
    Creates: /mnt/ramdisk_transcodes (tmpfs)
             /mnt/ramdisk_transcodes/transcoding-temp/
             /mnt/cache/Temp_Storage/Emby/Transcodes/ (SSD fallback)
             /mnt/ram-transcode → /mnt/ramdisk_transcodes (symlink)
    │
    ▼
Emby starts, reads transcode path from config
    Sees: /ext-ram-transcode (bind-mounted from /mnt/ram-transcode)
    All new sessions write to: /mnt/ram-transcode → /mnt/ramdisk_transcodes/


Every 3 minutes (transcode_management.sh):
    │
    ├─ transcode_cleanup.sh
    │    Remove files older than TRANSCODE_MAX_AGE, not open by any process
    │    transcoding-temp: never deleted
    │    If ramdisk recovered below RAMDISK_LOW_GB → trigger flip-back
    │
    └─ transcode_manager.sh
         Safety checks (symlink, ramdisk mount, transcoding-temp, permissions)
         smart mode: ramdisk > RAMDISK_WARN_GB → flip symlink to SSD
                     ramdisk < RAMDISK_LOW_GB  → flip symlink back to ramdisk
         Session display (all TRANSCODE_SERVERS)
         Append to TRANSCODE_DAILY_LOG