Varaverk/Transcodes/README-Transcoding.md

# ━━━━━ TRANSCODES ━━━━━

**Ramdisk-based transcode storage with automatic SSD fallback.** Emby transcodes to RAM
at full speed. When the ramdisk fills, new sessions shift to SSD automatically —
without interrupting anything already playing. When pressure drops, new sessions shift
back to RAM.

> **Three configuration requirements that are not obvious and were all discovered the
> hard way in production.** The `transcoding-temp` directory must be pre-created on
> the ramdisk or Emby finds the SSD version and routes all sessions there until
> restarted. GPU containers require `--gpus "device=UUID"` in Extra Parameters —
> not `--runtime=nvidia`. Do **not** use `bind-propagation=shared` on Unraid 7.3+
> (Docker 29.x / runc v1.3.5+) — it crashes container start; the ramdisk is already
> MS\_SHARED at the kernel level. Additionally: any GPU-accelerated sidecar (OCR plugins,
> credit detection) that holds VRAM and never releases it will starve Emby and Jellyfin
> of VRAM for transcoding — Jellyfin hard-fails, Emby silently falls back to CPU.
> All three are documented in Manual-Transcoding.md.

---

## ━━━ REQUIRED EXTRA PARAMETERS ━━━

> **Stop. Set this before starting Emby or Jellyfin. If you Google how to add GPU
> access to a Docker container on unRAID you will find the wrong answer.** Every
> forum post and guide shows `--runtime=nvidia` + `NVIDIA_VISIBLE_DEVICES`. That
> method breaks on container rebuilds. Use `--gpus` instead.

In the unRAID Docker template, open **Advanced View** and paste the following into
the **Extra Parameters** field. Do not use the path mapping UI for the transcode
directory — it does not support the `--mount` syntax.

**GPU-accelerated (Emby, Jellyfin with NVENC/NVDEC) — use this:**
```
--gpus "device=GPU-62e1659d-1ed4-935f-3df3-4bb4339438f1" --pids-limit=0 --mount type=bind,source=/mnt/ram-transcode,target=/ext-ram-transcode
```

**Non-GPU — use this:**
```
--mount type=bind,source=/mnt/ram-transcode,target=/ext-ram-transcode
```

In Emby and Jellyfin's transcoding settings, set the transcode temp path to `/ext-ram-transcode`.

Find your GPU UUID: `nvidia-smi -L`  
HOST1 GPU UUID (Quadro P2000): `GPU-62e1659d-1ed4-935f-3df3-4bb4339438f1`

**→ Full explanation: [Manual-Transcoding.md](Manual-Transcoding.md)**

---

## ━━━ THE PROBLEM THAT BUILT THIS ━━━

**Three Storage Options, None Perfect on Their Own**  
Hard drives: seek times cause buffering on multi-stream transcoding. SSD: fast enough,
but constant small file writes at Emby volume accelerate wear over months. RAM: fastest,
no wear, files vanish instantly on session end — but limited by available memory.  
Fix: RAM by default, SSD as a safety net. The system manages the transition automatically.

**Changing Transcode Location Requires Restarting Emby**  
Configuring Emby to switch between ramdisk and SSD requires a restart. Restarting
during active streams drops everyone. A 7-person household with 5 Live TV streams at
9pm is not a good moment to restart Emby.  
Fix: symlink indirection. Emby points at a fixed path. The symlink target changes.
ffmpeg resolves the symlink once at session start — existing sessions are completely
unaffected by flips. Only new sessions follow the new target.

**Docker Bind Mount Silently Ignored After First Flip** *(historical — resolved differently)*  
Symlink flip from ramdisk → SSD worked. Flip back: nothing. All new sessions still land
on SSD. The symlink on the host is correct. Emby doesn't see it.  
Original fix was `bind-propagation=shared` — but this crashes runc v1.3.5+ (Docker 29.x,
Unraid 7.3+) on any container start. The ramdisk tmpfs is `MS_SHARED` at the kernel level,
so propagation is inherited automatically without specifying it in Docker. Do not add
`bind-propagation=shared` to Extra Parameters.

**Sessions Drifting to SSD After a Day of Operation**  
System working correctly for hours, then sessions gradually drift to SSD despite the
ramdisk having plenty of space.  
Cause: cleanup was removing the empty `transcoding-temp` directory from the ramdisk.
Emby then found the SSD fallback version and routed all sessions there.  
Fix: `transcoding-temp` is excluded from cleanup by name. `ramdisk_setup.sh` pre-creates
it at mount time. Both protections together prevent this permanently.

**lsof Per File on a Live TV System**  
Early cleanup called `lsof filename` per file to check if anything had it open. On a busy
Live TV night with 5 simultaneous streams, the ramdisk contains thousands of HLS segment
files — thousands of subprocess calls every 7 minutes.  
Fix: lsof called once per location to build a complete open-file map. All subsequent
checks are O(1) lookups against that map.

---

## ━━━ WHAT THIS FOLDER DOES ━━━

Three scripts, one goal: keep transcodes on RAM, fall back to SSD when needed.

`ramdisk_setup.sh` runs at array start — creates the tmpfs, SSD fallback directory,
symlink, and pre-creates `transcoding-temp`. Everything that must exist before Emby starts.

`transcode_cleanup.sh` runs first in every 7-minute cycle — removes stale files from both
ramdisk and SSD. Cleans up before usage is measured, so the manager sees real load.

`transcode_manager.sh` runs second — measures ramdisk usage, flips the symlink if
thresholds are crossed, runs safety checks, displays active sessions, writes the daily log.

The symlink is the mechanism that makes this seamless. Emby writes to a fixed path. That
path is a symlink whose target is managed at runtime. Sessions in progress never notice.

---

## ━━━ RELATIONSHIP TO OTHER FOLDERS ━━━

```
System_Essentials/
  array_started.sh ──────────────────────────────► ramdisk_setup.sh (at array start)

Orchestrators/
  transcode_management.sh ──── cleanup first ──► transcode_cleanup.sh
                           ──── then manager ──► transcode_manager.sh
                           (every 7 minutes — order non-negotiable)

Monitors/
  weekly_health_digest.sh ◄─── reads ──────────── TRANSCODE_DAILY_LOG
```

Do not schedule `transcode_cleanup.sh` or `transcode_manager.sh` directly.
Both are called by `transcode_management.sh` in the correct order.

---

## ━━━ SCRIPTS IN THIS FOLDER ━━━

| Script | Role | When It Runs |
|--------|------|-------------|
| `ramdisk_setup.sh` | Create tmpfs, SSD fallback dir, symlink, transcoding-temp | At array start (via array_started.sh) |
| `transcode_cleanup.sh` | Remove stale files, check for flip-back opportunity | Every 7 minutes via transcode_management.sh — runs first |
| `transcode_manager.sh` | Check usage, flip symlink, safety checks, session display, daily log | Every 7 minutes via transcode_management.sh — runs second |

---

## ━━━ HOW THE SCRIPTS RELATE ━━━

```
Array starts
    │
    ▼
ramdisk_setup.sh
    Creates: /mnt/ramdisk_transcodes (tmpfs)
             /mnt/ramdisk_transcodes/transcoding-temp/
             /mnt/cache/Temp_Storage/Emby/Transcodes/ (SSD fallback)
             /mnt/ram-transcode → /mnt/ramdisk_transcodes (symlink)
    │
    ▼
Emby starts, reads transcode path from config
    Sees: /ext-ram-transcode (bind-mounted from /mnt/ram-transcode)
    All new sessions write to: /mnt/ram-transcode → /mnt/ramdisk_transcodes/


Every 7 minutes (transcode_management.sh):
    │
    ├─ transcode_cleanup.sh
    │    Remove files older than TRANSCODE_MAX_AGE, not open by any process
    │    transcoding-temp: never deleted
    │    If ramdisk recovered below RAMDISK_LOW_GB → trigger flip-back
    │
    └─ transcode_manager.sh
         Safety checks (symlink, ramdisk mount, transcoding-temp, permissions)
         smart mode: ramdisk > RAMDISK_WARN_GB → flip symlink to SSD
                     ramdisk < RAMDISK_LOW_GB  → flip symlink back to ramdisk
         Session display (all TRANSCODE_SERVERS)
         Append to TRANSCODE_DAILY_LOG
```