# ━━━━━ TRANSCODES — Manual ━━━━━ Configuration reference, Docker mount setup, threshold sizing, and troubleshooting for the ramdisk transcode system. Read the Docker mount section before anything else. --- ## ━━━ CONTENTS ━━━ - [Docker Mount — Required Configuration](#docker-mount--required-configuration) - [The Symlink Architecture](#the-symlink-architecture) - [ramdisk_setup.sh](#ramdisk_setupsh) - [transcode_manager.sh](#transcode_managersh) - [transcode_cleanup.sh](#transcode_cleanupsh) - [Full Configuration Reference](#full-configuration-reference) - [Schedule](#schedule) - [Troubleshooting](#troubleshooting) --- ## Output Tiers All three scripts follow a two-tier output model: `echo` lines are always visible; `log` lines only appear when `--log` is passed. **ramdisk_setup.sh** — one-shot at array start. Without `--log`, section headers and the final summary are visible. Per-step creation detail suppressed. **transcode_manager.sh** — runs every 7 minutes. Without `--log`, only state transitions (flips, warnings, errors) and active session display are shown. When nothing changes, a single one-line confirmation is printed. Per-check detail suppressed. **transcode_cleanup.sh** — runs every 7 minutes. Without `--log`, the cleanup summary (files removed, space freed) is always visible. Per-file deletion detail suppressed. --- ## Docker Mount — Required Configuration > **This is the most important configuration requirement in this folder.** > Get this wrong and symlink flips silently stop working after the first flip. > The system appears to work initially and fails subtly. ### Required Mount Everything goes in the unRAID Docker template **Extra Parameters** field. Do NOT use the path mapping UI for the transcode directory — it does not support `--mount` syntax. **Non-GPU containers:** ``` --mount type=bind,source=/mnt/ram-transcode,target=/ext-ram-transcode ``` **GPU-accelerated containers (Emby, Jellyfin with NVENC/NVDEC):** ``` --gpus "device=GPU-62e1659d-1ed4-935f-3df3-4bb4339438f1" --pids-limit=0 --mount type=bind,source=/mnt/ram-transcode,target=/ext-ram-transcode ``` > ⚠️ **Unraid 7.3+ / Docker 29.x / runc v1.3.5+:** Do NOT add `bind-propagation=shared`. > runc v1.3.5 crashes on it with `open /proc/self/mountinfo: no such file or directory`. > The ramdisk tmpfs is already `MS_SHARED` at the kernel level — propagation is automatic. Replace the UUID with your GPU's UUID (`nvidia-smi -L` to find it). In Emby and Jellyfin's transcoding settings, set the transcode path to `/ext-ram-transcode`. --- ### `--gpus` vs `--runtime=nvidia` There are two ways to give a container GPU access. **Always use `--gpus`.** `--runtime=nvidia` is the old approach. It requires the NVIDIA Container Toolkit configured at the Docker daemon level and spreads across multiple XML fields (runtime flag + `NVIDIA_VISIBLE_DEVICES` env var + `NVIDIA_DRIVER_CAPABILITIES` env var). When unRAID rebuilds a container from template (update, reinstall) these fields can break or get dropped — requiring manual XML repair to recover. `--runtime=nvidia` combined with `NVIDIA_VISIBLE_DEVICES` breaks on container rebuilds and has historically conflicted with bind mount propagation settings. `--gpus "device=UUID"` is Docker-native GPU support (Docker 19.03+). One field in Extra Parameters. Pins a specific GPU by UUID — no ambiguity on a single-GPU system. Survives container rebuilds cleanly. **HOST1 GPU UUID (Quadro P2000):** `GPU-62e1659d-1ed4-935f-3df3-4bb4339438f1` ### Mount Propagation — Unraid 7.3+ / Docker 29.x > **Do NOT use `bind-propagation=shared`** on Unraid 7.3+ (Docker 29.x / runc v1.3.5+). > runc v1.3.5 rejects it with `open /proc/self/mountinfo: no such file or directory` and > the container fails to start. This applies to all containers — GPU and non-GPU alike. The ramdisk tmpfs is mounted `MS_SHARED` at the kernel level (`shared:N` in `/proc/self/mountinfo`). Bind mounts from it inherit shared propagation automatically — specifying it in Docker is redundant and breaks on runc v1.3.5+. **Historical context (Unraid ≤ 7.2.5):** `bind-propagation=shared` was required because Docker's default `rprivate` resolves the symlink target at mount time and locks that inode, making subsequent flips invisible to the container. On Unraid 7.3+ this option crashes container start entirely. The ramdisk's kernel-level MS_SHARED makes it unnecessary. ### Verify the Mount ```bash # Check the mount is present: docker inspect Emby | grep -A4 "ext-ram" # Expected: "Destination": "/ext-ram-transcode" # Check Emby's transcode path setting: docker exec Emby cat /config/config/encoding.xml | grep TranscodingTempPath # Expected: /ext-ram-transcode ``` ### What NOT to Do Do NOT add a static SSD transcode path as a second path mapping in the template: ``` /mnt/cache/Temp_Storage/Emby/Transcodes → /ssd-transcode ← do not do this ``` If the SSD path is mounted inside the container, Emby can see it as an accessible transcode location. It will route sessions there independently of the symlink — bypassing the management system entirely. Sessions land on SSD regardless of symlink state. The whole system stops working. --- ## The Symlink Architecture ``` Emby is configured to write transcodes to TRANSCODE_LINK (/mnt/ram-transcode). TRANSCODE_LINK is a symlink — its target is managed at runtime. Normal operation (ramdisk has headroom): /mnt/ram-transcode → /mnt/ramdisk_transcodes/ (ramdisk — fast, no wear) Heavy load (ramdisk filling up): /mnt/ram-transcode → /mnt/cache/Temp_Storage/Emby/Transcodes/ (SSD) Emby doesn't know this symlink exists. It writes to /mnt/ram-transcode. ffmpeg resolves the symlink ONCE when a session starts. After that it holds a direct reference to the actual directory. Flipping the symlink has ZERO effect on sessions already in progress. Only NEW sessions care about where the symlink currently points. ``` ### What Lives Where ``` /mnt/ramdisk_transcodes/ ← tmpfs (HOST*_RAMDISK_SIZE ceiling) transcoding-temp/ ← pre-created by ramdisk_setup.sh — always here E0D8DC/ ← Emby session (Live TV HLS segments) F1A9BB/ ← another session /mnt/ram-transcode ← symlink — managed by transcode_manager.sh currently points at: /mnt/ramdisk_transcodes/ /mnt/cache/Temp_Storage/Emby/Transcodes/ ← SSD fallback transcoding-temp/ ← also pre-created — Emby finds ramdisk version first xyz789/ ← sessions that started when ramdisk was full ``` --- ## ramdisk_setup.sh Creates the ramdisk, SSD fallback directory, symlink, and `transcoding-temp` on the ramdisk. Run once at array start. Idempotent — already-mounted ramdisk exits cleanly. ### What It Creates ``` 1. RAMDISK_PATH (/mnt/ramdisk_transcodes) mount -t tmpfs -o size=HOST*_RAMDISK_SIZE tmpfs /mnt/ramdisk_transcodes tmpfs uses only as much RAM as actually needed — RAMDISK_SIZE is a ceiling. An empty ramdisk uses essentially zero RAM. 2. transcoding-temp/ inside the ramdisk Pre-created so Emby always finds it here first. Without this: Emby creates transcoding-temp at its first writable location, which may be the SSD fallback even when the symlink points at the ramdisk. chown nobody:users — correct ownership for Emby (PUID=99) 3. TRANSCODE_SSD (/mnt/cache/Temp_Storage/Emby/Transcodes/) mkdir -p — created if missing, silent if exists Also pre-creates transcoding-temp/ inside SSD fallback for consistency 4. TRANSCODE_LINK (/mnt/ram-transcode) ln -sfn /mnt/ramdisk_transcodes /mnt/ram-transcode Always reset to ramdisk at array start — clean state every boot ``` ### Verify After Setup ```bash # All three must pass: mountpoint /mnt/ramdisk_transcodes # Expected: /mnt/ramdisk_transcodes is a mountpoint readlink /mnt/ram-transcode # Expected: /mnt/ramdisk_transcodes ls /mnt/ramdisk_transcodes/ # Expected: transcoding-temp/ ``` ### Usage ```bash ramdisk_setup.sh # normal run (called by array_started.sh) ramdisk_setup.sh --dry-run # show what would be created without creating ramdisk_setup.sh --status # show current ramdisk, symlink, and SSD state ramdisk_setup.sh --log # verbose — show each creation step ``` --- ## transcode_manager.sh Monitors ramdisk usage and manages the symlink direction. Called second in every 7-minute cycle by `transcode_management.sh`. ### Three Modes ```bash # master.conf TRANSCODE_MANAGER_MODE="smart" # smart | ramdisk | ssd # smart (default — use in production): # Ramdisk above RAMDISK_WARN_GB → flip symlink to SSD # Ramdisk below RAMDISK_LOW_GB → flip symlink back to ramdisk # Hysteresis gap prevents flip-flopping under moderate load # ramdisk: # Always uses ramdisk. Warns if usage exceeds threshold. Never flips. # Use for: light load server, guaranteed RAM performance, testing ramdisk behaviour. # ssd: # Always uses SSD. Never uses ramdisk. # Use for: post-flip drain (waiting for ramdisk sessions to end naturally), # maintenance windows, ramdisk capacity testing. ``` ### Threshold Sizing ```bash # host*.conf HOST1_RAMDISK_SIZE="10G" HOST1_RAMDISK_WARN_GB=8.8 # flip to SSD above this HOST1_RAMDISK_LOW_GB=6.5 # flip back below this ``` Keep ~1.5–2.5GB hysteresis gap between WARN and LOW. Without the gap, usage hovering near WARN causes constant flip-flopping. The gap requires multiple sessions to end completely before flipping back — a genuine recovery, not a brief fluctuation. Production data from a 7-household Live TV system: ``` Normal (2–3 streams) → ~1.5–2.0 GB Busy evening (5–6 streams) → ~3.5–4.5 GB Peak (8 streams, Live TV) → ~5.2 GB Current setup: 10G ramdisk, 8.8 GB threshold → 1.2 GB safety headroom ``` Recommended thresholds by ramdisk size: | RAMDISK_SIZE | RAMDISK_WARN_GB | RAMDISK_LOW_GB | |-------------|----------------|----------------| | 6G | 4.8 | 3.5 | | 8G | 6.8 | 5.5 | | 10G | 8.8 | 6.5 | | 12G | 10.5 | 8.5 | ### Multi-Server Configuration ```bash # master.conf TRANSCODE_SERVERS=( "Emby|http://localhost:8096|your-api-key|emby" # Temporarily cover HOST2's Emby during maintenance: # "Emby-Jayred365|http://100.x.x.x:8096|HOST2-api-key|emby" # Jellyfin instance (separate port): # "Jellyfin|http://localhost:8097|jellyfin-api-key|jellyfin" ) ``` Type field: `emby` | `jellyfin` | `plex` — controls which API endpoint format is used. Entries with placeholder API keys are skipped automatically. Comment out unused entries rather than deleting — placeholders show what's available. **Tdarr does NOT belong here.** Tdarr encodes full video files — large working files fill the ramdisk rapidly and cause constant flips. Tdarr belongs on SSD permanently. ### Session Display ``` ━━━ Active Emby Sessions ━━━ Total: 7 | Live TV: 5 | Transcoding: 5 | Direct: 2 Storage: ramdisk Sunny — ABC (WTAE) — Live TV — Transcode Mama Bear — Cinemax — Live TV — Transcode Gmer4Lfe — WAN Show — TV Show — Transcode ``` Split state during a flip (ramdisk sessions draining, new sessions on SSD): ``` ⚠️ Split state — 4 folder(s) on ramdisk / 2 on SSD Storage: ramdisk (4) + SSD (2) ``` ### Usage ```bash transcode_manager.sh # normal run transcode_manager.sh --dry-run # preview flip decision without flipping transcode_manager.sh --status # show current state, usage, sessions, flip history transcode_manager.sh --log # verbose output per safety check transcode_manager.sh --no-log # suppress daily log write ``` --- ## transcode_cleanup.sh Removes stale transcode files from ramdisk and SSD fallback. Called first in every 7-minute cycle — cleanup before usage measurement is non-negotiable. ### Deletion Rules A file is eligible for deletion only when ALL conditions are true: 1. **Older than TRANSCODE_MAX_AGE minutes** (mtime — last write time). Active segments are written every few seconds. Not touched in 20 minutes = session ended. 2. **Not currently open by any process** (lsof pre-built map, O(1) lookup per file). If ffmpeg has a file open, it is not deleted regardless of age. `transcoding-temp/` is **never deleted**, even when empty. Protected by name exclusion in the find command — deleting it causes Emby to route all sessions to the SSD fallback. ### Usage ```bash transcode_cleanup.sh # normal cleanup run transcode_cleanup.sh --dry-run # show what would be deleted transcode_cleanup.sh --status # show file counts, ages, open-file status per location transcode_cleanup.sh --log # verbose per-file output ``` --- ## Full Configuration Reference ```bash # master.conf # ── Paths ────────────────────────────────────────────────────────────────────── TRANSCODE_LINK="/mnt/ram-transcode" # symlink Emby points at TRANSCODE_SSD="/mnt/cache/Temp_Storage/Emby/Transcodes/" # SSD fallback # ── Per-Host (host*.conf) ─────────────────────────────────────────────── HOST1_RAMDISK_PATH="/mnt/ramdisk_transcodes" # ramdisk mount point HOST1_RAMDISK_SIZE="10G" # tmpfs ceiling (not a reservation) HOST1_RAMDISK_WARN_GB=8.8 # flip to SSD above this HOST1_RAMDISK_LOW_GB=6.5 # flip back below this # ── Thresholds ───────────────────────────────────────────────────────────────── RAMDISK_SSD_MIN_GB=20 # minimum SSD free space before allowing SSD flip # prevents accidentally filling the SSD cache pool TRANSCODE_FLIP_WARN=3 # notify if symlink flips this many times in one hour # high flip count = ramdisk undersized for the load # ── Cleanup ──────────────────────────────────────────────────────────────────── TRANSCODE_MAX_AGE=20 # minutes — files older than this are stale TRANSCODE_ORPHAN_AGE=30 # minutes — orphaned session folders removed after this # ── Manager Mode ─────────────────────────────────────────────────────────────── TRANSCODE_MANAGER_MODE="smart" # smart | ramdisk | ssd TRANSCODE_CHECK_EMBY=true # skip threshold checks when Emby not running # prevents unnecessary flips overnight # ── Permissions ──────────────────────────────────────────────────────────────── TRANSCODE_OWNER="nobody:users" # matches PUID=99 PGID=100 container env TRANSCODE_CHMOD="755" # ── Daily Log ────────────────────────────────────────────────────────────────── TRANSCODE_DAILY_LOG="$DATA_DIR/transcode_daily.db" TRANSCODE_LOG_RETENTION=90 # days — trimmed on every write TRANSCODE_STATE_FILE="/tmp/transcode_state.db" # /tmp — resets on reboot correctly # ── Multi-Server ─────────────────────────────────────────────────────────────── TRANSCODE_SERVERS=( "ContainerName|http://host:port|api-key|emby" # type: emby | jellyfin | plex ) ``` --- ## Schedule ``` At array start (via array_started.sh in Orchestrators/): ramdisk_setup.sh — creates ramdisk, symlink, transcoding-temp Must run BEFORE Emby starts Every 7 minutes (via transcode_management.sh in Orchestrators/): 1. transcode_cleanup.sh — remove stale files, check flip-back 2. transcode_manager.sh — check usage, flip if needed, display sessions Do NOT schedule transcode_manager.sh or transcode_cleanup.sh directly. ``` --- ## Troubleshooting ### Sessions Landing on SSD Despite Symlink Pointing at Ramdisk ```bash # Check 1 — Docker mount is present: docker inspect Emby | grep -A4 "ext-ram" # Expected: "Destination": "/ext-ram-transcode" # Wrong: no ext-ram-transcode entry # # Fix: Update Emby Extra Parameters, restart Emby. # GPU containers (full string): # --gpus "device=GPU-62e1659d-1ed4-935f-3df3-4bb4339438f1" --pids-limit=0 --mount type=bind,source=/mnt/ram-transcode,target=/ext-ram-transcode # Non-GPU (mount only): # --mount type=bind,source=/mnt/ram-transcode,target=/ext-ram-transcode # Remove any standard path mapping for the transcode directory. # Do NOT add bind-propagation=shared — runc v1.3.5+ (Docker 29.x, Unraid 7.3+) rejects it. # Check 2 — transcoding-temp exists on ramdisk: ls /mnt/ramdisk_transcodes/ # Expected: transcoding-temp/ # # Fix if missing: mkdir -p /mnt/ramdisk_transcodes/transcoding-temp chown nobody:users /mnt/ramdisk_transcodes/transcoding-temp # Check 3 — no duplicate SSD mount in Emby template: docker inspect Emby | grep -A3 "Mounts" # Should show only /mnt/ram-transcode → /ext-ram-transcode # Should NOT show /mnt/cache/Temp_Storage/... as a second mount ``` ### Flip Count High — 3+ Per Hour ```bash # Ramdisk filling up regularly — load exceeds the current ceiling. # Check peak usage from the weekly health digest: Transcodes → "Week peak: X.XGB" # # If peak is close to RAMDISK_WARN_GB → increase ramdisk size: # host1.conf HOST1_RAMDISK_SIZE="12G" # increase by 2G HOST1_RAMDISK_WARN_GB=10.5 # adjust thresholds accordingly HOST1_RAMDISK_LOW_GB=8.5 # Remount at new size — run ramdisk_setup.sh manually: ramdisk_setup.sh --log # Ramdisk must be unmounted first if already mounted: # umount /mnt/ramdisk_transcodes && ramdisk_setup.sh --log ``` ### Ramdisk Not Mounting at Array Start ```bash # Check if tmpfs is mounted: mountpoint /mnt/ramdisk_transcodes # "not a mountpoint" → setup failed or not run yet # Run manually to see the error: ramdisk_setup.sh --log # Common causes: # /mnt/ramdisk_transcodes missing → mkdir -p /mnt/ramdisk_transcodes # Insufficient RAM → check free RAM: free -h # RAMDISK_SIZE too large → reduce HOST*_RAMDISK_SIZE ``` ### Emergency Manual Flip ```bash # Flip to SSD immediately — all new sessions go to SSD: ln -sfn /mnt/cache/Temp_Storage/Emby/Transcodes /mnt/ram-transcode # Flip back to ramdisk — all new sessions go to ramdisk: ln -sfn /mnt/ramdisk_transcodes /mnt/ram-transcode # Check current symlink target: readlink /mnt/ram-transcode # Existing sessions in progress are NEVER affected — only new sessions follow the flip. ``` ### Emby Won't Start After Dirty Shutdown After an unclean shutdown, the `--mount` bind-propagation entry in Extra Parameters can leave the container in a broken state where Emby refuses to start at all. ```bash # Symptom: Emby fails to start with the --mount extra parameter present. # Cause: dirty shutdown left the bind mount in a state Docker can't recover. # # Recovery: # 1. Remove the entire Extra Parameters line from Emby in unRAID Docker UI # 2. Start Emby and wait for it to fully load (check the WebUI is responsive) # 3. Add the full Extra Parameters line back: # GPU: --gpus "device=GPU-62e1659d-1ed4-935f-3df3-4bb4339438f1" --pids-limit=0 --mount type=bind,source=/mnt/ram-transcode,target=/ext-ram-transcode # Non-GPU: --mount type=bind,source=/mnt/ram-transcode,target=/ext-ram-transcode # 4. Save changes and restart Emby # # The container starts cleanly without the mount, which clears the broken state. # Re-adding the mount after a clean start works reliably. ``` ### GPU Memory Exhausted — Jellyfin Fails, Emby Falls Back to CPU **Symptoms:** - Jellyfin refuses to play anything — session fails immediately, no transcode starts - Emby plays but transcodes on CPU (logs show `libx265` or `libx264` instead of `h264_nvenc`/`hevc_nvenc`) Both symptoms can appear at the same time and have the same root cause: another process has consumed all available VRAM and not released it. The media servers respond differently to a failed GPU session init: - **Jellyfin:** hard fails — no session is created, playback stops entirely - **Emby:** falls back to CPU transcoding silently and keeps going ```bash # Confirm VRAM is exhausted: nvidia-smi # Find what is holding it: nvidia-smi --query-compute-apps=pid,used_memory,name --format=csv,noheader ``` **Common source — OCR plugin sidecars:** Credit detection and subtitle extraction plugins often talk to a GPU-accelerated OCR container running alongside the media server. The EmbyCredits plugin (`yocksers/EmbyCredits`) can be configured to use a PaddleOCR backend. PaddleOCR loads a neural network into VRAM at first use and **does not release it between runs**. After a single credit scan the GPU memory stays consumed, starving Emby and Jellyfin of VRAM for transcoding. This is not obvious because PaddleOCR is a separate container — `nvidia-smi` shows the process, but the connection to failing playback is not immediate. **Emby startup probe — the restart trap:** Emby runs a one-shot NVIDIA hardware detection when the container starts. If VRAM is exhausted at startup, NVIDIA is marked unavailable for the entire container session — there is no retry. Emby will use CPU for all transcoding until the container is restarted, and only after VRAM has been freed. Restarting Emby while VRAM is still exhausted causes the probe to fail again and NVIDIA is disabled again. Correct recovery sequence: 1. Free VRAM (stop the offending container/process) 2. Confirm VRAM is free: `nvidia-smi` — GPU memory used should drop to near zero 3. Restart Emby — startup probe now succeeds, NVENC available **Fix — switch OCR to CPU:** For EmbyCredits: use the `yock1/embycreditocr` Tesseract image instead of PaddleOCR. Tesseract is CPU-based, never touches VRAM, and is the plugin's own documented backend. ```bash docker run -d \ --name EmbyCredit-OCR \ -p 8884:8884 \ --restart unless-stopped \ yock1/embycreditocr ``` In EmbyCredits plugin settings, set the OCR endpoint to `http://localhost:8884`. The accuracy tradeoff is real (PaddleOCR is stronger on non-Latin scripts) but GPU starvation is not an acceptable failure mode for a live media server. **If you just switched from `--runtime=nvidia` to `--gpus` and it still fails:** Check `nvidia-smi` before concluding `--gpus` is wrong. The flag may be correct and something else may be holding all the VRAM. This is exactly what happens when the EmbyCredits PaddleOCR container is running — PaddleOCR steals all available VRAM while a credit scan runs and never releases it when the scan completes. Emby and Jellyfin then have no GPU access regardless of how the GPU is passed to their containers. The GPU assignment method is irrelevant when there is no VRAM left to assign. Always confirm VRAM is actually free before troubleshooting container GPU flags. --- ### Container Fails to Start — open /proc/self/mountinfo: no such file or directory > **Unraid 7.3+ / Docker 29.x / runc v1.3.5+** — any container with `bind-propagation=shared` > in Extra Parameters crashes with this error. Not GPU-specific — affects all containers. ``` OCI runtime create failed: runc create failed: unable to start container process: error during container init: error jailing process inside rootfs: open /proc/self/mountinfo: no such file or directory ``` **Root cause:** runc v1.3.5 tries to read `/proc/self/mountinfo` inside the new mount namespace before `/proc` is mounted, crashing when `bind-propagation=shared` is present. **Fix:** Remove `bind-propagation=shared` from Extra Parameters entirely. In the unRAID Docker template: 1. **Extra Parameters** — remove `bind-propagation=shared` from the `--mount` option: ``` --gpus "device=GPU-62e1659d-1ed4-935f-3df3-4bb4339438f1" --pids-limit=0 --mount type=bind,source=/mnt/ram-transcode,target=/ext-ram-transcode ``` 2. The ramdisk tmpfs is already `MS_SHARED` at the kernel level (`shared:N` in `/proc/self/mountinfo`). Propagation is inherited automatically — no Docker flag needed. **Historical note:** On Unraid ≤ 7.2.5 (Docker <29.x), `bind-propagation=shared` was required for symlink flips to propagate into running containers. On Unraid 7.3+ it crashes container start. The kernel-level MS_SHARED on the tmpfs makes it unnecessary. ### Increasing Ramdisk Size After Initial Setup ```bash # 1. Set new size and thresholds in host*.conf # 2. Unmount the existing ramdisk (no sessions should be active): umount /mnt/ramdisk_transcodes # 3. Re-run setup to mount at new size: ramdisk_setup.sh --log # 4. Verify: df -h /mnt/ramdisk_transcodes # Should show new size as total ```