Gmer4Lfe
ee086f309e
appdata-Failover → appdata-Fallback: All .md files updated — scripts and confs already used appdata-Fallback. Share was renamed; docs were the only stale references. critical_sync_maintenance.sh: */15 → */30, 96/day → 48/day: critical_sync_maintenance.sh header, master.conf, host1/2.conf comments, Orchestrators README, User Script Plug-in README, Partnership README/manual/script, Docker_Essentials README/downloaders_reset.sh, Fallback README, weekly_sync_maintenance.sh, user_script_plug-in.sh all updated. rsync_emby_failover User Script superseded: Emby dirty sync was already in HOST1_CRITICAL_SYNC_SHARES — it now runs inside critical_sync_maintenance.sh every 30min alongside auth sync and partnership check. Section in User Script Plug-in README marked for deletion from plugin. transcode_management.sh: */3 → */7: transcode_management.sh, transcode_manager.sh, transcode_cleanup.sh headers, Transcodes README/manual, Orchestrators README, User Script Plug-in README, user_script_plug-in.sh all updated.
138 lines
6.3 KiB
Markdown
138 lines
6.3 KiB
Markdown
# ━━━━━ TRANSCODES ━━━━━
|
|
|
|
**Ramdisk-based transcode storage with automatic SSD fallback.** Emby transcodes to RAM
|
|
at full speed. When the ramdisk fills, new sessions shift to SSD automatically —
|
|
without interrupting anything already playing. When pressure drops, new sessions shift
|
|
back to RAM.
|
|
|
|
> **Two configuration requirements that are not obvious and were both discovered the
|
|
> hard way in production.** The Docker mount must use `bind-propagation=shared` or
|
|
> symlink flips are silently ignored after the first flip. The `transcoding-temp`
|
|
> directory must be pre-created on the ramdisk or Emby finds the SSD version and
|
|
> routes all sessions there until restarted. Both are documented in Manual-Transcoding.md.
|
|
|
|
---
|
|
|
|
## ━━━ THE PROBLEM THAT BUILT THIS ━━━
|
|
|
|
**Three Storage Options, None Perfect on Their Own**
|
|
Hard drives: seek times cause buffering on multi-stream transcoding. SSD: fast enough,
|
|
but constant small file writes at Emby volume accelerate wear over months. RAM: fastest,
|
|
no wear, files vanish instantly on session end — but limited by available memory.
|
|
Fix: RAM by default, SSD as a safety net. The system manages the transition automatically.
|
|
|
|
**Changing Transcode Location Requires Restarting Emby**
|
|
Configuring Emby to switch between ramdisk and SSD requires a restart. Restarting
|
|
during active streams drops everyone. A 7-person household with 5 Live TV streams at
|
|
9pm is not a good moment to restart Emby.
|
|
Fix: symlink indirection. Emby points at a fixed path. The symlink target changes.
|
|
ffmpeg resolves the symlink once at session start — existing sessions are completely
|
|
unaffected by flips. Only new sessions follow the new target.
|
|
|
|
**Docker Bind Mount Silently Ignored After First Flip**
|
|
Symlink flip from ramdisk → SSD worked. Flip back: nothing. All new sessions still land
|
|
on SSD. The symlink on the host is correct. Emby doesn't see it.
|
|
Cause: Docker's default `rprivate` propagation resolves the symlink target at mount time
|
|
and locks that inode. Subsequent flips are invisible to the container.
|
|
Fix: `bind-propagation=shared` in Extra Parameters. Host mount changes propagate into
|
|
the container in real time. Requires `--mount` syntax — the path mapping UI doesn't
|
|
support propagation.
|
|
|
|
**Sessions Drifting to SSD After a Day of Operation**
|
|
System working correctly for hours, then sessions gradually drift to SSD despite the
|
|
ramdisk having plenty of space.
|
|
Cause: cleanup was removing the empty `transcoding-temp` directory from the ramdisk.
|
|
Emby then found the SSD fallback version and routed all sessions there.
|
|
Fix: `transcoding-temp` is excluded from cleanup by name. `ramdisk_setup.sh` pre-creates
|
|
it at mount time. Both protections together prevent this permanently.
|
|
|
|
**lsof Per File on a Live TV System**
|
|
Early cleanup called `lsof filename` per file to check if anything had it open. On a busy
|
|
Live TV night with 5 simultaneous streams, the ramdisk contains thousands of HLS segment
|
|
files — thousands of subprocess calls every 7 minutes.
|
|
Fix: lsof called once per location to build a complete open-file map. All subsequent
|
|
checks are O(1) lookups against that map.
|
|
|
|
---
|
|
|
|
## ━━━ WHAT THIS FOLDER DOES ━━━
|
|
|
|
Three scripts, one goal: keep transcodes on RAM, fall back to SSD when needed.
|
|
|
|
`ramdisk_setup.sh` runs at array start — creates the tmpfs, SSD fallback directory,
|
|
symlink, and pre-creates `transcoding-temp`. Everything that must exist before Emby starts.
|
|
|
|
`transcode_cleanup.sh` runs first in every 3-minute cycle — removes stale files from both
|
|
ramdisk and SSD. Cleans up before usage is measured, so the manager sees real load.
|
|
|
|
`transcode_manager.sh` runs second — measures ramdisk usage, flips the symlink if
|
|
thresholds are crossed, runs safety checks, displays active sessions, writes the daily log.
|
|
|
|
The symlink is the mechanism that makes this seamless. Emby writes to a fixed path. That
|
|
path is a symlink whose target is managed at runtime. Sessions in progress never notice.
|
|
|
|
---
|
|
|
|
## ━━━ RELATIONSHIP TO OTHER FOLDERS ━━━
|
|
|
|
```
|
|
unRAID_Essentials/
|
|
array_started.sh ──────────────────────────────► ramdisk_setup.sh (at array start)
|
|
|
|
Orchestrators/
|
|
transcode_management.sh ──── cleanup first ──► transcode_cleanup.sh
|
|
──── then manager ──► transcode_manager.sh
|
|
(every 7 minutes — order non-negotiable)
|
|
|
|
Monitors/
|
|
weekly_health_digest.sh ◄─── reads ──────────── TRANSCODE_DAILY_LOG
|
|
```
|
|
|
|
Do not schedule `transcode_cleanup.sh` or `transcode_manager.sh` directly.
|
|
Both are called by `transcode_management.sh` in the correct order.
|
|
|
|
---
|
|
|
|
## ━━━ SCRIPTS IN THIS FOLDER ━━━
|
|
|
|
| Script | Role | When It Runs |
|
|
|--------|------|-------------|
|
|
| `ramdisk_setup.sh` | Create tmpfs, SSD fallback dir, symlink, transcoding-temp | At array start (via array_started.sh) |
|
|
| `transcode_cleanup.sh` | Remove stale files, check for flip-back opportunity | Every 3 min via transcode_management.sh — runs first |
|
|
| `transcode_manager.sh` | Check usage, flip symlink, safety checks, session display, daily log | Every 3 min via transcode_management.sh — runs second |
|
|
|
|
---
|
|
|
|
## ━━━ HOW THE SCRIPTS RELATE ━━━
|
|
|
|
```
|
|
Array starts
|
|
│
|
|
▼
|
|
ramdisk_setup.sh
|
|
Creates: /mnt/ramdisk_transcodes (tmpfs)
|
|
/mnt/ramdisk_transcodes/transcoding-temp/
|
|
/mnt/cache/Temp_Storage/Emby/Transcodes/ (SSD fallback)
|
|
/mnt/ram-transcode → /mnt/ramdisk_transcodes (symlink)
|
|
│
|
|
▼
|
|
Emby starts, reads transcode path from config
|
|
Sees: /ext-ram-transcode (bind-mounted from /mnt/ram-transcode)
|
|
All new sessions write to: /mnt/ram-transcode → /mnt/ramdisk_transcodes/
|
|
|
|
|
|
Every 7 minutes (transcode_management.sh):
|
|
│
|
|
├─ transcode_cleanup.sh
|
|
│ Remove files older than TRANSCODE_MAX_AGE, not open by any process
|
|
│ transcoding-temp: never deleted
|
|
│ If ramdisk recovered below RAMDISK_LOW_GB → trigger flip-back
|
|
│
|
|
└─ transcode_manager.sh
|
|
Safety checks (symlink, ramdisk mount, transcoding-temp, permissions)
|
|
smart mode: ramdisk > RAMDISK_WARN_GB → flip symlink to SSD
|
|
ramdisk < RAMDISK_LOW_GB → flip symlink back to ramdisk
|
|
Session display (all TRANSCODE_SERVERS)
|
|
Append to TRANSCODE_DAILY_LOG
|
|
```
|