Gmer4Lfe
070013cb4f
array_start.sh → array_started.sh (bulk rename — actual file has always been array_started.sh; docs consistently used the wrong name across 9 files) failover.sh → fallback.sh (bulk rename — script is Fallback/fallback.sh; docs used the old name throughout) FAILOVER_HOST2_RUNS_FOR_HOST1_TIER1 → FALLBACK_HOST2_COVERS_HOST1_TIER1 (README.md — wrong variable name, actual vars use FALLBACK_ prefix and COVERS_ not RUNS_FOR_) docker_watchdog.sh attribution: "started by array_started.sh" → "called every minute by watchdog_orchestrator.sh" (docker_watchdog is not in ARRAY_START_SCRIPTS) server_reboot.sh shutdown sequence: step 7 "rc.docker stop" → calls array_stopping.sh (user_scripts, fallback, rsync, mover, docker_container_stop) Rsync manual ARRAY_START_SCRIPTS list: removed stability_watchdog and docker_watchdog (now in watchdog_orchestrator); failover → fallback
138 lines
6.3 KiB
Markdown
138 lines
6.3 KiB
Markdown
# ━━━━━ TRANSCODES ━━━━━
|
|
|
|
**Ramdisk-based transcode storage with automatic SSD fallback.** Emby transcodes to RAM
|
|
at full speed. When the ramdisk fills, new sessions shift to SSD automatically —
|
|
without interrupting anything already playing. When pressure drops, new sessions shift
|
|
back to RAM.
|
|
|
|
> **Two configuration requirements that are not obvious and were both discovered the
|
|
> hard way in production.** The Docker mount must use `bind-propagation=shared` or
|
|
> symlink flips are silently ignored after the first flip. The `transcoding-temp`
|
|
> directory must be pre-created on the ramdisk or Emby finds the SSD version and
|
|
> routes all sessions there until restarted. Both are documented in Manual-Transcoding.md.
|
|
|
|
---
|
|
|
|
## ━━━ THE PROBLEM THAT BUILT THIS ━━━
|
|
|
|
**Three Storage Options, None Perfect on Their Own**
|
|
Hard drives: seek times cause buffering on multi-stream transcoding. SSD: fast enough,
|
|
but constant small file writes at Emby volume accelerate wear over months. RAM: fastest,
|
|
no wear, files vanish instantly on session end — but limited by available memory.
|
|
Fix: RAM by default, SSD as a safety net. The system manages the transition automatically.
|
|
|
|
**Changing Transcode Location Requires Restarting Emby**
|
|
Configuring Emby to switch between ramdisk and SSD requires a restart. Restarting
|
|
during active streams drops everyone. A 7-person household with 5 Live TV streams at
|
|
9pm is not a good moment to restart Emby.
|
|
Fix: symlink indirection. Emby points at a fixed path. The symlink target changes.
|
|
ffmpeg resolves the symlink once at session start — existing sessions are completely
|
|
unaffected by flips. Only new sessions follow the new target.
|
|
|
|
**Docker Bind Mount Silently Ignored After First Flip**
|
|
Symlink flip from ramdisk → SSD worked. Flip back: nothing. All new sessions still land
|
|
on SSD. The symlink on the host is correct. Emby doesn't see it.
|
|
Cause: Docker's default `rprivate` propagation resolves the symlink target at mount time
|
|
and locks that inode. Subsequent flips are invisible to the container.
|
|
Fix: `bind-propagation=shared` in Extra Parameters. Host mount changes propagate into
|
|
the container in real time. Requires `--mount` syntax — the path mapping UI doesn't
|
|
support propagation.
|
|
|
|
**Sessions Drifting to SSD After a Day of Operation**
|
|
System working correctly for hours, then sessions gradually drift to SSD despite the
|
|
ramdisk having plenty of space.
|
|
Cause: cleanup was removing the empty `transcoding-temp` directory from the ramdisk.
|
|
Emby then found the SSD fallback version and routed all sessions there.
|
|
Fix: `transcoding-temp` is excluded from cleanup by name. `ramdisk_setup.sh` pre-creates
|
|
it at mount time. Both protections together prevent this permanently.
|
|
|
|
**lsof Per File on a Live TV System**
|
|
Early cleanup called `lsof filename` per file to check if anything had it open. On a busy
|
|
Live TV night with 5 simultaneous streams, the ramdisk contains thousands of HLS segment
|
|
files — thousands of subprocess calls every 3 minutes.
|
|
Fix: lsof called once per location to build a complete open-file map. All subsequent
|
|
checks are O(1) lookups against that map.
|
|
|
|
---
|
|
|
|
## ━━━ WHAT THIS FOLDER DOES ━━━
|
|
|
|
Three scripts, one goal: keep transcodes on RAM, fall back to SSD when needed.
|
|
|
|
`ramdisk_setup.sh` runs at array start — creates the tmpfs, SSD fallback directory,
|
|
symlink, and pre-creates `transcoding-temp`. Everything that must exist before Emby starts.
|
|
|
|
`transcode_cleanup.sh` runs first in every 3-minute cycle — removes stale files from both
|
|
ramdisk and SSD. Cleans up before usage is measured, so the manager sees real load.
|
|
|
|
`transcode_manager.sh` runs second — measures ramdisk usage, flips the symlink if
|
|
thresholds are crossed, runs safety checks, displays active sessions, writes the daily log.
|
|
|
|
The symlink is the mechanism that makes this seamless. Emby writes to a fixed path. That
|
|
path is a symlink whose target is managed at runtime. Sessions in progress never notice.
|
|
|
|
---
|
|
|
|
## ━━━ RELATIONSHIP TO OTHER FOLDERS ━━━
|
|
|
|
```
|
|
unRAID_Essentials/
|
|
array_started.sh ──────────────────────────────► ramdisk_setup.sh (at array start)
|
|
|
|
Orchestrators/
|
|
transcode_management.sh ──── cleanup first ──► transcode_cleanup.sh
|
|
──── then manager ──► transcode_manager.sh
|
|
(every 3 minutes — order non-negotiable)
|
|
|
|
Monitors/
|
|
weekly_health_digest.sh ◄─── reads ──────────── TRANSCODE_DAILY_LOG
|
|
```
|
|
|
|
Do not schedule `transcode_cleanup.sh` or `transcode_manager.sh` directly.
|
|
Both are called by `transcode_management.sh` in the correct order.
|
|
|
|
---
|
|
|
|
## ━━━ SCRIPTS IN THIS FOLDER ━━━
|
|
|
|
| Script | Role | When It Runs |
|
|
|--------|------|-------------|
|
|
| `ramdisk_setup.sh` | Create tmpfs, SSD fallback dir, symlink, transcoding-temp | At array start (via array_started.sh) |
|
|
| `transcode_cleanup.sh` | Remove stale files, check for flip-back opportunity | Every 3 min via transcode_management.sh — runs first |
|
|
| `transcode_manager.sh` | Check usage, flip symlink, safety checks, session display, daily log | Every 3 min via transcode_management.sh — runs second |
|
|
|
|
---
|
|
|
|
## ━━━ HOW THE SCRIPTS RELATE ━━━
|
|
|
|
```
|
|
Array starts
|
|
│
|
|
▼
|
|
ramdisk_setup.sh
|
|
Creates: /mnt/ramdisk_transcodes (tmpfs)
|
|
/mnt/ramdisk_transcodes/transcoding-temp/
|
|
/mnt/cache/Temp_Storage/Emby/Transcodes/ (SSD fallback)
|
|
/mnt/ram-transcode → /mnt/ramdisk_transcodes (symlink)
|
|
│
|
|
▼
|
|
Emby starts, reads transcode path from config
|
|
Sees: /ext-ram-transcode (bind-mounted from /mnt/ram-transcode)
|
|
All new sessions write to: /mnt/ram-transcode → /mnt/ramdisk_transcodes/
|
|
|
|
|
|
Every 3 minutes (transcode_management.sh):
|
|
│
|
|
├─ transcode_cleanup.sh
|
|
│ Remove files older than TRANSCODE_MAX_AGE, not open by any process
|
|
│ transcoding-temp: never deleted
|
|
│ If ramdisk recovered below RAMDISK_LOW_GB → trigger flip-back
|
|
│
|
|
└─ transcode_manager.sh
|
|
Safety checks (symlink, ramdisk mount, transcoding-temp, permissions)
|
|
smart mode: ramdisk > RAMDISK_WARN_GB → flip symlink to SSD
|
|
ramdisk < RAMDISK_LOW_GB → flip symlink back to ramdisk
|
|
Session display (all TRANSCODE_SERVERS)
|
|
Append to TRANSCODE_DAILY_LOG
|
|
```
|