Gmer4Lfe
ba3eed39e3
Fix discrepancies found by auditing docs against headers: - stability_watchdog.sh: wrong variable name (STRIKES → STRIKE_LIMIT) and wrong default (2hr → 12hr) in header - Watchdogs manual: REBOOT_WINDOW_HRS example value was 2, should be 12 - Transcodes README: transcode_cleanup/manager table said "Every 3 min", should be "Every 7 minutes" - Tools manual: fallback_state_reset.sh still marked "not yet built" — rewrite section to reflect current script - Tools manual: claude_startup --setup flag doesn't exist; modes were inverted — fix and add --launch - Tools README + manual: add docker_prune_images.sh (existed but undocumented) - Tools manual: add play_state_sync.sh --full to flag reference - Orchestrators README: arrs_failed_stalled_recovery runs via intermediate_sync (every 4hr), not standalone every 6hr - Plugin README: add build.sh to scripts table - Partnership README + manual: add partnership_transfer.sh and onboard_cancel.sh to script tables and flag reference
167 lines
9.8 KiB
Markdown
167 lines
9.8 KiB
Markdown
# ━━━━━ TOOLS ━━━━━
|
|
|
|
**Situational utilities — run when something needs fixing, not on a schedule.**
|
|
Recovery, repair, migration, and inspection tools for situations that arise outside
|
|
the scheduled maintenance model. These scripts sit ready for the moment you need them.
|
|
|
|
> **None of these scripts run on a schedule.** A script belongs here when it solves
|
|
> a specific operational situation — something you run in response to a problem, before
|
|
> a risky operation, or during a one-time setup task. Having a dedicated folder keeps
|
|
> the other folders clean and makes it obvious what runs routinely vs. situationally.
|
|
|
|
---
|
|
|
|
## ━━━ THE PROBLEM THAT BUILT THIS ━━━
|
|
|
|
**Fallback State Stuck After Testing**
|
|
Run a fallback test, something exits uncleanly, state file shows `FALLBACK`.
|
|
`fallback.sh` resumes and reads FALLBACK — starts containers it shouldn't, makes
|
|
decisions based on a state that doesn't reflect reality. Manual recovery means
|
|
knowing the exact file format and every field to reset. At 2am after a failed test.
|
|
Fix: `fallback_state_reset.sh` — shows current state, prompts for confirmation,
|
|
resets cleanly to NORMAL.
|
|
|
|
**Container Stuck on Watchdog Skip List After Fixing the Problem**
|
|
Authelia hit the restart loop limit — went on the skip list. Problem fixed. But
|
|
the watchdog still isn't monitoring it because the skip list persists on `/boot/config`
|
|
across reboots. Where's the file? What format? How do you clear restart history?
|
|
Fix: `watchdog_skip_list_manager.sh` — shows the skip list and which containers are
|
|
running vs. stopped, clears specific containers with confirmation.
|
|
|
|
**Emby Crashing With No Clear Cause After a Power Cut**
|
|
Server lost power with Emby running. Emby comes back, runs for 20 minutes, crashes.
|
|
Logs show database errors. Which database? library.db? users.db? Each has different
|
|
recovery implications — deleting the wrong one resets all user watch history.
|
|
Fix: `emby_database_repair.sh` — stops Emby, runs `PRAGMA integrity_check` on every
|
|
database, reports per-database with specific guidance on what to do about each one.
|
|
|
|
**Files Owned by Root After an Admin Copy**
|
|
`scp` a file into a media share. File arrives as `root:root`. Radarr fails to import —
|
|
permission denied. The daily permissions script won't run for another 20 hours. Running
|
|
`media_shares_permissions.sh` on the whole library takes 30 minutes just to fix one dir.
|
|
Fix: `bulk_permissions_repair.sh` — takes specific paths, applies correct ownership and
|
|
permissions in seconds.
|
|
|
|
**No Way to Back Up a Container Before a Risky Update**
|
|
Major version update, changelog says "database migration — no rollback." You want a
|
|
point-in-time backup. But `cp -r` while the container is running produces an
|
|
inconsistent backup, and tar without stopping the container is equally unreliable.
|
|
Fix: `container_data_export.sh` — stops the container cleanly, archives appdata to a
|
|
timestamped `.tar.gz`, verifies archive integrity, restarts the container.
|
|
|
|
**Fresh HOST2 Has Shares Configured But Directories Missing**
|
|
Fresh install on HOST2. Restored `/boot/config/shares/` from backup. Array starts.
|
|
Shares show in the UI. But the actual `/mnt/diskN/sharename` directories don't exist —
|
|
unRAID created the share definitions but not the directories. rsync.sh aborts.
|
|
Fix: `recreate_shares.sh` — reads every `.cfg` file, creates directories on each
|
|
included disk, places `.recovery` markers so the first rsync won't delete anything.
|
|
|
|
---
|
|
|
|
## ━━━ WHAT THIS FOLDER DOES ━━━
|
|
|
|
One role: hold scripts for situations the scheduled maintenance model can't handle.
|
|
|
|
Every script here was written because a specific situation arose that required bash
|
|
commands to resolve — and that situation is guaranteed to arise again. When you encounter
|
|
something new, write the tool. Store it here. Find it at 2am next time.
|
|
|
|
**Recovery Tools** — Restore known-good state after a failure
|
|
`fallback_state_reset.sh`, `watchdog_skip_list_manager.sh`
|
|
|
|
**Diagnostic Tools** — Inspect and verify before acting
|
|
`emby_database_repair.sh`, `continuous_scripts_status.sh`
|
|
|
|
**Repair Tools** — Fix a specific known problem
|
|
`bulk_permissions_repair.sh`, `zfs_pool_scrub.sh`
|
|
|
|
**Lifecycle Tools** — Backup, setup, and migration support
|
|
`container_data_export.sh`, `recreate_shares.sh`, `claude_startup.sh`, `ramdisk_stop.sh`
|
|
|
|
**Library Sync Bootstrap** — Close the gap between Emby and arr libraries
|
|
`emby_to_lidarr_sync.sh`, `emby_to_sonarr_sync.sh`, `emby_to_radarr_sync.sh`
|
|
|
|
---
|
|
|
|
## ━━━ RELATIONSHIP TO OTHER FOLDERS ━━━
|
|
|
|
```
|
|
System_Essentials/ ← regular system maintenance — scheduled
|
|
Docker_Essentials/ ← regular container management — scheduled
|
|
Monitors/ ← regular health reporting — scheduled
|
|
Orchestrators/ ← regular maintenance windows — scheduled
|
|
Fallback/ ← automated fallback/handback — event-driven
|
|
Tools/ ← situational utilities — run when needed
|
|
```
|
|
|
|
Some tools interact with state written by other folders:
|
|
|
|
```
|
|
Fallback/
|
|
fallback.sh ──────── writes FALLBACK_STATE_FILE ──► fallback_state_reset.sh reads/writes it
|
|
|
|
Docker_Essentials/
|
|
docker_watchdog.sh ── writes skip list + history ──► watchdog_skip_list_manager.sh manages them
|
|
|
|
Docker_Essentials/ + System_Essentials/ + Fallback/
|
|
All continuous scripts ──────────────────────────► continuous_scripts_status.sh reads their state
|
|
```
|
|
|
|
Tools never call scripts in other folders. Other folders never call Tools scripts.
|
|
The relationship is one-way: Tools act on state that other scripts have written.
|
|
|
|
---
|
|
|
|
## ━━━ SCRIPTS IN THIS FOLDER ━━━
|
|
|
|
| Script | What It Fixes | When to Run |
|
|
|--------|--------------|-------------|
|
|
| `fallback_state_reset.sh` | State file stuck in FALLBACK after test or failed handback | After fallback testing or manual intervention |
|
|
| `watchdog_skip_list_manager.sh` | Container stuck on watchdog skip list after fixing root cause | After fixing a container that hit the restart loop limit |
|
|
| `bulk_permissions_repair.sh` | Files owned by wrong user after admin copy or bad container config | When arr operations fail due to permissions |
|
|
| `container_data_export.sh` | Need a clean backup before a risky container update or migration | Before major updates, appdata migrations, or container removals |
|
|
| `emby_database_repair.sh` | Emby crashing with database errors after power loss or crash | When Emby logs show corruption or repeated crashes |
|
|
| `zfs_pool_scrub.sh` | Verify ZFS pool integrity — catch silent corruption before it spreads | Monthly, or after any disk or power event |
|
|
| `recreate_shares.sh` | Share directories missing after fresh install or disk rebuild | After fresh unRAID install or disk replacement on HOST2 |
|
|
| `continuous_scripts_status.sh` | Need a live view of watchdog and fallback state | Any time — manual dashboard, no schedule |
|
|
| `claude_startup.sh` | Claude Code session setup after reboot — symlinks persistent storage | After each unRAID reboot, or called by array_started.sh |
|
|
| `docker_prune_images.sh` | Remove dangling or unused Docker images accumulated after updates | After update cycles, or when disk space is low |
|
|
| `ramdisk_stop.sh` | Safely stop the transcode ramdisk — redirect symlink to SSD, unmount, update state | Before re-running ramdisk_setup.sh with new size or thresholds |
|
|
| `emby_to_lidarr_sync.sh` | Add all Emby album artists not yet tracked in Lidarr | After Lidarr setup, database wipe, or when you suspect gaps |
|
|
| `emby_to_sonarr_sync.sh` | Add all Emby TV series not yet tracked in Sonarr | After Sonarr setup, database wipe, or when you suspect gaps |
|
|
| `emby_to_radarr_sync.sh` | Add all Emby movies not yet tracked in Radarr | After Radarr setup, database wipe, or when you suspect gaps |
|
|
|
|
---
|
|
|
|
## ━━━ HOW THE SCRIPTS RELATE ━━━
|
|
|
|
All Tools scripts are independent — none call each other, none are called by other Tools.
|
|
|
|
```
|
|
Situation arises
|
|
│
|
|
▼
|
|
┌──────────────────────────────────────────────────────────────────────┐
|
|
│ Tools/ Run directly when needed │
|
|
│ │
|
|
│ fallback_state_reset.sh ◄── after fallback test / failed handback│
|
|
│ watchdog_skip_list_manager ◄── after fixing a crash-looping container│
|
|
│ bulk_permissions_repair ◄── wrong ownership after copy or rsync │
|
|
│ container_data_export ◄── before a risky update or migration │
|
|
│ emby_database_repair ◄── Emby logs show corruption │
|
|
│ zfs_pool_scrub ◄── monthly integrity check / post-event │
|
|
│ recreate_shares ◄── fresh HOST2 setup or disk rebuild │
|
|
│ continuous_scripts_status ◄── manual status check at any time │
|
|
│ claude_startup ◄── after each unRAID reboot │
|
|
│ ramdisk_stop ◄── before ramdisk resize / remount │
|
|
│ │
|
|
│ emby_to_lidarr_sync ◄── Lidarr setup / database wipe / gap │
|
|
│ emby_to_sonarr_sync ◄── Sonarr setup / database wipe / gap │
|
|
│ emby_to_radarr_sync ◄── Radarr setup / database wipe / gap │
|
|
└──────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
State files in other folders (Fallback/, Docker_Essentials/) may be read or written.
|
|
No other scripts call into Tools/.
|
|
```
|