doc audit — fix stale watchdog vars, fallback_state_reset, add monthly_maintenance and mesh_monitor sections

- Tools/Manual-Tools.md: fallback_state_reset.sh marked not yet built; manual
  workaround added; TOC entry updated; failover→fallback rename throughout
- Watchdogs/Manual-Watchdogs.md: System Watchdog config block updated — remove
  pre-split vars (MEM_WARN_GB, MEM_SHUTDOWN_GB, MEM_GB, MEM_RECOVER_GB,
  SYS_WATCHDOG_MEM_SHUTDOWN_EXCLUDED, SYS_WATCHDOG_REQUIRED_CONTAINERS);
  add SYS_WATCHDOG_MEM_GB (reboot trigger only); fix state file path names
  (watchdog_state.db → container_watchdog_state.db,
  sys_watchdog_state.db → system_watchdog_state.db)
- Orchestrators/README-Orchestrators.md: add monthly_maintenance.sh section
  with two-gate model, configuration reference, and usage examples
- Monitors/README-Monitors.md: add mesh_monitor.sh to both tables and diagram
- Monitors/Manual-Monitors.md: add mesh_monitor.sh section
- README-User_Script_Plug-in.md, Rsync/README-Rsync.md, Rsync/Manual-Rsync.md,
  README.md: emby-failover → emby-fallback (missed from prior batch)
This commit is contained in:
Gmer4Lfe
2026-05-22 22:07:46 -04:00
parent 070013cb4f
commit 2fa78a21b6
9 changed files with 131 additions and 52 deletions
+39
View File
@@ -343,6 +343,45 @@ long-term history. It avoids flash drive writes entirely.
--- ---
## ━━━ MESH MONITOR ━━━
At-a-glance view of the mesh — who the members are, what each node covers for the
others when they go down, and the current partnership state. Read-only. Parses conf
files directly — no SSH, no API calls, no sensitive data.
### Sections
```
MEMBERS — server name, owner, contact email for each HOST*
COVERAGE — what each host covers for others, by tier
PARTNERSHIP — enabled/disabled, current owner, sync interval
```
### When to Use
```
Onboarding a new mesh member — paste the output to show them the current topology
After adding a new container to a fallback tier — verify it appears in the right tier
When troubleshooting fallback — confirm coverage arrays are configured correctly
```
### Usage
```bash
mesh_monitor.sh # full mesh overview
mesh_monitor.sh --log # verbose output
mesh_monitor.sh --status # show config and exit
```
### Configuration
No dedicated config keys. Reads `HOST*_OWNER`, `HOST*_OWNER_EMAIL`, and
`FALLBACK_HOST*_COVERS_HOST*_TIER*` arrays from `host*.conf`. Scales automatically
as new nodes join — add a new `HOST3_*` block and it appears in output without
changing the script.
---
## ━━━ FULL CONFIGURATION REFERENCE ━━━ ## ━━━ FULL CONFIGURATION REFERENCE ━━━
### master.conf ### master.conf
+6
View File
@@ -108,6 +108,7 @@ folder is designed with this constraint in mind.
| `weekly_health_digest.sh` | Never | reads existing state files only | | `weekly_health_digest.sh` | Never | reads existing state files only |
| `bandwidth_monitor.sh` | One append + one trim per rsync run | Bounded — never exceeds `BANDWIDTH_LOG_RETENTION` lines | | `bandwidth_monitor.sh` | One append + one trim per rsync run | Bounded — never exceeds `BANDWIDTH_LOG_RETENTION` lines |
| `system_tuning_monitor.sh` | One append + one trim every 6 hours | Bounded — trimmed to `TUNING_LOG_RETENTION` days | | `system_tuning_monitor.sh` | One append + one trim every 6 hours | Bounded — trimmed to `TUNING_LOG_RETENTION` days |
| `mesh_monitor.sh` | Never | read-only conf parse — no API calls, no SSH |
--- ---
@@ -123,6 +124,7 @@ folder is designed with this constraint in mind.
| `bandwidth_monitor.sh` | rsync transfer history and trends | Auto (called by rsync.sh) + weekly report Sunday 11am | | `bandwidth_monitor.sh` | rsync transfer history and trends | Auto (called by rsync.sh) + weekly report Sunday 11am |
| `weekly_health_digest.sh` | Full ecosystem health aggregation | Daily 8am (DIGEST_PROFILE controls notify) | | `weekly_health_digest.sh` | Full ecosystem health aggregation | Daily 8am (DIGEST_PROFILE controls notify) |
| `system_tuning_monitor.sh` | inotify + PHP-FPM utilisation trends | Every 6 hours | | `system_tuning_monitor.sh` | inotify + PHP-FPM utilisation trends | Every 6 hours |
| `mesh_monitor.sh` | Mesh membership — who covers whom, partnership state | On demand |
--- ---
@@ -143,6 +145,10 @@ Sunday morning block (after nightly maintenance):
Every 6h: Every 6h:
system_tuning_monitor.sh ──────────────────► TUNING_MONITOR_LOG system_tuning_monitor.sh ──────────────────► TUNING_MONITOR_LOG
On demand:
mesh_monitor.sh ──────────────────────────── reads host*.conf only (no API, no SSH)
MEMBERS / COVERAGE / PARTNERSHIP sections
Daily 8am: Daily 8am:
weekly_health_digest.sh ── reads ──────────► FALLBACK_STATE_FILE weekly_health_digest.sh ── reads ──────────► FALLBACK_STATE_FILE
── reads ──────────► WATCHDOG_STATE_FILE ── reads ──────────► WATCHDOG_STATE_FILE
+47 -3
View File
@@ -104,7 +104,7 @@ array_started.sh
# Cron — one entry per maintenance window: # Cron — one entry per maintenance window:
*/3 * * * * transcode_management.sh */3 * * * * transcode_management.sh
0 */6 * * * arrs_failed_stalled_recovery.sh 0 */6 * * * arrs_failed_stalled_recovery.sh
*/30 * * * * rsync.sh emby-failover ← not an orchestrator, direct call */30 * * * * rsync.sh emby-fallback ← not an orchestrator, direct call
0 1 * * * daily_sync_maintenance.sh 0 1 * * * daily_sync_maintenance.sh
30 2 * * 0 weekly_sync_maintenance.sh 30 2 * * 0 weekly_sync_maintenance.sh
@@ -730,7 +730,7 @@ maintenance block before the 7am coffee report.
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
# Two Emby syncs run in parallel — dirty and clean: # Two Emby syncs run in parallel — dirty and clean:
# #
# emby-failover dirty sync (every 30 minutes, Emby running): # emby-fallback dirty sync (every 30 minutes, Emby running):
# watch states, library deltas, user activity — continuous coverage # watch states, library deltas, user activity — continuous coverage
# WAL files excluded — safe to copy while Emby writes # WAL files excluded — safe to copy while Emby writes
# HOST2 always within 30 minutes of HOST1 on playback state # HOST2 always within 30 minutes of HOST1 on playback state
@@ -838,6 +838,50 @@ weekly_sync_maintenance.sh --status # show configured shares, jobs, update t
--- ---
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
## 📆 monthly_maintenance.sh
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Uptime-triggered orchestrator for long-running system tasks — ZFS scrub, SMART
long tests — that should only run on stable systems that have been up for at least
30 days. Called daily by cron; most invocations are silent no-ops.
Two gates must both pass before any job runs:
1. Server uptime ≥ `MONTHLY_UPTIME_THRESHOLD_DAYS`
2. Last run ≥ `MONTHLY_RUN_INTERVAL_DAYS` ago (state file on `/boot/config/` — survives reboots)
If either gate fails, the script exits 0 with no output. This is expected — it runs
daily and most days are no-ops.
`--force` bypasses both gates and runs the job list immediately. Use for testing or
when a scrub was missed and the server hasn't reached the uptime threshold yet.
### Configuration (master.conf)
```bash
MONTHLY_MAINTENANCE_SCRIPTS=(
#"Tools/zfs_pool_scrub.sh"
#"Tools/smart_long_test.sh"
)
MONTHLY_UPTIME_THRESHOLD_DAYS=30
MONTHLY_RUN_INTERVAL_DAYS=30
MONTHLY_LAST_RUN_FILE="/boot/config/monthly_maintenance_last_run.db"
```
Scripts are commented out by default — uncomment what applies to your hardware.
### Usage
```bash
monthly_maintenance.sh # normal run (daily cron — silent no-op when gates not met)
monthly_maintenance.sh --force # bypass both gates — run immediately
monthly_maintenance.sh --dry-run # show what would run without running it
monthly_maintenance.sh --status # show gate state: uptime, last run, next eligible
monthly_maintenance.sh --log # verbose output from each child script
```
---
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
## 🧹 media_management.sh ## 🧹 media_management.sh
## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ## ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
@@ -906,7 +950,7 @@ array_started.sh
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
# Periodic — not an orchestrator, direct rsync call: # Periodic — not an orchestrator, direct rsync call:
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
*/30 * * * * rsync.sh /mnt/user/Media_Server/Emby --profile=emby-failover */30 * * * * rsync.sh /mnt/user/Media_Server/Emby --profile=emby-fallback
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
# Every 6 hours: # Every 6 hours:
+3 -3
View File
@@ -296,7 +296,7 @@ Background: YES
```bash ```bash
#!/bin/bash #!/bin/bash
/mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \ /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
/mnt/user/Media_Server/Emby --profile=emby-failover /mnt/user/Media_Server/Emby --profile=emby-fallback
``` ```
**What it does:** **What it does:**
@@ -304,7 +304,7 @@ Keeps HOST2's Emby within 30 minutes of HOST1 on watch states, play positions, a
library changes — with Emby running on both sides. Not a full sync (that's weekly) — library changes — with Emby running on both sides. Not a full sync (that's weekly) —
this is continuous coverage of what matters for failover. this is continuous coverage of what matters for failover.
**What syncs (emby-failover profile):** **What syncs (emby-fallback profile):**
``` ```
Syncs: users.db, library.db, authentication.db, config/ Syncs: users.db, library.db, authentication.db, config/
Skips: *.wal *.shm — WAL files being written by Emby, unsafe to copy mid-write Skips: *.wal *.shm — WAL files being written by Emby, unsafe to copy mid-write
@@ -800,7 +800,7 @@ Background: YES
**What syncs:** full clean Emby mirror — all databases checkpointed, metadata, plugins, **What syncs:** full clean Emby mirror — all databases checkpointed, metadata, plugins,
config. Both Emby instances stopped during sync. Excludes transcodes, logs, cache, crash config. Both Emby instances stopped during sync. Excludes transcodes, logs, cache, crash
files. This is the full-fidelity sync (vs emby-failover which is the dirty 30-minute sync). files. This is the full-fidelity sync (vs emby-fallback which is the dirty 30-minute sync).
--- ---
+1 -1
View File
@@ -580,7 +580,7 @@ Every 15 minutes:
critical_sync_maintenance.sh Critical-Data + Emby dirty sync + partnership check critical_sync_maintenance.sh Critical-Data + Emby dirty sync + partnership check
Every 30 minutes: Every 30 minutes:
rsync.sh --profile=emby-failover Emby watch state sync (separate cron entry) rsync.sh --profile=emby-fallback Emby watch state sync (separate cron entry)
Every 6 hours: Every 6 hours:
arrs_failed_stalled_recovery.sh blocklist + re-search failed imports arrs_failed_stalled_recovery.sh blocklist + re-search failed imports
+8 -8
View File
@@ -342,19 +342,19 @@ PROFILES["emby_EXCLUDE_DIRS"]=(
"transcodes/" "logs/" "crash*" "cache/" "transcodes/" "logs/" "crash*" "cache/"
) )
# ── emby-failover ───────────────────────────────────────────────────────────── # ── emby-fallback ─────────────────────────────────────────────────────────────
# Every 30 minutes, Emby STAYS RUNNING — dirty sync of critical state only # Every 30 minutes, Emby STAYS RUNNING — dirty sync of critical state only
PROFILES["emby-failover_CRITICAL_CONTAINER_NAMES"]=() # nothing stops PROFILES["emby-fallback_CRITICAL_CONTAINER_NAMES"]=() # nothing stops
PROFILES["emby-failover_EXCLUDE_DIRS"]=( PROFILES["emby-fallback_EXCLUDE_DIRS"]=(
"*.wal" "*.shm" # unsafe mid-write "*.wal" "*.shm" # unsafe mid-write
"transcodes/" "logs/" "crash*" "cache/" "transcodes/" "logs/" "crash*" "cache/"
) )
PROFILES["emby-failover_REMOTE_RESTART_CONTAINERS"]=("Emby") PROFILES["emby-fallback_REMOTE_RESTART_CONTAINERS"]=("Emby")
``` ```
### Two Emby Profiles — Why Both Exist ### Two Emby Profiles — Why Both Exist
**emby-failover** (every 30 minutes, Emby stays running): **emby-fallback** (every 30 minutes, Emby stays running):
- Syncs: users.db, library.db, authentication.db, config/ - Syncs: users.db, library.db, authentication.db, config/
- Skips: \*.wal, \*.shm, transcodes/, logs/, cache/ - Skips: \*.wal, \*.shm, transcodes/, logs/, cache/
- Why: WAL files are written while Emby runs — copying them would produce a corrupt database on HOST2 - Why: WAL files are written while Emby runs — copying them would produce a corrupt database on HOST2
@@ -366,7 +366,7 @@ PROFILES["emby-failover_REMOTE_RESTART_CONTAINERS"]=("Emby")
- Why: WAL is checkpointed on clean shutdown — safe to copy everything - Why: WAL is checkpointed on clean shutdown — safe to copy everything
- Result: HOST2 gets a gold-standard Emby state once per week - Result: HOST2 gets a gold-standard Emby state once per week
The two profiles work together. emby-failover keeps HOST2 current for immediate failover. The two profiles work together. emby-fallback keeps HOST2 current for immediate failover.
emby gives HOST2 full fidelity once per week. Neither alone is sufficient. emby gives HOST2 full fidelity once per week. Neither alone is sufficient.
--- ---
@@ -448,7 +448,7 @@ Watchdogs (resource, docker, system, stability) run separately via watchdog_orch
# Every 30 minutes — Emby dirty sync: # Every 30 minutes — Emby dirty sync:
*/30 * * * * */30 * * * *
bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \ bash /mnt/user/appdata/unraid_scripts/Rsync/rsync.sh \
/mnt/user/Media_Server/Emby --profile=emby-failover /mnt/user/Media_Server/Emby --profile=emby-fallback
# Every 6 hours — failed import + stalled download recovery: # Every 6 hours — failed import + stalled download recovery:
0 */6 * * * 0 */6 * * *
@@ -731,5 +731,5 @@ name doesn't match any profile key, or when testing a specific profile.
```bash ```bash
rsync.sh /mnt/user/appdata-Failover/Critical-Data --profile=critical-data rsync.sh /mnt/user/appdata-Failover/Critical-Data --profile=critical-data
rsync.sh /mnt/user/Media_Server/Emby --profile=emby-failover rsync.sh /mnt/user/Media_Server/Emby --profile=emby-fallback
``` ```
+3 -3
View File
@@ -68,7 +68,7 @@ Orchestrators/ ← decides what to sync, when, and in what order
critical_sync_maintenance.sh ────────────────────────────────► rsync.sh (partnership shares) critical_sync_maintenance.sh ────────────────────────────────► rsync.sh (partnership shares)
Cron (direct): Cron (direct):
*/30 * * * * ──────────────────────────────────► rsync.sh --profile=emby-failover */30 * * * * ──────────────────────────────────► rsync.sh --profile=emby-fallback
Monitors/ Monitors/
bandwidth_monitor.sh ◄─── called by rsync.sh after each sync (--log-transfer) bandwidth_monitor.sh ◄─── called by rsync.sh after each sync (--log-transfer)
@@ -94,7 +94,7 @@ Profile key = directory basename lowercased. `--profile=name` overrides.
| `critical-data` | Auth stack | Mariadb-Authelia, Redis-Authelia, NPM, Lldap | Authelia has restart delay | | `critical-data` | Auth stack | Mariadb-Authelia, Redis-Authelia, NPM, Lldap | Authelia has restart delay |
| `important-data` | NextCloud + Postgres | Postgres-NextCloud | NextCloud has restart delay | | `important-data` | NextCloud + Postgres | Postgres-NextCloud | NextCloud has restart delay |
| `emby` | Full Emby mirror | Emby (both sides) | Weekly — Sunday 2:30am | | `emby` | Full Emby mirror | Emby (both sides) | Weekly — Sunday 2:30am |
| `emby-failover` | Emby watch state delta | None | Dirty sync — Emby stays running | | `emby-fallback` | Emby watch state delta | None | Dirty sync — Emby stays running |
For full profile definitions see `Manual-Rsync.md`. For full profile definitions see `Manual-Rsync.md`.
@@ -116,7 +116,7 @@ Callers (Orchestrators/) ──────────────────
weekly_sync_maintenance.sh rsync.sh /path/to/share [--profile=name] │ weekly_sync_maintenance.sh rsync.sh /path/to/share [--profile=name] │
critical_sync_maintenance.sh ─────────────────────────────────────────────► │ critical_sync_maintenance.sh ─────────────────────────────────────────────► │
fallback.sh (writeback) │ fallback.sh (writeback) │
Direct cron (emby-failover) │ Direct cron (emby-fallback) │
┌─────────────────────────────────────┐ ┌─────────────────────────────────────┐
│ 1. Infer/accept profile │ │ 1. Infer/accept profile │
+19 -21
View File
@@ -11,7 +11,7 @@ making any changes.
- [emby_to_lidarr_sync.sh](#emby_to_lidarr_syncsh) - [emby_to_lidarr_sync.sh](#emby_to_lidarr_syncsh)
- [emby_to_sonarr_sync.sh](#emby_to_sonarr_syncsh) - [emby_to_sonarr_sync.sh](#emby_to_sonarr_syncsh)
- [emby_to_radarr_sync.sh](#emby_to_radarr_syncsh) - [emby_to_radarr_sync.sh](#emby_to_radarr_syncsh)
- [failover_state_reset.sh](#failover_state_resetsh) - [fallback_state_reset.sh](#fallback_state_resetsh) *(not yet built — manual workaround)*
- [watchdog_skip_list_manager.sh](#watchdog_skip_list_managersh) - [watchdog_skip_list_manager.sh](#watchdog_skip_list_managersh)
- [bulk_permissions_repair.sh](#bulk_permissions_repairsh) - [bulk_permissions_repair.sh](#bulk_permissions_repairsh)
- [container_data_export.sh](#container_data_exportsh) - [container_data_export.sh](#container_data_exportsh)
@@ -141,15 +141,17 @@ bash Tools/emby_to_radarr_sync.sh
--- ---
## failover_state_reset.sh ## fallback_state_reset.sh
Resets the fallback state file to NORMAL and clears all tier flags. State file only — > **Not yet built.** Use the manual workaround below.
does NOT start or stop any containers.
Planned: reset the fallback state file to NORMAL and clear all tier flags. State file
only — does NOT start or stop any containers.
### When to Use ### When to Use
``` ```
After failover_test.sh didn't complete cleanly After fallback_test.sh didn't complete cleanly
→ state left in FALLBACK but containers are actually back to normal → state left in FALLBACK but containers are actually back to normal
After a failed handback After a failed handback
@@ -162,13 +164,13 @@ After a dev/debug session
→ state left in a non-NORMAL state from testing → state left in a non-NORMAL state from testing
``` ```
### Verify Before Resetting ### Manual Workaround
Run `--status` first and check each of these before writing:
```bash ```bash
# Verify before resetting:
# Right containers on right server? # Right containers on right server?
continuous_scripts_status.sh # shows failover current state continuous_scripts_status.sh # shows fallback current state
# DDNS pointing correctly? # DDNS pointing correctly?
nslookup Gmer4Lfe.com # confirm it resolves to the right IP nslookup Gmer4Lfe.com # confirm it resolves to the right IP
@@ -178,24 +180,20 @@ pgrep -f "fallback.sh" # empty output = not running
# Both servers Tailscale connected? # Both servers Tailscale connected?
tailscale status # both hosts should show active tailscale status # both hosts should show active
# Check current state file:
cat /boot/config/fallback_state.db
# Reset to NORMAL (only after confirming containers and DDNS are correct):
echo "state=NORMAL" > /boot/config/fallback_state.db
``` ```
Resetting during an actual failover causes fallback.sh to think everything is normal Resetting during an actual fallback causes fallback.sh to think everything is normal
and stop covering the remote — services go offline until the next detection cycle. and stop covering the remote — services go offline until the next detection cycle.
### Usage ### What the State File Contains
```bash ```bash
fallback_state_reset.sh --status # show current state file — always check first
fallback_state_reset.sh --dry-run # show what would be written, no write
fallback_state_reset.sh # interactive reset — prompts for YES to confirm
fallback_state_reset.sh --force # non-interactive — for scripts, no terminal
```
### What Gets Written
```bash
# New state file after reset:
state=NORMAL state=NORMAL
fallback_start=0 fallback_start=0
handback_strikes=0 handback_strikes=0
+5 -13
View File
@@ -652,10 +652,8 @@ WATCHDOG_RESTART_CRASHED=true
WATCHDOG_BATCH_NOTIFY=true WATCHDOG_BATCH_NOTIFY=true
# State files: # State files:
WATCHDOG_STATE_FILE="/tmp/watchdog_state.db" WATCHDOG_STATE_FILE="/tmp/container_watchdog_state.db"
SYS_WATCHDOG_FAILED_FILE="/boot/config/system_watchdog_failed.db"
WATCHDOG_CONTAINER_RESTART_LOG="/boot/config/container_restart_history.db" WATCHDOG_CONTAINER_RESTART_LOG="/boot/config/container_restart_history.db"
SYS_WATCHDOG_STATE_FILE="/tmp/sys_watchdog_state.db"
# host*.conf # host*.conf
HOST1_WATCHDOG_CONTAINERS=() # "ContainerName:LimitMB" HOST1_WATCHDOG_CONTAINERS=() # "ContainerName:LimitMB"
@@ -679,6 +677,8 @@ STORAGE_WATCHDOG_STATE_FILE="/tmp/storage_watchdog_state.db"
# ) # )
# ── System Watchdog ──────────────────────────────────────────────────────────── # ── System Watchdog ────────────────────────────────────────────────────────────
# Reboot trigger only — warn/shutdown/recover RAM tiers handled by resource_watchdog.sh
# RW_RAM_HARD_GB > SYS_WATCHDOG_MEM_GB always (resource_watchdog acts before watchdog reboots)
SYS_WATCHDOG_STRIKE_LIMIT=2 SYS_WATCHDOG_STRIKE_LIMIT=2
SYSTEM_WATCHDOG_INTERVAL=300 SYSTEM_WATCHDOG_INTERVAL=300
@@ -686,10 +686,7 @@ SYS_WATCHDOG_REBOOT_WINDOW_HRS=2
SYS_WATCHDOG_MAX_REBOOTS=3 SYS_WATCHDOG_MAX_REBOOTS=3
SYS_WATCHDOG_OOM_LIMIT=3 SYS_WATCHDOG_OOM_LIMIT=3
MEM_WARN_GB=10 SYS_WATCHDOG_MEM_GB=4 # strike system → reboot (last resort — below resource_watchdog hard stop)
MEM_SHUTDOWN_GB=6
MEM_GB=4
MEM_RECOVER_GB=30
SYS_WATCHDOG_ROOTFS_CRITICAL_PCT=99 SYS_WATCHDOG_ROOTFS_CRITICAL_PCT=99
SYS_WATCHDOG_FD_CRITICAL_PCT=95 SYS_WATCHDOG_FD_CRITICAL_PCT=95
@@ -703,13 +700,8 @@ SYS_WATCHDOG_ABORT_ON_ZFS_UNHEALTHY=true
SYS_WATCHDOG_ABORT_ON_PARITY=true SYS_WATCHDOG_ABORT_ON_PARITY=true
SYS_WATCHDOG_ABORT_ON_MOVER=true SYS_WATCHDOG_ABORT_ON_MOVER=true
SYS_WATCHDOG_MEM_SHUTDOWN_EXCLUDED=(
"NginxProxyManager" "Authelia" "Mariadb" "Redis" "Emby" "Dispatcharr"
)
SYS_WATCHDOG_REQUIRED_CONTAINERS=()
# State files: # State files:
SYS_WATCHDOG_STATE_FILE="/tmp/sys_watchdog_state.db" SYS_WATCHDOG_STATE_FILE="/tmp/system_watchdog_state.db"
SYS_WATCHDOG_FAILED_FILE="/boot/config/system_watchdog_failed.db" SYS_WATCHDOG_FAILED_FILE="/boot/config/system_watchdog_failed.db"
SYS_WATCHDOG_REBOOT_LOG="/boot/config/system_watchdog_reboots.db" SYS_WATCHDOG_REBOOT_LOG="/boot/config/system_watchdog_reboots.db"
SYS_WATCHDOG_OOM_FILE="/tmp/system_watchdog_oom.db" SYS_WATCHDOG_OOM_FILE="/tmp/system_watchdog_oom.db"