FailedProxy/Varaverk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix stale RAM tier vars in stability_watchdog section of Watchdogs manual

Browse Source 
Update the Three-Tier Response and RAM Tiers sections to reflect the
resource_watchdog/stability_watchdog split: MEM_WARN_GB/MEM_SHUTDOWN_GB/
MEM_RECOVER_GB moved to RW_RAM_SOFT/MEDIUM/HARD/RECOVER_GB in resource_watchdog;
stability_watchdog now only holds SYS_WATCHDOG_MEM_GB (reboot trigger). Update
required containers reference from SYS_WATCHDOG_REQUIRED_CONTAINERS (removed)
to HOST*_WATCHDOG_REQUIRED_CONTAINERS in host*.conf.
This commit is contained in:
Gmer4Lfe
2026-05-22 22:09:13 -04:00
parent 2fa78a21b6
commit 2d0c91ea13
1 changed files with 13 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Watchdogs/Manual-Watchdogs.md
+13 -9
View File
@@ -524,14 +524,14 @@ healing at every other layer has failed or when the failure is non-recoverable.
| /boot read-only | write test fails | Config writes silently failing | | /boot read-only | write test fails | Config writes silently failing |
**Tier 2 — URGENT (bypass strikes with OOM confirmation)** **Tier 2 — URGENT (bypass strikes with OOM confirmation)**
RAM below `MEM_GB` AND OOM kills this cycle ≥ `SYS_WATCHDOG_OOM_LIMIT`. RAM below `SYS_WATCHDOG_MEM_GB` AND OOM kills this cycle ≥ `SYS_WATCHDOG_OOM_LIMIT`.
Both conditions required — RAM alone uses the standard strike system. Both conditions required — RAM alone uses the standard strike system.
OOM confirms the system is dying faster than watchdogs can heal. OOM confirms the system is dying faster than watchdogs can heal.
**Tier 3 — STANDARD (`SYS_WATCHDOG_STRIKES` consecutive failures → reboot)** **Tier 3 — STANDARD (`SYS_WATCHDOG_STRIKES` consecutive failures → reboot)**
| Check | Threshold | | Check | Threshold |
|-------|-----------| |-------|-----------|
| Free RAM | `MEM_WARN_GB` → `MEM_SHUTDOWN_GB` → `MEM_GB` | | Free RAM | `SYS_WATCHDOG_MEM_GB` (reboot trigger — earlier tiers handled by resource_watchdog) |
| Load average | `SYS_WATCHDOG_LOAD_MULTIPLIER` × cpu_count | | Load average | `SYS_WATCHDOG_LOAD_MULTIPLIER` × cpu_count |
| CPU temperature | `SYS_WATCHDOG_CPU_TEMP` | | CPU temperature | `SYS_WATCHDOG_CPU_TEMP` |
| Zombie processes | `SYS_WATCHDOG_ZOMBIES` | | Zombie processes | `SYS_WATCHDOG_ZOMBIES` |
@@ -539,19 +539,23 @@ OOM confirms the system is dying faster than watchdogs can heal.
| /tmp usage | `SYS_WATCHDOG_TMP_PCT` | | /tmp usage | `SYS_WATCHDOG_TMP_PCT` |
| Array disk errors | mdstat error delta > 0 | | Array disk errors | mdstat error delta > 0 |
| NIC state | interface operstate != "up" | | NIC state | interface operstate != "up" |
| Required containers | containers in `SYS_WATCHDOG_REQUIRED_CONTAINERS` | | Required containers | containers in `HOST*_WATCHDOG_REQUIRED_CONTAINERS` (host*.conf) |
### RAM Tiers ### RAM Tiers
RAM pressure is a graduated response split across resource_watchdog and stability_watchdog:
``` ```
MEM_WARN_GB (10GB) → warn + notify, no action RW_RAM_SOFT_GB (12GB) → throttle downloads, reduce background load (resource_watchdog)
MEM_SHUTDOWN_GB (6GB) → stop non-essential containers, wait for recovery RW_RAM_MEDIUM_GB (8GB) → pause background containers (resource_watchdog)
MEM_GB (4GB) → strike → reboot (URGENT bypass with OOM) RW_RAM_HARD_GB (6GB) → stop optional containers, wait for recovery (resource_watchdog)
MEM_RECOVER_GB (30GB) → RAM must reach this before stopped containers restart SYS_WATCHDOG_MEM_GB (4GB) → strike → reboot (last resort) (stability_watchdog)
RW_RAM_RECOVER_GB (20GB) → RAM must reach this before stopped containers restart
``` ```
At `MEM_SHUTDOWN_GB`, all containers NOT listed in `SYS_WATCHDOG_MEM_SHUTDOWN_EXCLUDED` `SYS_WATCHDOG_MEM_GB` must always be below `RW_RAM_HARD_GB` — resource_watchdog acts first.
are stopped. Adjust in master.conf for your critical services. Containers stopped at the hard tier use `HOST*_RW_STOP_CONTAINERS` in host*.conf.
Containers paused at the medium tier use `HOST*_RW_PAUSE_CONTAINERS` in host*.conf.
### Abort Conditions ### Abort Conditions