FailedProxy/Varaverk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Correct and extend folder docs for Docker Essentials, Watchdogs, Media and Arrs Stack

Browse Source 
The docs had drifted from the scripts — a script that no longer exists, three wrong variable
names, a reversed run order, and seven scheduled scripts that were never documented at all.
This commit is contained in:
Gmer4Lfe
2026-08-01 22:59:07 -04:00
parent c377ddfcca
commit 8a2707ee37
6 changed files with 331 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Docker_Essentials/Manual-Docker_Essentials.md
+75 -14
View File
@@ -94,20 +94,54 @@ HOST1_NETWORK_CONNECT_CONTAINERS=(
```bash
# master.conf
DAILY_CONTAINER_UPDATES=true # enable/disable daily image pull
DAILY_CONTAINER_UPDATES=true # enable/disable the daily image pull
# docker_daily_restart.sh still runs regardless
# update and restart are independent
WEEKLY_REMAINING_UPDATES=true # enable/disable weekly remainder pull + prune
# to disable: set false or remove from WEEKLY_MAINTENANCE_SCRIPTS
WEEKLY_CONTAINER_UPDATES=true # enable/disable the weekly image pull
# docker_weekly_restart.sh still runs regardless
MONTHLY_REMAINING_UPDATES=true # enable/disable the monthly remainder pull
# to disable: set false, or remove
# "docker_update.sh --remainder" from
# MONTHLY_MAINTENANCE_SCRIPTS
```
`docker_update.sh` in normal mode targets `DAILY_RESTART_CONTAINERS` — the same list
used by `docker_daily_restart.sh`. No second list to maintain.
There is one update script, `docker_update.sh`, with three modes. Each reuses the restart
list it pairs with, so there is no second list to maintain:
`docker_update_remaining.sh` derives its target list automatically:
all running containers minus `DAILY_RESTART_CONTAINERS` minus `WEEKLY_RESTART_CONTAINERS`.
Everything gets updated at least once per week with no explicit configuration.
| Mode | Targets | Runs |
|------|---------|------|
| *(default)* | `DAILY_RESTART_CONTAINERS` | Daily, before `docker_daily_restart.sh` |
| `--weekly` | `WEEKLY_RESTART_CONTAINERS` | Weekly, before `docker_weekly_restart.sh` |
| `--remainder` | derived, see below | Monthly, via `MONTHLY_MAINTENANCE_SCRIPTS` |
Remainder mode needs no configuration at all. It takes every **running** container and
subtracts:
```
DAILY_RESTART_CONTAINERS already updated daily
WEEKLY_RESTART_CONTAINERS already updated weekly
PROFILE_CRITICAL_CONTAINER_NAMES[emby] updated inline by the weekly sync window
PROFILE_CRITICAL_CONTAINER_NAMES[critical-data] updated inline by the weekly sync window
FALLBACK_<REMOTE>_TIER1..4 owned by the remote's update cycle
```
Stopped containers are never targeted in any mode — pulling for a stopped container adds
nothing, and it was most likely stopped deliberately.
The fallback exclusion is a correctness rule, not an optimisation. This server only runs
those containers during a fallback; the remote owns their version. If remainder updated them
independently and a handback then occurred, the remote's older image could meet data written
by the newer one.
**Ordering matters.** The update always runs *before* its matching restart so the restart
lands on the freshly pulled image. If a container's image actually changed, `docker_update.sh`
rebuilds it from its template (a plain `docker restart` reuses the image ID baked in at
creation time and would never pick up the new digest) and records it in
`DOCKER_UPDATE_REBUILT_*_FILE`. The restart script reads that file and skips those containers
rather than restarting them a second time — and discards the file as stale if it is older
than `DOCKER_UPDATE_REBUILT_STALE_HOURS`.
---
@@ -142,12 +176,22 @@ HOST1_NETWORK_CONNECT_CONTAINERS=()
# Shared — applies to both servers
# ── Container updates ──────────────────────────────────────────────────
DAILY_CONTAINER_UPDATES=true
WEEKLY_REMAINING_UPDATES=true
DAILY_CONTAINER_UPDATES=true # daily pull (DAILY_RESTART_CONTAINERS)
WEEKLY_CONTAINER_UPDATES=true # weekly pull (WEEKLY_RESTART_CONTAINERS)
MONTHLY_REMAINING_UPDATES=true # monthly pull (everything else)
# Handoff between update and restart — written by docker_update.sh,
# read by the matching restart script so it skips containers already
# rebuilt onto a new image this run.
DOCKER_UPDATE_REBUILT_DAILY_FILE
DOCKER_UPDATE_REBUILT_WEEKLY_FILE
DOCKER_UPDATE_REBUILT_STALE_HOURS=12 # older than this = discarded, restart all
# ── Retry behaviour (shared by restart scripts) ────────────────────────
RETRY_COUNT=3 # retry attempts before marking failed
SLEEP=5 # seconds between retry attempts
CONTAINER_DELAY # seconds between a dependency and its dependents
RESTART_VERIFY_WAIT=3 # settle time before verifying a restart stuck
# Watchdog thresholds (CPU, memory, HTTP, restart loop):
# → see Watchdogs/Manual-Watchdogs.md
@@ -211,7 +255,24 @@ All scripts support these standard flags:
summary block: identity, duration, counts, and a status line. Per-container detail only
appears with `--log`. Warnings and errors are always visible regardless of `--log`.
### `docker_update.sh --remainder`
Switches to remainder mode — updates all running containers not in the managed daily/weekly
lists. Called by `weekly_sync_maintenance.sh`. Can be run manually to sweep containers
that haven't been updated recently.
### Script-specific flags
| Script | Flag | What it does |
|--------|------|-------------|
| `docker_update.sh` | `--weekly` | Target `WEEKLY_RESTART_CONTAINERS`. Called by `weekly_sync_maintenance.sh` before the weekly restart. |
| `docker_update.sh` | `--remainder` | Target every running container not in the daily list, weekly list, emby/critical-data profiles, or fallback tiers. Called by `monthly_maintenance.sh`. Safe to run manually to sweep anything missed. |
| `media_cleaner.sh` *(Media/)* | `<anime\|media>` | Required positional profile — there is no default. |
`docker_update.sh` with no mode flag is normal mode: `DAILY_RESTART_CONTAINERS`, called by
`daily_sync_maintenance.sh` before the daily restart.
### Exit codes
| Code | Meaning |
|------|---------|
| `0` | Success, or nothing to do (empty list, disabled toggle, no running containers) |
| `1` | One or more containers failed — the summary names them |
`docker_update.sh` deliberately exits `0` even when pulls fail: a failed pull is not a reason
to abort the restart that follows, which simply proceeds on the existing image. The failure
is reported in the summary.
Docker_Essentials/README-Docker_Essentials.md
+64 -9
View File
@@ -178,12 +178,28 @@ windows so any downtime from restarts is absorbed by the window that's already h
---
### 🔄 Image Currency — `docker_update.sh` + `docker_update_remaining.sh`
### 🔄 Image Currency — `docker_update.sh`
Keeps all container images current without manual intervention. Daily updates for the
auth/proxy stack (the containers that restart daily anyway — no extra downtime). Weekly
remainder pass for everything else — derives the target list automatically from `docker ps`
minus what was already updated, so there is no second list to maintain.
One script, three modes — there is no separate remainder script.
| Mode | Targets | Runs |
|------|---------|------|
| *(default)* | `DAILY_RESTART_CONTAINERS` | Daily, **before** `docker_daily_restart.sh` |
| `--weekly` | `WEEKLY_RESTART_CONTAINERS` | Weekly, **before** `docker_weekly_restart.sh` |
| `--remainder` | Everything running that is in neither list | Monthly, via `monthly_maintenance.sh` |
The update always runs *before* its matching restart, so the restart lands on the freshly
pulled image. Reversing that order would restart onto the old image and leave the new one
sitting unused until the next window.
Each mode reuses the restart list it pairs with rather than maintaining its own — add a
container to `DAILY_RESTART_CONTAINERS` once and it gets both the restart and the image pull.
Remainder mode needs no list at all: it derives its targets from `docker ps` minus the daily
list, the weekly list, the emby/critical-data sync-window profiles, and the fallback tiers.
Fallback containers are deliberately excluded from remainder mode. This server only runs them
during a fallback; the remote owns their version. Updating them here would risk the remote's
older image meeting data written by a newer one after a handback.
---
@@ -229,7 +245,7 @@ full coordination model between all four watchdogs.
|--------|------|-------------|
| `docker_daily_restart.sh` | Nightly proactive restart of degradation-prone containers | 1am via `daily_sync_maintenance.sh` |
| `docker_weekly_restart.sh` | Weekly restart of less-critical services | 2:30am Sunday via `weekly_sync_maintenance.sh` |
| `docker_update.sh` | Container image updates — daily list + weekly remainder mode | Daily before restart; weekly remainder pass |
| `docker_update.sh` | Container image updates — three modes (default / `--weekly` / `--remainder`) | Daily and weekly before each restart; monthly for the remainder |
| `docker_network_connect.sh` | Network existence + container connection enforcement | Every array start |
| `docker_container_stop.sh` | Ordered container shutdown — graceful then forced | Called by `array_stopping.sh` |
| `downloaders_reset.sh` | Download client hygiene — slskd / SABnzbd / qBittorrent | Every 30min via `critical_sync_maintenance.sh` |
@@ -257,8 +273,13 @@ Daily maintenance window (1am):
Weekly maintenance window (2:30am Sunday):
weekly_sync_maintenance.sh
├── docker_weekly_restart.sh ──── restart less-critical services
└── docker_update.sh --remainder ─ update containers not in managed lists
├── docker_update.sh --weekly ─── pull latest (WEEKLY_RESTART_CONTAINERS)
└── docker_weekly_restart.sh ──── restart onto the fresh image
Monthly maintenance window:
monthly_maintenance.sh
├── docker_update.sh --remainder update everything not in the managed lists
└── Tools/docker_prune_images.sh --all
Critical maintenance window (every 30min):
critical_sync_maintenance.sh
@@ -266,5 +287,39 @@ Critical maintenance window (every 30min):
Array stopping:
array_stopping.sh
└── docker_container_stop.sh ──── ordered graceful shutdown
└── docker_container_stop.sh ──── ordered graceful shutdown, verified per container
```
---
## ━━━ SAFEGUARDS COMMON TO THIS FOLDER ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Every script here talks to the Docker daemon, so they share the same protections. Each
script's own header documents its full set; these are the ones worth knowing folder-wide.
**Daemon health is checked, not assumed.** A hung daemon returns an empty container list,
which is indistinguishable from "no containers running". Without the check,
`docker_container_stop.sh` would report a clean shutdown that never happened, and
`docker_update.sh --remainder` would report "nothing to update" while doing nothing.
**Every docker call is timeout-wrapped.** A wedged daemon cannot stall a maintenance window
or hold a lock open. The one deliberate exception is `docker pull` — a large image
legitimately outlasts any sane timeout, and killing it mid-layer wastes the transfer.
**State is respected.** Running containers get restarted; stopped ones stay stopped. A
stopped container was almost certainly stopped on purpose, and none of these scripts has the
authority to overrule that.
**Restarts are verified, not assumed.** After each restart the container is re-checked once
it has had time to settle. A container that starts and immediately crashes is recorded as a
failure and notified — a restart that did not stick is never reported as success.
**Dependency ordering is shared with the watchdog.** Restarts follow
`HOST*_WATCHDOG_DEPENDENCIES`, with `CONTAINER_DELAY` between a dependency and its dependents,
so a dependent is never brought up while what it needs is still initialising.
**Locks prevent overlap.** Long windows can outlast their interval;
`downloaders_reset.sh` uses wait-mode because it runs every 30 minutes and the previous pass
may still be finishing.
---