# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ # 🐳 DOCKER ESSENTIALS — Manual # ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Configuration reference, setup procedures, and operational workflows. For folder overview and design philosophy see `README-Docker_Essentials.md`. For per-script detail see the script headers directly. > **Watchdog configuration has moved.** `docker_watchdog.sh` is now in `Watchdogs/`. > Memory limits, CPU thresholds, HTTP health checks, dependency ordering, skip list > recovery, and all watchdog config vars are in `Watchdogs/Manual-Watchdogs.md`. --- ## ━━━ RESTART SCHEDULE CONFIGURATION ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ --- ### ── Daily Restart List ─────────────────────────────────────────────────────── ```bash # host1.conf # Restarted every night at 1am via daily_sync_maintenance.sh. # # Good candidates: # Reverse proxies — connection table fills slowly over weeks # Authentication services — session cache benefits from periodic clearing # Live TV schedulers — accumulated scheduling state slows decisions # Download managers — connection pool maintenance # HOST1_DAILY_RESTART_CONTAINERS=( "NginxProxyManager" # connection table fills slowly over weeks "Authelia" # session cache benefits from periodic clearing "Dispatcharr" # Live TV scheduler accumulates state "Dispatcharr-Basic" # secondary Live TV scheduler — same reason "ErsatzTV-Emby" # channel schedule builder, stale entries accumulate ) ``` This list also drives `docker_update.sh` in normal mode — containers added here get their images updated daily before the restart. Add a container once, it gets both. --- ### ── Weekly Restart List ────────────────────────────────────────────────────── ```bash # host1.conf # Restarted every Sunday at 2:30am via weekly_sync_maintenance.sh. # Runs AFTER the sync window's own restart of critical containers (Emby, auth stack). # # Daily vs Weekly decision: # Daily: connection-heavy infrastructure — degrades faster (proxy, auth, Live TV) # Weekly: productivity and media services — degrades slowly (NextCloud, AdGuard, Immich) # HOST1_WEEKLY_RESTART_CONTAINERS=( "NextCloud" # file sync — benefits from clean weekly start "AdGuard-Home" # DNS — cache and stat accumulation "Immich" # photo library — index/cache maintenance ) ``` --- ## ━━━ NETWORK CONFIGURATION ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ```bash # host1.conf # Networks to ensure exist + containers to connect to each network. # Many-to-many: every container connects to every network listed. # # Containers do not need to be running — script handles missing containers # gracefully (warns + skips). They connect on the next array start. # HOST1_NETWORK_CONNECT_NETWORKS=( "high-availability" # main internal network — most containers should be on this ) HOST1_NETWORK_CONNECT_CONTAINERS=( "memcached" # NextCloud's cache — needs to reach NextCloud AIO network "Npm-CrowdSec" # CrowdSec bouncer — needs to reach NPM's network ) ``` > **Timing dependency:** Networks created by Docker Compose stacks (e.g. NextCloud AIO) > only exist after those stacks start. If this script runs before the Compose stack, > the network won't exist yet and the connection fails this run. It will succeed on the > next array start. Schedule Compose stacks early in `ARRAY_START_SCRIPTS` order to > minimise the window. --- ## ━━━ CONTAINER UPDATE CONFIGURATION ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ```bash # master.conf DAILY_CONTAINER_UPDATES=true # enable/disable the daily image pull # docker_daily_restart.sh still runs regardless # update and restart are independent WEEKLY_CONTAINER_UPDATES=true # enable/disable the weekly image pull # docker_weekly_restart.sh still runs regardless MONTHLY_REMAINING_UPDATES=true # enable/disable the monthly remainder pull # to disable: set false, or remove # "docker_update.sh --remainder" from # MONTHLY_MAINTENANCE_SCRIPTS ``` There is one update script, `docker_update.sh`, with three modes. Each reuses the restart list it pairs with, so there is no second list to maintain: | Mode | Targets | Runs | |------|---------|------| | *(default)* | `DAILY_RESTART_CONTAINERS` | Daily, before `docker_daily_restart.sh` | | `--weekly` | `WEEKLY_RESTART_CONTAINERS` | Weekly, before `docker_weekly_restart.sh` | | `--remainder` | derived, see below | Monthly, via `MONTHLY_MAINTENANCE_SCRIPTS` | Remainder mode needs no configuration at all. It takes every **running** container and subtracts: ``` DAILY_RESTART_CONTAINERS already updated daily WEEKLY_RESTART_CONTAINERS already updated weekly PROFILE_CRITICAL_CONTAINER_NAMES[emby] updated inline by the weekly sync window PROFILE_CRITICAL_CONTAINER_NAMES[critical-data] updated inline by the weekly sync window FALLBACK__TIER1..4 owned by the remote's update cycle ``` Stopped containers are never targeted in any mode — pulling for a stopped container adds nothing, and it was most likely stopped deliberately. The fallback exclusion is a correctness rule, not an optimisation. This server only runs those containers during a fallback; the remote owns their version. If remainder updated them independently and a handback then occurred, the remote's older image could meet data written by the newer one. **Ordering matters.** The update always runs *before* its matching restart so the restart lands on the freshly pulled image. If a container's image actually changed, `docker_update.sh` rebuilds it from its template (a plain `docker restart` reuses the image ID baked in at creation time and would never pick up the new digest) and records it in `DOCKER_UPDATE_REBUILT_*_FILE`. The restart script reads that file and skips those containers rather than restarting them a second time — and discards the file as stale if it is older than `DOCKER_UPDATE_REBUILT_STALE_HOURS`. --- ## ━━━ FULL CONFIGURATION REFERENCE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ### ── host*.conf ──────────────────────────────────────────────────────── ```bash # Per-host — varies between HOST1 and HOST2 # Daily restart list (also drives docker_update.sh normal mode) HOST1_DAILY_RESTART_CONTAINERS=() # Weekly restart list HOST1_WEEKLY_RESTART_CONTAINERS=() # Networks to ensure exist HOST1_NETWORK_CONNECT_NETWORKS=() # Containers to connect to every configured network HOST1_NETWORK_CONNECT_CONTAINERS=() # Watchdog config (memory limits, HTTP checks, required, dependencies): # → see Watchdogs/Manual-Watchdogs.md ``` --- ### ── master.conf ────────────────────────────────────────────────────────────── ```bash # Shared — applies to both servers # ── Container updates ────────────────────────────────────────────────── DAILY_CONTAINER_UPDATES=true # daily pull (DAILY_RESTART_CONTAINERS) WEEKLY_CONTAINER_UPDATES=true # weekly pull (WEEKLY_RESTART_CONTAINERS) MONTHLY_REMAINING_UPDATES=true # monthly pull (everything else) # Handoff between update and restart — written by docker_update.sh, # read by the matching restart script so it skips containers already # rebuilt onto a new image this run. DOCKER_UPDATE_REBUILT_DAILY_FILE DOCKER_UPDATE_REBUILT_WEEKLY_FILE DOCKER_UPDATE_REBUILT_STALE_HOURS=12 # older than this = discarded, restart all # ── Retry behaviour (shared by restart scripts) ──────────────────────── RETRY_COUNT=3 # retry attempts before marking failed SLEEP=5 # seconds between retry attempts CONTAINER_DELAY # seconds between a dependency and its dependents RESTART_VERIFY_WAIT=3 # settle time before verifying a restart stuck # Watchdog thresholds (CPU, memory, HTTP, restart loop): # → see Watchdogs/Manual-Watchdogs.md ``` --- ## ━━━ PROCEDURES ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ### ── Adding a Container to Monitoring ──────────────────────────────────────── Adding a container to watchdog monitoring is purely additive — add lines to `host*.conf`. No script changes. `detect_hosts()` picks up the new config on the next watchdog cycle. See `Watchdogs/Manual-Watchdogs.md` for the full procedure. To add a container to **daily restarts** (and daily image updates): ```bash # host1.conf HOST1_DAILY_RESTART_CONTAINERS=( ...existing... "MyApp" # also adds it to the daily image update ) ``` To **exclude** a container from Tier 2 global scan (e.g. one-shot that exits normally): ```bash # master.conf WATCHDOG_SCAN_IGNORE=( "MyApp" # one-shot — exits cleanly, don't treat as crash ) ``` --- ### ── Skip List Recovery ──────────────────────────────────────────────────────── When `docker_watchdog.sh` skip-lists a container, use `Tools/watchdog_skip_list_manager.sh`. Full procedure in `Watchdogs/Manual-Watchdogs.md`. ```bash Tools/watchdog_skip_list_manager.sh --status # see skip list + container states Tools/watchdog_skip_list_manager.sh --clear MyApp # clear after fixing root cause Tools/watchdog_skip_list_manager.sh --clear-all # clear everything ``` --- ## ━━━ FLAG REFERENCE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ All scripts support these standard flags: | Flag | What it does | |------|-------------| | `--dry-run` | Preview actions without making changes. Shows exactly what would happen. | | `--status` | Show current config, container states, and relevant runtime info, then exit. | | `--log` | Verbose mode — adds per-container banners, action lines, pull output, and list details. | **Output tiers:** Without `--log`, each script processes silently and concludes with a summary block: identity, duration, counts, and a status line. Per-container detail only appears with `--log`. Warnings and errors are always visible regardless of `--log`. ### Script-specific flags | Script | Flag | What it does | |--------|------|-------------| | `docker_update.sh` | `--weekly` | Target `WEEKLY_RESTART_CONTAINERS`. Called by `weekly_sync_maintenance.sh` before the weekly restart. | | `docker_update.sh` | `--remainder` | Target every running container not in the daily list, weekly list, emby/critical-data profiles, or fallback tiers. Called by `monthly_maintenance.sh`. Safe to run manually to sweep anything missed. | | `media_cleaner.sh` *(Media/)* | `` | Required positional profile — there is no default. | `docker_update.sh` with no mode flag is normal mode: `DAILY_RESTART_CONTAINERS`, called by `daily_sync_maintenance.sh` before the daily restart. ### Exit codes | Code | Meaning | |------|---------| | `0` | Success, or nothing to do (empty list, disabled toggle, no running containers) | | `1` | One or more containers failed — the summary names them | `docker_update.sh` deliberately exits `0` even when pulls fail: a failed pull is not a reason to abort the restart that follows, which simply proceeds on the existing image. The failure is reported in the summary.