From f16c962ac061c8eb78896fa6e030691198d700ba Mon Sep 17 00:00:00 2001 From: Gmer4Lfe Date: Sun, 10 May 2026 22:12:08 -0400 Subject: [PATCH] feat: rename array orchestrators, add docker_update remainder mode MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Rename array_start.sh → array_started.sh, array_stop.sh → array_stopping.sh to clarify these are event-driven (array has started/is stopping), not imperative - Update all references across 9 files (master.conf, user_script_plug-in.sh, watchdogs, continuous_scripts_status.sh, claude_startup.sh) - Add --remainder mode to docker_update.sh: updates all running containers excluding daily containers, weekly sync-window containers (emby + critical-data), and fallback coverage containers (FALLBACK_${MY_ID}_COVERS_${REMOTE_ID}_TIER*) Fallback containers excluded because the remote server owns their version — independent updates risk writeback incompatibility on handback - weekly_sync_maintenance.sh calls docker_update.sh --remainder as final step - git_pull_execute.sh: add safe.directory config to fix dubious ownership error when running as root on a directory owned by uid 1000 Co-Authored-By: Claude Sonnet 4.6 --- Docker_Essentials/docker_container_stop.sh | 2 +- Docker_Essentials/docker_update.sh | 135 +++++++++++++++--- Docker_Essentials/docker_watchdog.sh | 2 +- Monitors/continuous_scripts_status.sh | 6 +- .../{array_start.sh => array_started.sh} | 8 +- .../{array_stop.sh => array_stopping.sh} | 10 +- Orchestrators/weekly_sync_maintenance.sh | 26 ++++ Tools/claude_startup.sh | 4 +- git_pull_execute.sh | 1 + master.conf | 12 +- unRAID_Essentials/system_watchdog.sh | 2 +- user_script_plug-in.sh | 24 ++-- 12 files changed, 176 insertions(+), 56 deletions(-) rename Orchestrators/{array_start.sh => array_started.sh} (96%) rename Orchestrators/{array_stop.sh => array_stopping.sh} (95%) diff --git a/Docker_Essentials/docker_container_stop.sh b/Docker_Essentials/docker_container_stop.sh index b402489..6d75f1d 100755 --- a/Docker_Essentials/docker_container_stop.sh +++ b/Docker_Essentials/docker_container_stop.sh @@ -3,7 +3,7 @@ # ============================= Docker Container Stop ========================================== # ============================================================================================== # Stops all running Docker containers one at a time, verifying each is stopped before -# moving to the next. Called by array_stop.sh as part of a planned shutdown sequence. +# moving to the next. Called by array_stopping.sh as part of a planned shutdown sequence. # # ── STOP SEQUENCE PER CONTAINER ────────────────────────────────────────────────────────────── # 1. docker stop -t 30 (SIGTERM + 30s grace period — docker sends SIGKILL if needed) diff --git a/Docker_Essentials/docker_update.sh b/Docker_Essentials/docker_update.sh index 00289fb..9740b81 100644 --- a/Docker_Essentials/docker_update.sh +++ b/Docker_Essentials/docker_update.sh @@ -2,11 +2,26 @@ # ============================================================================================== # ================================= Docker Update ============================================== # ============================================================================================== +# Two modes — normal (daily) and remainder (weekly). +# +# ── NORMAL MODE (daily) ─────────────────────────────────────────────────────────────────────── # Pulls the latest image for each container in HOST*_DAILY_RESTART_CONTAINERS. # Called by daily_sync_maintenance.sh before docker_daily_restart.sh — containers stay # running during the pull, so there is no extra downtime. # -# ── WHY SAME LIST AS DAILY RESTART ─────────────────────────────────────────────────────────── +# ── REMAINDER MODE (weekly) ─────────────────────────────────────────────────────────────────── +# Called by weekly_sync_maintenance.sh as the last step. +# Updates all currently running containers that are NOT in: +# DAILY_RESTART_CONTAINERS — already updated daily +# emby + critical-data profiles — already updated inline by the weekly sync window +# FALLBACK_${MY_ID}_COVERS_${REMOTE_ID}_TIER* — owned by the remote server's update cycle +# +# Fallback containers are excluded because this server only runs them during a failover. +# The remote server is the version owner — if remainder updates them independently and a +# handback writeback occurs, the remote's older version may not handle the newer data. +# This catches everything local-only (Organizr, AdGuard, etc.) once a week. +# +# ── WHY SAME LIST AS DAILY RESTART (normal mode) ───────────────────────────────────────────── # Containers that restart daily (auth stack: Authelia, NPM, Mariadb, Redis, etc.) are # exactly the containers that benefit from staying current. Reusing DAILY_RESTART_CONTAINERS # means no second list to maintain — add/remove a container once and both update + restart @@ -16,18 +31,21 @@ # docker pull — fetches the latest digest from the registry # Old vs new image ID comparison — distinguishes "updated" from "already current" # Containers keep running — pull does not affect the live container -# docker_daily_restart.sh runs after — containers restart on the fresh image +# docker_daily_restart.sh runs after (normal mode) — containers restart on the fresh image # # ── TOGGLE ──────────────────────────────────────────────────────────────────────────────────── -# DAILY_CONTAINER_UPDATES=false in master.conf — skips all pulls, exits cleanly +# DAILY_CONTAINER_UPDATES=false in master.conf — skips normal mode, exits cleanly # docker_daily_restart.sh still runs regardless — update and restart are independent +# Remainder mode has no toggle — exclude it from WEEKLY_MAINTENANCE_SCRIPTS to disable # # ── CONFIGURATION ───────────────────────────────────────────────────────────────────────────── -# master.conf: DAILY_CONTAINER_UPDATES — enable/disable (default: true) -# master_host*.conf: HOST*_DAILY_RESTART_CONTAINERS — containers to update +# master.conf: DAILY_CONTAINER_UPDATES — enable/disable normal mode (default: true) +# master_host*.conf: HOST*_DAILY_RESTART_CONTAINERS — containers to update (normal mode) +# master.conf: PROFILE_CRITICAL_CONTAINER_NAMES[emby|critical-data] — remainder exclusions # # ── USAGE ───────────────────────────────────────────────────────────────────────────────────── -# docker_update.sh — normal run +# docker_update.sh — normal mode: update DAILY_RESTART_CONTAINERS +# docker_update.sh --remainder — remainder mode: update all except daily + weekly sync containers # docker_update.sh --dry-run — show what would be pulled # docker_update.sh --log — verbose output # docker_update.sh --status — show config and exit @@ -37,7 +55,20 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" source "$SCRIPT_DIR/../load_config.sh" -parse_args "$@" +# Pre-parse --remainder before parse_args (unknown args pass through to PARSED_ARGS) +REMAINDER_MODE=false +_filtered_args=() +for _arg in "$@"; do + if [[ "$_arg" == "--remainder" ]]; then + REMAINDER_MODE=true + else + _filtered_args+=("$_arg") + fi +done +unset _arg + +parse_args "${_filtered_args[@]}" +unset _filtered_args # ============================================================================================== # ━━━ Setup ━━━ @@ -54,15 +85,57 @@ fi detect_hosts -if [[ "${DAILY_CONTAINER_UPDATES:-true}" != "true" ]]; then - log "DAILY_CONTAINER_UPDATES=false — skipping container updates" - exit 0 -fi +# ============================================================================================== +# ━━━ Container Discovery ━━━ +# ============================================================================================== +if [[ "$REMAINDER_MODE" == true ]]; then + declare -A _exclude=() -if [[ ${#DAILY_RESTART_CONTAINERS[@]} -eq 0 ]]; then - warn "DAILY_RESTART_CONTAINERS is empty for $MY_ID — nothing to update" - warn "Check HOST${MY_ID#HOST}_DAILY_RESTART_CONTAINERS in master_host*.conf" - exit 0 + # Daily containers — updated by docker_update.sh normal mode + for _c in "${DAILY_RESTART_CONTAINERS[@]}"; do + [[ -n "$_c" ]] && _exclude["$_c"]=1 + done + + # Weekly sync-window containers (emby + critical-data) — updated inline by weekly_sync_maintenance.sh + _weekly_str="${PROFILE_CRITICAL_CONTAINER_NAMES[emby]:-} ${PROFILE_CRITICAL_CONTAINER_NAMES[critical-data]:-}" + read -r -a _weekly_arr <<< "$_weekly_str" + for _c in "${_weekly_arr[@]}"; do + [[ -n "$_c" ]] && _exclude["$_c"]=1 + done + unset _weekly_str _weekly_arr + + # Fallback coverage containers — owned by the remote server's update cycle. + # This server runs them during failover but should never update them independently. + # Updating them here risks version divergence: if remote's writeback after handback + # encounters data written by a newer version, it may not handle it correctly. + for _tier in 1 2 3 4; do + _tier_var="FALLBACK_${MY_ID}_COVERS_${REMOTE_ID}_TIER${_tier}" + eval "_tier_arr=(\"\${${_tier_var}[@]:-}\")" 2>/dev/null + for _c in "${_tier_arr[@]}"; do + [[ -n "$_c" ]] && _exclude["$_c"]=1 + done + done + unset _tier _tier_var _tier_arr _c + + mapfile -t _all_running < <(docker ps --format '{{.Names}}' | sort) + TARGET_CONTAINERS=() + for _c in "${_all_running[@]}"; do + [[ -z "${_exclude[$_c]+x}" ]] && TARGET_CONTAINERS+=("$_c") + done + unset _all_running _exclude _c +else + if [[ "${DAILY_CONTAINER_UPDATES:-true}" != "true" ]]; then + log "DAILY_CONTAINER_UPDATES=false — skipping container updates" + exit 0 + fi + + if [[ ${#DAILY_RESTART_CONTAINERS[@]} -eq 0 ]]; then + warn "DAILY_RESTART_CONTAINERS is empty for $MY_ID — nothing to update" + warn "Check HOST${MY_ID#HOST}_DAILY_RESTART_CONTAINERS in master_host*.conf" + exit 0 + fi + + TARGET_CONTAINERS=("${DAILY_RESTART_CONTAINERS[@]}") fi # ============================================================================================== @@ -72,8 +145,14 @@ if [[ "$SHOW_STATUS" == true ]]; then echo "" echo "━━━━━ $ICON_SUMMARY STATUS ━━━━━" echo "$ICON_HOST Identity: $MY_ID ($LOCAL_SERVER_NAME)" - echo "$ICON_CONTAINERS Containers: ${DAILY_RESTART_CONTAINERS[*]}" - echo "$ICON_GEAR Enabled: ${DAILY_CONTAINER_UPDATES:-true}" + echo "$ICON_GEAR Mode: $([[ "$REMAINDER_MODE" == true ]] && echo "remainder" || echo "normal (daily)")" + if [[ "$REMAINDER_MODE" == true ]]; then + echo "$ICON_CONTAINERS Containers: ${#TARGET_CONTAINERS[@]} running (excluding daily, weekly sync, and fallback)" + for _c in "${TARGET_CONTAINERS[@]}"; do echo " $_c"; done + else + echo "$ICON_CONTAINERS Containers: ${TARGET_CONTAINERS[*]}" + echo "$ICON_GEAR Enabled: ${DAILY_CONTAINER_UPDATES:-true}" + fi echo "$ICON_GEAR Dry Run: $DRY_RUN" echo "━━━━━━━━━━━━━━━━━━━━━━━" exit 0 @@ -81,12 +160,22 @@ fi [[ "$DRY_RUN" == true ]] && warn "DRY RUN — no images will be pulled" +if [[ ${#TARGET_CONTAINERS[@]} -eq 0 ]]; then + log "No containers to update" + exit 0 +fi + # ============================================================================================== # ━━━ Pull Updates ━━━ # ============================================================================================== echo "" -echo "━━━ $ICON_CONTAINERS Docker Update — $(date '+%Y-%m-%d %H:%M:%S') ━━━" -echo "$ICON_CONTAINERS Containers: ${DAILY_RESTART_CONTAINERS[*]}" +if [[ "$REMAINDER_MODE" == true ]]; then + echo "━━━ $ICON_CONTAINERS Docker Update (remainder) — $(date '+%Y-%m-%d %H:%M:%S') ━━━" + echo "$ICON_CONTAINERS Updating ${#TARGET_CONTAINERS[@]} container(s) (not in daily or weekly sync)" +else + echo "━━━ $ICON_CONTAINERS Docker Update — $(date '+%Y-%m-%d %H:%M:%S') ━━━" + echo "$ICON_CONTAINERS Containers: ${TARGET_CONTAINERS[*]}" +fi echo "" START=$(date +%s) @@ -95,7 +184,7 @@ UP_TO_DATE=() FAILED=() SKIPPED=() -for container in "${DAILY_RESTART_CONTAINERS[@]}"; do +for container in "${TARGET_CONTAINERS[@]}"; do [[ -z "$container" ]] && continue echo "━━━ $ICON_CONTAINERS $container ━━━" @@ -150,7 +239,11 @@ END=$(date +%s) # ============================================================================================== # ━━━ Summary ━━━ # ============================================================================================== -echo "━━━━━ $ICON_SUMMARY DOCKER UPDATE SUMMARY ━━━━━" +if [[ "$REMAINDER_MODE" == true ]]; then + echo "━━━━━ $ICON_SUMMARY DOCKER UPDATE (REMAINDER) SUMMARY ━━━━━" +else + echo "━━━━━ $ICON_SUMMARY DOCKER UPDATE SUMMARY ━━━━━" +fi echo "$ICON_HOST Identity: $MY_ID ($LOCAL_SERVER_NAME)" echo "$ICON_TIME Duration: $(format_duration $(( END - START )))" [[ ${#UPDATED[@]} -gt 0 ]] && echo "$ICON_DONE Updated: ${UPDATED[*]}" diff --git a/Docker_Essentials/docker_watchdog.sh b/Docker_Essentials/docker_watchdog.sh index 2723a9d..7fb0414 100755 --- a/Docker_Essentials/docker_watchdog.sh +++ b/Docker_Essentials/docker_watchdog.sh @@ -3,7 +3,7 @@ # ================================= Docker Watchdog ============================================ # ============================================================================================== # Two-tier self-healing container monitoring system. -# Runs continuously as a background process — started by array_start.sh at array start. +# Runs continuously as a background process — started by array_started.sh at array start. # Shuts down cleanly on SIGTERM/SIGINT when array stops. # # ── TIER 1 — STRICT MONITORING ──────────────────────────────────────────────────────────────── diff --git a/Monitors/continuous_scripts_status.sh b/Monitors/continuous_scripts_status.sh index fcad8f7..ff7d433 100644 --- a/Monitors/continuous_scripts_status.sh +++ b/Monitors/continuous_scripts_status.sh @@ -140,7 +140,7 @@ if is_script_running "system_watchdog"; then echo " ⏱️ Interval: ${SYSTEM_WATCHDOG_INTERVAL}s │ Heartbeat every: ${SYSTEM_WATCHDOG_HEARTBEAT_HOURS}hr" else echo " ❌ NOT RUNNING — system_watchdog.sh is not active" - echo " Start via: bash Orchestrators/array_start.sh" + echo " Start via: bash Orchestrators/array_started.sh" fi echo "" @@ -254,7 +254,7 @@ if is_script_running "docker_watchdog"; then echo " ⏱️ Interval: ${DOCKER_WATCHDOG_INTERVAL}s │ Heartbeat every: ${DOCKER_WATCHDOG_HEARTBEAT_HOURS}hr" else echo " ❌ NOT RUNNING — docker_watchdog.sh is not active" - echo " Start via: bash Orchestrators/array_start.sh" + echo " Start via: bash Orchestrators/array_started.sh" fi echo "" @@ -400,7 +400,7 @@ else echo " ⏸️ Disabled — FALLBACK_ENABLED=false in master.conf" else echo " ❌ NOT RUNNING — fallback.sh is not active" - echo " Start via: bash Orchestrators/array_start.sh" + echo " Start via: bash Orchestrators/array_started.sh" fi fi diff --git a/Orchestrators/array_start.sh b/Orchestrators/array_started.sh similarity index 96% rename from Orchestrators/array_start.sh rename to Orchestrators/array_started.sh index 37d6cc1..b6968d4 100644 --- a/Orchestrators/array_start.sh +++ b/Orchestrators/array_started.sh @@ -51,10 +51,10 @@ # ARRAY_START_SCRIPTS — ordered list of scripts to launch at array start # # ── USAGE ───────────────────────────────────────────────────────────────────────────────────── -# array_start.sh — normal launch (called by User Scripts at array start) -# array_start.sh --dry-run — show what would be launched without launching -# array_start.sh --status — show configured scripts and their current state -# array_start.sh --log — verbose output per script +# array_started.sh — normal launch (called by User Scripts at array start) +# array_started.sh --dry-run — show what would be launched without launching +# array_started.sh --status — show configured scripts and their current state +# array_started.sh --log — verbose output per script # ============================================================================================== SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/Orchestrators/array_stop.sh b/Orchestrators/array_stopping.sh similarity index 95% rename from Orchestrators/array_stop.sh rename to Orchestrators/array_stopping.sh index f26cea2..096866d 100755 --- a/Orchestrators/array_stop.sh +++ b/Orchestrators/array_stopping.sh @@ -17,7 +17,7 @@ # Containers last — apps should stay available as long as possible during shutdown prep. # # ── SEQUENTIAL vs BACKGROUND ───────────────────────────────────────────────────────────────── -# Unlike array_start.sh, all scripts run in the foreground. Each must complete (pass or fail) +# Unlike array_started.sh, all scripts run in the foreground. Each must complete (pass or fail) # before the next starts — a failed stop is noted but does not prevent remaining steps. # # ── SAFEGUARDS ──────────────────────────────────────────────────────────────────────────────── @@ -32,10 +32,10 @@ # ARRAY_STOP_SCRIPTS — ordered list of stop scripts to run # # ── USAGE ───────────────────────────────────────────────────────────────────────────────────── -# array_stop.sh — run full stop sequence -# array_stop.sh --dry-run — preview without stopping anything -# array_stop.sh --status — show configured scripts and exit -# array_stop.sh --log — verbose output +# array_stopping.sh — run full stop sequence +# array_stopping.sh --dry-run — preview without stopping anything +# array_stopping.sh --status — show configured scripts and exit +# array_stopping.sh --log — verbose output # ============================================================================================== SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/Orchestrators/weekly_sync_maintenance.sh b/Orchestrators/weekly_sync_maintenance.sh index 9a0db04..92c9f24 100644 --- a/Orchestrators/weekly_sync_maintenance.sh +++ b/Orchestrators/weekly_sync_maintenance.sh @@ -14,6 +14,7 @@ # 6. Start remote containers — correct order, delayed start respected # 7. Start local containers — correct order, delayed start respected # 8. WEEKLY_MAINTENANCE_SCRIPTS — weekly restarts etc. (docker_weekly_restart.sh) +# 9. docker_update.sh --remainder — update all containers not in daily or weekly sync window # # ── WHY WEEKLY NOT NIGHTLY FOR EMBY ────────────────────────────────────────────────────────── # Emby builds a warm image cache on HOST2 throughout the week. @@ -330,6 +331,31 @@ if [[ ${#WEEKLY_MAINTENANCE_SCRIPTS[@]} -gt 0 ]]; then done fi +# ============================================================================================== +# ━━━ Remainder Container Updates ━━━ +# ============================================================================================== +# Updates all running containers not already covered by daily or the weekly sync window. +# Runs last — weekly sync-window containers are back up before this pulls their peers. +echo "" +echo "━━━ $ICON_CONTAINERS Remainder Container Updates ━━━" + +DOCKER_UPDATE_SCRIPT="$SCRIPTS_ROOT/Docker_Essentials/docker_update.sh" +if [[ ! -f "$DOCKER_UPDATE_SCRIPT" ]]; then + warn "docker_update.sh not found — skipping remainder updates" + JOB_FAIL+=("docker_update.sh --remainder") +else + _remainder_args=("--remainder") + [[ "$DRY_RUN" == true ]] && _remainder_args+=("--dry-run") + if bash "$DOCKER_UPDATE_SCRIPT" "${_remainder_args[@]}"; then + log "Remainder updates complete ✅" + JOB_PASS+=("docker_update.sh --remainder") + else + warn "Remainder updates completed with errors" + JOB_FAIL+=("docker_update.sh --remainder") + fi + unset _remainder_args +fi + WINDOW_END=$(date +%s) # ============================================================================================== diff --git a/Tools/claude_startup.sh b/Tools/claude_startup.sh index daa8e5e..89f1c68 100755 --- a/Tools/claude_startup.sh +++ b/Tools/claude_startup.sh @@ -16,7 +16,7 @@ # # ── USAGE ───────────────────────────────────────────────────────────────────────────────────── # claude_startup.sh — set up persistent symlinks and launch Claude -# claude_startup.sh --setup — set up only, do not launch (for array_start.sh use) +# claude_startup.sh --setup — set up only, do not launch (for array_started.sh use) # ============================================================================================== PERSIST_DIR="/mnt/user/appdata/claude-code" @@ -89,7 +89,7 @@ _log "claude v$LATEST ready" echo "" -# ── Setup-only mode (used by array_start.sh or other callers) ───────────────────────────────── +# ── Setup-only mode (used by array_started.sh or other callers) ───────────────────────────────── if [[ "$LAUNCH" == false ]]; then _log "Setup complete — run 'claude' to start" echo "" diff --git a/git_pull_execute.sh b/git_pull_execute.sh index d1ba5a8..3e2513f 100755 --- a/git_pull_execute.sh +++ b/git_pull_execute.sh @@ -195,6 +195,7 @@ if [[ "$DRY_RUN" == true ]]; then SYNC_SUCCESS=true else mkdir -p "$TARGET_DIR" + git config --global --add safe.directory "$TARGET_DIR" cd "$TARGET_DIR" || { error "Cannot cd into $TARGET_DIR"; exit 1; } if [[ -d ".git" ]]; then diff --git a/master.conf b/master.conf index efb2d59..28ac553 100644 --- a/master.conf +++ b/master.conf @@ -36,7 +36,7 @@ # GIT / REPO Gitea repository and SSH settings # # ── ORCHESTRATORS ────────────────────────────────────────────────────────────────────────── -# ARRAY START Scripts launched at array start (array_start.sh) +# ARRAY START Scripts launched at array start (array_started.sh) # DAILY SYNC MAINTENANCE Job list + media shares (daily_sync_maintenance.sh) # WEEKLY SYNC MAINTENANCE Job list + sync shares + update toggles (weekly_sync_maintenance.sh) # CRITICAL SYNC MAINTENANCE 15-minute jobs + sync shares + partnership check (critical_sync_maintenance.sh) @@ -264,7 +264,7 @@ # No changes to orchestrator scripts needed when adding or removing jobs. # ━━━ Array Stop ━━━ -# Scripts run by array_stop.sh for a planned shutdown — stops everything cleanly in order. +# Scripts run by array_stopping.sh for a planned shutdown — stops everything cleanly in order. # Run sequentially (foreground) — each must complete before the next starts. # Order matters: user scripts first (prevents new ops), then data movement, then containers. ARRAY_STOP_SCRIPTS=( @@ -275,7 +275,7 @@ ) # ━━━ Array Start ━━━ -# Scripts launched by array_start.sh when the array comes online. +# Scripts launched by array_started.sh when the array comes online. # Launched in order — each as a background process. # One-shot scripts (ramdisk, syslog, fpm, inotify, network) run and exit naturally. # Continuous scripts (watchdogs, failover) run until array stops. @@ -561,7 +561,7 @@ # ── FALLBACK ────────────────────────────────────────────────────────────────────────────────── # ============================================================================================== # Mutual container failover between two unRAID servers. -# Each server runs Fallback/fallback.sh independently via array_start.sh. +# Each server runs Fallback/fallback.sh independently via array_started.sh. # All decisions based on two pings: remote reachable + internet reachable. # # States: NORMAL | FALLBACK | NO_INTERNET | DARK @@ -617,7 +617,7 @@ # ━━━ Docker Watchdog ━━━ # Continuous two-tier self-healing container monitoring. -# Started by array_start.sh — runs until array stops. +# Started by array_started.sh — runs until array stops. # Re-sources all three conf files each cycle — add/remove containers without restarting watchdog. # # Tier 1 — strict monitoring of explicitly configured containers: @@ -1118,7 +1118,7 @@ # ── SYSTEM WATCHDOG ─────────────────────────────────────────────────────────────────────────── # ============================================================================================== # Continuous system health monitoring — last line of defense before a crash. -# Started by array_start.sh — runs until array stops. +# Started by array_started.sh — runs until array stops. # Re-sources all three conf files each cycle — config changes take effect on next cycle. # # ── THREE-TIER RESPONSE SYSTEM ──────────────────────────────────────────────────────────────── diff --git a/unRAID_Essentials/system_watchdog.sh b/unRAID_Essentials/system_watchdog.sh index dde85a4..2992d39 100755 --- a/unRAID_Essentials/system_watchdog.sh +++ b/unRAID_Essentials/system_watchdog.sh @@ -3,7 +3,7 @@ # ================================= System Watchdog ============================================ # ============================================================================================== # Last line of defense — reboots the system cleanly if it is about to become unstable. -# Runs continuously as a background process — started by array_start.sh at array start. +# Runs continuously as a background process — started by array_started.sh at array start. # Works alongside docker_watchdog.sh which handles container-level healing first. # # ── THREE-TIER RESPONSE SYSTEM ──────────────────────────────────────────────────────────────── diff --git a/user_script_plug-in.sh b/user_script_plug-in.sh index e465739..8f17496 100644 --- a/user_script_plug-in.sh +++ b/user_script_plug-in.sh @@ -14,7 +14,7 @@ # # You do not need to schedule every script below. The orchestrators cover it all: # -# array_start.sh at array start — launches ALL startup scripts in order +# array_started.sh at array start — launches ALL startup scripts in order # transcode_management.sh every 3 min — cleanup then manager (order critical) # critical_sync_maintenance.sh every 15 min — auth + Emby dirty sync + partnership # intermediate_sync_maintenance.sh every 4 hours — arr library sync + artwork fetch @@ -61,7 +61,7 @@ # v1.7 — cert_monitor.sh added # v1.8 — Monitors/ folder with all monitoring scripts # v1.9 — transcode_management.sh added to Orchestrators/ -# v2.0 — Major restructure: array_start.sh single entry point, daily/weekly orchestrators, +# v2.0 — Major restructure: array_started.sh single entry point, daily/weekly orchestrators, # sunday_morning_coffee_report.sh, docker_network_connect combined, # inotify_tuning.sh, arr path translation, version checking, nuclear mode flags # v2.1 — critical_sync_maintenance.sh, Partnership/ folder, system_tuning_monitor.sh, @@ -112,7 +112,7 @@ # docker_watchdog.sh [continuous] two-tier container self-healing watchdog # fallback.sh [continuous] mutual fallback state machine # -# bash /mnt/user/appdata/unraid_scripts/Orchestrators/array_start.sh +# bash /mnt/user/appdata/unraid_scripts/Orchestrators/array_started.sh # ── TRANSCODE MANAGEMENT ────────────────────────────────────────────────────────────────────── @@ -416,7 +416,7 @@ # ────────────────────────────────────────────────────────────────────────────────────────────── # fallback.sh — [continuous] mutual fallback state machine -# Started by array_start.sh on both servers independently. +# Started by array_started.sh on both servers independently. # Every FALLBACK_CHECK_INTERVAL (120s) pings: remote Tailscale IP + 8.8.8.8 # States: NORMAL / FALLBACK / NO_INTERNET / DARK # FALLBACK: starts remote containers in tiers across 24 hours: @@ -510,7 +510,7 @@ # ────────────────────────────────────────────────────────────────────────────────────────────── # docker_watchdog.sh — [continuous] two-tier container self-healing monitor -# Started by array_start.sh. Every DOCKER_WATCHDOG_INTERVAL (900s = 15min). +# Started by array_started.sh. Every DOCKER_WATCHDOG_INTERVAL (900s = 15min). # # Tier 1 — explicit per-container (configured in master_host*.conf): # Memory hard limits: immediate restart when exceeded — no strikes, no waiting @@ -577,7 +577,7 @@ # bash /mnt/user/appdata/unraid_scripts/Docker_Essentials/docker_weekly_restart.sh # docker_network_connect.sh — ensure custom networks exist and containers are connected -# Called by array_start.sh. Idempotent — silent when everything is already correct. +# Called by array_started.sh. Idempotent — silent when everything is already correct. # Network missing → create (bridge driver, auto-assigned subnet) → notify. # Network creation should only happen after an unRAID update wiped networks — notify tells you. # Configured via HOST*_NETWORK_CONNECT_NETWORKS and HOST*_NETWORK_CONNECT_CONTAINERS. @@ -637,7 +637,7 @@ # ────────────────────────────────────────────────────────────────────────────────────────────── # system_watchdog.sh — [continuous] three-tier server stability last-resort watchdog -# Started by array_start.sh. Every SYSTEM_WATCHDOG_INTERVAL (300s = 5min). +# Started by array_started.sh. Every SYSTEM_WATCHDOG_INTERVAL (300s = 5min). # All 18 checks independently toggleable per host in master_host*.conf. # # Tier 1 CRITICAL — bypass ALL strikes, reboot immediately: @@ -664,7 +664,7 @@ # bash /mnt/user/appdata/unraid_scripts/unRAID_Essentials/system_watchdog.sh --dry-run # inotify_tuning.sh — raise Linux inotify kernel limits at array start -# Called by array_start.sh FIRST — must run before containers start (they inherit limits). +# Called by array_started.sh FIRST — must run before containers start (they inherit limits). # Settings reset on each reboot — script reapplies on every array start. Idempotent. # # Limits set: @@ -679,7 +679,7 @@ # bash /mnt/user/appdata/unraid_scripts/unRAID_Essentials/inotify_tuning.sh # php_fpm_max_children.sh — set PHP-FPM pm.max_children at array start -# Called by array_start.sh. Default is 4-8 workers — inadequate for a busy multi-user server. +# Called by array_started.sh. Default is 4-8 workers — inadequate for a busy multi-user server. # Sets PHP_MAX_CHILDREN (250). 250 × ~2MB idle = ~500MB. Acceptable on 64GB+. # Symptom of saturation: WebGUI slow, settings saves hang, container UI starts timeout. # Resets on each reboot — reapplied at array start. Idempotent: silent when already correct. @@ -688,7 +688,7 @@ # bash /mnt/user/appdata/unraid_scripts/unRAID_Essentials/php_fpm_max_children.sh # docker_syslog_filter.sh — suppress Docker veth/docker0 interface log noise -# Called by array_start.sh before containers start. Creates rsyslog drop rule. +# Called by array_started.sh before containers start. Creates rsyslog drop rule. # Without this: 50+ containers at array start = 200-400 lines of kernel veth messages. # Real events (mount failures, permission errors) are invisible in that noise. # Idempotent: compares expected filter content exactly — only writes when changed. @@ -895,7 +895,7 @@ # ────────────────────────────────────────────────────────────────────────────────────────────── # ramdisk_setup.sh — create tmpfs ramdisk and transcode symlink at array start -# Called by array_start.sh. MUST run before Emby starts. +# Called by array_started.sh. MUST run before Emby starts. # Idempotent: if ramdisk already mounted → report status and exit cleanly, do not remount. # # Creates: @@ -1198,7 +1198,7 @@ # ============================================================================================== # # At Startup of Array: -# Orchestrators/array_start.sh (single entry — handles everything) +# Orchestrators/array_started.sh (single entry — handles everything) # # */3 * * * * every 3 minutes: # Orchestrators/transcode_management.sh