Add full banner headers to all scripts across the codebase

Every script now has the established header format: PURPOSE with ─────── separator,
OPERATIONAL MODEL, DESIGN PRINCIPLES, OPERATIONAL SAFEGUARDS, CONFIGURATION, and
RUNTIME MODES — structured with full ====== banner sections throughout.

Orchestrators converted from compact ── inline format to full banners. Stale
emby-fallback and dirty sync references removed from Plugin/user_script_plug-in.sh.
This commit is contained in:
Gmer4Lfe
2026-06-26 18:50:05 -04:00
parent 1003bee72a
commit f92ee4064b
51 changed files with 1822 additions and 563 deletions
+8 -3
View File
@@ -26,15 +26,20 @@
# rebuild cron when the array mounts. Scripts stay on appdata (git clone).
#
# ==============================================================================================
# USAGE
# RUNTIME MODES
# ==============================================================================================
# ./build.sh # version = today's date (YYYY.MM.DD)
# ./build.sh 2026.09.01 # explicit version
#
# ./build.sh
# Build with today's date as version (YYYY.MM.DD).
#
# ./build.sh 2026.09.01
# Build with an explicit version string.
#
# After building: commit Plugin/dist/<txz> + the updated .plg, then attach the
# .txz to a GitHub release tagged <version> so the .plg URL resolves for
# downloaders. (The .plg also works offline if the .txz is already cached on
# flash with a matching SHA256.)
#
# ==============================================================================================
set -euo pipefail
+14 -4
View File
@@ -3,15 +3,25 @@
# ====================== Partnership — Unraid Container Adapter ================================
# ==============================================================================================
#
# Sourced by partnership_onboard.sh and partnership_offboard.sh via:
# PURPOSE
# ─────────────────────────────────────────────────────────────────────────────
# Platform adapter providing container deploy/cleanup functions for the Unraid
# partnership system. Sourced (not executed) by partnership_onboard.sh and
# partnership_offboard.sh via:
# source "$SCRIPTS_ROOT/Plugin/$PLATFORM/Partnership/containers.sh"
#
# Provides container deploy/cleanup functions specific to the Unraid platform:
# - Docker container deployment from Unraid CA XML templates
# ==============================================================================================
# OPERATIONAL MODEL
# ==============================================================================================
#
# Functions use variables from the calling script's scope (sourced, not exec'd):
# Functions operate on variables from the calling script's scope:
# MIRROR, MIRROR_IP, MIRROR_SSH_KEY, SSH_TIMEOUT, DRY_RUN, SCRIPTS_ROOT
#
# Capabilities:
# - Docker container deployment from Unraid CA XML templates
# - Remote GPU type detection (cached per session — one SSH call per onboard)
# - Deployed stack tracking via _STACK_DEPLOYED / _STACK_FAILED counters
#
# ==============================================================================================
TEMPLATES_DIR="/boot/config/plugins/dockerMan/templates-user"
@@ -14,6 +14,29 @@
# page always reflects the live key value.
#
# ==============================================================================================
# DESIGN PRINCIPLES
# ==============================================================================================
#
# Self-Healing at Boot
# The unraid-api registry is ephemeral — OS updates and service restarts clear
# it without warning. Running at every array start means the key is always
# present after boot without any manual intervention.
#
# Conf Stays Current
# HOST*_UNRAID_API_KEY in the local host conf is updated after every renewal.
# The partnership page reads the conf — it always reflects the live key value
# without a separate sync step.
#
# ==============================================================================================
# OPERATIONAL SAFEGUARDS
# ==============================================================================================
#
# acquire_lock — prevents concurrent renewal attempts at boot
# detect_hosts() — sets MY_ID to derive the correct conf var name
# Conf file check — aborts before any writes if the host conf is missing
# dry-run mode — shows what would happen without touching anything
#
# ==============================================================================================
# RUNTIME MODES
# ==============================================================================================
#
+23 -1
View File
@@ -15,7 +15,29 @@
# Accepts --host=HOST2 to refresh a single host (used by the UI refresh button).
#
# ==============================================================================================
# USAGE
# DESIGN PRINCIPLES
# ==============================================================================================
#
# Cache-First, Never Live on Page Load
# Remote arr APIs have non-trivial latency — calling them on every page view
# would make the arrs page slow and fragile. Writing to /tmp/vv_cache/ on a
# 2-hour schedule decouples page load time from network availability.
#
# Single-Host Refresh for UI
# The UI refresh button passes --host=HOSTN to update one host without waiting
# for the full 2-hour cycle. Keeps the cache fresh when a user requests it.
#
# ==============================================================================================
# OPERATIONAL SAFEGUARDS
# ==============================================================================================
#
# acquire_lock — prevents concurrent cache write runs
# detect_hosts() — MY_ID and partner host list
# SSH reachability — skips a host cleanly if it cannot be reached
# /tmp/vv_cache/ — auto-created if missing; cleared on reboot (intentional)
#
# ==============================================================================================
# RUNTIME MODES
# ==============================================================================================
#
# remote_arr_cache_writer.sh — refresh all remote hosts
+23
View File
@@ -24,6 +24,29 @@
# varaverk.cron rebuilt via PHP (job paths regenerated for new SCRIPTS_DIR)
#
# ==============================================================================================
# DESIGN PRINCIPLES
# ==============================================================================================
#
# Atomic Path Transition
# All four files (varaverk.cfg, master.conf, host*.conf, varaverk.cron) are
# updated in a single pass. A partial migration would leave cron entries
# pointing at the wrong SCRIPTS_DIR — all or nothing.
#
# PHP Rebuilds Cron
# Job paths in varaverk.cron are derived from SCRIPTS_DIR. Rather than
# text-substituting the cron file, the script regenerates it via PHP using
# the new SCRIPTS_DIR as the source of truth.
#
# ==============================================================================================
# OPERATIONAL SAFEGUARDS
# ==============================================================================================
#
# acquire_lock — prevents concurrent migration attempts
# dry-run mode — shows all changes without touching any file
# --status mode — reports current mode without requiring a target
# --to= required — refuses to run without an explicit target mode
#
# ==============================================================================================
# RUNTIME MODES
# ==============================================================================================
#
+41 -31
View File
@@ -2,42 +2,52 @@
# ==============================================================================================
# ================================= Unraid Platform Adapter ====================================
# ==============================================================================================
# Sourced by load_config.sh when PLATFORM=unraid.
# Provides the platform_*() API — bash scripts call these instead of OS-specific commands.
#
# ── API CONTRACT ──────────────────────────────────────────────────────────────────────────────
# Every function returns 0 on success / 1 on failure unless noted.
# Functions that produce output write to stdout; callers capture with $().
# No function calls exit — callers decide what failure means for their flow.
# PURPOSE
# ─────────────────────────────────────────────────────────────────────────────
# Platform abstraction layer for Unraid. Sourced by load_config.sh when
# PLATFORM=unraid. Scripts call platform_*() functions instead of OS-specific
# commands — the adapter isolates all OS-dependent logic in one place.
#
# ── ADDING A PLATFORM ─────────────────────────────────────────────────────────────────────────
# Create Plugin/truenas/adapter.sh (or ubuntu/adapter.sh) implementing the same function names.
# load_config.sh sources Plugin/$PLATFORM/adapter.sh — no other changes needed.
# ==============================================================================================
# OPERATIONAL MODEL
# ==============================================================================================
#
# ── FUNCTIONS ─────────────────────────────────────────────────────────────────────────────────
# platform_require_cmd — verify a platform command exists and is executable
# platform_storage_healthy — array mounted and shfs active on /mnt/user
# platform_disk_states_path — path to the platform disk state file
# platform_get_disk_states — raw disk state content from platform
# platform_get_temp_thresholds — disk warn/crit °C from platform config
# API CONTRACT
# Every function returns 0 on success / 1 on failure unless noted.
# Functions that produce output write to stdout; callers capture with $().
# No function calls exit — callers decide what failure means for their flow.
#
# ADDING A PLATFORM
# Create Plugin/truenas/adapter.sh (or ubuntu/adapter.sh) implementing the
# same function names. load_config.sh sources Plugin/$PLATFORM/adapter.sh —
# no other changes needed anywhere in the codebase.
#
# FUNCTIONS
# platform_require_cmd — verify a platform command exists and is executable
# platform_storage_healthy — array mounted and shfs active on /mnt/user
# platform_disk_states_path — path to the platform disk state file
# platform_get_disk_states — raw disk state content from platform
# platform_get_temp_thresholds — disk warn/crit °C from platform config
# platform_is_maintenance_running — parity check/sync in progress
# platform_is_service_enabled — docker or libvirt enabled in boot config
# platform_restart_service — restart a named service via rc.d
# platform_stop_service — stop a named service via rc.d
# platform_is_service_running — check if a named service process is alive
# platform_is_mover_running — unRAID mover process check
# platform_stop_user_scripts — kill all user.scripts background processes
# platform_send_os_notification — native unRAID notify (dynamix)
# platform_storage_path — root path for user shares/storage (e.g. /mnt/user)
# platform_webui_install_path — where the platform serves the WebGUI plugin files from
# platform_is_service_enabled — docker or libvirt enabled in boot config
# platform_restart_service — restart a named service via rc.d
# platform_stop_service — stop a named service via rc.d
# platform_is_service_running — check if a named service process is alive
# platform_is_mover_running — unRAID mover process check
# platform_stop_user_scripts — kill all user.scripts background processes
# platform_send_os_notification — native unRAID notify (dynamix)
# platform_storage_path — root path for user shares/storage (e.g. /mnt/user)
# platform_webui_install_path — where the platform serves the WebGUI plugin files from
# platform_scripts_dir_probe_cmd — shell command to run on a remote to discover its SCRIPTS_DIR
# platform_get_templates_dir — path to Unraid CA docker templates-user directory
# platform_setup_db_path — path to the persistent Varaverk setup/wizard state database
# platform_get_os_version — local OS version string (e.g. "7.2.3")
# platform_os_version_probe_cmd — shell command to run on a remote to retrieve its OS version
# platform_rebuild_container — rebuild a container from its stored XML template
# platform_push_conf — push master.conf to all listed hosts via WebGUI PHP
# platform_push_setup_state — push wizard setup state to WebGUI PHP
# platform_get_templates_dir — path to Unraid CA docker templates-user directory
# platform_setup_db_path — path to the persistent Varaverk setup/wizard state database
# platform_get_os_version — local OS version string (e.g. "7.2.3")
# platform_os_version_probe_cmd — shell command to run on a remote to retrieve its OS version
# platform_rebuild_container — rebuild a container from its stored XML template
# platform_push_conf — push master.conf to all listed hosts via WebGUI PHP
# platform_push_setup_state — push wizard setup state to WebGUI PHP
#
# ==============================================================================================
# ──────────────────────────────────────────────────────────────────────────────────────────────
+68 -10
View File
@@ -1,19 +1,77 @@
#!/bin/bash
# Varaverk job runner — wraps script execution with JSON status tracking.
# Called by /etc/cron.d/varaverk for every scheduled job.
# ==============================================================================================
# ============================= Job Runner =====================================================
# ==============================================================================================
#
# Usage: bash run_job.sh <job_id> <script_path> [flags...]
# PURPOSE
# ─────────────────────────────────────────────────────────────────────────────
# Wraps every scheduled script execution with JSON status tracking and log
# management. Called by /etc/cron.d/varaverk for every scheduled job. The PHP
# dashboard polls the JSON files to show live job status without running scripts.
#
# Flags consumed by run_job.sh (stripped before passing to script):
# --manual — marks a UI-triggered run; writes a sentinel on completion so
# the next cron fire is suppressed if it falls within the job's
# own cron interval (prevents double-firing after manual run).
# ==============================================================================================
# OPERATIONAL MODEL
# ==============================================================================================
#
# Writes: /var/log/varaverk/<id>.json — status, timestamps, exit code, pid
# /var/log/varaverk/<id>.log — appended per run, trimmed to LOG_MAX_LINES
# /var/log/varaverk/<id>.manual_ts — sentinel: epoch of last manual completion
# Invocation: bash run_job.sh <job_id> <script_path> [flags...]
#
# Writes three files per job to /var/log/varaverk/:
# <id>.json — status, timestamps, exit code, pid (polled by WebGUI)
# <id>.log — appended per run, trimmed to LOG_MAX_LINES lines
# <id>.manual_ts — sentinel: epoch of last manual completion (interval suppression)
#
# Status values: running → ok (exit 0) | warn (exit 1) | error (exit 2+)
#
# MANUAL FLAG
# --manual marks a UI-triggered run. On completion, writes a manual_ts sentinel.
# The next cron fire reads the sentinel and suppresses itself if the elapsed time
# is within the job's own cron interval — prevents double-firing after a manual run.
# Static schedules (e.g. "30 2 * * 0") are never suppressed — only */N intervals.
#
# ==============================================================================================
# DESIGN PRINCIPLES
# ==============================================================================================
#
# Suppress Double-Fire
# When a user triggers a job from the UI, the next cron fire within the job's
# own interval is skipped. A 30-minute cron job triggered at HH:14 won't fire
# again at HH:30 — it waits for HH:44. Static schedules are never suppressed.
#
# Status as Ground Truth
# The JSON file is overwritten atomically on every state change (start → end).
# The WebGUI polls it directly — no additional IPC or database needed.
#
# Log Trim on Every Write
# The log file is trimmed to LOG_MAX_LINES after every run. Never grows
# unbounded regardless of how long the server runs or how often the job fires.
#
# ==============================================================================================
# OPERATIONAL SAFEGUARDS
# ==============================================================================================
#
# --manual stripped — flag consumed here, never passed to the wrapped script
# mkdir -p — log dir created if missing before any write
# Log trim — tail -n LOG_MAX_LINES via tmp file + mv (atomic)
# Sentinel cleanup — manual_ts removed after it's used or expired
#
# ==============================================================================================
# CONFIGURATION
# ==============================================================================================
#
# LOG_DIR /var/log/varaverk (hardcoded — tmpfs on Unraid, cleared on reboot)
# LOG_MAX_LINES 1000 (hardcoded — trim threshold per job log)
#
# ==============================================================================================
# RUNTIME MODES
# ==============================================================================================
#
# bash run_job.sh <job_id> <script_path> [script_flags...]
# Normal cron-triggered run.
#
# bash run_job.sh <job_id> <script_path> --manual [script_flags...]
# UI-triggered run. Suppresses next cron fire within the job's interval.
#
# ==============================================================================================
JOB_ID="$1"
SCRIPT="$2"
+28 -42
View File
@@ -3,30 +3,36 @@
# ============================= USER SCRIPTS MASTER TEMPLATE ===================================
# ==============================================================================================
#
# Paste this file into a User Script entry. Uncomment ONE script block and set the schedule.
# Every script in the ecosystem is listed here — from the orchestrators that run it all,
# down to the individual scripts you can run standalone for specific tasks.
# PURPOSE
# ─────────────────────────────────────────────────────────────────────────────
# Paste this file into a User Script entry. Uncomment ONE script block and set
# the schedule. Every script in the ecosystem is listed here — from the
# orchestrators that run it all, down to the individual scripts you can run
# standalone for specific tasks.
#
# ==============================================================================================
# OPERATIONAL MODEL
# ==============================================================================================
#
# ── HOW THIS ECOSYSTEM WORKS ──────────────────────────────────────────────────────────────────
# When used as intended, only a handful of orchestrators need to be scheduled.
# The orchestrators handle everything else in the correct order — they call child scripts,
# manage timing and dependencies, track pass/fail, and send one notification per window.
# The orchestrators handle everything else in the correct order — they call child
# scripts, manage timing and dependencies, track pass/fail, and send one
# notification per window.
#
# You do not need to schedule every script below. The orchestrators cover it all:
#
# array_started.sh at array start — launches ALL startup scripts in order
# array_stopping.sh at array stop — graceful shutdown sequence
# watchdog_orchestrator.sh every 15 min — resource + docker + system watchdog
# transcode_management.sh every 7 min — cleanup then manager (order critical)
# critical_sync_maintenance.sh every 30 min — auth + Emby dirty sync + partnership
# intermediate_sync_maintenance.sh every 4 hours — arr library sync + artwork fetch
# daily_sync_maintenance.sh 1am daily — git + rsync + media + restart
# rsync.sh --profile=emby-fallback every 30 min — Emby watch state dirty sync
# weekly_sync_maintenance.sh 2:30am Sunday — clean sync + image updates
# sunday_morning_coffee_report.sh 7am Sunday — full weekly digest
# weekly_health_digest.sh 8am daily — profile-controlled health notification
# system_tuning_monitor.sh every 6 hours — inotify + php-fpm trend tracking
# monthly_maintenance.sh 15th monthly — uptime-gated heavy tasks (ZFS scrub, SMART long test)
# array_stopping.sh at array stop — graceful shutdown sequence
# watchdog_orchestrator.sh every 15 min — resource + docker + system watchdog
# transcode_management.sh every 7 min — cleanup then manager (order critical)
# critical_sync_maintenance.sh every 30 min — auth + play_state_sync + partnership
# intermediate_sync_maintenance.sh every 4 hours — arr library sync + artwork fetch
# daily_sync_maintenance.sh 1am daily — git + rsync + media + restart
# weekly_sync_maintenance.sh 2:30am Sunday — clean sync + image updates
# sunday_morning_coffee_report.sh 7am Sunday — full weekly digest
# weekly_health_digest.sh 8am daily — profile-controlled health notification
# system_tuning_monitor.sh every 6 hours — inotify + php-fpm trend tracking
# monthly_maintenance.sh 15th monthly — uptime-gated heavy tasks (ZFS scrub, SMART long test)
#
# ── INDIVIDUAL SCRIPTS ────────────────────────────────────────────────────────────────────────
# Every child script is also listed below, individually.
@@ -227,10 +233,9 @@
# Auth stack to HOST2: NPM proxy rules, TLS certs, LLDAP user accounts,
# Authelia config and policies. HOST2 auth always within 30min of HOST1.
#
# rsync Emby dirty sync:
# Watch states, user activity, library delta — Emby stays running on both sides.
# WAL/SHM files excluded (unsafe to copy mid-write). HOST2 Emby restarts after sync
# to pick up config changes.
# play_state_sync:
# API-based watch state and resume position sync. Runs as a CRITICAL_MAINTENANCE_SCRIPT
# in this window. No rsync — uses Emby API to sync per-user watch state directly.
#
# partnership --check:
# Reads remote state file. Increments offline counter on failed sync.
@@ -295,22 +300,6 @@
# bash /boot/config/plugins/varaverk/Orchestrators/daily_sync_maintenance.sh
# ── RSYNC EMBY FALLBACK ───────────────────────────────────────────────────────────────────────
# Schedule: */30 * * * * (every 30 minutes)
# Background: YES
#
# Keeps HOST2 Emby within 30 minutes of HOST1 on watch states and library changes.
# Direct rsync.sh call (not an orchestrator). Emby stays running on both sides.
#
# Syncs: users.db, library.db, authentication.db, config/
# Skips: *.wal *.shm (unsafe mid-write), transcodes/, logs/, cache/ (volatile/local only)
# After: HOST2 Emby restarts to pick up any config changes from the sync.
# Result: if HOST1 fails, users resume from at most 30 minutes stale.
#
# bash /boot/config/plugins/varaverk/Rsync/rsync.sh \
# /mnt/user/Media_Server/Emby --profile=emby-fallback
# ── WEEKLY SYNC MAINTENANCE ───────────────────────────────────────────────────────────────────
# Schedule: 30 2 * * 0 (Sunday 2:30am — 4.5 hours before coffee report)
# Background: YES
@@ -524,7 +513,6 @@
# important-data Postgres-NextCloud + delayed: NextCloud
# arrs_stack Sonarr, Radarr, Lidarr, Prowlarr, Bazarr, Pinchflat
# emby Emby both sides (weekly clean sync — both instances stopped)
# emby-fallback nothing stopped (Emby stays running — dirty sync, WAL/SHM excluded)
# [no profile] no containers stopped (media shares, plain data)
#
# bash /boot/config/plugins/varaverk/Rsync/rsync.sh \
@@ -536,8 +524,6 @@
# bash /boot/config/plugins/varaverk/Rsync/rsync.sh \
# /mnt/user/Media_Server/Emby --profile=emby
# bash /boot/config/plugins/varaverk/Rsync/rsync.sh \
# /mnt/user/Media_Server/Emby --profile=emby-fallback
# bash /boot/config/plugins/varaverk/Rsync/rsync.sh \
# /mnt/user/appdata-Fallback/Gmer4Lfe
# bash /boot/config/plugins/varaverk/Rsync/rsync.sh /mnt/user/Movies
# bash /boot/config/plugins/varaverk/Rsync/rsync.sh /mnt/user/Tv_Shows
@@ -1422,7 +1408,7 @@
#
# */30 * * * * every 30 minutes:
# Orchestrators/critical_sync_maintenance.sh
# Rsync/rsync.sh /mnt/user/Media_Server/Emby --profile=emby-fallback
# └─ Critical-Data rsync → play_state_sync → partnership --check
#
# 0 */4 * * * every 4 hours:
# Orchestrators/intermediate_sync_maintenance.sh