Standardize orchestrator child-script execution and logging

Every orchestrator invoked its children differently — four near-duplicate
run_job() copies, a differently-shaped run_watchdog(), or plain inline bash
calls, each with its own take on path resolution, pass/fail naming, and
dry-run threading. Extracted one shared run_orch_child() into common.sh so
there's a single place to fix or extend this behavior going forward.

Along the way: watchdog_orchestrator.sh and monthly_maintenance.sh were
checking $VERBOSE, a variable nothing in the codebase ever assigns, so --log
silently did nothing beyond basic logging on those two. Fixed to
$ENABLE_LOGGING. watchdog_orchestrator.sh and array_started.sh had no
trailing exit, so their exit codes reflected whatever the last command
happened to return rather than actual success/failure. transcode_management.sh
had no failure notification and no summary at all. Also made
transcode_management.sh's two-script pipeline config-driven
(TRANSCODE_MANAGEMENT_SCRIPTS in master.conf) instead of hardcoded, for room
to extend it later without editing the orchestrator itself.
This commit is contained in:
Gmer4Lfe
2026-07-03 10:57:52 -04:00
parent 6fd22ae4ee
commit 2a062e5140
14 changed files with 303 additions and 370 deletions
+21 -22
View File
@@ -942,41 +942,40 @@ array_started.sh
## ━━━ ADDING A NEW ORCHESTRATOR ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
If you find yourself running 3+ related scripts on the same schedule, wrap them
in a new orchestrator. Model directly on `media_management.sh` which has the
in a new orchestrator. Model directly on `daily_sync_maintenance.sh` which has the
complete pattern — dry-run passthrough, status display, pass/fail tracking, summary.
Child-script execution goes through the shared `run_orch_child()` helper in
`common.sh` — never hand-roll a per-file `run_job()` loop. It resolves the entry
against `$ECOSYSTEM_ROOT`, threads `--dry-run`/`--log` from `$DRY_RUN`/`$ENABLE_LOGGING`
automatically (never `$VERBOSE` — nothing in this codebase assigns it), and tracks
into `JOB_PASS`/`JOB_FAIL` arrays the caller declares.
```bash
# Minimal skeleton — the full pattern in its simplest form:
#!/bin/bash
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../load_config.sh"
ECOSYSTEM_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
source "$ECOSYSTEM_ROOT/load_config.sh"
parse_args "$@"
SCRIPTS_ROOT="$SCRIPT_DIR/.."
PASS=()
FAIL=()
JOB_PASS=()
JOB_FAIL=()
# Read job list from master.conf — never hardcode jobs in the orchestrator
for script_entry in "${MY_MAINTENANCE_JOBS[@]:-}"; do
[[ -z "$script_entry" ]] && continue
read -r -a parts <<< "$script_entry"
script_path="$SCRIPTS_ROOT/${parts[0]}"
script_name=$(basename "${parts[0]}")
extra_args=("${parts[@]:1}")
[[ "$DRY_RUN" == true ]] && extra_args+=("--dry-run")
if bash "$script_path" "${extra_args[@]}"; then
PASS+=("$script_name")
else
FAIL+=("$script_name")
fi
run_orch_child "$script_entry"
done
# One summary — one notification
echo "Passed: ${#PASS[@]} Failed: ${#FAIL[@]}"
[[ ${#FAIL[@]} -gt 0 ]] && \
notify "My maintenance failed on $(hostname) ($MY_ID) — ${FAIL[*]}" \
# One summary — one notification. On a frequent (sub-daily) cadence, keep the
# healthy path to a single line and reserve the full breakdown for failure/--log —
# see watchdog_orchestrator.sh or transcode_management.sh for that split.
echo "Passed: ${#JOB_PASS[@]} Failed: ${#JOB_FAIL[@]}"
if [[ ${#JOB_FAIL[@]} -gt 0 ]]; then
notify "My maintenance failed on $(hostname) ($MY_ID) — ${JOB_FAIL[*]}" \
"My Orchestrator" "warning"
exit 1
fi
exit 0
```