Compare commits
5
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Gmer4Lfe
|
2a062e5140 | ||
|
Gmer4Lfe
|
6fd22ae4ee | ||
|
Gmer4Lfe
|
93aa134aaa | ||
|
Gmer4Lfe
|
14ecf0c08a | ||
|
Gmer4Lfe
|
5d5b60a8ed |
@@ -4,7 +4,6 @@
|
||||
Configurations/host*.conf
|
||||
Configurations/master.conf
|
||||
Configurations/*.bak
|
||||
.claude
|
||||
.vscode
|
||||
|
||||
# ── Runtime state, data, logs ─────────────────────────────────────────────────
|
||||
@@ -28,9 +27,6 @@ varaverk-*.txz
|
||||
# .txz packages are attached to GitHub releases, not committed to the repo.
|
||||
Plugin/dist/
|
||||
|
||||
# ── Claude Code installation (lives alongside repo on flash, not source) ──────
|
||||
claude-bin/
|
||||
claude-data/
|
||||
|
||||
# ── OS / editor ───────────────────────────────────────────────────────────────
|
||||
.DS_Store
|
||||
|
||||
@@ -1,162 +0,0 @@
|
||||
# Varaverk — Claude Code Context
|
||||
|
||||
## Working Rules (read first)
|
||||
|
||||
- **Dev workspace** — `/mnt/cloud-storage/Important Shit/Git/Development/Varaverk/`. All editing happens here.
|
||||
- **Prod** — `/boot/config/plugins/varaverk/`. Never edited directly. Only updated via `git pull` (daily orchestrator or the pull button in the UI).
|
||||
- **No Co-Authored-By** in commit messages unless explicitly asked.
|
||||
- **No comments** unless the WHY is genuinely non-obvious.
|
||||
|
||||
### Current workflow: dev/prod split
|
||||
|
||||
Edit in dev, push to Gitea, pull prod when ready (daily orchestrator or manual UI trigger).
|
||||
Push is the only bridge — no deploy hooks, no rsync-on-save, no direct path references between dev and prod.
|
||||
|
||||
`plugin_setup.sh` stays pointing at the prod path. Dev never touches `/boot/` directly.
|
||||
|
||||
### Hard limits — do not cross these
|
||||
|
||||
- **Never create or modify `.claude/settings.json`** in this repo. No workspace hooks, ever. The stale hook that existed here previously fired `Deployment/deploy.sh` (now deleted) on every file edit and caused unintended deploys. If you think a hook would help, ask first.
|
||||
- **Never change `HOST1_STORAGE_MODE_INTERNAL`** in `host1.conf`. Claude data belongs in `/mnt/user/appdata/claude-code/` — not inside this repo.
|
||||
- **Never move files between `Configurations/` and `Deployment/`** without explicit instruction. `Configurations/` = live runtime confs (gitignored). `Deployment/` = templates and setup tooling (tracked).
|
||||
|
||||
---
|
||||
|
||||
## Project: What Varaverk Is
|
||||
|
||||
Self-healing, self-maintaining, mutually-redundant two-server Unraid home media ecosystem.
|
||||
One codebase runs on both servers. No primary/standby — both run independently and cover each other.
|
||||
|
||||
**HOST1 — unRAID-Gmer4Lfe** (`gmer4lfe@gmail.com`)
|
||||
- Hardware: Threadripper 1950X, 128 GB RAM, ZFS cache pools
|
||||
- Domain: Gmer4Lfe.com
|
||||
- Runs: full arr stack (Sonarr/Radarr/Lidarr), auth stack (source of truth), Emby primary
|
||||
|
||||
**HOST2 — unRAID-Jayred365**
|
||||
- Hardware: Intel i5 10th gen, 64 GB RAM
|
||||
- Domain: Gmer4Lfe.us
|
||||
- Status: being rebuilt — most host2.conf sections scaffolded, not yet fully online
|
||||
|
||||
Networking between hosts: Tailscale mesh. No hardcoded IPs — hostnames resolve via Tailscale.
|
||||
|
||||
---
|
||||
|
||||
## Configuration System (three-file model)
|
||||
|
||||
Every script sources all three at startup:
|
||||
|
||||
```
|
||||
master.conf ← shared: thresholds, toggles, profiles, orchestrator job lists
|
||||
host1.conf ← HOST1 credentials, shares, container names, keys
|
||||
host2.conf ← HOST2 credentials, shares, container names, keys
|
||||
```
|
||||
|
||||
Sparse checkout (git) means each server only pulls its own `host*.conf`.
|
||||
HOST1 never sees HOST2 credentials and vice versa.
|
||||
|
||||
**Rule:** thresholds/toggles → `master.conf`; credentials/paths/container names → `host*.conf`.
|
||||
|
||||
`detect_hosts()` in `common.sh` matches `$(hostname)` against `HOST1`/`HOST2` in `master.conf`
|
||||
and sets `MY_ID` / `REMOTE_ID` for the rest of the script.
|
||||
|
||||
---
|
||||
|
||||
## Platform Adapter Layer
|
||||
|
||||
`Plugin/unraid/adapter.sh` isolates all OS-specific calls.
|
||||
Scripts never branch on OS directly — always call adapter functions.
|
||||
This is intentional architecture — don't bypass it.
|
||||
|
||||
---
|
||||
|
||||
## Key Paths
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `master.conf` | Shared config — all thresholds, toggles, profiles |
|
||||
| `host1.conf` / `host2.conf` | Per-host credentials, shares, container lists |
|
||||
| `common.sh` | Shared functions — `detect_hosts()`, `log()`, `notify()`, etc. |
|
||||
| `load_config.sh` | Sources all three conf files + common.sh |
|
||||
| `State_Files/` | Runtime state (watchdogs, fallback, transcode) — survives reboots |
|
||||
| `data/` | Historical logs and stats |
|
||||
| `Plugin/unraid/` | Unraid WebGUI plugin (PHP pages, API endpoints, adapter) |
|
||||
| `Orchestrators/` | Top-level schedulers (array_started, daily, weekly, watchdog) |
|
||||
| `Watchdogs/` | docker_watchdog, system_watchdog, resource_watchdog, stability |
|
||||
| `Fallback/` | Mutual container failover logic |
|
||||
| `Rsync/` | rsync.sh + profile system |
|
||||
| `Media/` | Arr cleanup, discovery, permissions, play state sync |
|
||||
| `Tools/` | Manual one-off tools including `claude_startup.sh` |
|
||||
|
||||
---
|
||||
|
||||
## Orchestrator Schedule
|
||||
|
||||
| When | What |
|
||||
|------|------|
|
||||
| Array start | `Orchestrators/array_started.sh` → runs `ARRAY_START_SCRIPTS` |
|
||||
| Every minute | `watchdog_orchestrator.sh` → resource → docker → system → stability watchdogs |
|
||||
| Every 30 min | `critical_sync_maintenance.sh` → downloaders_reset, play_state_sync, critical rsync |
|
||||
| Every 4 hours | `intermediate_sync_maintenance.sh` → arr_sync, arrs_failed_stalled_recovery |
|
||||
| Daily 1am | `daily_sync_maintenance.sh` → git pull, permissions, cleaners, arr cleanup, docker updates |
|
||||
| Sunday 2:30am | `weekly_sync_maintenance.sh` → full Emby + Critical-Data sync, weekly restarts |
|
||||
| Sunday 3am+ | `monthly_maintenance.sh` (self-gated on 30-day uptime) → ZFS scrub, SMART tests |
|
||||
| Sunday 7am | `sunday_morning_coffee_report.sh` → ZFS, SMART, certs, backup verify, bandwidth, Emby report |
|
||||
|
||||
---
|
||||
|
||||
## Rsync Toggle State (current)
|
||||
|
||||
```bash
|
||||
RSYNC_ENABLED=true
|
||||
CRITICAL_RSYNC_ENABLED=true
|
||||
INTERMEDIATE_RSYNC_ENABLED=true
|
||||
DAILY_RSYNC_ENABLED=true
|
||||
WEEKLY_RSYNC_ENABLED=true
|
||||
FALLBACK_RSYNC_ENABLED=true
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Fallback System
|
||||
|
||||
`fallback.sh` runs continuously from array start.
|
||||
States: `NORMAL | FALLBACK | NO_INTERNET | DARK`
|
||||
|
||||
DDNS rules are absolute:
|
||||
- Internet loss → stop own DDNS immediately
|
||||
- Failover → start remote's DDNS as Tier 1 first
|
||||
- Handback → stop remote DDNS → rsync → start containers → start local DDNS last
|
||||
|
||||
Tier delays before activating higher tiers are in `host*.conf` (`HOST1_TIER*_DELAY`, `HOST2_TIER*_DELAY`).
|
||||
|
||||
---
|
||||
|
||||
## Port Notes
|
||||
|
||||
- **NPM admin API (`HOST1_NPM_URL`)** — port **7818**. Port 81 is the partnership WebUI port (`HOST1_PARTNERSHIP_AUTH_WEBUIS`), not the API. Easy to confuse.
|
||||
- **HOST1_NETWORK_WATCHDOG_NPM_URL** — external HTTPS domain, completely separate from the admin API.
|
||||
|
||||
## Known Gaps / Active Work
|
||||
|
||||
- HOST2 NPM/lldap credentials (`HOST2_NPM_USER`, `HOST2_NPM_PASS`, `HOST2_LLDAP_PASS`) are empty in `host2.conf` — fill in when HOST2 is back online.
|
||||
- `PARTNERSHIP_ENABLED=false` — not yet active.
|
||||
- `FALLBACK_ENABLED=true` — fallback is running.
|
||||
|
||||
---
|
||||
|
||||
## Claude Code Persistence on Unraid
|
||||
|
||||
`/root` is a RAM filesystem — wiped on every reboot.
|
||||
`Tools/claude_startup.sh` runs at array start (via `ARRAY_START_SCRIPTS`) and:
|
||||
- Symlinks `/root/.claude` → `/mnt/user/appdata/claude-code/.claude`
|
||||
- Symlinks `/root/.local/share/claude` → `/mnt/user/appdata/claude-code/local/share/claude`
|
||||
- Symlinks `/root/CLAUDE.md` → `/boot/config/plugins/varaverk/CLAUDE.md` (this file)
|
||||
|
||||
This file lives on `/boot` (USB flash) and is always available regardless of array state.
|
||||
|
||||
---
|
||||
|
||||
## Commit Style
|
||||
|
||||
Plain, concise messages. No Co-Authored-By trailers. No bullet-point summaries in the body.
|
||||
One sentence on the why, not the what.
|
||||
@@ -303,7 +303,6 @@
|
||||
"Plugin/unraid/System_Essentials/php_fpm_max_children.sh" # WebGUI performance tuning
|
||||
"System_Essentials/inotify_tuning.sh" # bump inotify limits — containers miss events if exhausted
|
||||
"Docker_Essentials/docker_network_connect.sh" # ensure networks exist + connect containers
|
||||
"Tools/claude_startup.sh" # persist Claude data + binary to appdata; re-symlink on boot
|
||||
"Arrs_Stack/start_webhook_listener.sh" # arr upgrade webhook listener — continuous
|
||||
"Fallback/fallback.sh" # mutual failover — continuous
|
||||
)
|
||||
@@ -321,6 +320,16 @@
|
||||
"Docker_Essentials/docker_container_stop.sh" # stop all containers last
|
||||
)
|
||||
|
||||
# ━━━ Transcode Management ━━━
|
||||
# transcode_management.sh runs TRANSCODE_MANAGEMENT_SCRIPTS in order each cron cycle.
|
||||
# Schedule: */7 * * * * (every 7 minutes)
|
||||
# Order matters — cleanup first so the manager measures real current ramdisk usage,
|
||||
# not usage inflated by stale segment files from ended sessions.
|
||||
TRANSCODE_MANAGEMENT_SCRIPTS=(
|
||||
"Transcodes/transcode_cleanup.sh" # remove aged segment files before usage is measured
|
||||
"Transcodes/transcode_manager.sh" # flip ramdisk/SSD symlink, write daily log entry
|
||||
)
|
||||
|
||||
# ━━━ Watchdog Orchestrator ━━━
|
||||
# watchdog_orchestrator.sh runs WATCHDOG_ORCHESTRATOR_SCRIPTS in order each cron cycle.
|
||||
# Schedule: */15 * * * * (every 15 minutes)
|
||||
|
||||
@@ -317,6 +317,16 @@
|
||||
"Docker_Essentials/docker_container_stop.sh" # stop all containers last
|
||||
)
|
||||
|
||||
# ━━━ Transcode Management ━━━
|
||||
# transcode_management.sh runs TRANSCODE_MANAGEMENT_SCRIPTS in order each cron cycle.
|
||||
# Schedule: */7 * * * * (every 7 minutes)
|
||||
# Order matters — cleanup first so the manager measures real current ramdisk usage,
|
||||
# not usage inflated by stale segment files from ended sessions.
|
||||
TRANSCODE_MANAGEMENT_SCRIPTS=(
|
||||
"Transcodes/transcode_cleanup.sh" # remove aged segment files before usage is measured
|
||||
"Transcodes/transcode_manager.sh" # flip ramdisk/SSD symlink, write daily log entry
|
||||
)
|
||||
|
||||
# ━━━ Watchdog Orchestrator ━━━
|
||||
# watchdog_orchestrator.sh runs WATCHDOG_ORCHESTRATOR_SCRIPTS in order each cron cycle.
|
||||
# Schedule: */15 * * * * (every 15 minutes)
|
||||
|
||||
@@ -305,7 +305,31 @@ entries needed. The plugin handles all triggers natively:
|
||||
- **Cron** → `Plugin/unraid/event/disks_mounted/rebuild_cron` rebuilds the cron file from `schedule.json` on every boot
|
||||
|
||||
Configure via the Varaverk plugin Scheduler tab (or edit `schedule.json` directly).
|
||||
Individual scripts are never scheduled — only orchestrators.
|
||||
The built-in job list schedules orchestrators, never individual repo scripts directly —
|
||||
but the Scheduler tab's **Custom Scripts** card is the one place individual scripts
|
||||
*are* scheduled directly (see below).
|
||||
|
||||
---
|
||||
|
||||
### ── Custom Scripts ────────────────────────────────────────────────────────────
|
||||
|
||||
The Scheduler tab has a **Custom Scripts** card for one-off scripts that aren't part of
|
||||
the repo's orchestrator pipeline — personal tooling, quick fixes, anything you don't
|
||||
want to wire into `master.conf`.
|
||||
|
||||
Scripts live in `/boot/config/plugins/user.scripts/Varaverk/Scripts/` — deliberately
|
||||
**outside** the Varaverk git repo (that folder is never pushed to GitHub), in the same
|
||||
place the Unraid User Scripts plugin keeps its own scripts, so it's a folder location
|
||||
admins are already used to.
|
||||
|
||||
Two ways to get a script there:
|
||||
|
||||
- Click **+ Create Script** on the Scheduler tab — opens an inline editor, writes the
|
||||
file to that folder, and adds a `schedule.json` entry automatically.
|
||||
- Drop any `.sh` file into the folder yourself (e.g. via terminal, or Unraid's own
|
||||
Custom Scripts / User Scripts plugin pointed at the same path). The Scheduler tab
|
||||
**auto-detects** it — discovery is a folder scan, not a registry, so it doesn't matter
|
||||
how the file got there. It shows up disabled with no cron until you configure one.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,9 +1,3 @@
|
||||
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
|
||||
source ~/.bashrc
|
||||
claude
|
||||
|
||||
|
||||
|
||||
need a script that check file locations. like if a kids movie is in movies. it needs to detect this and update the location in the arrs. so it would tell radarr to move shrek from movies to kids movies and trck the new file
|
||||
|
||||
|
||||
|
||||
@@ -222,7 +222,6 @@ ARRAY_START_SCRIPTS=(
|
||||
# containers inherit limits at startup
|
||||
"Docker_Essentials/docker_network_connect.sh" # ensure networks + connections BEFORE
|
||||
# watchdogs check container states
|
||||
"Tools/claude_startup.sh" # symlink Claude persistent storage on /boot
|
||||
|
||||
# ── Continuous scripts — run until array stops ─────────────────────────────
|
||||
"Arrs_Stack/start_webhook_listener.sh" # start webhook listener before arrs POST events
|
||||
@@ -943,41 +942,40 @@ array_started.sh
|
||||
## ━━━ ADDING A NEW ORCHESTRATOR ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
If you find yourself running 3+ related scripts on the same schedule, wrap them
|
||||
in a new orchestrator. Model directly on `media_management.sh` which has the
|
||||
in a new orchestrator. Model directly on `daily_sync_maintenance.sh` which has the
|
||||
complete pattern — dry-run passthrough, status display, pass/fail tracking, summary.
|
||||
|
||||
Child-script execution goes through the shared `run_orch_child()` helper in
|
||||
`common.sh` — never hand-roll a per-file `run_job()` loop. It resolves the entry
|
||||
against `$ECOSYSTEM_ROOT`, threads `--dry-run`/`--log` from `$DRY_RUN`/`$ENABLE_LOGGING`
|
||||
automatically (never `$VERBOSE` — nothing in this codebase assigns it), and tracks
|
||||
into `JOB_PASS`/`JOB_FAIL` arrays the caller declares.
|
||||
|
||||
```bash
|
||||
# Minimal skeleton — the full pattern in its simplest form:
|
||||
#!/bin/bash
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
source "$SCRIPT_DIR/../load_config.sh"
|
||||
ECOSYSTEM_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
source "$ECOSYSTEM_ROOT/load_config.sh"
|
||||
parse_args "$@"
|
||||
|
||||
SCRIPTS_ROOT="$SCRIPT_DIR/.."
|
||||
PASS=()
|
||||
FAIL=()
|
||||
JOB_PASS=()
|
||||
JOB_FAIL=()
|
||||
|
||||
# Read job list from master.conf — never hardcode jobs in the orchestrator
|
||||
for script_entry in "${MY_MAINTENANCE_JOBS[@]:-}"; do
|
||||
[[ -z "$script_entry" ]] && continue
|
||||
|
||||
read -r -a parts <<< "$script_entry"
|
||||
script_path="$SCRIPTS_ROOT/${parts[0]}"
|
||||
script_name=$(basename "${parts[0]}")
|
||||
extra_args=("${parts[@]:1}")
|
||||
|
||||
[[ "$DRY_RUN" == true ]] && extra_args+=("--dry-run")
|
||||
|
||||
if bash "$script_path" "${extra_args[@]}"; then
|
||||
PASS+=("$script_name")
|
||||
else
|
||||
FAIL+=("$script_name")
|
||||
fi
|
||||
run_orch_child "$script_entry"
|
||||
done
|
||||
|
||||
# One summary — one notification
|
||||
echo "Passed: ${#PASS[@]} Failed: ${#FAIL[@]}"
|
||||
[[ ${#FAIL[@]} -gt 0 ]] && \
|
||||
notify "My maintenance failed on $(hostname) ($MY_ID) — ${FAIL[*]}" \
|
||||
# One summary — one notification. On a frequent (sub-daily) cadence, keep the
|
||||
# healthy path to a single line and reserve the full breakdown for failure/--log —
|
||||
# see watchdog_orchestrator.sh or transcode_management.sh for that split.
|
||||
echo "Passed: ${#JOB_PASS[@]} Failed: ${#JOB_FAIL[@]}"
|
||||
if [[ ${#JOB_FAIL[@]} -gt 0 ]]; then
|
||||
notify "My maintenance failed on $(hostname) ($MY_ID) — ${JOB_FAIL[*]}" \
|
||||
"My Orchestrator" "warning"
|
||||
exit 1
|
||||
fi
|
||||
exit 0
|
||||
```
|
||||
@@ -163,9 +163,8 @@ log "Launching ${#ARRAY_START_SCRIPTS[@]} script(s)..."
|
||||
echo ""
|
||||
|
||||
START=$(date +%s)
|
||||
LAUNCHED=0
|
||||
FAILED=0
|
||||
FAILED_SCRIPTS=()
|
||||
JOB_PASS=()
|
||||
JOB_FAIL=()
|
||||
|
||||
for relative_path in "${ARRAY_START_SCRIPTS[@]}"; do
|
||||
[[ -z "$relative_path" ]] && continue
|
||||
@@ -177,8 +176,7 @@ for relative_path in "${ARRAY_START_SCRIPTS[@]}"; do
|
||||
if [[ ! -f "$SCRIPT_PATH" ]]; then
|
||||
error "$SCRIPT_NAME — not found"
|
||||
error " Expected: $SCRIPT_PATH"
|
||||
(( FAILED++ ))
|
||||
FAILED_SCRIPTS+=("$SCRIPT_NAME")
|
||||
JOB_FAIL+=("$SCRIPT_NAME")
|
||||
continue
|
||||
fi
|
||||
|
||||
@@ -187,15 +185,14 @@ for relative_path in "${ARRAY_START_SCRIPTS[@]}"; do
|
||||
warn "$SCRIPT_NAME — not executable, fixing..."
|
||||
chmod +x "$SCRIPT_PATH" || {
|
||||
error "$SCRIPT_NAME — chmod +x failed"
|
||||
(( FAILED++ ))
|
||||
FAILED_SCRIPTS+=("$SCRIPT_NAME")
|
||||
JOB_FAIL+=("$SCRIPT_NAME")
|
||||
continue
|
||||
}
|
||||
fi
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
warn "DRY RUN — would launch: $SCRIPT_NAME"
|
||||
(( LAUNCHED++ ))
|
||||
JOB_PASS+=("$SCRIPT_NAME")
|
||||
continue
|
||||
fi
|
||||
|
||||
@@ -209,19 +206,18 @@ for relative_path in "${ARRAY_START_SCRIPTS[@]}"; do
|
||||
if kill -0 "$PID" 2>/dev/null; then
|
||||
# Still running → continuous script
|
||||
log "$SCRIPT_NAME — running (PID $PID) ✅"
|
||||
(( LAUNCHED++ ))
|
||||
JOB_PASS+=("$SCRIPT_NAME")
|
||||
else
|
||||
# Exited — check if one-shot success or failure
|
||||
wait "$PID"
|
||||
EXIT_CODE=$?
|
||||
if [[ "$EXIT_CODE" -eq 0 ]]; then
|
||||
echo "$SCRIPT_NAME — completed (one-shot) ✅"
|
||||
(( LAUNCHED++ ))
|
||||
JOB_PASS+=("$SCRIPT_NAME")
|
||||
else
|
||||
error "$SCRIPT_NAME — exited with code $EXIT_CODE"
|
||||
error " Path: $SCRIPT_PATH"
|
||||
(( FAILED++ ))
|
||||
FAILED_SCRIPTS+=("$SCRIPT_NAME")
|
||||
JOB_FAIL+=("$SCRIPT_NAME")
|
||||
fi
|
||||
fi
|
||||
|
||||
@@ -235,18 +231,21 @@ END=$(date +%s)
|
||||
echo ""
|
||||
echo "━━━━━ $ICON_SUMMARY ARRAY START SUMMARY ━━━━━"
|
||||
echo "$ICON_HOST Identity: $MY_ID ($LOCAL_SERVER_NAME)"
|
||||
echo "$ICON_SUCCESS Launched: $LAUNCHED"
|
||||
[[ "$FAILED" -gt 0 ]] && echo "$ICON_ERROR Failed: $FAILED — ${FAILED_SCRIPTS[*]}"
|
||||
echo "$ICON_SUCCESS Launched: ${#JOB_PASS[@]}"
|
||||
[[ ${#JOB_FAIL[@]} -gt 0 ]] && echo "$ICON_ERROR Failed: ${#JOB_FAIL[@]} — ${JOB_FAIL[*]}"
|
||||
echo "$ICON_TIME Duration: $(format_duration $(( END - START )))"
|
||||
echo ""
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
warn "DRY RUN — no scripts launched"
|
||||
elif [[ "$FAILED" -gt 0 ]]; then
|
||||
warn "Status: $FAILED script(s) failed — ${FAILED_SCRIPTS[*]}"
|
||||
notify "Array start on $(hostname) ($MY_ID) — $FAILED script(s) failed: ${FAILED_SCRIPTS[*]}" \
|
||||
elif [[ ${#JOB_FAIL[@]} -gt 0 ]]; then
|
||||
warn "Status: ${#JOB_FAIL[@]} script(s) failed — ${JOB_FAIL[*]}"
|
||||
notify "Array start on $(hostname) ($MY_ID) — ${#JOB_FAIL[@]} script(s) failed: ${JOB_FAIL[*]}" \
|
||||
"Array Start" "warning"
|
||||
else
|
||||
echo "$ICON_DONE Status: all $LAUNCHED script(s) launched ✅"
|
||||
echo "$ICON_DONE Status: all ${#JOB_PASS[@]} script(s) launched ✅"
|
||||
fi
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
|
||||
[[ ${#JOB_FAIL[@]} -gt 0 ]] && exit 1
|
||||
exit 0
|
||||
@@ -134,8 +134,8 @@ echo "$ICON_GEAR Running ${#ARRAY_STOP_SCRIPTS[@]} stop script(s) sequentially..
|
||||
echo ""
|
||||
|
||||
START=$(date +%s)
|
||||
PASSED=()
|
||||
FAILED=()
|
||||
JOB_PASS=()
|
||||
JOB_FAIL=()
|
||||
STEP=0
|
||||
|
||||
for entry in "${ARRAY_STOP_SCRIPTS[@]}"; do
|
||||
@@ -143,44 +143,11 @@ for entry in "${ARRAY_STOP_SCRIPTS[@]}"; do
|
||||
(( STEP++ ))
|
||||
|
||||
read -r -a parts <<< "$entry"
|
||||
script_path="$ECOSYSTEM_ROOT/${parts[0]}"
|
||||
script_name=$(basename "${parts[0]}")
|
||||
extra_args=("${parts[@]:1}")
|
||||
|
||||
echo "━━━ $ICON_GEAR Step $STEP: $script_name${extra_args:+ ${extra_args[*]}} ━━━"
|
||||
|
||||
if [[ ! -f "$script_path" ]]; then
|
||||
error "$script_name — not found at $script_path"
|
||||
FAILED+=("$script_name")
|
||||
echo ""
|
||||
continue
|
||||
fi
|
||||
|
||||
if [[ ! -x "$script_path" ]]; then
|
||||
warn "$script_name — not executable, fixing..."
|
||||
chmod +x "$script_path" || {
|
||||
error "$script_name — chmod +x failed"
|
||||
FAILED+=("$script_name")
|
||||
echo ""
|
||||
continue
|
||||
}
|
||||
fi
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
warn "DRY RUN — would run: $script_name ${extra_args[*]}"
|
||||
PASSED+=("$script_name")
|
||||
echo ""
|
||||
continue
|
||||
fi
|
||||
|
||||
if bash "$script_path" "${extra_args[@]}"; then
|
||||
echo "$script_name — done ✅"
|
||||
PASSED+=("$script_name")
|
||||
else
|
||||
warn "$script_name — failed (exit $?) — continuing to next step"
|
||||
FAILED+=("$script_name")
|
||||
fi
|
||||
|
||||
run_orch_child "$entry"
|
||||
echo ""
|
||||
done
|
||||
|
||||
@@ -192,22 +159,22 @@ END=$(date +%s)
|
||||
echo "━━━━━ $ICON_SUMMARY ARRAY STOP SUMMARY ━━━━━"
|
||||
echo "$ICON_HOST Identity: $MY_ID ($LOCAL_SERVER_NAME)"
|
||||
echo "$ICON_TIME Duration: $(format_duration $(( END - START )))"
|
||||
[[ ${#PASSED[@]} -gt 0 ]] && echo "$ICON_DONE Passed: ${PASSED[*]}"
|
||||
[[ ${#FAILED[@]} -gt 0 ]] && echo "$ICON_ERROR Failed: ${FAILED[*]}"
|
||||
[[ ${#JOB_PASS[@]} -gt 0 ]] && echo "$ICON_DONE Passed: ${JOB_PASS[*]}"
|
||||
[[ ${#JOB_FAIL[@]} -gt 0 ]] && echo "$ICON_ERROR Failed: ${JOB_FAIL[*]}"
|
||||
echo ""
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
warn "DRY RUN — no changes made"
|
||||
elif [[ ${#FAILED[@]} -eq 0 ]]; then
|
||||
elif [[ ${#JOB_FAIL[@]} -eq 0 ]]; then
|
||||
echo "$ICON_DONE Status: all $STEP step(s) complete ✅"
|
||||
notify "Array stop complete on $(hostname) ($MY_ID) — $STEP step(s) done" \
|
||||
"Array Stop" "normal"
|
||||
else
|
||||
warn "Status: ${#FAILED[@]} step(s) failed — ${FAILED[*]}"
|
||||
notify "Array stop on $(hostname) ($MY_ID) — ${#FAILED[@]} step(s) failed: ${FAILED[*]}" \
|
||||
warn "Status: ${#JOB_FAIL[@]} step(s) failed — ${JOB_FAIL[*]}"
|
||||
notify "Array stop on $(hostname) ($MY_ID) — ${#JOB_FAIL[@]} step(s) failed: ${JOB_FAIL[*]}" \
|
||||
"Array Stop" "warning"
|
||||
fi
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
|
||||
[[ ${#FAILED[@]} -gt 0 ]] && exit 1
|
||||
[[ ${#JOB_FAIL[@]} -gt 0 ]] && exit 1
|
||||
exit 0
|
||||
|
||||
@@ -80,6 +80,7 @@
|
||||
# ==============================================================================================
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ECOSYSTEM_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
|
||||
source "$SCRIPT_DIR/../load_config.sh"
|
||||
|
||||
@@ -199,29 +200,12 @@ fi
|
||||
# ==============================================================================================
|
||||
# ━━━ Critical Maintenance Scripts ━━━
|
||||
# ==============================================================================================
|
||||
if [[ ${#CRITICAL_MAINTENANCE_SCRIPTS[@]} -gt 0 ]]; then
|
||||
for script_entry in "${CRITICAL_MAINTENANCE_SCRIPTS[@]}"; do
|
||||
[[ -z "$script_entry" || "$script_entry" == \#* ]] && continue
|
||||
|
||||
SCRIPT_PATH="$SCRIPT_DIR/../${script_entry%% *}"
|
||||
SCRIPT_ARGS="${script_entry#* }"
|
||||
[[ "$SCRIPT_ARGS" == "$script_entry" ]] && SCRIPT_ARGS=""
|
||||
[[ "$DRY_RUN" == true ]] && SCRIPT_ARGS="$SCRIPT_ARGS --dry-run"
|
||||
|
||||
SCRIPT_NAME=$(basename "$SCRIPT_PATH")
|
||||
|
||||
if [[ ! -f "$SCRIPT_PATH" ]]; then
|
||||
warn "$SCRIPT_NAME not found at $SCRIPT_PATH — skipping"
|
||||
continue
|
||||
fi
|
||||
|
||||
log "Running: $SCRIPT_NAME"
|
||||
bash "$SCRIPT_PATH" $SCRIPT_ARGS
|
||||
EXIT_CODE=$?
|
||||
[[ "$EXIT_CODE" -ne 0 ]] && \
|
||||
warn "$SCRIPT_NAME exited with code $EXIT_CODE"
|
||||
done
|
||||
fi
|
||||
JOB_PASS=()
|
||||
JOB_FAIL=()
|
||||
for script_entry in "${CRITICAL_MAINTENANCE_SCRIPTS[@]}"; do
|
||||
[[ -z "$script_entry" || "$script_entry" == \#* ]] && continue
|
||||
run_orch_child "$script_entry"
|
||||
done
|
||||
|
||||
# ==============================================================================================
|
||||
# ━━━ Partnership Check ━━━
|
||||
@@ -247,20 +231,23 @@ fi
|
||||
END=$(date +%s)
|
||||
DURATION=$(format_duration $(( END - START )))
|
||||
|
||||
# Silent when healthy — only show summary if there were failures or notable events
|
||||
if [[ ${#FAIL[@]} -gt 0 ]]; then
|
||||
TOTAL_FAIL=$(( ${#FAIL[@]} + ${#JOB_FAIL[@]} ))
|
||||
|
||||
# Minimal one-liner when healthy — 30-min cadence, keep it quiet. Full detail on failure.
|
||||
if [[ "$TOTAL_FAIL" -gt 0 ]]; then
|
||||
echo ""
|
||||
echo "━━━━━ $ICON_SUMMARY CRITICAL SYNC SUMMARY ━━━━━"
|
||||
echo "$ICON_HOST Identity: $MY_ID ($LOCAL_SERVER_NAME)"
|
||||
echo "$ICON_TIME Duration: $DURATION"
|
||||
[[ ${#PASS[@]} -gt 0 ]] && echo "Synced: ${PASS[*]}"
|
||||
echo "$ICON_ERROR Failed: ${FAIL[*]}"
|
||||
[[ ${#PASS[@]} -gt 0 ]] && echo "Synced: ${PASS[*]}"
|
||||
[[ ${#FAIL[@]} -gt 0 ]] && echo "$ICON_ERROR Failed shares: ${FAIL[*]}"
|
||||
[[ ${#JOB_FAIL[@]} -gt 0 ]] && echo "$ICON_ERROR Failed jobs: ${JOB_FAIL[*]}"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
notify "Critical sync failed on $(hostname) ($MY_ID) — ${FAIL[*]}" \
|
||||
notify "Critical sync failed on $(hostname) ($MY_ID) — ${FAIL[*]} ${JOB_FAIL[*]}" \
|
||||
"Critical Sync" "warning"
|
||||
exit 1
|
||||
else
|
||||
echo "Critical sync complete — $MY_ID — ${DURATION} — ${#PASS[@]} share(s)"
|
||||
echo "Critical sync complete — $MY_ID — ${DURATION} — ${#PASS[@]} share(s), ${#JOB_PASS[@]} job(s)"
|
||||
fi
|
||||
|
||||
exit 0
|
||||
@@ -103,8 +103,8 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
source "$SCRIPT_DIR/../load_config.sh"
|
||||
|
||||
RSYNC_SCRIPT="$SCRIPT_DIR/../Rsync/rsync.sh"
|
||||
SCRIPTS_ROOT="$SCRIPT_DIR/.."
|
||||
ECOSYSTEM_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
RSYNC_SCRIPT="$ECOSYSTEM_ROOT/Rsync/rsync.sh"
|
||||
|
||||
parse_args "$@"
|
||||
|
||||
@@ -184,35 +184,6 @@ for script_entry in "${DAILY_MAINTENANCE_SCRIPTS[@]}"; do
|
||||
fi
|
||||
done
|
||||
|
||||
# Helper — run a maintenance script, track pass/fail
|
||||
run_job() {
|
||||
local script_entry="$1"
|
||||
local extra_dry=""
|
||||
[[ "$DRY_RUN" == true ]] && extra_dry="--dry-run"
|
||||
|
||||
read -r -a script_args <<< "$script_entry"
|
||||
local script_path="$SCRIPTS_ROOT/${script_args[0]}"
|
||||
local script_name
|
||||
script_name=$(basename "${script_args[0]}")
|
||||
local extra_args=("${script_args[@]:1}")
|
||||
|
||||
if [[ ! -f "$script_path" ]]; then
|
||||
error "$script_name — not found at $script_path"
|
||||
JOB_FAIL+=("$script_name")
|
||||
return 1
|
||||
fi
|
||||
|
||||
log "Running: $script_name ${extra_args[*]}"
|
||||
# shellcheck disable=SC2086
|
||||
if bash "$script_path" "${extra_args[@]}" $extra_dry; then
|
||||
echo "$script_name — done ✅"
|
||||
JOB_PASS+=("$script_name ${extra_args[*]}")
|
||||
else
|
||||
error "$script_name — failed (exit $?)"
|
||||
JOB_FAIL+=("$script_name ${extra_args[*]}")
|
||||
fi
|
||||
}
|
||||
|
||||
WINDOW_START=$(date +%s)
|
||||
JOB_PASS=()
|
||||
JOB_FAIL=()
|
||||
@@ -230,7 +201,7 @@ if [[ ${#PRE_SYNC_SCRIPTS[@]} -gt 0 ]]; then
|
||||
echo ""
|
||||
echo "━━━ $ICON_GIT Pre-sync ━━━"
|
||||
for script_entry in "${PRE_SYNC_SCRIPTS[@]}"; do
|
||||
run_job "$script_entry"
|
||||
run_orch_child "$script_entry"
|
||||
done
|
||||
fi
|
||||
|
||||
@@ -240,7 +211,7 @@ fi
|
||||
echo ""
|
||||
echo "━━━ $ICON_SYNC Arr Sync ━━━"
|
||||
|
||||
ARR_SYNC_SCRIPT="$SCRIPTS_ROOT/Arrs_Stack/arr_sync.sh"
|
||||
ARR_SYNC_SCRIPT="$ECOSYSTEM_ROOT/Arrs_Stack/arr_sync.sh"
|
||||
if [[ "${ARR_SYNC_ENABLED:-true}" != "true" ]]; then
|
||||
echo "ARR_SYNC_ENABLED=false — skipping"
|
||||
elif [[ ! -f "$ARR_SYNC_SCRIPT" ]]; then
|
||||
@@ -336,7 +307,7 @@ if [[ ${#POST_SYNC_SCRIPTS[@]} -gt 0 ]]; then
|
||||
echo "━━━ $ICON_CLEAN Post-sync Maintenance ━━━"
|
||||
for script_entry in "${POST_SYNC_SCRIPTS[@]}"; do
|
||||
echo ""
|
||||
run_job "$script_entry"
|
||||
run_orch_child "$script_entry"
|
||||
done
|
||||
fi
|
||||
|
||||
|
||||
@@ -50,7 +50,7 @@
|
||||
# check_connectivity — verified before any rsync (skipped if no shares)
|
||||
# check_remote_rootfs — aborts rsync if remote rootfs nearly full
|
||||
# Non-fatal jobs — a failed arr_sync warns but does not block rsync or artwork fetch
|
||||
# Silent on success — runs 4x/day, only failures warrant notification
|
||||
# Minimal on success — runs 6x/day; full breakdown only on failure or --log
|
||||
#
|
||||
# ==============================================================================================
|
||||
# CONFIGURATION
|
||||
@@ -88,8 +88,8 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
source "$SCRIPT_DIR/../load_config.sh"
|
||||
|
||||
RSYNC_SCRIPT="$SCRIPT_DIR/../Rsync/rsync.sh"
|
||||
SCRIPTS_ROOT="$SCRIPT_DIR/.."
|
||||
ECOSYSTEM_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
RSYNC_SCRIPT="$ECOSYSTEM_ROOT/Rsync/rsync.sh"
|
||||
|
||||
parse_args "$@"
|
||||
|
||||
@@ -109,35 +109,6 @@ resolve_remote_ip
|
||||
|
||||
[[ "$DRY_RUN" == true ]] && warn "DRY RUN — no changes will be made"
|
||||
|
||||
# ── Helper — run a maintenance job, track pass/fail ───────────────────────────────────────────
|
||||
run_job() {
|
||||
local script_entry="$1"
|
||||
local extra_dry=""
|
||||
[[ "$DRY_RUN" == true ]] && extra_dry="--dry-run"
|
||||
|
||||
read -r -a script_args <<< "$script_entry"
|
||||
local script_path="$SCRIPTS_ROOT/${script_args[0]}"
|
||||
local script_name
|
||||
script_name=$(basename "${script_args[0]}")
|
||||
local extra_args=("${script_args[@]:1}")
|
||||
|
||||
if [[ ! -f "$script_path" ]]; then
|
||||
error "$script_name — not found at $script_path"
|
||||
JOB_FAIL+=("$script_name")
|
||||
return 1
|
||||
fi
|
||||
|
||||
log "Running: $script_name ${extra_args[*]}"
|
||||
# shellcheck disable=SC2086
|
||||
if bash "$script_path" "${extra_args[@]}" $extra_dry; then
|
||||
echo "$script_name — done ✅"
|
||||
JOB_PASS+=("$script_name ${extra_args[*]}")
|
||||
else
|
||||
warn "$script_name — failed (exit $?)"
|
||||
JOB_FAIL+=("$script_name ${extra_args[*]}")
|
||||
fi
|
||||
}
|
||||
|
||||
# ==============================================================================================
|
||||
# ━━━ Status ━━━
|
||||
# ==============================================================================================
|
||||
@@ -187,7 +158,7 @@ echo "━━━ $ICON_GEAR Intermediate Sync — $MY_ID — $(date '+%Y-%m-%d %H
|
||||
echo ""
|
||||
echo "━━━ $ICON_GEAR Conf Pull ━━━"
|
||||
|
||||
CONF_SYNC_SCRIPT="$SCRIPTS_ROOT/System_Essentials/conf_sync.sh"
|
||||
CONF_SYNC_SCRIPT="$ECOSYSTEM_ROOT/System_Essentials/conf_sync.sh"
|
||||
if [[ ! -f "$CONF_SYNC_SCRIPT" ]]; then
|
||||
warn "conf_sync.sh not found — skipping partner conf refresh"
|
||||
else
|
||||
@@ -209,7 +180,7 @@ fi
|
||||
echo ""
|
||||
echo "━━━ $ICON_SYNC Arr Sync ━━━"
|
||||
|
||||
ARR_SYNC_SCRIPT="$SCRIPTS_ROOT/Arrs_Stack/arr_sync.sh"
|
||||
ARR_SYNC_SCRIPT="$ECOSYSTEM_ROOT/Arrs_Stack/arr_sync.sh"
|
||||
if [[ "${ARR_SYNC_ENABLED:-true}" != "true" ]]; then
|
||||
echo "ARR_SYNC_ENABLED=false — skipping"
|
||||
elif [[ ! -f "$ARR_SYNC_SCRIPT" ]]; then
|
||||
@@ -306,7 +277,7 @@ if [[ ${#INTERMEDIATE_MAINTENANCE_SCRIPTS[@]} -gt 0 ]]; then
|
||||
for script_entry in "${INTERMEDIATE_MAINTENANCE_SCRIPTS[@]}"; do
|
||||
[[ -z "$script_entry" ]] && continue
|
||||
echo ""
|
||||
run_job "$script_entry"
|
||||
run_orch_child "$script_entry"
|
||||
done
|
||||
fi
|
||||
|
||||
@@ -315,47 +286,57 @@ WINDOW_END=$(date +%s)
|
||||
# ==============================================================================================
|
||||
# ━━━ Summary ━━━
|
||||
# ==============================================================================================
|
||||
echo ""
|
||||
echo "━━━━━ $ICON_SUMMARY INTERMEDIATE SYNC SUMMARY ━━━━━"
|
||||
echo "$ICON_HOST Identity: $MY_ID ($LOCAL_SERVER_NAME)"
|
||||
echo "$ICON_TIME Window: $(date -d @"$WINDOW_START" '+%Y-%m-%d %H:%M:%S') → $(date -d @"$WINDOW_END" '+%H:%M:%S')"
|
||||
echo "$ICON_TIME Duration: $(format_duration $(( WINDOW_END - WINDOW_START )))"
|
||||
echo ""
|
||||
|
||||
if [[ "$SHARE_COUNT" -gt 0 ]]; then
|
||||
echo "$ICON_SYNC Shares ($SHARE_COUNT):"
|
||||
for entry in "${SHARE_TIMES[@]}"; do
|
||||
sname="${entry%%:*}"
|
||||
sdur="${entry##*:}"
|
||||
if printf '%s\n' "${FAIL[@]}" | grep -q "^${sname}"; then
|
||||
echo " $ICON_ERROR $sname — $(format_duration "$sdur")"
|
||||
else
|
||||
echo " $ICON_DONE $sname — $(format_duration "$sdur")"
|
||||
fi
|
||||
done
|
||||
echo " Passed: ${#PASS[@]}/$SHARE_COUNT Failed: ${#FAIL[@]}/$SHARE_COUNT"
|
||||
echo ""
|
||||
fi
|
||||
|
||||
if [[ ${#JOB_PASS[@]} -gt 0 || ${#JOB_FAIL[@]} -gt 0 ]]; then
|
||||
echo "$ICON_GEAR Jobs:"
|
||||
for job in "${JOB_PASS[@]}"; do echo " $ICON_DONE $job"; done
|
||||
for job in "${JOB_FAIL[@]}"; do echo " $ICON_ERROR $job"; done
|
||||
echo ""
|
||||
fi
|
||||
|
||||
TOTAL_FAIL=$(( ${#FAIL[@]} + ${#JOB_FAIL[@]} ))
|
||||
|
||||
# Full breakdown on failure or --log; minimal one-liner otherwise (4-hour cadence — keep it quiet).
|
||||
SHOW_FULL=false
|
||||
[[ "$TOTAL_FAIL" -gt 0 || "$ENABLE_LOGGING" == true ]] && SHOW_FULL=true
|
||||
|
||||
if [[ "$SHOW_FULL" == true ]]; then
|
||||
echo ""
|
||||
echo "━━━━━ $ICON_SUMMARY INTERMEDIATE SYNC SUMMARY ━━━━━"
|
||||
echo "$ICON_HOST Identity: $MY_ID ($LOCAL_SERVER_NAME)"
|
||||
echo "$ICON_TIME Window: $(date -d @"$WINDOW_START" '+%Y-%m-%d %H:%M:%S') → $(date -d @"$WINDOW_END" '+%H:%M:%S')"
|
||||
echo "$ICON_TIME Duration: $(format_duration $(( WINDOW_END - WINDOW_START )))"
|
||||
echo ""
|
||||
|
||||
if [[ "$SHARE_COUNT" -gt 0 ]]; then
|
||||
echo "$ICON_SYNC Shares ($SHARE_COUNT):"
|
||||
for entry in "${SHARE_TIMES[@]}"; do
|
||||
sname="${entry%%:*}"
|
||||
sdur="${entry##*:}"
|
||||
if printf '%s\n' "${FAIL[@]}" | grep -q "^${sname}"; then
|
||||
echo " $ICON_ERROR $sname — $(format_duration "$sdur")"
|
||||
else
|
||||
echo " $ICON_DONE $sname — $(format_duration "$sdur")"
|
||||
fi
|
||||
done
|
||||
echo " Passed: ${#PASS[@]}/$SHARE_COUNT Failed: ${#FAIL[@]}/$SHARE_COUNT"
|
||||
echo ""
|
||||
fi
|
||||
|
||||
if [[ ${#JOB_PASS[@]} -gt 0 || ${#JOB_FAIL[@]} -gt 0 ]]; then
|
||||
echo "$ICON_GEAR Jobs:"
|
||||
for job in "${JOB_PASS[@]}"; do echo " $ICON_DONE $job"; done
|
||||
for job in "${JOB_FAIL[@]}"; do echo " $ICON_ERROR $job"; done
|
||||
echo ""
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
warn "DRY RUN — no changes made"
|
||||
elif [[ "$TOTAL_FAIL" -eq 0 ]]; then
|
||||
echo "$ICON_DONE Status: all complete ✅ — ${#JOB_PASS[@]} job(s) run, ${#PASS[@]}/$SHARE_COUNT share(s) synced"
|
||||
if [[ "$SHOW_FULL" == true ]]; then
|
||||
echo "$ICON_DONE Status: all complete ✅ — ${#JOB_PASS[@]} job(s) run, ${#PASS[@]}/$SHARE_COUNT share(s) synced"
|
||||
else
|
||||
echo "$ICON_DONE Intermediate sync — ${#JOB_PASS[@]} job(s), ${#PASS[@]}/$SHARE_COUNT share(s) ($(format_duration $(( WINDOW_END - WINDOW_START ))))"
|
||||
fi
|
||||
else
|
||||
warn "Status: $TOTAL_FAIL failure(s)"
|
||||
notify "Intermediate sync failed on $(hostname) ($MY_ID) — shares: ${#FAIL[@]}/$SHARE_COUNT failed, jobs: ${#JOB_FAIL[@]} failed" \
|
||||
"Intermediate Sync" "warning"
|
||||
fi
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
[[ "$SHOW_FULL" == true ]] && echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
|
||||
[[ "$TOTAL_FAIL" -gt 0 ]] && exit 1
|
||||
exit 0
|
||||
|
||||
@@ -234,8 +234,8 @@ echo "$ICON_GEAR Scripts: ${#MONTHLY_MAINTENANCE_SCRIPTS[@]}"
|
||||
echo ""
|
||||
|
||||
START=$(date +%s)
|
||||
PASSED=()
|
||||
FAILED=()
|
||||
JOB_PASS=()
|
||||
JOB_FAIL=()
|
||||
STEP=0
|
||||
|
||||
# ==============================================================================================
|
||||
@@ -246,47 +246,11 @@ for entry in "${MONTHLY_MAINTENANCE_SCRIPTS[@]}"; do
|
||||
(( STEP++ ))
|
||||
|
||||
read -r -a parts <<< "$entry"
|
||||
script_path="$ECOSYSTEM_ROOT/${parts[0]}"
|
||||
script_name=$(basename "${parts[0]}")
|
||||
extra_args=("${parts[@]:1}")
|
||||
|
||||
echo "━━━ $ICON_GEAR Step $STEP: $script_name${extra_args:+ ${extra_args[*]}} ━━━"
|
||||
|
||||
if [[ ! -f "$script_path" ]]; then
|
||||
error "$script_name — not found at $script_path"
|
||||
FAILED+=("$script_name")
|
||||
echo ""
|
||||
continue
|
||||
fi
|
||||
|
||||
if [[ ! -x "$script_path" ]]; then
|
||||
warn "$script_name — not executable, fixing..."
|
||||
chmod +x "$script_path" || {
|
||||
error "$script_name — chmod +x failed"
|
||||
FAILED+=("$script_name")
|
||||
echo ""
|
||||
continue
|
||||
}
|
||||
fi
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
warn "DRY RUN — would run: $script_name${extra_args:+ ${extra_args[*]}}"
|
||||
PASSED+=("$script_name")
|
||||
echo ""
|
||||
continue
|
||||
fi
|
||||
|
||||
local_args=()
|
||||
[[ "$VERBOSE" == true ]] && local_args+=("--log")
|
||||
|
||||
if bash "$script_path" "${extra_args[@]}" "${local_args[@]}"; then
|
||||
echo "$script_name — done ✅"
|
||||
PASSED+=("$script_name")
|
||||
else
|
||||
warn "$script_name — failed (exit $?) — continuing to next step"
|
||||
FAILED+=("$script_name")
|
||||
fi
|
||||
|
||||
run_orch_child "$entry"
|
||||
echo ""
|
||||
done
|
||||
|
||||
@@ -307,22 +271,22 @@ fi
|
||||
echo "━━━━━ $ICON_SUMMARY MONTHLY MAINTENANCE SUMMARY ━━━━━"
|
||||
echo "$ICON_HOST Identity: $MY_ID ($LOCAL_SERVER_NAME)"
|
||||
echo "$ICON_TIME Duration: $(format_duration $(( END - START )))"
|
||||
[[ ${#PASSED[@]} -gt 0 ]] && echo "$ICON_DONE Passed: ${PASSED[*]}"
|
||||
[[ ${#FAILED[@]} -gt 0 ]] && echo "$ICON_ERROR Failed: ${FAILED[*]}"
|
||||
[[ ${#JOB_PASS[@]} -gt 0 ]] && echo "$ICON_DONE Passed: ${JOB_PASS[*]}"
|
||||
[[ ${#JOB_FAIL[@]} -gt 0 ]] && echo "$ICON_ERROR Failed: ${JOB_FAIL[*]}"
|
||||
echo ""
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
warn "DRY RUN — no changes made"
|
||||
elif [[ ${#FAILED[@]} -eq 0 ]]; then
|
||||
elif [[ ${#JOB_FAIL[@]} -eq 0 ]]; then
|
||||
echo "$ICON_DONE Status: all $STEP step(s) complete ✅"
|
||||
notify "Monthly maintenance complete on $(hostname) ($MY_ID) — $STEP step(s) done" \
|
||||
"Monthly Maintenance" "normal"
|
||||
else
|
||||
warn "Status: ${#FAILED[@]} step(s) failed — ${FAILED[*]}"
|
||||
notify "Monthly maintenance on $(hostname) ($MY_ID) — ${#FAILED[@]} step(s) failed: ${FAILED[*]}" \
|
||||
warn "Status: ${#JOB_FAIL[@]} step(s) failed — ${JOB_FAIL[*]}"
|
||||
notify "Monthly maintenance on $(hostname) ($MY_ID) — ${#JOB_FAIL[@]} step(s) failed: ${JOB_FAIL[*]}" \
|
||||
"Monthly Maintenance" "warning"
|
||||
fi
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
|
||||
[[ ${#FAILED[@]} -gt 0 ]] && exit 1
|
||||
[[ ${#JOB_FAIL[@]} -gt 0 ]] && exit 1
|
||||
exit 0
|
||||
|
||||
@@ -39,9 +39,12 @@
|
||||
# OPERATIONAL SAFEGUARDS
|
||||
# ==============================================================================================
|
||||
#
|
||||
# Root check — required for the docker/system reads used in the report
|
||||
# acquire_lock — prevents overlapping weekly runs
|
||||
# detect_hosts() — MY_ID in banner and summary
|
||||
# Non-fatal steps — a failed script is logged; remaining scripts still run
|
||||
# Flag pass-through — --dry-run and --log forwarded to all child scripts
|
||||
# notify() on failure — pushed only outside --dry-run, matching the runtime-mode contract below
|
||||
#
|
||||
# ==============================================================================================
|
||||
# CONFIGURATION
|
||||
@@ -70,12 +73,22 @@
|
||||
# ==============================================================================================
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ECOSYSTEM_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
|
||||
source "$SCRIPT_DIR/../load_config.sh"
|
||||
|
||||
SCRIPTS_ROOT="$SCRIPT_DIR/.."
|
||||
source "$ECOSYSTEM_ROOT/load_config.sh"
|
||||
|
||||
parse_args "$@"
|
||||
|
||||
# ==============================================================================================
|
||||
# ━━━ Setup ━━━
|
||||
# ==============================================================================================
|
||||
if [[ "$EUID" -ne 0 ]]; then
|
||||
error "Must be run as root"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
acquire_lock
|
||||
|
||||
detect_hosts
|
||||
|
||||
# ==============================================================================================
|
||||
@@ -84,35 +97,6 @@ detect_hosts
|
||||
JOB_PASS=()
|
||||
JOB_FAIL=()
|
||||
|
||||
run_job() {
|
||||
local script_entry="$1"
|
||||
local extra_dry=""
|
||||
local extra_log=""
|
||||
[[ "$DRY_RUN" == true ]] && extra_dry="--dry-run"
|
||||
[[ "$ENABLE_LOGGING" == true ]] && extra_log="--log"
|
||||
|
||||
read -r -a script_args <<< "$script_entry"
|
||||
local script_path="$SCRIPTS_ROOT/${script_args[0]}"
|
||||
local script_name
|
||||
script_name=$(basename "${script_args[0]}")
|
||||
local extra_args=("${script_args[@]:1}")
|
||||
|
||||
if [[ ! -f "$script_path" ]]; then
|
||||
error "$script_name — not found at $script_path"
|
||||
JOB_FAIL+=("$script_name")
|
||||
return 1
|
||||
fi
|
||||
|
||||
log "Running: $script_name ${extra_args[*]}"
|
||||
if bash "$script_path" "${extra_args[@]}" $extra_dry $extra_log; then
|
||||
echo "$script_name — done ✅"
|
||||
JOB_PASS+=("$script_name")
|
||||
else
|
||||
error "$script_name — failed (exit $?)"
|
||||
JOB_FAIL+=("$script_name")
|
||||
fi
|
||||
}
|
||||
|
||||
# ==============================================================================================
|
||||
# ━━━ Status ━━━
|
||||
# ==============================================================================================
|
||||
@@ -158,7 +142,7 @@ fi
|
||||
for script_entry in "${COFFEE_REPORT_SCRIPTS[@]}"; do
|
||||
[[ -z "$script_entry" ]] && continue
|
||||
echo ""
|
||||
run_job "$script_entry"
|
||||
run_orch_child "$script_entry"
|
||||
done
|
||||
|
||||
# ==============================================================================================
|
||||
@@ -176,5 +160,10 @@ if [[ ${#JOB_FAIL[@]} -gt 0 ]]; then
|
||||
echo "❌ Failed: ${JOB_FAIL[*]}"
|
||||
fi
|
||||
|
||||
if [[ ${#JOB_FAIL[@]} -gt 0 && "$DRY_RUN" != true ]]; then
|
||||
notify "Sunday coffee report had failures on $(hostname) ($MY_ID) — ${JOB_FAIL[*]}" \
|
||||
"Sunday Morning Coffee Report" "warning"
|
||||
fi
|
||||
|
||||
[[ ${#JOB_FAIL[@]} -gt 0 ]] && exit 1
|
||||
exit 0
|
||||
|
||||
@@ -5,14 +5,17 @@
|
||||
#
|
||||
# PURPOSE
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Runs transcode_cleanup.sh then transcode_manager.sh in the correct order.
|
||||
# Replaces individual cron entries for each — this is the single cron entry.
|
||||
# Schedule: */7 * * * * (every 7 minutes via User Scripts plugin)
|
||||
# Runs TRANSCODE_MANAGEMENT_SCRIPTS in order each cron cycle — this is the
|
||||
# single cron entry replacing individual entries for each child script.
|
||||
# Schedule: */7 * * * * (every 7 minutes)
|
||||
#
|
||||
# ==============================================================================================
|
||||
# OPERATIONAL MODEL
|
||||
# ==============================================================================================
|
||||
#
|
||||
# Order driven by TRANSCODE_MANAGEMENT_SCRIPTS in master.conf.
|
||||
# Default: transcode_cleanup.sh → transcode_manager.sh
|
||||
#
|
||||
# transcode_cleanup.sh
|
||||
# Removes aged segment files not open by any process. Uses lsof for O(1)
|
||||
# per-file active check. Triggers flip-back to ramdisk after cleanup if
|
||||
@@ -36,6 +39,8 @@
|
||||
# Stale segment files from ended sessions inflate the ramdisk usage reading
|
||||
# and trigger unnecessary SSD flips even when active sessions would fit on
|
||||
# the ramdisk. Cleanup runs first so the manager measures real current usage.
|
||||
# Order is config-driven (TRANSCODE_MANAGEMENT_SCRIPTS) but this dependency
|
||||
# is real — reordering the array changes what the manager measures.
|
||||
#
|
||||
# Delegated Logging
|
||||
# This orchestrator does not write its own log — transcode_manager.sh owns
|
||||
@@ -48,8 +53,9 @@
|
||||
# Root check — mount and docker operations require root
|
||||
# acquire_lock — prevents concurrent 7-minute cycles overlapping
|
||||
# detect_hosts() — aliases RAMDISK_PATH, TRANSCODE_SSD, RAMDISK_WARN_GB per host
|
||||
# --dry-run — passed through to both child scripts
|
||||
# Exit code — worst exit code of both scripts returned to cron
|
||||
# --dry-run — passed through to every script in TRANSCODE_MANAGEMENT_SCRIPTS
|
||||
# Exit code — worst exit code across all scripts returned to cron
|
||||
# notify() — pushed on failure, skipped in --dry-run
|
||||
#
|
||||
# ==============================================================================================
|
||||
# CONFIGURATION
|
||||
@@ -57,6 +63,7 @@
|
||||
#
|
||||
# master.conf
|
||||
#
|
||||
# TRANSCODE_MANAGEMENT_SCRIPTS — scripts to run, in order
|
||||
# TRANSCODE_DAILY_LOG — daily stats log (written by transcode_manager.sh)
|
||||
# TRANSCODE_LOG_RETENTION — days to keep (trimmed by manager)
|
||||
# TRANSCODE_STATE_FILE — current state (ramdisk_used, flip_count, etc.)
|
||||
@@ -70,13 +77,13 @@
|
||||
# Normal run (every 7 minutes via cron).
|
||||
#
|
||||
# transcode_management.sh --dry-run
|
||||
# Preview without changes (passed to both child scripts).
|
||||
# Preview without changes (passed to every script in TRANSCODE_MANAGEMENT_SCRIPTS).
|
||||
#
|
||||
# transcode_management.sh --status
|
||||
# Show configuration and current state.
|
||||
#
|
||||
# transcode_management.sh --log
|
||||
# Verbose output from both child scripts.
|
||||
# Verbose output from every child script.
|
||||
#
|
||||
# ==============================================================================================
|
||||
|
||||
@@ -86,9 +93,6 @@ source "$SCRIPT_DIR/../load_config.sh"
|
||||
|
||||
parse_args "$@"
|
||||
|
||||
CLEANUP_SCRIPT="$SCRIPT_DIR/../Transcodes/transcode_cleanup.sh"
|
||||
MANAGER_SCRIPT="$SCRIPT_DIR/../Transcodes/transcode_manager.sh"
|
||||
|
||||
# ==============================================================================================
|
||||
# ━━━ Setup ━━━
|
||||
# ==============================================================================================
|
||||
@@ -122,12 +126,15 @@ if [[ "$SHOW_STATUS" == true ]]; then
|
||||
echo "$ICON_TIME Schedule: every 7 minutes"
|
||||
echo ""
|
||||
echo "━━━ Child Scripts ━━━"
|
||||
[[ -f "$CLEANUP_SCRIPT" ]] && \
|
||||
echo " $ICON_SUCCESS transcode_cleanup.sh — found" || \
|
||||
echo " $ICON_ERROR transcode_cleanup.sh — NOT FOUND at $CLEANUP_SCRIPT"
|
||||
[[ -f "$MANAGER_SCRIPT" ]] && \
|
||||
echo " $ICON_SUCCESS transcode_manager.sh — found" || \
|
||||
echo " $ICON_ERROR transcode_manager.sh — NOT FOUND at $MANAGER_SCRIPT"
|
||||
for _entry in "${TRANSCODE_MANAGEMENT_SCRIPTS[@]}"; do
|
||||
_script_path="$SCRIPT_DIR/../$_entry"
|
||||
_script_name=$(basename "$_entry")
|
||||
if [[ -f "$_script_path" ]]; then
|
||||
echo " $ICON_SUCCESS $_script_name — found"
|
||||
else
|
||||
echo " $ICON_ERROR $_script_name — NOT FOUND at $_script_path"
|
||||
fi
|
||||
done
|
||||
echo ""
|
||||
echo "━━━ Daily Log ━━━"
|
||||
if [[ -f "${TRANSCODE_DAILY_LOG:-}" ]] && [[ -s "$TRANSCODE_DAILY_LOG" ]]; then
|
||||
@@ -154,15 +161,13 @@ fi
|
||||
# ==============================================================================================
|
||||
# ━━━ Validate Child Scripts ━━━
|
||||
# ==============================================================================================
|
||||
if [[ ! -f "$CLEANUP_SCRIPT" ]]; then
|
||||
error "transcode_cleanup.sh not found: $CLEANUP_SCRIPT"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ ! -f "$MANAGER_SCRIPT" ]]; then
|
||||
error "transcode_manager.sh not found: $MANAGER_SCRIPT"
|
||||
exit 1
|
||||
fi
|
||||
for _entry in "${TRANSCODE_MANAGEMENT_SCRIPTS[@]}"; do
|
||||
_script_path="$SCRIPT_DIR/../$_entry"
|
||||
if [[ ! -f "$_script_path" ]]; then
|
||||
error "$(basename "$_entry") not found: $_script_path"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
# ==============================================================================================
|
||||
# ━━━ Pre-run State Snapshot ━━━
|
||||
@@ -179,29 +184,46 @@ else
|
||||
fi
|
||||
|
||||
# ==============================================================================================
|
||||
# ━━━ Run Cleanup ━━━
|
||||
# ━━━ Run Scripts ━━━
|
||||
# ==============================================================================================
|
||||
# transcode_manager.sh writes to TRANSCODE_DAILY_LOG after each run — no flag here
|
||||
# suppresses that; it owns the log write for this cycle regardless of position ✅
|
||||
DRY_FLAG=""
|
||||
[[ "$DRY_RUN" == true ]] && DRY_FLAG="--dry-run"
|
||||
|
||||
_cleanup_start=$(date +%s)
|
||||
bash "$CLEANUP_SCRIPT" $DRY_FLAG
|
||||
CLEANUP_EXIT=$?
|
||||
log "cleanup: $(format_duration $(( $(date +%s) - _cleanup_start ))) (exit $CLEANUP_EXIT)"
|
||||
WORST_EXIT=0
|
||||
PASS_COUNT=0
|
||||
FAIL_NAMES=()
|
||||
for _entry in "${TRANSCODE_MANAGEMENT_SCRIPTS[@]}"; do
|
||||
_script_path="$SCRIPT_DIR/../$_entry"
|
||||
_script_name=$(basename "$_entry" .sh)
|
||||
_start=$(date +%s)
|
||||
bash "$_script_path" $DRY_FLAG
|
||||
_exit=$?
|
||||
log "$_script_name: $(format_duration $(( $(date +%s) - _start ))) (exit $_exit)"
|
||||
if [[ "$_exit" -ne 0 ]]; then
|
||||
WORST_EXIT=1
|
||||
FAIL_NAMES+=("$_script_name")
|
||||
else
|
||||
PASS_COUNT=$(( PASS_COUNT + 1 ))
|
||||
fi
|
||||
done
|
||||
|
||||
# ==============================================================================================
|
||||
# ━━━ Run Manager ━━━
|
||||
# ━━━ Summary — minimal one-liner by default (7-min cadence — keep it quiet when healthy) ━━━
|
||||
# ==============================================================================================
|
||||
# transcode_manager.sh writes to TRANSCODE_DAILY_LOG after each run
|
||||
# No --no-log flag here — manager owns the log write for this cycle ✅
|
||||
_manager_start=$(date +%s)
|
||||
bash "$MANAGER_SCRIPT" $DRY_FLAG
|
||||
MANAGER_EXIT=$?
|
||||
log "manager: $(format_duration $(( $(date +%s) - _manager_start ))) (exit $MANAGER_EXIT)"
|
||||
if [[ "$WORST_EXIT" -eq 0 ]]; then
|
||||
echo "$ICON_SUCCESS Transcode cycle — $PASS_COUNT/${#TRANSCODE_MANAGEMENT_SCRIPTS[@]} passed"
|
||||
else
|
||||
error "Transcode cycle — failed: ${FAIL_NAMES[*]}"
|
||||
if [[ "$DRY_RUN" != true ]]; then
|
||||
notify "Transcode management failure on $(hostname) ($MY_ID) — ${FAIL_NAMES[*]}" \
|
||||
"Transcode Management" "warning"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ==============================================================================================
|
||||
# ━━━ Exit ━━━
|
||||
# ==============================================================================================
|
||||
# Return worst exit code — caller knows if either script failed
|
||||
[[ "$CLEANUP_EXIT" -ne 0 || "$MANAGER_EXIT" -ne 0 ]] && exit 1
|
||||
exit 0
|
||||
# Return worst exit code — caller knows if any script failed
|
||||
exit "$WORST_EXIT"
|
||||
@@ -193,7 +193,7 @@ run_watchdog() {
|
||||
|
||||
local extra_args=()
|
||||
[[ "$DRY_RUN" == true ]] && extra_args+=("--dry-run")
|
||||
[[ "$VERBOSE" == true ]] && extra_args+=("--log")
|
||||
[[ "$ENABLE_LOGGING" == true ]] && extra_args+=("--log")
|
||||
|
||||
local _ws
|
||||
_ws=$(date +%s)
|
||||
@@ -234,18 +234,23 @@ if [[ "${WATCHDOG_ORCHESTRATOR_HEARTBEAT:-true}" == true ]]; then
|
||||
fi
|
||||
|
||||
# ==============================================================================================
|
||||
# ━━━ Summary — only shown on failures or --log ━━━
|
||||
# ━━━ Summary — minimal one-liner by default, full breakdown on failure or --log ━━━
|
||||
# ==============================================================================================
|
||||
if [[ "${#FAIL[@]}" -gt 0 || "$VERBOSE" == true ]]; then
|
||||
if [[ "${#FAIL[@]}" -gt 0 || "$ENABLE_LOGGING" == true ]]; then
|
||||
echo ""
|
||||
echo "━━━━━ $ICON_SUMMARY WATCHDOG CYCLE — $MY_ID — $(date '+%H:%M:%S') ━━━━━"
|
||||
for p in "${PASS[@]}"; do log " $ICON_DONE $p"; done
|
||||
for f in "${FAIL[@]}"; do error " $ICON_ERROR $f"; done
|
||||
echo "$ICON_TIME Duration: $(format_duration $DURATION)"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
|
||||
if [[ "${#FAIL[@]}" -gt 0 ]]; then
|
||||
notify "Watchdog cycle failure on $(hostname) ($MY_ID) — ${FAIL[*]}" \
|
||||
"Watchdog Orchestrator" "warning"
|
||||
fi
|
||||
else
|
||||
echo "$ICON_DONE Watchdog cycle — ${#PASS[@]}/${#WATCHDOG_ORCHESTRATOR_SCRIPTS[@]} passed ($(format_duration $DURATION))"
|
||||
fi
|
||||
|
||||
if [[ "${#FAIL[@]}" -gt 0 ]]; then
|
||||
notify "Watchdog cycle failure on $(hostname) ($MY_ID) — ${FAIL[*]}" \
|
||||
"Watchdog Orchestrator" "warning"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
exit 0
|
||||
|
||||
@@ -84,8 +84,8 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
source "$SCRIPT_DIR/../load_config.sh"
|
||||
|
||||
RSYNC_SCRIPT="$SCRIPT_DIR/../Rsync/rsync.sh"
|
||||
SCRIPTS_ROOT="$SCRIPT_DIR/.."
|
||||
ECOSYSTEM_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
RSYNC_SCRIPT="$ECOSYSTEM_ROOT/Rsync/rsync.sh"
|
||||
|
||||
parse_args "$@"
|
||||
|
||||
@@ -123,35 +123,6 @@ read -r -a MAINTENANCE_CONTAINERS <<< \
|
||||
|
||||
[[ "$DRY_RUN" == true ]] && warn "DRY RUN — no containers will be stopped, no sync, no updates"
|
||||
|
||||
# ── Helper — run a post-sync maintenance script ────────────────────────────────────────────────
|
||||
run_job() {
|
||||
local script_entry="$1"
|
||||
local extra_dry=""
|
||||
[[ "$DRY_RUN" == true ]] && extra_dry="--dry-run"
|
||||
|
||||
read -r -a script_args <<< "$script_entry"
|
||||
local script_path="$SCRIPTS_ROOT/${script_args[0]}"
|
||||
local script_name
|
||||
script_name=$(basename "${script_args[0]}")
|
||||
local extra_args=("${script_args[@]:1}")
|
||||
|
||||
if [[ ! -f "$script_path" ]]; then
|
||||
error "$script_name — not found at $script_path"
|
||||
JOB_FAIL+=("$script_name")
|
||||
return 1
|
||||
fi
|
||||
|
||||
log "Running: $script_name ${extra_args[*]}"
|
||||
# shellcheck disable=SC2086
|
||||
if bash "$script_path" "${extra_args[@]}" $extra_dry; then
|
||||
echo "$script_name — done ✅"
|
||||
JOB_PASS+=("$script_name ${extra_args[*]}")
|
||||
else
|
||||
error "$script_name — failed (exit $?)"
|
||||
JOB_FAIL+=("$script_name ${extra_args[*]}")
|
||||
fi
|
||||
}
|
||||
|
||||
# ==============================================================================================
|
||||
# ━━━ Status ━━━
|
||||
# ==============================================================================================
|
||||
@@ -390,7 +361,7 @@ if [[ ${#WEEKLY_MAINTENANCE_SCRIPTS[@]} -gt 0 ]]; then
|
||||
for script_entry in "${WEEKLY_MAINTENANCE_SCRIPTS[@]}"; do
|
||||
[[ -z "$script_entry" ]] && continue
|
||||
echo ""
|
||||
run_job "$script_entry"
|
||||
run_orch_child "$script_entry"
|
||||
done
|
||||
fi
|
||||
|
||||
|
||||
+14
-2
@@ -73,8 +73,8 @@ to Community Applications.
|
||||
|
||||
## ━━━ SCRIPTS DIRECTORY SETTING ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
`SCRIPTS_DIR` is the only plugin-level setting. It tells the plugin where to find the
|
||||
Configurations directory and all scripts.
|
||||
`SCRIPTS_DIR` tells the plugin where to find the Configurations directory and all
|
||||
repo scripts.
|
||||
|
||||
**Set it via:** Settings → Other Settings → Varaverk → Scripts directory
|
||||
|
||||
@@ -88,6 +88,18 @@ SCRIPTS_DIR="/boot/config/plugins/varaverk"
|
||||
|
||||
All other configuration lives in `Configurations/master.conf` and `Configurations/host*.conf`.
|
||||
|
||||
`CUSTOM_SCRIPTS_DIR` is a separate, optional override for where the Scheduler tab's
|
||||
Custom Scripts feature reads/writes user-authored scripts (see Manual.md → Custom
|
||||
Scripts). It's intentionally **not** under `SCRIPTS_DIR` — Custom Scripts are personal,
|
||||
non-repo tooling and must never end up inside the git-tracked plugin folder.
|
||||
|
||||
Default: `/boot/config/plugins/user.scripts/Varaverk/Scripts`
|
||||
|
||||
```bash
|
||||
# in varaverk.cfg, alongside SCRIPTS_DIR
|
||||
CUSTOM_SCRIPTS_DIR="/boot/config/plugins/user.scripts/Varaverk/Scripts"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ━━━ REPO MOVE PROCEDURE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
<?php
|
||||
// Import Script — lets the Scheduler page's "+ Import Script" browser move an existing
|
||||
// .sh file from anywhere on the server into CUSTOM_SCRIPTS_DIR. This is a MOVE: the
|
||||
// source is deleted once the copy is verified, so no stale duplicate is left behind.
|
||||
header('Content-Type: application/json');
|
||||
header('Cache-Control: no-store, no-cache');
|
||||
require_once dirname(__DIR__) . '/include/config.php';
|
||||
|
||||
// ── browse (GET): list subdirectories and .sh files at $path, rooted at / ─────
|
||||
if ($_SERVER['REQUEST_METHOD'] === 'GET' && ($_GET['action'] ?? '') === 'browse') {
|
||||
$path = trim($_GET['path'] ?? '/');
|
||||
if (!preg_match('#^/[^\0]*$#', $path) || str_contains($path, '..')) {
|
||||
echo json_encode(['ok' => false, 'error' => 'Invalid path']);
|
||||
exit;
|
||||
}
|
||||
$clean = rtrim($path, '/') ?: '/';
|
||||
if (!is_dir($clean)) {
|
||||
echo json_encode(['ok' => false, 'error' => 'Not a directory: ' . $clean]);
|
||||
exit;
|
||||
}
|
||||
|
||||
$dirOut = shell_exec('find ' . escapeshellarg($clean) . ' -maxdepth 1 -mindepth 1 -type d 2>/dev/null | sort | head -300') ?: '';
|
||||
$dirs = array_values(array_filter(array_map('trim', explode("\n", $dirOut))));
|
||||
|
||||
$fileOut = shell_exec('find ' . escapeshellarg($clean) . ' -maxdepth 1 -mindepth 1 -type f -iname "*.sh" 2>/dev/null | sort | head -300') ?: '';
|
||||
$files = array_values(array_filter(array_map('trim', explode("\n", $fileOut))));
|
||||
|
||||
$parent = ($clean !== '/') ? (dirname($clean) ?: '/') : null;
|
||||
echo json_encode(['ok' => true, 'path' => $clean, 'dirs' => $dirs, 'files' => $files, 'parent' => $parent]);
|
||||
exit;
|
||||
}
|
||||
|
||||
// ── import (POST): move the chosen .sh file into CUSTOM_SCRIPTS_DIR ───────────
|
||||
if ($_SERVER['REQUEST_METHOD'] === 'POST' && ($_POST['action'] ?? '') === 'import') {
|
||||
$src = trim($_POST['path'] ?? '');
|
||||
if (!preg_match('#^/[^\0]*\.sh$#i', $src) || str_contains($src, '..')) {
|
||||
echo json_encode(['ok' => false, 'error' => 'Invalid script path']);
|
||||
exit;
|
||||
}
|
||||
if (!is_file($src)) {
|
||||
echo json_encode(['ok' => false, 'error' => 'Not found: ' . $src]);
|
||||
exit;
|
||||
}
|
||||
|
||||
$srcReal = realpath($src);
|
||||
if ($srcReal === false) {
|
||||
echo json_encode(['ok' => false, 'error' => 'Could not resolve path']);
|
||||
exit;
|
||||
}
|
||||
|
||||
// Refuse to move a file out of the git-tracked repo — that would delete a
|
||||
// tracked file out from under git without a commit recording it.
|
||||
$repoReal = realpath(SCRIPTS_DIR);
|
||||
if ($repoReal && str_starts_with($srcReal, $repoReal . '/')) {
|
||||
echo json_encode(['ok' => false, 'error' => 'Refusing to import from inside the Varaverk repo — that would delete a git-tracked file.']);
|
||||
exit;
|
||||
}
|
||||
|
||||
// Already there — nothing to do.
|
||||
$customReal = realpath(CUSTOM_SCRIPTS_DIR) ?: CUSTOM_SCRIPTS_DIR;
|
||||
if (str_starts_with($srcReal, rtrim($customReal, '/') . '/')) {
|
||||
echo json_encode(['ok' => false, 'error' => 'Already in Custom Scripts.']);
|
||||
exit;
|
||||
}
|
||||
|
||||
if (!is_dir(CUSTOM_SCRIPTS_DIR)) mkdir(CUSTOM_SCRIPTS_DIR, 0755, true);
|
||||
|
||||
$name = basename($srcReal);
|
||||
$dest = CUSTOM_SCRIPTS_DIR . '/' . $name;
|
||||
if (file_exists($dest)) {
|
||||
echo json_encode(['ok' => false, 'error' => "A script named \"$name\" already exists in Custom Scripts."]);
|
||||
exit;
|
||||
}
|
||||
|
||||
// Copy across filesystems, verify, THEN delete the source — never remove the
|
||||
// only copy on a failed or partial copy.
|
||||
if (!copy($srcReal, $dest)) {
|
||||
@unlink($dest);
|
||||
echo json_encode(['ok' => false, 'error' => 'Copy failed']);
|
||||
exit;
|
||||
}
|
||||
if (filesize($srcReal) !== filesize($dest) || hash_file('sha256', $srcReal) !== hash_file('sha256', $dest)) {
|
||||
@unlink($dest);
|
||||
echo json_encode(['ok' => false, 'error' => 'Copy verification failed — source left untouched']);
|
||||
exit;
|
||||
}
|
||||
chmod($dest, 0755);
|
||||
|
||||
if (!@unlink($srcReal)) {
|
||||
// Copied and verified but couldn't remove the original (permissions, read-only
|
||||
// mount). The script is usable from its new home either way — surface a warning
|
||||
// rather than failing the import outright.
|
||||
echo json_encode([
|
||||
'ok' => true,
|
||||
'id' => 'Custom/' . $name,
|
||||
'warning' => 'Imported, but could not delete the original at ' . $srcReal . ' — remove it manually.',
|
||||
]);
|
||||
exit;
|
||||
}
|
||||
|
||||
echo json_encode(['ok' => true, 'id' => 'Custom/' . $name]);
|
||||
exit;
|
||||
}
|
||||
|
||||
echo json_encode(['ok' => false, 'error' => 'Invalid request']);
|
||||
@@ -8,7 +8,7 @@ if ($_SERVER['REQUEST_METHOD'] === 'GET') {
|
||||
echo json_encode(['ok' => false, 'error' => 'Invalid id']);
|
||||
exit;
|
||||
}
|
||||
$path = SCRIPTS_DIR . '/' . $id;
|
||||
$path = CUSTOM_SCRIPTS_DIR . '/' . substr($id, strlen('Custom/'));
|
||||
echo json_encode(['ok' => true, 'content' => file_exists($path) ? file_get_contents($path) : '']);
|
||||
exit;
|
||||
}
|
||||
@@ -24,7 +24,7 @@ if ($_SERVER['REQUEST_METHOD'] === 'POST') {
|
||||
}
|
||||
|
||||
$id = 'Custom/' . $name . '.sh';
|
||||
$path = SCRIPTS_DIR . '/Custom/' . $name . '.sh';
|
||||
$path = CUSTOM_SCRIPTS_DIR . '/' . $name . '.sh';
|
||||
|
||||
if ($action === 'delete') {
|
||||
if (!file_exists($path)) {
|
||||
@@ -40,7 +40,7 @@ if ($_SERVER['REQUEST_METHOD'] === 'POST') {
|
||||
exit;
|
||||
}
|
||||
|
||||
$dir = SCRIPTS_DIR . '/Custom';
|
||||
$dir = CUSTOM_SCRIPTS_DIR;
|
||||
if (!is_dir($dir)) mkdir($dir, 0755, true);
|
||||
if (file_put_contents($path, $content) === false) {
|
||||
echo json_encode(['ok' => false, 'error' => 'Failed to write script']);
|
||||
|
||||
@@ -11,6 +11,11 @@ define('DEPLOY_DIR', SCRIPTS_DIR . '/Deployment');
|
||||
define('DATA_DIR', SCRIPTS_DIR . '/data');
|
||||
define('STATE_DIR', SCRIPTS_DIR . '/State_Files');
|
||||
define('LOG_DIR', '/var/log/varaverk');
|
||||
// User-authored custom scripts (scheduler page "+ Create Script") — kept outside the git
|
||||
// repo entirely, alongside the User Scripts plugin's own storage. Any *.sh file placed
|
||||
// directly in this folder is auto-detected and listed — it doesn't have to be created
|
||||
// through the page's editor.
|
||||
define('CUSTOM_SCRIPTS_DIR', $_vv_cfg['CUSTOM_SCRIPTS_DIR'] ?? '/boot/config/plugins/user.scripts/Varaverk/Scripts');
|
||||
unset($_vv_cfg);
|
||||
|
||||
define('VV_SETUP_STATE_FILE', STATE_DIR . '/varaverk_setup.db');
|
||||
|
||||
@@ -97,7 +97,10 @@ function vv_cron_rebuild(array $schedule): bool {
|
||||
$id = $entry['id'];
|
||||
// When a child's orch is enabled it is the sole trigger — suppress independent cron.
|
||||
if (isset($childToOrch[$id]) && !empty($schedule[$childToOrch[$id]]['enabled'])) continue;
|
||||
$script = "$scriptsDir/$id";
|
||||
// Custom Scripts live outside the repo (CUSTOM_SCRIPTS_DIR) — everything else resolves under SCRIPTS_DIR.
|
||||
$script = str_starts_with($id, 'Custom/')
|
||||
? CUSTOM_SCRIPTS_DIR . '/' . substr($id, strlen('Custom/'))
|
||||
: "$scriptsDir/$id";
|
||||
$flags = !empty($entry['log_enabled']) ? ' --log' : '';
|
||||
$lines[] = "{$entry['cron']} bash \"$runner\" \"$id\" \"$script\"$flags";
|
||||
}
|
||||
@@ -278,6 +281,10 @@ function vv_tools_scripts(): array {
|
||||
return $scripts;
|
||||
}
|
||||
|
||||
// Lists every Custom Script for the scheduler page. Discovery is glob-based, not a
|
||||
// registry — any *.sh file dropped directly into CUSTOM_SCRIPTS_DIR (or a platform
|
||||
// adapter's own Custom/ folder) shows up here, whether or not it was created via the
|
||||
// page's "+ Create Script" editor or has a schedule.json entry yet.
|
||||
function vv_custom_scripts(): array {
|
||||
$schedule = vv_schedule_load();
|
||||
$scripts = [];
|
||||
@@ -297,7 +304,7 @@ function vv_custom_scripts(): array {
|
||||
}
|
||||
};
|
||||
|
||||
$collect(SCRIPTS_DIR . '/Custom', 'Custom/');
|
||||
$collect(CUSTOM_SCRIPTS_DIR, 'Custom/');
|
||||
|
||||
// Platform adapter custom scripts (Plugin/<platform>/Custom/)
|
||||
foreach (glob(SCRIPTS_DIR . '/Plugin/*/Custom') ?: [] as $customDir) {
|
||||
|
||||
@@ -272,7 +272,7 @@ $runningScripts = array_unique($runningScripts);
|
||||
</div>
|
||||
<div class="vv-children" id="vv-custom-children" style="display:none;">
|
||||
<?php if (empty($customs)): ?>
|
||||
<p class="vv-custom-empty">No custom scripts yet — click <strong>+ Add Script</strong> to create one.</p>
|
||||
<p class="vv-custom-empty">No custom scripts yet — click <strong>+ Create Script</strong> to create one.</p>
|
||||
<?php else: ?>
|
||||
|
||||
<?php // ── Folder groups ──────────────────────────────────────────────
|
||||
@@ -362,7 +362,8 @@ $runningScripts = array_unique($runningScripts);
|
||||
</div><!-- /#vv-sched-cards -->
|
||||
|
||||
<div class="vv-sched-footer">
|
||||
<button class="vv-save-btn vv-add-script-btn" onclick="vvAddScript()">+ Add Script</button>
|
||||
<button class="vv-save-btn vv-add-script-btn" onclick="vvAddScript()">+ Create Script</button>
|
||||
<button class="vv-save-btn vv-import-script-btn" onclick="vvImportScriptOpen()" title="Move an existing script from anywhere on the server into Custom Scripts">+ Import Script</button>
|
||||
<button class="vv-save-btn vv-new-folder-btn" onclick="vvNewFolder()" title="Create a folder in Custom Scripts">+ Folder</button>
|
||||
<button id="vv-arrange-btn" class="vv-save-btn" onclick="vvToggleArrange()" title="Drag scripts between orchestrators">Arrange</button>
|
||||
<button id="vv-arrange-save-btn" class="vv-save-btn vv-arrange-save-btn" onclick="vvSaveArrange()" style="display:none">Save Arrangement</button>
|
||||
@@ -1044,6 +1045,9 @@ function vvLH() {
|
||||
let vvSetupConf = <?= json_encode($vv_setup_conf) ?>;
|
||||
const vvLocalHostConf = <?= json_encode($_vv_local_host_conf) ?>;
|
||||
|
||||
// Where Custom Scripts (Create + Import) actually live — shown in the Import Script dialog
|
||||
window.__vvCustomScriptsDir = <?= json_encode(CUSTOM_SCRIPTS_DIR) ?>;
|
||||
|
||||
if (vvSetupConf) {
|
||||
// Auto-open the setup conf file once the page is ready
|
||||
document.addEventListener('DOMContentLoaded', () => {
|
||||
@@ -1718,6 +1722,191 @@ function vvDeleteScript() {
|
||||
.catch(() => { btn.disabled = false; btn.textContent = '\u{1F5D1} Delete'; });
|
||||
}
|
||||
|
||||
// ── Import Script — browse the whole server, move a chosen .sh into Custom Scripts ──
|
||||
|
||||
let _vvImpSelected = null; // full path of the currently-selected file, or null
|
||||
|
||||
function vvImportScriptOpen() {
|
||||
let modal = document.getElementById('vv-import-modal');
|
||||
if (!modal) modal = _vvImpBuildModal();
|
||||
modal.style.display = 'flex';
|
||||
_vvImpSelected = null;
|
||||
_vvImpUpdateFooter();
|
||||
_vvImpBrowse('/');
|
||||
}
|
||||
|
||||
function vvImportScriptClose() {
|
||||
const modal = document.getElementById('vv-import-modal');
|
||||
if (modal) modal.style.display = 'none';
|
||||
_vvImpSelected = null;
|
||||
}
|
||||
|
||||
function _vvImpBuildModal() {
|
||||
const modal = document.createElement('div');
|
||||
modal.id = 'vv-import-modal';
|
||||
modal.style.cssText = 'display:none;position:fixed;inset:0;z-index:9000;background:rgba(0,0,0,.6);'
|
||||
+ 'align-items:center;justify-content:center;';
|
||||
modal.addEventListener('mousedown', e => { if (e.target === modal) vvImportScriptClose(); });
|
||||
|
||||
const box = document.createElement('div');
|
||||
box.style.cssText = 'background:#111;border:1px solid #222;border-radius:6px;width:560px;max-width:92vw;'
|
||||
+ 'max-height:80vh;display:flex;flex-direction:column;overflow:hidden;';
|
||||
|
||||
box.innerHTML = `
|
||||
<div style="padding:10px 14px;border-bottom:1px solid #1e1e1e;display:flex;align-items:center;justify-content:space-between;">
|
||||
<span style="font-size:13px;font-weight:bold;color:#ccc;">Import Script</span>
|
||||
<button onclick="vvImportScriptClose()" style="background:none;border:none;color:#666;cursor:pointer;font-size:14px;">✕</button>
|
||||
</div>
|
||||
<div style="padding:10px 14px;border-bottom:1px solid #1e1e1e;">
|
||||
<div style="font-size:10px;color:#555;margin-bottom:6px;">
|
||||
Moves the selected script into Custom Scripts (<code>${_vvImpEsc(window.__vvCustomScriptsDir || '')}</code>).
|
||||
The original is removed once the copy is verified.
|
||||
</div>
|
||||
<div style="display:flex;gap:6px;align-items:center;">
|
||||
<input id="vv-imp-path" type="text" value="/" style="flex:1;min-width:0;background:#0a0a0a;border:1px solid #222;
|
||||
color:#aaa;border-radius:3px;padding:4px 8px;font-size:11px;font-family:monospace;"
|
||||
onkeydown="if(event.key==='Enter')_vvImpBrowse(document.getElementById('vv-imp-path').value.trim()||'/')">
|
||||
<button onclick="_vvImpNavUp()" title="Parent" style="background:#111;border:1px solid #222;color:#555;
|
||||
border-radius:3px;padding:4px 8px;cursor:pointer;font-size:12px;flex-shrink:0;">↑</button>
|
||||
<button onclick="_vvImpBrowse(document.getElementById('vv-imp-path').value.trim()||'/')" style="background:#111;
|
||||
border:1px solid #222;color:#4a9eff;border-radius:3px;padding:4px 9px;cursor:pointer;font-size:11px;
|
||||
white-space:nowrap;flex-shrink:0;">↻ Browse</button>
|
||||
</div>
|
||||
<span id="vv-imp-err" style="font-size:9px;color:#ef5350;display:none;margin-top:4px;"></span>
|
||||
</div>
|
||||
<div id="vv-imp-list" style="flex:1;overflow-y:auto;background:#080808;min-height:220px;"></div>
|
||||
<div style="padding:10px 14px;border-top:1px solid #1e1e1e;display:flex;align-items:center;justify-content:space-between;gap:10px;">
|
||||
<span id="vv-imp-selected" style="font-size:10px;color:#555;font-family:monospace;flex:1;min-width:0;
|
||||
overflow:hidden;text-overflow:ellipsis;white-space:nowrap;"></span>
|
||||
<div style="display:flex;gap:6px;flex-shrink:0;">
|
||||
<button onclick="vvImportScriptClose()" class="vv-btn-sm">Cancel</button>
|
||||
<button id="vv-imp-do-btn" class="vv-btn-sm vv-save-script-btn-style" disabled onclick="_vvImpDoImport()">Import</button>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
modal.appendChild(box);
|
||||
document.body.appendChild(modal);
|
||||
return modal;
|
||||
}
|
||||
|
||||
function _vvImpEsc(s) {
|
||||
const d = document.createElement('div');
|
||||
d.textContent = s;
|
||||
return d.innerHTML;
|
||||
}
|
||||
|
||||
function _vvImpBrowse(path) {
|
||||
const listEl = document.getElementById('vv-imp-list');
|
||||
const errEl = document.getElementById('vv-imp-err');
|
||||
const pathEl = document.getElementById('vv-imp-path');
|
||||
errEl.style.display = 'none';
|
||||
listEl.innerHTML = '<div style="padding:10px;color:#444;font-size:11px;">Loading…</div>';
|
||||
|
||||
fetch(`/plugins/varaverk/api/import_script.php?action=browse&path=${encodeURIComponent(path)}&_=${Date.now()}`)
|
||||
.then(r => r.json())
|
||||
.then(d => {
|
||||
if (!d.ok) {
|
||||
listEl.innerHTML = '';
|
||||
errEl.textContent = d.error || 'Browse failed';
|
||||
errEl.style.display = '';
|
||||
return;
|
||||
}
|
||||
pathEl.value = d.path;
|
||||
_vvImpRender(d);
|
||||
})
|
||||
.catch(e => {
|
||||
listEl.innerHTML = '';
|
||||
errEl.textContent = 'Request failed: ' + e;
|
||||
errEl.style.display = '';
|
||||
});
|
||||
}
|
||||
|
||||
function _vvImpNavUp() {
|
||||
const cur = (document.getElementById('vv-imp-path').value || '/').replace(/\/$/, '') || '/';
|
||||
const up = cur === '/' ? '/' : (cur.substring(0, cur.lastIndexOf('/')) || '/');
|
||||
_vvImpBrowse(up);
|
||||
}
|
||||
|
||||
function _vvImpRender(d) {
|
||||
const listEl = document.getElementById('vv-imp-list');
|
||||
let html = '';
|
||||
|
||||
if (d.parent !== null) {
|
||||
const pname = d.parent === '/' ? '/' : (d.parent.replace(/^.*\//, '') || d.parent) + '/';
|
||||
html += `<div class="vv-imp-row" onclick="_vvImpBrowse(${JSON.stringify(d.parent)})"
|
||||
style="padding:5px 12px;font-size:11px;color:#444;font-style:italic;cursor:pointer;font-family:monospace;">↑ ${pname}</div>`;
|
||||
}
|
||||
|
||||
for (const dir of d.dirs) {
|
||||
const name = dir.replace(/^.*\//, '') || dir;
|
||||
html += `<div class="vv-imp-row" onclick="_vvImpBrowse(${JSON.stringify(dir)})" title="${_vvImpEsc(dir)}"
|
||||
style="padding:5px 12px;font-size:11px;color:#666;cursor:pointer;font-family:monospace;">▶ ${_vvImpEsc(name)}</div>`;
|
||||
}
|
||||
|
||||
for (const file of d.files) {
|
||||
const name = file.replace(/^.*\//, '') || file;
|
||||
const sel = file === _vvImpSelected;
|
||||
html += `<div class="vv-imp-row vv-imp-file${sel ? ' vv-imp-file-sel' : ''}" data-path="${_vvImpEsc(file)}"
|
||||
onclick="_vvImpSelectFile(${JSON.stringify(file)})" title="${_vvImpEsc(file)}"
|
||||
style="padding:5px 12px;font-size:11px;cursor:pointer;font-family:monospace;
|
||||
color:${sel ? '#4caf50' : '#4a9eff'};background:${sel ? '#0f1f0f' : 'transparent'};">📄 ${_vvImpEsc(name)}</div>`;
|
||||
}
|
||||
|
||||
if (!d.dirs.length && !d.files.length) {
|
||||
html += '<div style="padding:10px 12px;color:#333;font-size:11px;">— empty —</div>';
|
||||
}
|
||||
|
||||
listEl.innerHTML = html;
|
||||
}
|
||||
|
||||
function _vvImpSelectFile(path) {
|
||||
_vvImpSelected = path;
|
||||
_vvImpUpdateFooter();
|
||||
// Re-render just the highlight without a re-fetch.
|
||||
document.querySelectorAll('#vv-imp-list .vv-imp-file').forEach(el => {
|
||||
const isSel = el.dataset.path === path;
|
||||
el.classList.toggle('vv-imp-file-sel', isSel);
|
||||
el.style.color = isSel ? '#4caf50' : '#4a9eff';
|
||||
el.style.background = isSel ? '#0f1f0f' : 'transparent';
|
||||
});
|
||||
}
|
||||
|
||||
function _vvImpUpdateFooter() {
|
||||
const sel = document.getElementById('vv-imp-selected');
|
||||
const btn = document.getElementById('vv-imp-do-btn');
|
||||
if (!sel || !btn) return;
|
||||
sel.textContent = _vvImpSelected || '';
|
||||
btn.disabled = !_vvImpSelected;
|
||||
}
|
||||
|
||||
function _vvImpDoImport() {
|
||||
if (!_vvImpSelected) return;
|
||||
const dest = (window.__vvCustomScriptsDir || 'Custom Scripts') + '/' + _vvImpSelected.replace(/^.*\//, '');
|
||||
if (!confirm('Move\n ' + _vvImpSelected + '\n→ ' + dest + '\n\nThe original will be deleted once the copy is verified. Continue?')) return;
|
||||
|
||||
const btn = document.getElementById('vv-imp-do-btn');
|
||||
btn.disabled = true;
|
||||
btn.textContent = 'Importing…';
|
||||
|
||||
vvPost('/plugins/varaverk/api/import_script.php', {action: 'import', path: _vvImpSelected})
|
||||
.then(d => {
|
||||
if (!d.ok) {
|
||||
alert('Import failed: ' + (d.error || 'Unknown error'));
|
||||
btn.disabled = false;
|
||||
btn.textContent = 'Import';
|
||||
return;
|
||||
}
|
||||
if (d.warning) alert(d.warning);
|
||||
window.location.reload();
|
||||
})
|
||||
.catch(e => {
|
||||
alert('Import failed: ' + e);
|
||||
btn.disabled = false;
|
||||
btn.textContent = 'Import';
|
||||
});
|
||||
}
|
||||
|
||||
function vvShowConfMode(title) {
|
||||
document.getElementById('vv-suggestions').style.display = 'none';
|
||||
document.getElementById('vv-log-pre').style.display = 'none';
|
||||
|
||||
@@ -593,7 +593,6 @@ Array start (Varaverk event hook):
|
||||
→ php_fpm_max_children.sh WebGUI tuning before first request
|
||||
→ inotify_tuning.sh raise kernel limits before containers start
|
||||
→ docker_network_connect.sh connect containers to extra networks
|
||||
→ claude_startup.sh symlink Claude persistent storage (one-shot)
|
||||
→ start_webhook_listener.sh continuous — Node.js webhook server for arr upgrades
|
||||
→ fallback.sh continuous — mutual fallback state machine
|
||||
|
||||
|
||||
@@ -56,7 +56,6 @@ ARRAY_START_SCRIPTS=(
|
||||
"Plugin/unraid/System_Essentials/php_fpm_max_children.sh" # WebGUI tuning
|
||||
"System_Essentials/inotify_tuning.sh" # before docker_network_connect and continuous scripts
|
||||
"Docker_Essentials/docker_network_connect.sh" # ensure networks + container connections
|
||||
"Tools/claude_startup.sh" # persist Claude data to appdata; re-symlink on boot
|
||||
"Arrs_Stack/start_webhook_listener.sh" # arr upgrade webhook — continuous
|
||||
"Fallback/fallback.sh" # mutual failover — continuous
|
||||
)
|
||||
|
||||
@@ -21,7 +21,6 @@ making any changes.
|
||||
- [smart_long_test.sh](#smart_long_testsh)
|
||||
- [arr_profile_enforcer.sh](#arr_profile_enforcersh)
|
||||
- [webhook_setup.sh](#webhook_setupsh)
|
||||
- [claude_startup.sh](#claude_startupsh)
|
||||
- [ramdisk_stop.sh](#ramdisk_stopsh)
|
||||
- [Adding a New Tool](#adding-a-new-tool)
|
||||
|
||||
@@ -666,41 +665,6 @@ webhook_setup.sh --dry-run
|
||||
|
||||
---
|
||||
|
||||
## claude_startup.sh
|
||||
|
||||
Restores Claude Code's persistent data after an unRAID reboot and optionally launches
|
||||
Claude. Standalone script — no common.sh dependency.
|
||||
|
||||
### Why This Exists
|
||||
|
||||
unRAID's root filesystem lives in RAM — `/root/.claude` and `/root/.local` are wiped on
|
||||
every reboot. This script symlinks both directories back to persistent appdata storage
|
||||
at `/mnt/user/appdata/claude-code/` before launching Claude.
|
||||
|
||||
### First Run Migration
|
||||
|
||||
On first run, if persistent storage is empty, the script migrates from current live locations:
|
||||
|
||||
```
|
||||
/root/.claude → /mnt/user/appdata/claude-code/.claude
|
||||
/root/.local/share/claude → /mnt/user/appdata/claude-code/local/share/claude
|
||||
```
|
||||
|
||||
Subsequent runs skip the migration and only create the symlinks.
|
||||
|
||||
### Calling from array_started.sh
|
||||
|
||||
`array_started.sh` calls `claude_startup.sh` directly (no flags). This sets up the
|
||||
symlinks only — no interactive session is launched. That is the default behavior.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
claude_startup.sh # set up persistent symlinks only (default — used by array_started.sh)
|
||||
claude_startup.sh --launch # set up symlinks and launch Claude interactively
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ramdisk_stop.sh
|
||||
|
||||
|
||||
@@ -69,7 +69,7 @@ something new, write the tool. Store it here. Find it at 2am next time.
|
||||
`bulk_permissions_repair.sh`, `arr_profile_enforcer.sh`
|
||||
|
||||
**Lifecycle Tools** — Backup, setup, and migration support
|
||||
`container_data_export.sh`, `claude_startup.sh`, `ramdisk_stop.sh`, `webhook_setup.sh`
|
||||
`container_data_export.sh`, `ramdisk_stop.sh`, `webhook_setup.sh`
|
||||
|
||||
**Library Sync Bootstrap** — Close the gap between Emby and arr libraries
|
||||
`emby_to_lidarr_sync.sh`, `emby_to_sonarr_sync.sh`, `emby_to_radarr_sync.sh`
|
||||
@@ -113,7 +113,6 @@ The relationship is one-way: Tools act on state that other scripts have written.
|
||||
| `emby_database_repair.sh` | Emby crashing with database errors after power loss or crash | When Emby logs show corruption or repeated crashes |
|
||||
| `zfs_pool_scrub.sh` | Verify ZFS pool integrity — catch silent corruption before it spreads | Monthly, or after any disk or power event |
|
||||
| `smart_long_test.sh` | Run SMART extended self-test on all drives — full sector scan | Monthly via monthly_maintenance.sh, or after any disk event |
|
||||
| `claude_startup.sh` | Claude Code session setup after reboot — symlinks persistent storage | After each unRAID reboot, or called by array_started.sh |
|
||||
| `docker_prune_images.sh` | Remove dangling or unused Docker images accumulated after updates | After update cycles, or when disk space is low |
|
||||
| `ramdisk_stop.sh` | Safely stop the transcode ramdisk — redirect symlink to SSD, unmount, update state | Before re-running ramdisk_setup.sh with new size or thresholds |
|
||||
| `arr_profile_enforcer.sh` | Enforce correct quality profiles across all Sonarr/Radarr libraries | After arr setup, profile changes, or when library was imported with wrong profile |
|
||||
@@ -144,7 +143,6 @@ Situation arises
|
||||
│ smart_long_test ◄── monthly SMART scan / post-disk event │
|
||||
│ arr_profile_enforcer ◄── wrong profiles after import or setup │
|
||||
│ webhook_setup ◄── after install or adding a new arr │
|
||||
│ claude_startup ◄── after each unRAID reboot │
|
||||
│ ramdisk_stop ◄── before ramdisk resize / remount │
|
||||
│ │
|
||||
│ emby_to_lidarr_sync ◄── Lidarr setup / database wipe / gap │
|
||||
|
||||
@@ -1,181 +0,0 @@
|
||||
#!/bin/bash
|
||||
# ==============================================================================================
|
||||
# ================================= Claude Code Startup ========================================
|
||||
# ==============================================================================================
|
||||
#
|
||||
# PURPOSE
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Restores Claude Code's persistent data after an Unraid reboot.
|
||||
# /root is RAM — wiped on every boot. This script re-creates symlinks so
|
||||
# Claude's memory, sessions, settings, and binary survive across reboots.
|
||||
#
|
||||
# Storage mode is read from the host conf file:
|
||||
#
|
||||
# HOST*_STORAGE_MODE_INTERNAL=true → Internal (boot) mode
|
||||
# .claude data → /boot/config/claude
|
||||
# binary → /boot/config/claude-bin
|
||||
# No array dependency — runs even before array mounts.
|
||||
#
|
||||
# HOST*_STORAGE_MODE_INTERNAL=false → Appdata mode
|
||||
# .claude data → /mnt/user/appdata/claude-code/.claude
|
||||
# binary → /mnt/user/appdata/claude-code/local/share/claude
|
||||
# Requires array to be mounted.
|
||||
#
|
||||
# On first run in either mode, migrates any existing live data to persistent
|
||||
# storage. Subsequent runs only re-create the symlinks.
|
||||
#
|
||||
# Standalone script — no common.sh dependency. Safe to run directly from
|
||||
# terminal or from array_started.sh.
|
||||
#
|
||||
# ==============================================================================================
|
||||
# DESIGN PRINCIPLES
|
||||
# ==============================================================================================
|
||||
#
|
||||
# Mode-Aware Paths
|
||||
# Storage mode is read from host*.conf before any symlink is created. Internal
|
||||
# mode (boot) requires no array — symlinks resolve immediately. Appdata mode
|
||||
# requires the array to be mounted before symlinks are useful.
|
||||
#
|
||||
# Migrate on First Run
|
||||
# If live data already exists at /root/.claude and the persistent store is
|
||||
# empty, the live data is moved to persistent storage on first run. Subsequent
|
||||
# runs only re-create the symlinks — migration is one-time.
|
||||
#
|
||||
# No common.sh Dependency
|
||||
# Runs before load_config.sh is available (early in ARRAY_START_SCRIPTS).
|
||||
# All logic is self-contained — no ecosystem functions used.
|
||||
#
|
||||
# ==============================================================================================
|
||||
# OPERATIONAL SAFEGUARDS
|
||||
# ==============================================================================================
|
||||
#
|
||||
# Conf probe — reads storage mode from Configurations/host*.conf directly
|
||||
# Migration guard — only migrates if persistent store is empty; never overwrites
|
||||
# Symlink-safe — removes existing symlink before re-creating; won't error on re-run
|
||||
#
|
||||
# ==============================================================================================
|
||||
# RUNTIME MODES
|
||||
# ==============================================================================================
|
||||
#
|
||||
# claude_startup.sh
|
||||
# Set up persistent symlinks only — default, used by array_started.sh on boot.
|
||||
#
|
||||
# claude_startup.sh --launch
|
||||
# Set up persistent symlinks and launch Claude interactively.
|
||||
#
|
||||
# ==============================================================================================
|
||||
|
||||
LAUNCH=false
|
||||
[[ "$1" == "--launch" ]] && LAUNCH=true
|
||||
|
||||
_log() { echo " ✅ $*"; }
|
||||
_warn() { echo " ⚠️ $*"; }
|
||||
_err() { echo " ❌ $*" >&2; }
|
||||
|
||||
echo ""
|
||||
echo "━━━ Claude Code Startup ━━━"
|
||||
echo ""
|
||||
|
||||
# ── Detect storage mode ───────────────────────────────────────────────────────────────────────
|
||||
CONF_DIR="/boot/config/plugins/varaverk/Configurations"
|
||||
STORAGE_INTERNAL=false
|
||||
for _conf in "$CONF_DIR"/host*.conf; do
|
||||
[[ -f "$_conf" ]] || continue
|
||||
if grep -q "_STORAGE_MODE_INTERNAL=true" "$_conf" 2>/dev/null; then
|
||||
STORAGE_INTERNAL=true
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ "$STORAGE_INTERNAL" == true ]]; then
|
||||
CLAUDE_DATA="/boot/config/plugins/varaverk/claude-data"
|
||||
CLAUDE_BIN="/boot/config/plugins/varaverk/claude-bin"
|
||||
_log "Storage mode: internal boot"
|
||||
else
|
||||
PERSIST_DIR="/mnt/user/appdata/claude-code"
|
||||
CLAUDE_DATA="$PERSIST_DIR/.claude"
|
||||
CLAUDE_BIN="$PERSIST_DIR/local/share/claude"
|
||||
_log "Storage mode: appdata"
|
||||
fi
|
||||
|
||||
# ── Array check (appdata mode only) ──────────────────────────────────────────────────────────
|
||||
if [[ "$STORAGE_INTERNAL" == false ]]; then
|
||||
if ! mountpoint -q /mnt/user 2>/dev/null; then
|
||||
_err "Array not mounted — /mnt/user not available (required for appdata mode)"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── Create persistent dirs ────────────────────────────────────────────────────────────────────
|
||||
mkdir -p "$CLAUDE_DATA" "$CLAUDE_BIN"
|
||||
|
||||
# ── Migrate from old /boot/config/claude location (internal mode only) ───────────────────────
|
||||
if [[ "$STORAGE_INTERNAL" == true && -d /boot/config/claude && "$CLAUDE_DATA" != "/boot/config/claude" ]]; then
|
||||
if [[ -z "$(ls -A "$CLAUDE_DATA" 2>/dev/null)" ]]; then
|
||||
_warn "Migrating old /boot/config/claude → $CLAUDE_DATA"
|
||||
cp -a /boot/config/claude/. "$CLAUDE_DATA/"
|
||||
_log "Migrated .claude data from old location"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── Migrate .claude on first run ──────────────────────────────────────────────────────────────
|
||||
if [[ ! -L /root/.claude && -d /root/.claude ]]; then
|
||||
_warn "First run — migrating /root/.claude → $CLAUDE_DATA"
|
||||
cp -a /root/.claude/. "$CLAUDE_DATA/"
|
||||
rm -rf /root/.claude
|
||||
_log "Migrated .claude (memory, sessions, settings)"
|
||||
elif [[ -z "$(ls -A "$CLAUDE_DATA" 2>/dev/null)" && -d /root/.claude ]]; then
|
||||
_warn "Persistent storage empty — copying current .claude data"
|
||||
cp -a /root/.claude/. "$CLAUDE_DATA/"
|
||||
_log "Copied .claude data to persistent storage"
|
||||
fi
|
||||
|
||||
# ── Migrate Claude binaries on first run ──────────────────────────────────────────────────────
|
||||
if [[ ! -L /root/.local/share/claude && -d /root/.local/share/claude ]]; then
|
||||
_warn "First run — migrating Claude binaries → $CLAUDE_BIN"
|
||||
cp -a /root/.local/share/claude/. "$CLAUDE_BIN/"
|
||||
_log "Migrated Claude binaries"
|
||||
fi
|
||||
|
||||
# ── Create symlinks ───────────────────────────────────────────────────────────────────────────
|
||||
mkdir -p /root/.local/share /root/.local/bin
|
||||
|
||||
[[ -d /root/.claude && ! -L /root/.claude ]] && rm -rf /root/.claude
|
||||
ln -sfn "$CLAUDE_DATA" /root/.claude
|
||||
_log ".claude → $CLAUDE_DATA"
|
||||
|
||||
[[ -d /root/.local/share/claude && ! -L /root/.local/share/claude ]] && rm -rf /root/.local/share/claude
|
||||
ln -sfn "$CLAUDE_BIN" /root/.local/share/claude
|
||||
_log "claude binary → $CLAUDE_BIN"
|
||||
|
||||
# ── Symlink CLAUDE.md ─────────────────────────────────────────────────────────────────────────
|
||||
CLAUDE_MD="/boot/config/plugins/varaverk/CLAUDE.md"
|
||||
if [[ -f "$CLAUDE_MD" ]]; then
|
||||
ln -sfn "$CLAUDE_MD" /root/CLAUDE.md
|
||||
_log "CLAUDE.md → $CLAUDE_MD"
|
||||
else
|
||||
_warn "CLAUDE.md not found at $CLAUDE_MD — skipping symlink"
|
||||
fi
|
||||
|
||||
# ── Point the claude binary at the latest installed version ───────────────────────────────────
|
||||
LATEST=$(ls "$CLAUDE_BIN/versions/" 2>/dev/null | sort -V | tail -1)
|
||||
if [[ -z "$LATEST" ]]; then
|
||||
_err "No Claude versions found in $CLAUDE_BIN/versions/"
|
||||
_err "Install Claude Code first: npm install -g @anthropic-ai/claude-code"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
ln -sfn "$CLAUDE_BIN/versions/$LATEST" /root/.local/bin/claude
|
||||
_log "claude v$LATEST ready"
|
||||
|
||||
echo ""
|
||||
|
||||
if [[ "$LAUNCH" == false ]]; then
|
||||
_log "Setup complete — run 'claude' to start"
|
||||
echo ""
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# ── Launch ────────────────────────────────────────────────────────────────────────────────────
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
exec claude
|
||||
@@ -1537,6 +1537,64 @@ _release_rsync_on_exit() {
|
||||
fi
|
||||
}
|
||||
|
||||
# ==============================================================================================
|
||||
# ── ORCHESTRATOR CHILD EXECUTION ──────────────────────────────────────────────────────────────
|
||||
# ==============================================================================================
|
||||
# Shared child-script runner for Orchestrators/*.sh. Replaces the run_job()/run_watchdog()
|
||||
# copies that used to be hand-duplicated (with small inconsistencies) into each orchestrator —
|
||||
# one implementation now, one place to fix a bug or extend behavior (e.g. per-child logging).
|
||||
#
|
||||
# Usage — caller declares its own tracking arrays before the loop:
|
||||
# JOB_PASS=()
|
||||
# JOB_FAIL=()
|
||||
# for entry in "${SOME_SCRIPTS[@]}"; do
|
||||
# run_orch_child "$entry"
|
||||
# done
|
||||
#
|
||||
# $entry is "relative/path.sh [extra args...]" — the same format every master.conf
|
||||
# script-list array already uses. Resolved against $ECOSYSTEM_ROOT, which the caller
|
||||
# must set before calling this (the absolute repo root — "$(cd "$SCRIPT_DIR/.." && pwd)").
|
||||
#
|
||||
# --dry-run / --log are threaded down automatically from $DRY_RUN / $ENABLE_LOGGING —
|
||||
# never $VERBOSE, which nothing in this codebase ever assigns.
|
||||
|
||||
run_orch_child() {
|
||||
local entry="$1"
|
||||
local script_args script_path script_name label extra_args run_args
|
||||
|
||||
read -r -a script_args <<< "$entry"
|
||||
script_path="$ECOSYSTEM_ROOT/${script_args[0]}"
|
||||
script_name=$(basename "${script_args[0]}")
|
||||
extra_args=("${script_args[@]:1}")
|
||||
label="$script_name"
|
||||
[[ -n "${extra_args[*]}" ]] && label="$script_name ${extra_args[*]}"
|
||||
|
||||
if [[ ! -f "$script_path" ]]; then
|
||||
error "$script_name — not found at $script_path"
|
||||
JOB_FAIL+=("$label")
|
||||
return 1
|
||||
fi
|
||||
[[ ! -x "$script_path" ]] && chmod +x "$script_path"
|
||||
|
||||
run_args=("${extra_args[@]}")
|
||||
[[ "$DRY_RUN" == true ]] && run_args+=("--dry-run")
|
||||
[[ "$ENABLE_LOGGING" == true ]] && run_args+=("--log")
|
||||
|
||||
local _start _ec
|
||||
_start=$(date +%s)
|
||||
log "Running: $label"
|
||||
if bash "$script_path" "${run_args[@]}"; then
|
||||
log "$script_name — done in $(format_duration $(( $(date +%s) - _start )))"
|
||||
JOB_PASS+=("$label")
|
||||
return 0
|
||||
else
|
||||
_ec=$?
|
||||
error "$label — failed (exit $_ec, $(format_duration $(( $(date +%s) - _start ))))"
|
||||
JOB_FAIL+=("$label")
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# ==============================================================================================
|
||||
# ── PATH TRANSLATION ──────────────────────────────────────────────────────────────────────────
|
||||
# ==============================================================================================
|
||||
|
||||
Reference in New Issue
Block a user