Varaverk: arrange mode, folder management, rsync standalone, layout fixes

Scheduler UI:
- Arrange mode: drag scripts between orchs and reorder within arrays; right
  panel shows unassigned script pool; Save Arrangement commits to master.conf
- + Folder: named collapsible subfolders for Custom Scripts stored in schedule.json
- Rsync children: hide Run/Dry Run/Log/location when orch is ON; show standalone
  location + cron controls when orch is OFF; cron only fires when both filled
- Non-conf-managed children (transcode): toggles now show enabled when orch is on
- Right panel height sync: fix ResizeObserver feedback loop via align-self:flex-start
  on left panel and left.offsetHeight in vvFitRight
- How do I use this: updated to cover arrange, folders, rsync standalone, transcode

New API endpoints:
- board.php, clearlock.php, movescript.php, rawconf.php, readscript.php
- reorderarray.php, rsync_standalone.php, savefolders.php

run.php / dryrun.php: accept optional --location= arg for standalone rsync calls
This commit is contained in:
Gmer4Lfe
2026-05-25 21:46:12 -04:00
parent 6b30768853
commit 6078af0dbb
17 changed files with 2667 additions and 196 deletions
Regular → Executable
+30
View File
@@ -45,3 +45,33 @@ when a owner offboards and there is more than 1 server left, one needs to become
look into an overall, setup script, pull as many vars as possible without user having to add. like docker names, and so on.
fallback, should partnership be authoritive. like no partners fallback fails and after x amount of time dissables untill a new remote is found
i need a way to get rid of git. the apllication becomes the source for each mesh cluster... so it gets dl from github, or ca. i need the app layer to be able to open and edit settings, like open conf first. things have to filled out for it to work. , and partnership deploy to second server, third server........... and push the correct host.conf. and only update that file if new things are added or completely wiped....... cant mod a var nut can rearrang the script if structure changes again......... even if git it used as a core and is auto set from the start..... the app layer is king moving forward, and scipts are cores that just get called. i have all the flexabiliy in the world being able to change things and the app layers call and use it............
we may need a startup guide.. first launch, open master.conf. then host1.
we need to updata the folder Unraid_Scripts to Varaverk every where and update git script, and update all traces and scripts that call it and update the user script plugin......
with this app layer i think we modle the user script to our system and no longer user facing..... maybe
uide. owner sets up first.. establishes himself in master.conf as host1.... sets up the rest of master.conf.. little changes may be needed.. then fills out host1.conf.... then makes sure machines are shared in tail scale.. start onboard.......... then host2, host3, what ever is in master.conf it will reachout, uinstall correct containers,and leave the state file... install varaverk on host 2 and host 3. on install it sees the state file and defuaults to mirror. they fill out their setting and all should start working.......
working on getting this down to just web ui. on intitial no state file opens master.conf, locks you out of enabling any orchs untill set up.....
+3 -3
View File
@@ -3,7 +3,7 @@
# ============================ Watchdog Orchestrator ===========================================
# ==============================================================================================
# Runs WATCHDOG_ORCHESTRATOR_SCRIPTS in order each cron cycle.
# Schedule: * * * * * (every minute via User Scripts plugin)
# Schedule: */15 * * * * (every 15 minutes)
#
# ── EXECUTION ORDER ───────────────────────────────────────────────────────────────────────────
# Driven by WATCHDOG_ORCHESTRATOR_SCRIPTS in master.conf — add, remove, or reorder there.
@@ -37,7 +37,7 @@
# WATCHDOG_ORCHESTRATOR_HEARTBEAT_HOURS — heartbeat interval in hours
#
# ── USAGE ─────────────────────────────────────────────────────────────────────────────────────
# watchdog_orchestrator.sh — normal run (called by cron every minute)
# watchdog_orchestrator.sh — normal run (called by cron every 15 minutes)
# watchdog_orchestrator.sh --dry-run — pass --dry-run to all sub-scripts
# watchdog_orchestrator.sh --status — show script paths and current grace state
# watchdog_orchestrator.sh --log — verbose output from all sub-scripts
@@ -104,7 +104,7 @@ if [[ "$SHOW_STATUS" == true ]]; then
done
echo ""
echo " Schedule: * * * * * (every minute via User Scripts)"
echo " Schedule: */15 * * * * (every 15 minutes)"
echo " Heartbeat: ${WATCHDOG_ORCHESTRATOR_HEARTBEAT:-true} / every ${WATCHDOG_ORCHESTRATOR_HEARTBEAT_HOURS:-1}hr"
echo "━━━━━━━━━━━━━━━━━━━━━━━"
exit 0
@@ -9,7 +9,7 @@ $pluginDir = "$docroot/plugins/$plugin";
// Determine active tab
$tab = $_GET['tab'] ?? 'monitor';
$validTabs = ['monitor', 'scheduler', 'config', 'docs'];
$validTabs = ['monitor', 'scheduler'];
if (!in_array($tab, $validTabs)) $tab = 'monitor';
?>
@@ -0,0 +1,88 @@
<?php
// Live board data: locks, recent errors, partner reachability.
header('Content-Type: application/json');
require_once dirname(__DIR__) . '/include/config.php';
$out = ['ok' => true];
// ── Active locks ──────────────────────────────────────────────────────────
$lockDir = '/tmp/unraid_locks';
$locks = [];
if (is_dir($lockDir)) {
foreach (glob($lockDir . '/*.lock') ?: [] as $lf) {
$name = basename($lf, '.lock');
$age = time() - (int)filemtime($lf);
$content = trim(file_get_contents($lf) ?: '');
// content is "PID:scriptname" — extract PID
$pid = preg_match('/^(\d+)/', $content, $pm) ? $pm[1] : '';
// Skip if PID is still alive (it's legitimately running)
if ($pid && file_exists("/proc/$pid")) continue;
$locks[] = ['name' => $name, 'file' => basename($lf), 'age' => $age];
}
}
$out['locks'] = $locks;
// ── Recent errors ──────────────────────────────────────────────────────────
$errors = [];
if (is_dir(LOG_DIR)) {
$cutoff = time() - 7 * 86400; // only logs touched in last 7 days
foreach (glob(LOG_DIR . '/*.log') ?: [] as $lf) {
if (filemtime($lf) < $cutoff) continue;
$script = basename($lf, '.log');
$lines = array_slice(@file($lf) ?: [], -200);
$lastErr = null;
foreach (array_reverse($lines) as $raw) {
// Strip ANSI escape codes
$clean = preg_replace('/\033\[[0-9;]*[mK]/', '', rtrim($raw));
if (!$clean) continue;
if (preg_match('/\[(?:ERROR|WARN|CRITICAL|FAILED)\]/i', $clean) ||
preg_match('/\b(?:ERROR|CRITICAL|FAILED):\s/i', $clean) ||
str_contains($clean, '✗') ||
(str_contains($clean, '⚠') && !str_contains($clean, '♥'))) {
$lastErr = mb_substr($clean, 0, 220);
break;
}
}
if ($lastErr !== null) {
$errors[] = ['script' => $script, 'line' => $lastErr, 'ts' => (int)filemtime($lf)];
}
}
usort($errors, fn($a, $b) => $b['ts'] - $a['ts']);
}
$out['errors'] = array_slice($errors, 0, 20);
// ── Partner reachability ───────────────────────────────────────────────────
$cacheFile = '/tmp/vv_partner_cache.json';
$cacheTtl = 30;
$partnerData = null;
if (file_exists($cacheFile) && (time() - (int)filemtime($cacheFile)) < $cacheTtl) {
$partnerData = json_decode(file_get_contents($cacheFile), true);
} else {
$confRaw = vv_read_conf_raw('master.conf');
preg_match('/^\s*HOST1(?:_NAME)?\s*=\s*["\']?(\S+?)["\']?\s*(?:#.*)?$/m', $confRaw, $m1);
preg_match('/^\s*HOST2(?:_NAME)?\s*=\s*["\']?(\S+?)["\']?\s*(?:#.*)?$/m', $confRaw, $m2);
$host1 = $m1[1] ?? '';
$host2 = $m2[1] ?? '';
$mine = vv_get_hostname();
$partnerHost = null;
if ($mine && $host1 && strcasecmp($mine, $host1) === 0) $partnerHost = $host2;
if ($mine && $host2 && strcasecmp($mine, $host2) === 0) $partnerHost = $host1;
if ($partnerHost) {
$start = microtime(true);
$result = shell_exec('ping -c1 -W1 ' . escapeshellarg($partnerHost) . ' 2>&1');
$elapsed = (int)round((microtime(true) - $start) * 1000);
$reached = str_contains((string)$result, '1 received')
|| str_contains((string)$result, '1 packets received');
$partnerData = [
'host' => $partnerHost,
'reachable' => $reached,
'latency' => $reached ? $elapsed : null,
];
@file_put_contents($cacheFile, json_encode($partnerData));
}
}
$out['partner'] = $partnerData;
echo json_encode($out);
@@ -0,0 +1,14 @@
<?php
header('Content-Type: application/json');
if ($_SERVER['REQUEST_METHOD'] !== 'POST') {
echo json_encode(['ok' => false, 'error' => 'POST only']);
exit;
}
$file = basename($_POST['file'] ?? '');
if (!$file || !preg_match('/^[a-zA-Z0-9_\-]+\.lock$/', $file)) {
echo json_encode(['ok' => false, 'error' => 'Invalid filename']);
exit;
}
$path = '/tmp/unraid_locks/' . $file;
if (file_exists($path)) @unlink($path);
echo json_encode(['ok' => true]);
@@ -21,7 +21,14 @@ if (!is_dir($logDir)) mkdir($logDir, 0755, true);
file_put_contents($logFile, "\n── " . date('Y-m-d H:i:s') . " [DRY RUN] ──────────────────────\n", FILE_APPEND);
$flags = vv_job_flags($id);
exec('nohup env DRY_RUN=1 bash ' . escapeshellarg($script) . ($flags ? " $flags" : '') . ' >> ' . escapeshellarg($logFile) . ' 2>&1 </dev/null &');
$location = trim($_POST['location'] ?? '');
if ($location && (!str_starts_with($location, '/') || str_contains($location, '..') || preg_match('/[\x00\n\r]/', $location))) {
echo json_encode(['ok' => false, 'error' => 'Invalid location']);
exit;
}
$flags = vv_job_flags($id);
$locArg = $location ? ' ' . escapeshellarg('--location=' . $location) : '';
exec('nohup env DRY_RUN=1 bash ' . escapeshellarg($script) . ($flags ? " $flags" : '') . $locArg . ' >> ' . escapeshellarg($logFile) . ' 2>&1 </dev/null &');
echo json_encode(['ok' => true]);
@@ -0,0 +1,78 @@
<?php
// Move a script between *_SCRIPTS arrays in master.conf.
// POST: script (rel path), to_array (var name, or '' to remove from all arrays).
header('Content-Type: application/json');
require_once dirname(__DIR__) . '/include/config.php';
if ($_SERVER['REQUEST_METHOD'] !== 'POST') {
echo json_encode(['ok' => false, 'error' => 'POST only']);
exit;
}
$script = trim($_POST['script'] ?? '');
$toArray = trim($_POST['to_array'] ?? '');
if (!$script || str_contains($script, '..') || !preg_match('/^[A-Za-z0-9_.\-\/]+\.sh$/', $script)) {
echo json_encode(['ok' => false, 'error' => 'Invalid script']);
exit;
}
if ($toArray && !preg_match('/^[A-Z_]+_SCRIPTS$/', $toArray)) {
echo json_encode(['ok' => false, 'error' => 'Invalid array name']);
exit;
}
$confPath = CONF_DIR . '/master.conf';
if (!file_exists($confPath)) {
echo json_encode(['ok' => false, 'error' => 'master.conf not found']);
exit;
}
$lines = file($confPath, FILE_KEEP_BLANK_LINES);
if (!$lines) {
echo json_encode(['ok' => false, 'error' => 'Could not read master.conf']);
exit;
}
$scriptEsc = preg_quote($script, '/');
$removedLine = null;
$inArray = false;
// Step 1: find and remove the script line from whatever array it is currently in.
$newLines = [];
foreach ($lines as $line) {
if (preg_match('/^\s*[A-Z_]+_SCRIPTS\s*=\s*\(/', $line)) $inArray = true;
if ($inArray && preg_match('/^\s*\)\s*(?:#.*)?$/', $line) && !str_contains($line, '(')) $inArray = false;
if ($inArray && preg_match('/^\s*(?:#\s*)?"' . $scriptEsc . '(?:\s[^"]*)?"/', $line)) {
$removedLine = ' "' . $script . '"' . "\n"; // normalise indentation when re-inserting
continue; // drop from current location
}
$newLines[] = $line;
}
// Step 2: insert into target array (if specified).
if ($toArray) {
$resultLines = [];
$inTarget = false;
$inserted = false;
foreach ($newLines as $line) {
if (preg_match('/^\s*' . preg_quote($toArray, '/') . '\s*=\s*\(/', $line)) $inTarget = true;
if ($inTarget && !$inserted && preg_match('/^\s*\)\s*(?:#.*)?$/', $line) && !str_contains($line, '(')) {
$resultLines[] = $removedLine ?? (' "' . $script . '"' . "\n");
$inTarget = false;
$inserted = true;
}
$resultLines[] = $line;
}
if (!$inserted) {
echo json_encode(['ok' => false, 'error' => 'Target array "' . $toArray . '" not found in master.conf']);
exit;
}
$newLines = $resultLines;
}
if (file_put_contents($confPath, implode('', $newLines)) === false) {
echo json_encode(['ok' => false, 'error' => 'Write failed']);
exit;
}
echo json_encode(['ok' => true]);
@@ -0,0 +1,29 @@
<?php
// Raw conf read/write — respects per-host file visibility from vv_get_conf_files().
header('Content-Type: application/json');
require_once dirname(__DIR__) . '/include/config.php';
$allowed = vv_get_conf_files();
if ($_SERVER['REQUEST_METHOD'] === 'GET') {
$file = trim($_GET['file'] ?? 'master.conf');
if (!in_array($file, $allowed, true) || str_contains($file, '..')) {
echo json_encode(['ok' => false, 'error' => 'Not allowed']);
exit;
}
echo json_encode(['ok' => true, 'content' => vv_read_conf_raw($file), 'file' => $file, 'allowed' => $allowed]);
exit;
}
if ($_SERVER['REQUEST_METHOD'] === 'POST') {
$file = trim($_POST['file'] ?? '');
$content = $_POST['content'] ?? '';
if (!in_array($file, $allowed, true) || str_contains($file, '..')) {
echo json_encode(['ok' => false, 'error' => 'Not allowed']);
exit;
}
echo json_encode(['ok' => vv_write_conf_raw($file, $content)]);
exit;
}
echo json_encode(['ok' => false, 'error' => 'Method not allowed']);
@@ -0,0 +1,20 @@
<?php
// Read-only endpoint: return full content of any script in SCRIPTS_DIR.
header('Content-Type: application/json');
require_once dirname(__DIR__) . '/include/config.php';
$id = trim($_GET['id'] ?? '');
// Must be relative path within SCRIPTS_DIR, no traversal, must end in .sh or .md
if (!$id || str_contains($id, '..') || !preg_match('/^[A-Za-z0-9_.\-\/]+\.(sh|md)$/', $id)) {
echo json_encode(['ok' => false, 'error' => 'Invalid id']);
exit;
}
$path = SCRIPTS_DIR . '/' . $id;
if (!file_exists($path)) {
echo json_encode(['ok' => false, 'error' => 'Not found']);
exit;
}
echo json_encode(['ok' => true, 'content' => file_get_contents($path)]);
@@ -0,0 +1,109 @@
<?php
// Rewrite a *_SCRIPTS array in master.conf with a new script order.
// POST: array_name (e.g. "DAILY_SCRIPTS"), scripts (JSON: [{"id":"rel/path.sh","enabled":true}, ...])
// Preserves original entry lines (including inline flags/args) where possible.
// Scripts absent from the new list are dropped; new scripts are added as fresh entries.
header('Content-Type: application/json');
require_once dirname(__DIR__) . '/include/config.php';
if ($_SERVER['REQUEST_METHOD'] !== 'POST') {
echo json_encode(['ok' => false, 'error' => 'POST only']);
exit;
}
$arrayName = trim($_POST['array_name'] ?? '');
$raw = $_POST['scripts'] ?? '';
$decoded = json_decode($raw, true);
if (!$arrayName || !preg_match('/^[A-Z_]+_SCRIPTS$/', $arrayName)) {
echo json_encode(['ok' => false, 'error' => 'Invalid array_name']);
exit;
}
if (!is_array($decoded)) {
echo json_encode(['ok' => false, 'error' => 'Invalid scripts JSON']);
exit;
}
// Validate each entry
$order = [];
foreach ($decoded as $item) {
$id = trim((string)($item['id'] ?? ''));
$enabled = (bool)($item['enabled'] ?? true);
if (!$id || str_contains($id, '..') || !preg_match('/^[A-Za-z0-9_.\-\/]+\.sh$/', $id)) continue;
$order[] = ['id' => $id, 'enabled' => $enabled];
}
$confPath = CONF_DIR . '/master.conf';
if (!file_exists($confPath)) {
echo json_encode(['ok' => false, 'error' => 'master.conf not found']);
exit;
}
$lines = file($confPath, FILE_KEEP_BLANK_LINES);
if (!$lines) {
echo json_encode(['ok' => false, 'error' => 'Could not read master.conf']);
exit;
}
// Find the array block and extract original entry lines keyed by script path.
$arrayEsc = preg_quote($arrayName, '/');
$blockStart = null;
$blockEnd = null;
$depth = 0;
$origEntries = []; // path → original trimmed content line (e.g. '"Daily/script.sh --flag"')
foreach ($lines as $i => $line) {
if ($blockStart === null) {
if (preg_match('/^\s*' . $arrayEsc . '\s*=\s*\(/', $line)) {
$blockStart = $i;
$depth = 1;
}
continue;
}
$depth += substr_count($line, '(');
$depth -= substr_count($line, ')');
if ($depth <= 0) {
$blockEnd = $i;
break;
}
// Collect entries (enabled and commented)
if (preg_match('/^\s*(?:#\s*)?"([^"]+)"/', $line, $m)) {
$parts = preg_split('/\s+/', trim($m[1]));
$path = $parts[0] ?? '';
if (substr($path, -3) === '.sh' && !isset($origEntries[$path])) {
// Store the full quoted expression (may include flags after the path)
$origEntries[$path] = '"' . $m[1] . '"';
}
}
}
if ($blockStart === null || $blockEnd === null) {
echo json_encode(['ok' => false, 'error' => "Array $arrayName not found in master.conf"]);
exit;
}
// Build replacement block lines
$newBlockLines = [];
// Preserve the opening line exactly (e.g. "DAILY_SCRIPTS=(")
$newBlockLines[] = $lines[$blockStart];
foreach ($order as $item) {
$id = $item['id'];
$enabled = $item['enabled'];
$entry = $origEntries[$id] ?? '"' . $id . '"';
$prefix = $enabled ? ' ' : ' # ';
$newBlockLines[] = $prefix . $entry . "\n";
}
// Preserve the closing line exactly
$newBlockLines[] = $lines[$blockEnd];
// Replace the original block in $lines
array_splice($lines, $blockStart, $blockEnd - $blockStart + 1, $newBlockLines);
if (file_put_contents($confPath, implode('', $lines)) === false) {
echo json_encode(['ok' => false, 'error' => 'Write failed']);
exit;
}
echo json_encode(['ok' => true]);
@@ -0,0 +1,47 @@
<?php
// Save rsync standalone config (location + cron) for a specific rsync tier.
// POST: flag_name (e.g. "DAILY_RSYNC_ENABLED"), orch_id, location, cron
// Stored in schedule.json under "__rsync_{FLAG_NAME}".
// Triggers a cron rebuild so the standalone entry takes effect immediately.
header('Content-Type: application/json');
require_once dirname(__DIR__) . '/include/config.php';
require_once dirname(__DIR__) . '/include/scheduler.php';
if ($_SERVER['REQUEST_METHOD'] !== 'POST') {
echo json_encode(['ok' => false, 'error' => 'POST only']);
exit;
}
$flagName = trim($_POST['flag_name'] ?? '');
$orchId = trim($_POST['orch_id'] ?? '');
$location = trim($_POST['location'] ?? '');
$cron = trim($_POST['cron'] ?? '');
if (!$flagName || !preg_match('/^[A-Z_]+_RSYNC_ENABLED$/', $flagName)) {
echo json_encode(['ok' => false, 'error' => 'Invalid flag_name']);
exit;
}
if ($orchId && (str_contains($orchId, '..') || !preg_match('/^[A-Za-z0-9_.\-\/]+\.sh$/', $orchId))) {
echo json_encode(['ok' => false, 'error' => 'Invalid orch_id']);
exit;
}
if ($location && (!str_starts_with($location, '/') || str_contains($location, '..') || preg_match('/[\x00\n\r]/', $location))) {
echo json_encode(['ok' => false, 'error' => 'Invalid location']);
exit;
}
$key = '__rsync_' . $flagName;
$schedule = vv_schedule_load();
$schedule[$key] = [
'flag_name' => $flagName,
'orch_id' => $orchId,
'location' => $location,
'cron' => $cron,
];
if (!vv_schedule_save($schedule)) {
echo json_encode(['ok' => false, 'error' => 'Write failed']);
exit;
}
vv_cron_rebuild($schedule);
echo json_encode(['ok' => true]);
@@ -21,7 +21,14 @@ if (!is_dir($logDir)) mkdir($logDir, 0755, true);
file_put_contents($logFile, "\n── " . date('Y-m-d H:i:s') . " ──────────────────────\n", FILE_APPEND);
$flags = vv_job_flags($id);
exec('nohup bash ' . escapeshellarg($script) . ($flags ? " $flags" : '') . ' >> ' . escapeshellarg($logFile) . ' 2>&1 </dev/null &');
$location = trim($_POST['location'] ?? '');
if ($location && (!str_starts_with($location, '/') || str_contains($location, '..') || preg_match('/[\x00\n\r]/', $location))) {
echo json_encode(['ok' => false, 'error' => 'Invalid location']);
exit;
}
$flags = vv_job_flags($id);
$locArg = $location ? ' ' . escapeshellarg('--location=' . $location) : '';
exec('nohup bash ' . escapeshellarg($script) . ($flags ? " $flags" : '') . $locArg . ' >> ' . escapeshellarg($logFile) . ' 2>&1 </dev/null &');
echo json_encode(['ok' => true]);
@@ -0,0 +1,41 @@
<?php
// Save custom-script folder assignments to schedule.json (__folders key).
// POST: folders (JSON-encoded object: {"FolderName": ["Custom/script.sh", ...]})
header('Content-Type: application/json');
require_once dirname(__DIR__) . '/include/config.php';
require_once dirname(__DIR__) . '/include/scheduler.php';
if ($_SERVER['REQUEST_METHOD'] !== 'POST') {
echo json_encode(['ok' => false, 'error' => 'POST only']);
exit;
}
$raw = $_POST['folders'] ?? '';
$decoded = json_decode($raw, true);
if (!is_array($decoded)) {
echo json_encode(['ok' => false, 'error' => 'Invalid JSON']);
exit;
}
$clean = [];
foreach ($decoded as $name => $scripts) {
$name = trim((string)$name);
if (!$name || strlen($name) > 80) continue;
if (!is_array($scripts)) continue;
$cleanScripts = [];
foreach ($scripts as $s) {
$s = trim((string)$s);
if (!$s || str_contains($s, '..') || !preg_match('/^[A-Za-z0-9_.\-\/]+\.sh$/', $s)) continue;
$cleanScripts[] = $s;
}
$clean[$name] = $cleanScripts;
}
$schedule = vv_schedule_load();
$schedule['__folders'] = $clean;
if (!vv_schedule_save($schedule)) {
echo json_encode(['ok' => false, 'error' => 'Write failed']);
exit;
}
echo json_encode(['ok' => true]);
@@ -98,21 +98,47 @@
/* Two-panel layout */
#vv-sched-layout { display: flex; gap: 16px; align-items: stretch; }
#vv-sched-left { flex: 1 1 0; min-width: 0; display: flex; flex-direction: column; }
#vv-sched-left { flex: 1 1 0; min-width: 0; display: flex; flex-direction: column; align-self: flex-start; }
#vv-sched-cards { flex: 1; }
#vv-sched-right { display: none; flex: 1 1 0; min-width: 0; flex-direction: column; }
#vv-sched-right.vv-panel-visible { display: flex; }
.vv-log-card { flex: 1; display: flex; flex-direction: column; padding-bottom: 0; }
.vv-log-right-pre { max-height: none; overflow-y: auto; }
/* Mobile: stack vertically, right panel full-width below cards */
@media (max-width: 880px) {
/* Scheduler stacked layout (narrow viewport) */
@media (max-width: 900px) {
#vv-sched-layout { flex-direction: column; align-items: stretch; }
#vv-sched-left { flex: none; width: 100%; }
#vv-sched-right { flex-direction: column; width: 100%; }
.vv-log-right-pre { min-height: 260px; max-height: 340px; }
}
/* Plugin settings row (Advanced mode) */
.vv-nb-settings { display: flex; align-items: center; gap: 8px; flex-wrap: wrap;
padding: 5px 10px; background: #101010; border-bottom: 1px solid #1a1a1a; }
/* Monitor responsive — 4-column grid at medium width */
@media (max-width: 1024px) {
#vv-monitor { grid-template-columns: repeat(4, 1fr) !important; }
#vv-docker { grid-column: span 4 !important; }
}
/* Phone layout — scheduler row/actions fixes + monitor single-column */
@media (max-width: 600px) {
/* Allow action buttons to wrap rather than overflow the card */
.vv-job-actions { flex-wrap: wrap; padding-left: 0; }
/* Kill the inline margin-left:auto that pushes Advanced off-screen */
.vv-advanced-toggle { margin-left: 0 !important; width: auto !important; }
/* Let the cron input flex rather than hold a fixed 110px */
.vv-cron { flex: 1 1 80px; width: auto; min-width: 80px; }
/* Slightly tighter label on narrow screens */
.vv-job-label { font-size: 14px; }
/* Monitor single-column */
#vv-monitor { grid-template-columns: 1fr !important; }
#vv-monitor > .vv-card { grid-column: 1 / -1 !important; }
}
/* Shared footer (Save Schedule left, info right) — same min-height so log card ends level with script cards */
.vv-sched-footer { display: flex; align-items: center; gap: 10px;
margin-top: 12px; padding: 10px 0; border-top: 1px solid #333;
@@ -173,6 +199,17 @@
border: 1px solid #333; border-radius: 4px; padding: 10px 12px; resize: none;
line-height: 1.5; scroll-behavior: auto; tab-size: 2; width: 100%; box-sizing: border-box; }
/* Syntax-highlighted conf editor overlay */
#vv-editor-wrap { position: relative; display: block; }
#vv-hl-overlay { display: none; position: absolute; top: 1px; left: 1px; right: 1px; bottom: 1px;
margin: 0; padding: 10px 12px; box-sizing: border-box;
font-family: monospace; font-size: 12px; line-height: 1.5; tab-size: 2;
white-space: pre-wrap; word-break: break-all; overflow: hidden;
pointer-events: none; user-select: none;
background: #0d0d0d; border: none; border-radius: 3px; }
.vv-editor-hl #vv-hl-overlay { display: block; }
.vv-editor-hl #vv-editor-body { color: transparent; caret-color: #ddd; background: transparent; }
/* Suggestions panel — accordion */
#vv-suggestions { overflow-y: auto; }
.vv-sug-block { border-bottom: 1px solid #1e1e1e; }
@@ -210,6 +247,150 @@
.vv-info-divider { font-size: 10px; text-transform: uppercase; letter-spacing: 0.1em; color: #444;
padding: 10px 12px 4px; border-top: 1px solid #2a2a2a; margin-top: 2px; }
/* How do I use this — pinned at top of suggestions panel */
#vv-how-to-use {
position: sticky;
top: 0;
z-index: 3;
background: #1e1e1e;
border-bottom: 1px solid #333;
}
/* Board blocks — Next Runs, Errors, Locks, Partner, Disabled */
.vv-board-placeholder { color: #555; font-size: 11px; padding: 3px 0; }
.vv-hdr-badge { display: inline-block; padding: 1px 7px; border-radius: 10px;
font-size: 10px; font-weight: bold; flex-shrink: 0; }
.vv-hdr-badge-red { background: #7f0000; color: #ef9a9a; }
.vv-hdr-badge-orange { background: #5d2000; color: #ffcc80; }
.vv-hdr-badge-gray { background: #2a2a2a; color: #aaa; }
/* Next Runs */
.vv-nextrun-list { display: flex; flex-direction: column; }
.vv-nextrun-row { display: flex; align-items: center; gap: 8px; padding: 4px 0;
border-bottom: 1px solid #1a1a1a; font-size: 12px; }
.vv-nextrun-row:last-child { border-bottom: none; }
.vv-nr-label { color: #ccc; flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.vv-nr-cron { color: #555; font-family: monospace; font-size: 10px; flex-shrink: 0; }
.vv-nr-in { color: #4caf50; font-size: 11px; white-space: nowrap; flex-shrink: 0; }
.vv-nr-at { color: #666; font-size: 11px; white-space: nowrap; flex-shrink: 0; }
/* Errors */
.vv-errors-list { display: flex; flex-direction: column; gap: 1px; }
.vv-err-row { padding: 5px 0; border-bottom: 1px solid #1a1a1a; }
.vv-err-row:last-child { border-bottom: none; }
.vv-err-top { display: flex; align-items: baseline; gap: 8px; margin-bottom: 2px; }
.vv-err-script { color: #e07070; font-size: 11px; font-weight: bold; }
.vv-err-age { color: #555; font-size: 10px; }
.vv-err-line { color: #888; font-size: 11px; word-break: break-all; line-height: 1.4; }
/* Locks */
.vv-locks-list { display: flex; flex-direction: column; }
.vv-lock-row { display: flex; align-items: center; gap: 8px; padding: 4px 0;
border-bottom: 1px solid #1a1a1a; font-size: 12px; }
.vv-lock-row:last-child { border-bottom: none; }
.vv-lk-name { color: #e8a87c; flex: 1; }
.vv-lk-age { color: #777; font-size: 11px; white-space: nowrap; }
.vv-lock-clear { font-size: 11px !important; padding: 1px 7px !important;
border-color: #b71c1c !important; color: #ef9a9a !important; }
.vv-lock-clear:hover { background: #7f0000 !important; color: #fff !important; }
/* Partner */
.vv-partner-row { display: flex; align-items: center; gap: 8px; padding: 4px 0; font-size: 12px; }
.vv-partner-name { color: #ccc; }
.vv-partner-detail { color: #666; font-size: 11px; }
.vv-partner-down { color: #f44336 !important; }
/* Disabled scripts */
.vv-disabled-list { display: flex; flex-direction: column; gap: 1px; }
.vv-disabled-row { display: flex; align-items: center; gap: 8px; padding: 3px 0;
border-bottom: 1px solid #1a1a1a; font-size: 12px; }
.vv-disabled-row:last-child { border-bottom: none; }
.vv-disabled-name { color: #888; flex: 1; }
.vv-disabled-grp { color: #444; font-size: 10px; font-family: monospace; white-space: nowrap; }
/* Notification board */
.vv-nb-board { display: flex; align-items: center; gap: 8px; flex-wrap: wrap;
padding: 7px 10px; background: #141414; border-bottom: 1px solid #222;
font-size: 12px; min-height: 34px; }
.vv-nb-stat { color: #aaa; }
.vv-nb-sep { color: #444; }
.vv-nb-running { color: #f0a040; }
.vv-nb-conf-btns { display: flex; gap: 6px; margin-left: auto; }
.vv-nb-conf-btn { font-size: 11px; font-family: monospace; }
/* Advanced mode button */
.vv-adv-mode-btn { border: 1px solid #555; color: #999; transition: background 0.15s, color 0.15s, border-color 0.15s; }
.vv-adv-mode-btn:hover { border-color: #888; color: #fff; }
.vv-adv-mode-btn.vv-adv-mode-on { background: #1565c0; border-color: #1565c0; color: #fff; }
/* Script browser in suggestions panel */
.vv-sb-desc { font-size: 12px; color: #888; margin: 4px 0 6px; line-height: 1.5; }
.vv-sb-child-desc { margin-left: 16px; }
.vv-sb-hdr { font-family: monospace; font-size: 11px; color: #666; white-space: pre-wrap;
word-break: break-word; background: none; border: none; margin: 4px 0 8px;
padding: 0; line-height: 1.5; border-left: 2px solid #222; padding-left: 8px; }
.vv-sb-full { font-family: monospace; font-size: 11px; color: #888; white-space: pre-wrap;
word-break: break-word; background: #0a0a0a; border: 1px solid #222;
border-radius: 3px; margin: 6px 0 8px; padding: 8px 10px; line-height: 1.5;
max-height: 480px; overflow-y: auto; }
.vv-sb-child-block { border-top: 1px solid #1a1a1a; margin-top: 8px; padding-top: 8px; }
.vv-sb-child-name { display: flex; align-items: center; gap: 6px; margin-bottom: 3px; flex-wrap: wrap; }
/* README content display */
.vv-readme-body { font-family: monospace; font-size: 11px; color: #777; white-space: pre-wrap;
word-break: break-word; background: none; border: none; margin: 0;
padding: 0 4px; line-height: 1.6; }
/* Script browser tree rows */
#vv-sb-tree { padding: 0; }
.vv-sb-entry { }
.vv-sb-row { display: flex; align-items: center; gap: 6px; padding: 5px 6px;
cursor: pointer; border-radius: 3px; user-select: none; }
.vv-sb-row:hover { background: rgba(255,255,255,0.04); }
.vv-sb-selected { background: rgba(100,149,237,0.12) !important; outline: 1px solid #3a5a8a; }
.vv-sb-orch-row { border-bottom: 1px solid #1c1c1c; }
.vv-sb-child-row { padding-left: 2px; }
.vv-sb-expand { width: 14px; flex-shrink: 0; color: #555; font-size: 10px; text-align: center; }
.vv-sb-expand:hover { color: #aaa; }
.vv-sb-leaf { cursor: default; pointer-events: none; }
.vv-sb-name { flex: 1; font-size: 13px; color: #b0c4d0; min-width: 0;
white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
.vv-sb-orch-row .vv-sb-name { font-weight: bold; color: #9ab; }
.vv-sb-children { padding-left: 10px; }
.vv-sb-child-indent { width: 14px; flex-shrink: 0; color: #333; font-size: 11px;
text-align: center; pointer-events: none; }
.vv-sb-badge { flex-shrink: 0; font-size: 11px; }
.vv-sb-cron { color: #555; background: #111; border: 1px solid #222;
padding: 0 4px; border-radius: 2px; font-size: 10px;
font-family: monospace; flex-shrink: 0; }
.vv-sb-status { font-size: 10px; flex-shrink: 0; }
/* Script info content area */
.vv-si-hdr { font-family: monospace; font-size: 12px; color: #4d894d; white-space: pre-wrap;
word-break: break-word; background: none; border: none; margin: 0;
padding: 8px 12px; line-height: 1.6; }
.vv-si-src { font-family: monospace; font-size: 11px; white-space: pre-wrap;
word-break: break-word; background: none; border: none; margin: 0;
padding: 8px 12px; line-height: 1.5; }
/* Syntax highlight tokens — dark theme (VSCode-inspired) */
.vv-hl-sep { color: #2d2d2d; }
.vv-hl-shebang { color: #4a4a4a; }
.vv-hl-hash { color: #4a7340; }
.vv-hl-comment { color: #6a9955; }
.vv-hl-section { color: #9cdcfe; font-weight: bold; letter-spacing: 0.04em; }
.vv-hl-key { color: #9cdcfe; }
.vv-hl-value { color: #ce9178; }
.vv-hl-cron { color: #d7ba7d; font-weight: bold; }
.vv-hl-text { color: #4d894d; }
.vv-hl-keyword { color: #569cd6; }
.vv-hl-builtin { color: #4ec9b0; }
.vv-hl-string { color: #ce9178; }
.vv-hl-var { color: #d7ba7d; }
.vv-hl-number { color: #b5cea8; }
.vv-hl-op { color: #808080; }
/* Log panel */
.vv-log-panel { margin-top: 10px; border-top: 1px solid #333; padding-top: 8px; }
.vv-log-toolbar { display: flex; justify-content: space-between; align-items: center;
@@ -288,3 +469,63 @@
/* Live var substitution colours */
code.vv-live-var { color: #4caf50; background: #0d1f0d; }
code.vv-unknown-var { color: #ff9800; background: #1f130d; }
/* ── Arrange mode ─────────────────────────────────────────────────────────── */
.vv-drag-handle { cursor: grab; color: #555; font-size: 14px; padding: 0 5px 0 0; user-select: none; flex-shrink: 0; }
.vv-drag-handle:hover { color: #888; }
.vv-drag-ghost { opacity: 0.35; }
.vv-drop-line { height: 2px; background: #4caf50; border-radius: 2px; margin: 2px 0; pointer-events: none; }
.vv-arrange-active .vv-children { min-height: 28px; border: 1px dashed #2a2a2a; border-radius: 4px;
padding: 4px 2px; transition: border-color 0.12s, background 0.12s; }
.vv-arrange-active .vv-children.vv-drop-target { border-color: #4caf50; background: rgba(76,175,80,0.07); }
.vv-arrange-btn-active { background: #1a3a1e !important; color: #4caf50 !important; border-color: #2d5c33 !important; }
.vv-arrange-save-btn { background: #1a3a1e; border-color: #2d5c33; color: #4caf50; }
.vv-arrange-save-btn:hover { background: #22502a; }
/* ── Arrange workspace panel ─────────────────────────────────────────────── */
.vv-arrange-ws-hdr { font-size: 11px; font-weight: bold; color: #888; text-transform: uppercase;
letter-spacing: 0.6px; margin-bottom: 8px; display: flex; align-items: center; gap: 8px; }
#vv-pending-badge { background: #ff9800; color: #000; font-size: 10px; padding: 1px 7px;
border-radius: 10px; font-weight: bold; }
.vv-arrange-pending-hdr { font-size: 10px; color: #555; text-transform: uppercase; letter-spacing: 0.5px;
margin-bottom: 5px; }
.vv-pending-row { display: flex; align-items: center; gap: 8px; padding: 3px 0;
font-size: 11px; border-bottom: 1px solid #1e1e1e; }
.vv-pending-script { color: #ddd; font-weight: 500; }
.vv-pending-arrow { color: #555; font-size: 10px; }
.vv-library-zone { border: 1px dashed #2a2a2a; border-radius: 4px; padding: 6px;
min-height: 60px; transition: border-color 0.12s, background 0.12s; }
.vv-library-zone.vv-drop-target { border-color: #c62828; background: rgba(198,40,40,0.07); }
.vv-arrange-drop-hint { font-size: 10px; color: #444; text-align: center; padding: 2px 0 7px; }
.vv-lib-card { background: #1c1c1c; border: 1px solid #2e2e2e; border-radius: 3px;
padding: 4px 8px; margin-bottom: 4px; cursor: grab; display: flex;
align-items: center; justify-content: space-between; gap: 8px; }
.vv-lib-card:hover { border-color: #444; }
.vv-lib-card.vv-drag-ghost { opacity: 0.35; }
.vv-lib-card-name { font-size: 16px; color: #ccc; font-weight: bold; }
.vv-lib-card-path { font-size: 10px; color: #444; font-family: monospace; }
/* ── Custom script folders ────────────────────────────────────────────────── */
.vv-folder-group { margin-bottom: 1px; }
.vv-folder-row { display: flex; align-items: center; gap: 6px; padding: 3px 6px;
cursor: pointer; border-radius: 3px; color: #888; font-size: 12px; user-select: none; }
.vv-folder-row:hover { background: #1e1e1e; }
.vv-folder-chevron { font-size: 10px; color: #555; width: 10px; flex-shrink: 0; }
.vv-folder-name { flex: 1; font-weight: 500; color: #aaa; }
.vv-folder-count { font-size: 10px; color: #555; background: #1c1c1c;
padding: 0 5px; border-radius: 8px; border: 1px solid #2a2a2a; }
.vv-folder-children { padding-left: 12px; min-height: 4px; }
.vv-folder-children.vv-drop-target { background: rgba(76,175,80,0.07); border-radius: 4px;
outline: 1px dashed #3a6a3e; }
.vv-folder-new-row { display: flex; align-items: center; gap: 6px; padding: 4px 6px; }
.vv-new-folder-btn { background: #1c1e1c; border-color: #2e3a2e; color: #6a9e6a; }
.vv-new-folder-btn:hover { background: #222e22; }
/* ── Rsync standalone controls ────────────────────────────────────────────── */
.vv-rsync-location { width: 120px; flex-shrink: 1; min-width: 60px; font-size: 11px; font-family: monospace;
background: #111; border: 1px solid #333; color: #aaa;
padding: 2px 6px; border-radius: 3px; }
.vv-rsync-location:focus { border-color: #555; outline: none; }
.vv-rsync-location::placeholder { color: #444; }
.vv-rsync-save-btn { background: #1a2e1a; border-color: #2a4a2a; color: #6aaa6a; }
.vv-rsync-save-btn:hover { background: #22382a; }
@@ -82,6 +82,24 @@ function vv_cron_rebuild(array $schedule): bool {
}
$lines[] = "";
// Standalone rsync entries: fire when orch is disabled but location + cron are both configured.
$scriptsDir = SCRIPTS_DIR;
foreach ($schedule as $key => $entry) {
if (!str_starts_with((string)$key, '__rsync_')) continue;
$orchId = $entry['orch_id'] ?? '';
$location = $entry['location'] ?? '';
$cron = $entry['cron'] ?? '';
if (!$orchId || !$location || !$cron) continue;
// Skip if orch is still enabled
if (!empty($schedule[$orchId]['enabled'])) continue;
$rsyncScript = "$scriptsDir/Rsync/rsync.sh";
if (!file_exists($rsyncScript)) continue;
$locArg = escapeshellarg('--location=' . $location);
$logFlag = !empty($entry['log_enabled']) ? ' --log' : '';
$lines[] = "$cron bash \"$runner\" \"Rsync/rsync.sh\" \"$rsyncScript\" $locArg$logFlag";
}
$lines[] = "";
// Write to the plugin cron file; update_cron merges all plugin *.cron files into /etc/cron.d/root.
if (file_put_contents(CRON_FILE, implode("\n", $lines)) === false) return false;
exec('/usr/local/sbin/update_cron');
@@ -210,6 +228,59 @@ function vv_custom_scripts(): array {
return $scripts;
}
// Load rsync standalone config (location + cron) for a flag name from schedule.json.
function vv_rsync_standalone(string $flagName): array {
$s = vv_schedule_load();
$r = $s['__rsync_' . $flagName] ?? [];
return [
'location' => (string)($r['location'] ?? ''),
'cron' => (string)($r['cron'] ?? ''),
];
}
// Extract *_SCRIPTS array variable names that an orchestrator iterates over.
function vv_orch_conf_arrays(string $orchPath): array {
$content = file_get_contents($orchPath) ?: '';
preg_match_all('/\$\{([A-Z_]+_SCRIPTS)\[@\]\}/', $content, $refs);
return array_unique($refs[1] ?? []);
}
// Return .sh scripts that exist in SCRIPTS_DIR but are not referenced in any
// master.conf *_SCRIPTS array and are not orchestrators or custom scripts.
function vv_script_library(): array {
$scriptsDir = SCRIPTS_DIR;
$confMap = vv_conf_script_map();
$orchIds = [];
foreach (glob("$scriptsDir/Orchestrators/*.sh") ?: [] as $p) {
$orchIds[] = 'Orchestrators/' . basename($p);
}
$exclude = ['Plugin', '.git', 'Orchestrators', 'Custom', 'Configurations'];
$library = [];
try {
$ri = new RecursiveIteratorIterator(
new RecursiveDirectoryIterator($scriptsDir, RecursiveDirectoryIterator::SKIP_DOTS)
);
$base = rtrim($scriptsDir, '/') . '/';
foreach ($ri as $rf) {
if (!$rf->isFile() || strtolower($rf->getExtension()) !== 'sh') continue;
$rel = ltrim(str_replace($base, '', $rf->getPathname()), '/');
$parts = explode('/', $rel);
if (count($parts) < 2 || in_array($parts[0], $exclude)) continue;
if (in_array($rel, $orchIds) || isset($confMap[$rel])) continue;
$library[] = ['id' => $rel, 'label' => basename($rel, '.sh')];
}
} catch (Exception $e) {}
usort($library, fn($a, $b) => strcmp($a['id'], $b['id']));
return $library;
}
// Load custom-script folder assignments from schedule.json (__folders key).
function vv_folders_load(): array {
$s = vv_schedule_load();
$f = $s['__folders'] ?? [];
return is_array($f) ? $f : [];
}
// Walk the scripts repo and return the job tree:
// hardcoded array-event entries first, then cron-scheduled orchestrators
function vv_job_tree(): array {
@@ -260,6 +331,7 @@ function vv_job_tree(): array {
'suggested_cron' => $suggested['cron'],
'suggested_label' => $suggested['label'],
'children' => vv_script_children($path, $schedule),
'conf_arrays' => vv_orch_conf_arrays($path),
];
}
return $orchs;
@@ -411,6 +483,7 @@ function vv_script_children(string $orchPath, array $schedule): array {
'log_enabled' => (bool)($entry['log_enabled'] ?? false),
'conf_managed' => $conf['managed'],
'conf_enabled' => $conf['enabled'], // null if not in any *_SCRIPTS array
'conf_array' => $conf['array'],
'suggested_cron' => $suggested['cron'],
'suggested_label' => $suggested['label'],
];
@@ -447,3 +520,63 @@ function vv_script_children(string $orchPath, array $schedule): array {
return $children;
}
// Extract the comment header block from a bash script (shebang + all leading comment lines).
// Returns raw lines with # markers intact.
function vv_script_header(string $path): string {
if (!file_exists($path)) return '';
$lines = array_slice(file($path) ?: [], 0, 80);
$out = [];
foreach ($lines as $line) {
$t = rtrim($line);
if (str_starts_with($t, '#') || ($out === [] && str_starts_with($t, '#!'))) {
$out[] = $t;
} elseif ($t === '' && !empty($out)) {
$out[] = $t; // allow blank lines within header
} else {
break;
}
}
// Trim trailing blank lines
while (!empty($out) && trim(end($out)) === '') array_pop($out);
return implode("\n", $out);
}
// Strip the leading # marker from each line of a script header for cleaner display.
// Also drops the shebang line (#!/bin/bash) since it's not informative in this context.
function vv_script_header_clean(string $path): string {
$raw = vv_script_header($path);
if (!$raw) return '';
$lines = explode("\n", $raw);
$out = [];
foreach ($lines as $line) {
if (str_starts_with($line, '#!')) continue; // shebang — not useful in header display
$out[] = preg_replace('/^#\s?/', '', $line); // strip # and optional space
}
while (!empty($out) && trim(end($out)) === '') array_pop($out);
return implode("\n", $out);
}
// Read a named section from a markdown file.
// Calls $matcher(heading, isIntro) where isIntro=true for content before the first heading.
// Returns the first matching section body, capped at $maxChars.
function vv_readme_section(string $readmePath, callable $matcher, int $maxChars = 3000): string {
if (!file_exists($readmePath)) return '';
$content = file_get_contents($readmePath) ?: '';
$parts = preg_split('/^(#{1,4}[^\n]*)/m', $content, -1, PREG_SPLIT_DELIM_CAPTURE);
$heading = '';
$isIntro = true;
foreach ($parts as $i => $part) {
if ($i % 2 === 1) {
$heading = trim(preg_replace('/^#{1,4}\s*/', '', $part));
$isIntro = false;
continue;
}
$body = trim($part);
if ($body === '') continue;
if ($matcher($heading, $isIntro)) {
return strlen($body) > $maxChars ? substr($body, 0, $maxChars) . "\n[…]" : $body;
}
}
return '';
}
File diff suppressed because it is too large Load Diff
+8 -8
View File
@@ -16,7 +16,7 @@
#
# array_started.sh at array start — launches ALL startup scripts in order
# array_stopping.sh at array stop — graceful shutdown sequence
# watchdog_orchestrator.sh every minute — resource + docker + system watchdog
# watchdog_orchestrator.sh every 15 min — resource + docker + system watchdog
# transcode_management.sh every 7 min — cleanup then manager (order critical)
# critical_sync_maintenance.sh every 30 min — auth + Emby dirty sync + partnership
# intermediate_sync_maintenance.sh every 4 hours — arr library sync + artwork fetch
@@ -132,7 +132,7 @@
# fallback.sh [continuous] mutual fallback state machine
#
# NOTE: system_watchdog.sh and docker_watchdog.sh are no longer started by array_started.sh.
# They are now single-pass scripts called every minute by watchdog_orchestrator.sh (see below).
# They are now single-pass scripts called every 15 minutes by watchdog_orchestrator.sh (see below).
# Remove them from ARRAY_START_SCRIPTS in master.conf if present.
#
# bash /mnt/user/appdata/unraid_scripts/Orchestrators/array_started.sh
@@ -161,7 +161,7 @@
# ── WATCHDOG ORCHESTRATOR ─────────────────────────────────────────────────────────────────────
# Schedule: * * * * * (every minute)
# Schedule: */15 * * * * (every 15 minutes)
# Background: YES
#
# Runs the three-layer watchdog system in the correct sequence each cycle. Replaces the
@@ -178,9 +178,9 @@
# 4. stability_watchdog.sh reboot if all else failed — last line of defense
#
# Startup grace: no action until uptime >= WATCHDOG_STARTUP_GRACE (600s). Exits early
# so all three sub-scripts don't each log startup-grace skips on every minute.
# so all three sub-scripts don't each log startup-grace skips on every cycle.
# Overlap protection: acquire_lock() exits immediately if a prior cycle is still running —
# prevents pile-up when a restart attempt or daemon check takes longer than 60 seconds.
# prevents pile-up when a restart attempt or daemon check takes longer than 15 minutes.
#
# bash /mnt/user/appdata/unraid_scripts/Orchestrators/watchdog_orchestrator.sh
# bash /mnt/user/appdata/unraid_scripts/Orchestrators/watchdog_orchestrator.sh --dry-run
@@ -560,7 +560,7 @@
# ──────────────────────────────────────────────────────────────────────────────────────────────
# docker_watchdog.sh — two-tier container self-healing monitor (single-pass)
# Called every minute by watchdog_orchestrator.sh — NOT started by array_started.sh. Single-pass.
# Called every 15 minutes by watchdog_orchestrator.sh — NOT started by array_started.sh. Single-pass.
#
# Tier 1 — explicit per-container (configured in host*.conf):
# Memory hard limits: immediate restart when exceeded — no strikes, no waiting
@@ -814,7 +814,7 @@
# bash /mnt/user/appdata/unraid_scripts/unRAID_Essentials/docker_syslog_filter.sh
# webgui_watchdog.sh — WebGUI availability watchdog (called by system_watchdog via SYSTEM_WATCHDOG_SCRIPTS)
# Not scheduled directly — runs as part of the every-minute watchdog chain.
# Not scheduled directly — runs as part of the every-15-minute watchdog chain.
# Escalation: nginx → php-fpm → emhttp, each with recheck before proceeding to next level.
# bash /mnt/user/appdata/unraid_scripts/Watchdogs/System/webgui_watchdog.sh --dry-run
# bash /mnt/user/appdata/unraid_scripts/Watchdogs/System/webgui_watchdog.sh --status
@@ -1414,7 +1414,7 @@
# At Stopping of Array:
# Orchestrators/array_stopping.sh (single entry — graceful shutdown)
#
# * * * * * every minute:
# */15 * * * * every 15 minutes:
# Orchestrators/watchdog_orchestrator.sh
# └─ resource_watchdog → docker_watchdog → system_watchdog (storage/webgui/network) → stability_watchdog
#