Stop the media seed from holding the onboard, and therefore the partnership, open for weeks

A first seed is ~28 TB behind a 12.5 MB/s bwlimit, and it ran inline as Step 9d, so the phase-2 flag every status reader depends on was written only after it finished.
2026-08-17 07:40:00 -04:00
parent 9d2610a911
commit ad623353bc
6 changed files with 466 additions and 61 deletions
@@ -0,0 +1,131 @@
+<?php
+// ═══════════════════════════════════════════════════════════════════════════════════════════════
+// PURPOSE
+//   Start and stop the background media seed — Rsync/media_seed.sh, the multi-week first push
+//   of every DAILY_SYNC_SHARES entry to the partner. Onboard Step 13 dispatches it once; this
+//   endpoint is how the Partnership tab restarts it after a failure and stops it on demand.
+//
+// OPERATIONAL MODEL
+//   The seed used to run inline inside partnership_onboard.sh, which meant the only control a
+//   person had over ~28 TB of transfer was killing the onboard. It is its own job now, with its
+//   own record at /var/log/varaverk/Rsync/media_seed.json, and these two actions are its whole
+//   control surface. Progress is read by include/partnership.php and rendered on the partner's
+//   card; nothing here reports progress.
+//
+// DESIGN PRINCIPLES
+//   Stopping is safe by construction, and that is why a Stop button exists at all.
+//       rsync.sh runs --inplace --partial, so a stopped seed loses the file in flight, not the
+//       share, and a later start resumes rather than restarting. Anything else and the button
+//       would be a trap on a transfer measured in weeks.
+//
+//   The job id is a literal.
+//       Nothing about which job runs comes from the request, so there is no path, no script
+//       name and no argument for a caller to influence. This endpoint can start exactly one
+//       script and signal exactly one recorded pid.
+//
+// OPERATIONAL SAFEGUARDS
+//   POST only, for both actions. Unraid's CSRF guard is POST-only and jQuery-injected, so a
+//   GET here would be both unguarded and, from native fetch(), silently unauthenticated.
+//   See README-unraid.md.
+//
+//   Stop signals the process group, not the pid.
+//       run_job.sh is dispatched under setsid, so its pid is its process group leader and
+//       kill -TERM -<pgid> reaches the rsync and the ssh beneath it. Signalling the pid alone
+//       would reap the wrapper and leave the transfer running with no record pointing at it.
+//
+//   The recorded pid is verified to still be that job before it is signalled.
+//       A stale json from a run killed by a reboot can name a pid the kernel has since reused.
+//       /proc/<pid>/cmdline is checked for the seed script's own path first, so at worst this
+//       refuses to stop something; it cannot kill an unrelated process.
+//
+//   Starting is delegated, not duplicated.
+//       run_job.sh already refuses to start a job that is running, and media_seed.sh takes its
+//       own lock. This endpoint does not re-implement either check — it dispatches and reports
+//       what the record then says.
+//
+// REQUEST
+//   POST action=start     dispatch the seed detached
+//   POST action=stop      terminate a running seed
+//
+// RESPONSE
+//   {"ok":true,"status":"running"}   start: the job record went live within the wait window
+//   {"ok":true,"stopped":true}       stop: the process group was signalled
+//   {"ok":false,"error":string}      wrong method, unknown action, or nothing to act on
+//
+// DEPENDS ON
+//   Rsync/media_seed.sh            the job itself
+//   Plugin/unraid/run_job.sh       the wrapper that writes the record
+//   include/config.php             SCRIPTS_DIR
+// ═══════════════════════════════════════════════════════════════════════════════════════════════
+header('Content-Type: application/json');
+require_once dirname(__DIR__) . '/include/config.php';
+
+if ($_SERVER['REQUEST_METHOD'] !== 'POST') {
+    echo json_encode(['ok' => false, 'error' => 'POST only']);
+    exit;
+}
+
+$action = $_POST['action'] ?? '';
+$dir    = rtrim(SCRIPTS_DIR, '/');
+$script = $dir . '/Rsync/media_seed.sh';
+$runner = $dir . '/Plugin/unraid/run_job.sh';
+$stat   = '/var/log/varaverk/Rsync/media_seed.json';
+
+if ($action === 'start') {
+    if (!is_file($script) || !is_file($runner)) {
+        echo json_encode(['ok' => false, 'error' => 'media_seed.sh not found on this host']);
+        exit;
+    }
+    $cmd = 'setsid /bin/bash ' . escapeshellarg($runner)
+         . ' ' . escapeshellarg('Rsync/media_seed.sh')
+         . ' ' . escapeshellarg($script)
+         . ' --manual >/dev/null 2>&1 </dev/null &';
+    shell_exec($cmd);
+
+    // Report what the record says, not that the command was issued. run_job.sh writes its
+    // stat file before running the script, so a live record is the difference between a seed
+    // that started and one that was refused for already running or died on its gate check.
+    for ($i = 0; $i < 10; $i++) {
+        if (is_file($stat)) {
+            $j = json_decode((string)file_get_contents($stat), true);
+            if (is_array($j) && ($j['status'] ?? '') === 'running'
+                && time() - filemtime($stat) < 60) {
+                echo json_encode(['ok' => true, 'status' => 'running']);
+                exit;
+            }
+        }
+        usleep(500000);
+    }
+    echo json_encode(['ok' => false, 'error' => 'Seed did not start — check Rsync/media_seed.log']);
+    exit;
+}
+
+if ($action === 'stop') {
+    if (!is_file($stat)) {
+        echo json_encode(['ok' => false, 'error' => 'No seed has been run on this host']);
+        exit;
+    }
+    $j   = json_decode((string)file_get_contents($stat), true);
+    $pid = (int)($j['pid'] ?? 0);
+    if (($j['status'] ?? '') !== 'running' || !$pid || !is_dir("/proc/$pid")) {
+        echo json_encode(['ok' => false, 'error' => 'Seed is not running']);
+        exit;
+    }
+
+    // See "The recorded pid is verified" above — cmdline is NUL-separated, so the script path
+    // is matched against the raw bytes rather than a split.
+    $cmdline = @file_get_contents("/proc/$pid/cmdline") ?: '';
+    if (strpos($cmdline, 'media_seed.sh') === false) {
+        echo json_encode(['ok' => false, 'error' => 'Recorded PID is no longer the seed — record is stale']);
+        exit;
+    }
+
+    shell_exec('kill -TERM -' . $pid . ' 2>/dev/null');
+    usleep(400000);
+    if (is_dir("/proc/$pid")) shell_exec('kill -KILL -' . $pid . ' 2>/dev/null');
+
+    echo json_encode(['ok' => true, 'stopped' => true]);
+    exit;
+}
+
+echo json_encode(['ok' => false, 'error' => 'Unknown action']);