Bring the seed docs in line with Phase 3
This commit is contained in:
@@ -2,8 +2,9 @@
|
|||||||
// ═══════════════════════════════════════════════════════════════════════════════════════════════
|
// ═══════════════════════════════════════════════════════════════════════════════════════════════
|
||||||
// PURPOSE
|
// PURPOSE
|
||||||
// Start and stop the background media seed — Rsync/media_seed.sh, the multi-week first push
|
// Start and stop the background media seed — Rsync/media_seed.sh, the multi-week first push
|
||||||
// of every DAILY_SYNC_SHARES entry to the partner. Onboard Step 13 dispatches it once; this
|
// of every DAILY_SYNC_SHARES entry to the partner. Phase 3 (partnership_onboard.sh
|
||||||
// endpoint is how the Partnership tab restarts it after a failure and stops it on demand.
|
// --phase3-only) is how it is normally started; this endpoint is the direct control the
|
||||||
|
// Partnership tab uses to stop a running seed, and to start one without re-running the phase.
|
||||||
//
|
//
|
||||||
// OPERATIONAL MODEL
|
// OPERATIONAL MODEL
|
||||||
// The seed used to run inline inside partnership_onboard.sh, which meant the only control a
|
// The seed used to run inline inside partnership_onboard.sh, which meant the only control a
|
||||||
|
|||||||
@@ -232,7 +232,7 @@ function vv_pt_remote_system(string $ip, string $sshKey): array {
|
|||||||
|
|
||||||
// ── Media seed progress ───────────────────────────────────────────────────────
|
// ── Media seed progress ───────────────────────────────────────────────────────
|
||||||
//
|
//
|
||||||
// Onboard Step 13 dispatches Rsync/media_seed.sh detached, because a first seed of a full
|
// Phase 3 dispatches Rsync/media_seed.sh detached, because a first seed of a full
|
||||||
// media library is a multi-week transfer and used to hold the onboard — and therefore the
|
// media library is a multi-week transfer and used to hold the onboard — and therefore the
|
||||||
// phase-2 flag, and therefore this whole page — open for the duration.
|
// phase-2 flag, and therefore this whole page — open for the duration.
|
||||||
//
|
//
|
||||||
|
|||||||
+11
-6
@@ -104,18 +104,23 @@ For full profile definitions see `Manual-Rsync.md`.
|
|||||||
| Script | Role | When It Runs |
|
| Script | Role | When It Runs |
|
||||||
|--------|------|-------------|
|
|--------|------|-------------|
|
||||||
| `rsync.sh` | Core transfer engine — profile resolution, pre-flights, container management, transfer, bandwidth logging | Called by orchestrators; directly for manual syncs |
|
| `rsync.sh` | Core transfer engine — profile resolution, pre-flights, container management, transfer, bandwidth logging | Called by orchestrators; directly for manual syncs |
|
||||||
| `media_seed.sh` | First-fill of a new partner — loops every `DAILY_SYNC_SHARES` entry through `rsync.sh --seed` | Dispatched detached by partnership onboard Step 13; restartable from the Partnership tab |
|
| `media_seed.sh` | First-fill of a new partner — loops every `DAILY_SYNC_SHARES` entry through `rsync.sh --seed` | Phase 3 — dispatched by `partnership_onboard.sh --phase3-only`, triggered from the Partnership tab |
|
||||||
|
|
||||||
`media_seed.sh` is the one script here that is expected to run for weeks. A first seed of
|
`media_seed.sh` is the one script here that is expected to run for weeks. A first seed of
|
||||||
HOST1's library is ~28 TB against a 12.5 MB/s `--bwlimit`, which is why onboard dispatches it
|
HOST1's library is ~28 TB against a 12.5 MB/s `--bwlimit`, which is why it is a phase of its own
|
||||||
rather than waiting on it, and why the Partnership tab gives it a Stop button — `rsync.sh` runs
|
rather than a step inside the onboard, and why the Partnership tab gives it a Stop button —
|
||||||
`--inplace --partial`, so stopping costs the file in flight, not the share.
|
`rsync.sh` runs `--inplace --partial`, so stopping costs the file in flight, not the share.
|
||||||
|
|
||||||
It answers to `MEDIA_SEED_ENABLED` in `master.conf` — a Tier 2 toggle beside the per-orchestrator
|
It answers to `MEDIA_SEED_ENABLED` in `master.conf` — a Tier 2 toggle beside the per-orchestrator
|
||||||
ones, still under Tier 1 `RSYNC_ENABLED`. Off means onboard finishes without ever dispatching it,
|
ones, still under Tier 1 `RSYNC_ENABLED`. **Off means there is no Phase 3 at all** and onboarding
|
||||||
for a partner being filled from a moved disk or one that already holds the library. An unset
|
is two phases: the toggle removes the phase rather than disabling a button. Use it for a partner
|
||||||
|
being filled from a moved disk or one that already holds the library. An unset
|
||||||
`MEDIA_SEED_ENABLED` reads as on, so a conf that predates the toggle keeps its behaviour.
|
`MEDIA_SEED_ENABLED` reads as on, so a conf that predates the toggle keeps its behaviour.
|
||||||
|
|
||||||
|
Phase 3 also sets the gate posture it needs and leaves it there: Tier 1 open so `rsync.sh` will
|
||||||
|
move anything at all, every Tier 2 gate closed so the scheduled orchestrators are not competing
|
||||||
|
for the same link and the same disks for the weeks the seed takes.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ━━━ HOW THE SCRIPTS RELATE ━━━
|
## ━━━ HOW THE SCRIPTS RELATE ━━━
|
||||||
|
|||||||
+10
-5
@@ -6,14 +6,16 @@
|
|||||||
# PURPOSE
|
# PURPOSE
|
||||||
# ─────────────────────────────────────────────────────────────────────────────
|
# ─────────────────────────────────────────────────────────────────────────────
|
||||||
# Pushes every share in this host's DAILY_SYNC_SHARES to the partner with
|
# Pushes every share in this host's DAILY_SYNC_SHARES to the partner with
|
||||||
# Rsync/rsync.sh --seed. Run once after a partnership onboard, when the mirror's
|
# Rsync/rsync.sh --seed. This is Phase 3 — run after a partnership onboard has
|
||||||
# arr databases know about all the content but the disks behind them are empty.
|
# completed, when the mirror's arr databases know about all the content but the
|
||||||
|
# disks behind them are empty. The partnership does not wait on it.
|
||||||
#
|
#
|
||||||
# ==============================================================================================
|
# ==============================================================================================
|
||||||
# WHY THIS IS ITS OWN SCRIPT
|
# WHY THIS IS ITS OWN SCRIPT
|
||||||
# ==============================================================================================
|
# ==============================================================================================
|
||||||
#
|
#
|
||||||
# This was Step 9d inside partnership_onboard.sh, running inline. On HOST1 that
|
# This was Step 9d inside partnership_onboard.sh, running inline, and is now Phase 3 —
|
||||||
|
# reached only by partnership_onboard.sh --phase3-only. On HOST1 that
|
||||||
# is thirteen shares and roughly 28 TB, and DEFAULT_RSYNC_OPTS caps the transfer
|
# is thirteen shares and roughly 28 TB, and DEFAULT_RSYNC_OPTS caps the transfer
|
||||||
# at --bwlimit=12500 (12.5 MB/s), so a first seed is a multi-week transfer.
|
# at --bwlimit=12500 (12.5 MB/s), so a first seed is a multi-week transfer.
|
||||||
#
|
#
|
||||||
@@ -50,7 +52,7 @@
|
|||||||
#
|
#
|
||||||
# Gate Read From Disk
|
# Gate Read From Disk
|
||||||
# Both are read out of master.conf here rather than trusted from the sourced
|
# Both are read out of master.conf here rather than trusted from the sourced
|
||||||
# environment. Onboard's Step 9c rewrites that file moments before dispatching
|
# environment. Phase 3 rewrites that file moments before dispatching
|
||||||
# this script, and each rsync.sh below sources it fresh anyway. With Tier 1
|
# this script, and each rsync.sh below sources it fresh anyway. With Tier 1
|
||||||
# closed rsync.sh moves nothing and still exits 0, so every share would be
|
# closed rsync.sh moves nothing and still exits 0, so every share would be
|
||||||
# counted as seeded — refuse once instead of reporting fourteen no-ops.
|
# counted as seeded — refuse once instead of reporting fourteen no-ops.
|
||||||
@@ -79,7 +81,10 @@
|
|||||||
# RUNTIME MODES
|
# RUNTIME MODES
|
||||||
# ==============================================================================================
|
# ==============================================================================================
|
||||||
#
|
#
|
||||||
# Rsync/media_seed.sh — seed every DAILY_SYNC_SHARES entry
|
# Partnership/partnership_onboard.sh --phase3-only — the normal entry point: checks
|
||||||
|
# Phase 2 is done, sets the gate posture, dispatches this
|
||||||
|
#
|
||||||
|
# Rsync/media_seed.sh — seed every DAILY_SYNC_SHARES entry directly
|
||||||
# Rsync/media_seed.sh --dry-run — pass --dry-run down to rsync.sh
|
# Rsync/media_seed.sh --dry-run — pass --dry-run down to rsync.sh
|
||||||
# Rsync/media_seed.sh --log — verbose output
|
# Rsync/media_seed.sh --log — verbose output
|
||||||
#
|
#
|
||||||
|
|||||||
Reference in New Issue
Block a user