# Scheduler

Everything that runs on a timer is decided here. The page is the front end for two files:
`schedule.json` holds when things fire, and `master.conf` holds which steps an orchestrator
actually calls. Every control below writes to one of those two — nothing on this page needs a
conf file opened by hand.

**An orchestrator is a job that calls other scripts in order.** The orchestrator has the cron
entry; its steps do not. That one fact explains most of the behaviour below.

---

## Stopping one step without disabling the whole job

The common case: a nightly job is fine, but one script inside it should stop running.

1. Find the orchestrator's card — for example **Daily Sync Maintenance**.
2. Click **Steps ▸** on the right of its row. The list of scripts it calls expands.
3. Toggle off the one you want stopped.

The orchestrator keeps running on its schedule and simply skips that step. Under the hood the
toggle comments the script out of that orchestrator's array in `master.conf`, which is the same
edit you would make by hand — the toggle is just the safe way to make it.

Turn it back on with the same toggle. The commented line is left in place, so the step keeps its
position in the sequence rather than being appended to the end.

> The button is called **Steps**, not Advanced. There is a separate **Advanced** button at the
> top right of the panel that does something quite different — see *Advanced mode* below.

## Turning an orchestrator off entirely

Toggle the orchestrator itself off. Its cron entry stops firing and **every step inside it is
suppressed** — an enabled step under a disabled orchestrator does not run, because the
orchestrator is what calls it.

If you still want one of those scripts running while its orchestrator is off, give that step its
own cron expression. With the orchestrator off, a step's cron field becomes live and it fires
standalone.

## Running something right now

**Run** fires the script immediately and resets its cron timer, so the next scheduled fire is a
full interval later rather than a few minutes away.

**Dry Run** passes `--dry-run`. Nothing is written to disk, and the full output is shown rather
than a summary — with the cleanup scripts, reading the list of what *would* be removed is the
entire point. Anything destructive offers both, and Run asks for confirmation first.

## Changing when something runs

Type into the cron field. It saves when the field loses focus, shows the schedule back to you in
plain English, and turns red if the expression is invalid.

Two shortcuts worth knowing:

- **Suggested cron** — in the Orch tree on the right, each script shows a suggested cron badge.
  Click it to apply and save that schedule instantly.
- **Cron Calculator** — accepts an expression or plain English, shows a description plus the next
  five fire times, and **Apply** pushes it into the cron field you last used.

For array events, put `array_start` or `array_stop` in the cron field instead of an expression.
A badge appears and the script fires on Unraid's array start or stop. This works for any
orchestrator or custom script.

## Finding out why something failed

**Log** opens that script's log in the right-hand panel. It auto-scrolls to the newest line,
pauses when you scroll up, and resumes when you return to the bottom. **Invert** puts newest
lines at the top. The **Search** box in the toolbar filters and highlights matches while dimming
everything else.

Two blocks on the Scheduler Info panel get you there faster:

- **Recent Activity** — the last 24 runs. Click any row to open that log. Expands itself when
  there are errors.
- **Recent Errors** — the most recent error per script over 7 days, so a failure that happened
  once at 3am is still visible days later.

The status dot in the Orch tree is green, orange or red for the last run's outcome, dim if the
script has never run. Hover for detail.

## Stopping a script that is stuck

**Stop** sends SIGTERM, waits 3 seconds, then SIGKILL, and clears any lock files the script left
behind.

Locks are never cleared automatically anywhere in Varaverk, and that is deliberate — a lock that
looks stale may belong to a job that is still working. Stop is the explicit way to say you have
decided otherwise.

## Moving a script between orchestrators

Use **Arrange** to drag scripts between orchestrators. Dropping one onto the right-hand panel
removes it from its orchestrator. Nothing is written until you click **Save Arrangement**, which
commits the new order to `master.conf`.

Order matters — an orchestrator runs its steps in the sequence shown.

## Adding your own script

Custom scripts live in `$CUSTOM_SCRIPTS_DIR`, outside the git repo. Any `*.sh` placed there is
detected and listed automatically; it does not have to be registered anywhere. Being outside the
repo means a `git pull` never touches your scripts and they are never committed by accident.

**+ Folder** creates a collapsible subfolder in Custom Scripts that you can drag scripts into.

While editing, **Cancel** discards your changes and returns to the Scheduler Info panel. The file
on disk is never touched until you press Save.

---

## Reference — the controls on a job row

| Control | What it does |
|---|---|
| **Toggle** | Enable/disable. Saves immediately to `schedule.json` |
| **Cron field** | Schedule. Auto-saves on focus-out; plain-English hint; red border means invalid |
| **⚙ Cog** | Settings panel — the script's own header, matching README/Manual sections, and its config fields |
| **Script name** | Jumps to that script's entry in the Orch tree on the right |
| **Run** | Execute now; resets the cron timer |
| **Dry Run** | Execute with `--dry-run`; writes nothing |
| **Log** | Open this script's log in the right panel |
| **Verbose** | Appends `--log` for per-item detail in the output |
| **Stop** | SIGTERM, 3s, then SIGKILL. Clears any lock the script left behind |
| **Steps ▸** | Expand the scripts this orchestrator calls |

## Reference — controls elsewhere on the page

| Control | Where | What it does |
|---|---|---|
| **Suggested cron** | Orch tree | Click the cron badge to apply and save that schedule instantly |
| **Status dot** | Orch tree | Green / orange / red for the last run; dim if never run. Hover for detail |
| **Arrange** | Orch tree | Drag scripts between orchestrators. **Save Arrangement** commits to `master.conf` |
| **Array events** | Cron field | Type `array_start` or `array_stop` instead of an expression |
| **Log search** | Toolbar | Filters the open log, highlighting matches and dimming the rest |
| **Invert** | Toolbar | Newest log lines at the top |
| **Auto Scroll** | Toolbar | Follows the newest line; pauses if you scroll up |
| **Cancel** | Toolbar, editing | Discards changes and returns here. The file is untouched until Save |
| **+ Folder** | Custom Scripts | A collapsible subfolder you can drag scripts into |

## Reference — orchestrator and step behaviour

| State | Behaviour |
|---|---|
| Orchestrator **ON** | Sole trigger. Cron fires the orchestrator, which calls its steps in sequence |
| Orchestrator **OFF** | Never runs automatically. Steps are suppressed; each can take its own cron |
| Step, orchestrator ON | Toggle comments/uncomments the script in `master.conf` |
| Step, orchestrator ON, no conf array | The orchestrator hardcodes the call — the toggle is display-only |
| Step, orchestrator OFF | Enter a cron to run it standalone |
| Rsync badge | Toggle writes `TIER_RSYNC_ENABLED` straight to `master.conf` |
| Rsync, orchestrator OFF | Needs both a location and a standalone cron before it will fire independently |

## Reference — the Scheduler Info panel

The right-hand panel is the default view and everything returns to it. **← Scheduler Info** comes
back here from any log, editor, conf form or script view.

- **Notification board** — how many jobs are scheduled out of the total, and what is running now.
- **Next Runs** — what fires next, and when.
- **Recent Activity** / **Recent Errors** — described above.
- **Cron Calculator** — described above.

## Reference — Advanced mode

The **Advanced** button at the top right turns blue when active. It is a display and editing
mode, unrelated to the **Steps ▸** expander on each orchestrator:

- The ⚙ cog switches to an enriched view — full script source, documentation and config together.
- Raw conf editing unlocks, including direct buttons for each conf file.
- **Git Pull** appears on the notification board, which pulls the latest scripts and runs a conf
  upgrade.

Raw conf editing writes the file directly. The toggles elsewhere on this page are surgical — they
preserve the comment blocks that document every threshold — so prefer them where one exists.

## Reference — what this page writes

| File | Written by |
|---|---|
| `schedule.json` | Every toggle and cron field |
| `master.conf` | Step toggles, Save Arrangement, rsync tier flags, raw conf editing |
| `CUSTOM_SCRIPTS_DIR` | The custom script editor |

Cron changes take effect on the next cron rebuild. Nothing here edits the running crontab
directly — `varaverk.cron` is regenerated from `schedule.json`.
