Not enterprise HA. No SLAs, no quorum nodes, no guaranteed uptime — don't put your billing system on this.

What it is: mutual automatic failover between two independent unRAID servers. When one goes down the other starts its containers, cuts over DNS, and keeps users online. When it comes back, everything hands back in the correct sequence — DDNS stops, containers stop, rsync writeback runs, containers start on the primary, primary DDNS starts last — so users hit the returning server only after it's actually ready.

Built from scratch. Refined through a year of production testing. The DDNS sequencing and handback order were the hardest parts to get right. Both directions are exercised regularly with fallback_test.sh.

Varaverk owns the cron layer. schedule.json stores every orchestrator's enabled state, cron expression, and log setting. When you save, the plugin rebuilds /etc/cron.d/varaverk — one entry per enabled orchestrator. That's all cron knows about.

Orchestrators are the only scripts with cron schedules. Each one reads its SCRIPTS list out of master.conf and runs each child in sequence. The children do the actual work. The orchestrator controls the order and the timing. A child failure stops the chain.

master.conf is the per-host control plane. Comment out a line to disable that script on this host without touching any code. Both servers run the same repository — different master.conf comments produce different behaviour per machine.

Children only get their own cron when their orchestrator is OFF. When the orch is on, it owns the timing completely.

common.sh provides the shared runtime every script sources: acquire_lock(), log_msg(), info(), success(), warn(), error(). Lock files live in /tmp/varaverk_locks/. acquire_lock() exits cleanly if a prior run is still in progress — nothing piles up.

Logs are written per script to the appdata path. The Log button shows the latest run. Verbose mode passes --log for per-step detail.

Array events bypass cron entirely. array_start fires when Unraid brings the array up; array_stop fires before shutdown. array_started handles all post-start setup in sequence; array_stopping handles the graceful teardown.

The watchdog tier runs every 15 minutes as a single-pass chain: resource_watchdog frees memory pressure first, docker_watchdog restarts down containers into that headroom, system_watchdog checks storage/WebGUI/network, stability_watchdog reboots as a last resort. The order matters — restarting containers before freeing memory just restarts them into the same pressure.

One script. A few lines — checking whether a container was running and restarting it if not. Backgrounded through User Scripts, sleeping a minute between checks, essentially forgotten. It did its job.

Then a conversation with a buddy: wouldn't it be cool if it emailed when something crashed? Sure, added it. Wouldn't it be cool if it handled more containers? Easy — grow the list. Wouldn't it be cool if both servers stayed in sync automatically so neither of us had to remember to do it manually?

That one took a while.

Order started to matter. Freeing memory before attempting container restarts meant the containers actually stayed up. Sync had to happen before the media database refresh, not after. Watchdogs couldn't block each other. The big loop that tried to do everything started breaking into pieces — one piece per job, and a sequencer that knew which piece went first.

master.conf became the control plane: one file, two different sets of comments for two different hosts. Same codebase, different behaviour per machine. The orchestrator pattern came out of not wanting to duplicate logic.

By the time the User Scripts plugin couldn't clearly show what was actually scheduled, whether something was running, or why a sync had quietly stopped working two weeks ago — the interface became the next thing to build.

Varaverk started as a status table and a schedule editor. The scheduler became the main thing. The monitor followed.

Still the same two servers, two households, the same media stack running itself. The scripts just grew up around it.