# `data/` — everything Varaverk keeps on disk

This directory is the single on-disk root. If Varaverk persists something across a reboot, it is
under here. Move `DATA_DIR` in `master.conf` and the whole tree follows.

This file is the only tracked thing in here. Everything else is runtime data, gitignored, and
per-host — none of it syncs, none of it belongs in the repo, and losing any of it costs at most
a rebuild.

---

## Layout

```
data/
├── db/       statistics, histories, counters, blocklists — things that accumulate
├── state/    runtime state for every script: watchdogs, fallback, transcode, setup
├── ai/       retrieval index, operator memory, token ledger, filed bugs, saved chats
├── cache/    persistent backups of the tmpfs caches — and only those
│   ├── arr/    *_tracked_cache.json, restored into tmpfs on demand
│   └── conf/   partner host*.conf snapshot (0700 — holds credentials)
└── logs/     retained log output
```

Each has a conf variable. Never hardcode a path into a script; use the variable, because the
variable is what a storage-mode migration rewrites.

| Directory | Variable | Notes |
|---|---|---|
| `db/` | `DB_DIR` | |
| `state/` | `STATE_DIR` | ALL state files must use this. No `/tmp`, no repo root. |
| `ai/` | `AI_DATA_DIR` | |
| `cache/` | `CACHE_BACKUP_DIR` | `ARR_CACHE_BACKUP_DIR`, `PERSISTENT_CONF_CACHE` sit under it |
| `logs/` | `LOG_ARCHIVE_DIR` | live logging still goes to `LOG_DIR` (`/var/log/varaverk`) |

---

## What is NOT here

**The tmpfs caches.** They live under `VV_CACHE_ROOT` (`/tmp/varaverk/`) and must stay there.
The WebGUI payload cache is read every one to five seconds by every open tab, and the arr item
caches are rewritten by the hundred megabytes. On flash that is pointless write load for data
whose entire purpose is to be cheap and disposable.

```
/tmp/varaverk/
├── api/    WebGUI payloads — monitor, arrs, ai
├── arr/    arr payloads (backed up to data/cache/arr/)
├── conf/   partner confs   (backed up to data/cache/conf/)
├── ai/     partner token ledgers — deliberately never backed up
└── jobs/   in-flight AI answers and container actions (0700)
```

**Locks.** `/tmp/unraid_locks/` — deliberately outside both roots, and deliberately on tmpfs so a
lock cannot outlive the boot that took it.

**Configuration.** `Configurations/` holds the confs. Data and config stay apart: one is written
by scripts, the other by you.

---

## Where a new file goes

Ask what happens if it is deleted.

- *Something is permanently lost* → it is a source of truth. `db/` if it accumulates, `state/`
  if it describes right now.
- *It gets re-fetched and nothing else changes* → it is a cache. If it also lives in tmpfs, its
  backup goes in `cache/`. If it only lives here, it is not really a cache — put it in `db/`.
- *Nothing at all* → it should not be written to disk in the first place.

`cache/` is the one that gets misused. It means "persistent backup of a tmpfs cache", not
"anything cache-shaped". A file that is only ever here is a source of truth no matter what it is
called — `lidarr_art_miss_cache.tsv` has "cache" in its name and lives in `db/` for exactly that
reason.

---

## History

Until 2026-08-08 this was two roots and two strays: `data/` and `State_Files/` as siblings, the
conf-cache backup off in `SCRIPTS_DIR/.cache/vv/d`, and the arr backups loose in `data/`'s root.
No single decision there was wrong. Together they meant nothing answered "what does Varaverk keep
on disk", and the PHP layer — which cannot source bash — restated the paths it needed, so the two
layers agreed only by hand.

State is data. It is the data that happens to describe right now, so it belongs under the same
root as the rest.

`STATE_DIR` kept its name and changed only its value, which is why that restructure did not touch
the 15 conf entries, 18 shell paths and 23 PHP paths built on it.

**Migrating an installation:** `Deployment/migrate_data_layout.sh --dry-run` first, then without
the flag. It is idempotent, it moves rather than copies, and it rewrites the conf values that
`conf_upgrade` deliberately will not touch. Each host runs it itself — this directory is
gitignored, so the restructure travels as code and conf while the files stay put.