Document today's restructures, and give data/ a README that ships with it

This commit is contained in:
Gmer4Lfe
2026-08-09 00:01:05 -04:00
parent 38cd3edf99
commit 3a45c18555
12 changed files with 243 additions and 29 deletions
+49 -4
View File
@@ -119,7 +119,7 @@ questions this answers, intent retrieves better and costs far less.
**Only `git ls-files` is ever indexed.** This is not a convenience — it is the security model.
`Configurations/`, `State_Files/` and `data/` are gitignored, so every file holding a credential
`Configurations/` and `data/` are gitignored, so every file holding a credential
was never in the repo to begin with. The index therefore describes the full conf schema (via the
tracked templates, which carry all the explanatory comments) while structurally **never
containing a secret**.
@@ -228,9 +228,54 @@ and a pull that changed twelve files costs a few seconds.
---
## ━━━ PROFILES ━━━
A profile is a contract plus a set of inputs. `Plugin/unraid/include/ai_profiles.php` is the one
definition of both, read by the endpoint, the worker, the shared chat include and the Scheduler
dock.
| Profile | Turns | Retrieves | Notes |
|---|---|---|---|
| `varaverk` | 3 | yes | answers only from the index, with citations. The default. |
| `chat` | 8 | **no** | ordinary conversation. Holds zero capabilities, deliberately. |
| `code` | 4 | no | drafts shell for Custom Scripts; scans its own output for destructive ops |
| `troubleshoot` | 2 | yes | reasons from an open log first, docs second. May file bug reports. |
Capabilities are granted per profile — retrieval, live health, run evidence, scoped log,
incidents, conf lookup, bug filing, code scanning. `chat` holding an empty list is a guarantee,
not an oversight: anything added to it stops being general chat and becomes an assistant that
sometimes invents claims about this installation.
`chat` escalates to `varaverk` on its own when a question is genuinely about Varaverk, and
reverts if the index turns out to have nothing — so the loose profile is safe to sit in.
This used to live in five places — history depth in the endpoint, capabilities in `include/ai.php`,
label and depth again in JavaScript, a prompt branch in the worker, and a label map on the
Scheduler page. They had already drifted: the JavaScript knew three profiles where PHP knew four.
The system prompts still live in `Tools/ai_chat_worker.php`, because they have one reader and
moving them would relocate the most delicate text in the subsystem without removing a duplicate.
## ━━━ CONVERSATIONS ━━━
Chats are stored server-side under `AI_DATA_DIR/ai_chats/`, one JSON file each, saved
automatically when a turn completes and pruned to `AI_CHAT_HISTORY_MAX` (default 10, oldest
first by creation).
There is no Save button. A conversation worth keeping is not reliably one you knew was worth
keeping while you were having it.
The same store backs the AI tab and the Monitor tab's AI row, so a thread started on the
dashboard is the one you carry on in the tab. Messages are re-validated per message on the way
in — a stored chat is replayed into a later prompt when reopened, so an unchecked role written
there would be an injection that survives a reload rather than one turn.
Reopened chats render as plain turns: sources, reasoning and timings describe one generation and
are not stored, because redrawing them beside a transcript that may be continued under a
different profile would be citing evidence for an answer no longer being made.
## ━━━ TOKEN ACCOUNTING ━━━
Every completed `ask` appends one row to `AI_TOKEN_DB` (`data/ai_token_history.db`):
Every completed `ask` appends one row to `AI_TOKEN_DB` (`data/ai/ai_token_history.db`):
```
date|time|host|profile|source|prompt_tokens|completion_tokens|tok_s
@@ -247,13 +292,13 @@ straight from the shell, since it is just a delimited file:
```bash
# tokens used today
awk -F'|' -v d="$(date +%F)" '$1==d {p+=$6; c+=$7} END {print p+c}' data/ai_token_history.db
awk -F'|' -v d="$(date +%F)" '$1==d {p+=$6; c+=$7} END {print p+c}' data/ai/ai_token_history.db
```
**The host column is where the turn ran, not where the file is read.** Each host writes only its
own rows.
`ai_token_sync.sh` pulls each partner's ledger into `/tmp/.cache/vv/ai/<slot>.tokens.db` — the
`ai_token_sync.sh` pulls each partner's ledger into `$AI_TOKEN_CACHE_DIR/<slot>.tokens.db` (`/tmp/varaverk/ai/`) — the
same trick `conf_sync.sh` uses for partner confs, and it runs from
`INTERMEDIATE_MAINTENANCE_SCRIPTS` every four hours. The tab then reads every ledger it can see,
so a fleet total is a fleet total.