FailedProxy/Varaverk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
d8e3c09b577120060272c6199dd23cbfbc46f987
Varaverk/Plugin/unraid/include/ai.php
T
Gmer4Lfe 2bda4cdaf8 Declare the AI owner, and let every node borrow its model over the mesh 
vv_is_ai_host() was `=== 'host1'`, which made a physical fact — that is where the
GPU is — look like a rule. AI_OWNER_HOST declares it, so the card can move to a
rebuilt host3 or a friend's spare.

The gate was also doing two jobs. Assistant docks and findings strips now ask
whether a model is reachable, so a node without a GPU gets them by borrowing;
the AI tab asks whether this is the owner, because that page carries the bug
reports, the index and the model configuration — the surface where the
vocabulary assumes you built the mesh.

Resolution is local, then owner, then anyone else declaring a model, pinned once
it answers. Pinned rather than re-derived per call: a mesh that re-decides every
request eventually decides differently mid-conversation, and a chat whose second
turn lands on another machine has no history there. Cleared only on a transport
failure, and only when there is somewhere else to go — a single-node mesh whose
model is down should say so, not report AI as unconfigured.
2026-08-14 22:54:26 -04:00

1879 lines
93 KiB
PHP
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<?php
// ═══════════════════════════════════════════════════════════════════════════════════════════════
// PURPOSE
// The AI tab's library. Resolves AI configuration, reports the health of the retrieval
// subsystem for the status banner, runs retrieval against the index, and owns the job-file
// contract the chat worker and the polling endpoint share.
//
// OPERATIONAL MODEL
// Retrieval is delegated, not reimplemented. AI/lib/search.js already embeds the query,
// scans the index and applies intent routing; this file shells to that CLI with --json. A
// second PHP implementation of vector search would be a second set of scoring bugs, and the
// two would drift the first time the chunker changed.
//
// Generation is not performed here. It takes 25-76 seconds and belongs in a detached worker;
// this file only describes where that worker writes and what the states mean.
//
// DESIGN PRINCIPLES
// Every reported statistic distinguishes "unknown" from "bad".
// A banner that shows a healthy default when it cannot reach Ollama is worse than one
// that shows nothing, because the operator stops checking. Each probe returns null when
// it could not determine the answer.
//
// Offload percentage is a first-class metric.
// size_vram against size is the difference between 74 tok/s and 19 on this hardware, and
// nothing else on the host surfaces it. It is computed here so the banner and any future
// watchdog read the same number.
//
// Staleness is measured against tracked files, not the filesystem.
// The index only ever contains git-tracked content, so comparing to an untracked scratch
// file would report permanent staleness. git ls-files is the same source the indexer uses.
//
// Health checks compare configuration against reality, and carry their remedy.
// "Retrieval failed" is not actionable; "HOST1_OLLAMA_MODEL names a tag that is not
// installed" is. Every check names the setting to change, because the failures this
// subsystem actually has are configuration drift — a model pulled, wired into conf, then
// removed — rather than software faults.
//
// Logs are read live, never indexed.
// They churn constantly, would dominate a 3341-chunk corpus, and embedding similarity
// retrieves log lines poorly compared with recency and a severity filter. Indexing them
// would also mean re-embedding every few minutes for content that is stale immediately.
//
// OPERATIONAL SAFEGUARDS
// Read-only. Nothing here writes to the index, the conf, or the job files — it reads state
// and runs a retrieval query. The worker owns every write.
//
// Every external call is time-boxed.
// Retrieval and the Ollama probes carry explicit timeouts, because this library is
// loaded by a page render and by a polling endpoint; a hung Ollama must degrade the
// banner, not hang the tab.
//
// Retrieval arguments are validated before they reach the shell.
// kind is checked against the five real values and k is clamped, then every value is
// escapeshellarg'd. An unknown kind would match zero rows in SQL and read as "the index
// has no answer", which is the most misleading failure this subsystem can produce.
//
// Tokens are validated as hex before they compose a path.
// Job files live in a fixed directory and are named from a caller-supplied token; the
// pattern is what keeps that token from escaping the directory.
//
// A missing index is reported, never treated as empty.
// Empty results and an absent index look identical to a caller that only counts rows,
// and they need completely different responses from the user.
//
// EXPORTS
// Config vv_ai_enabled(), vv_ai_config()
// Status vv_ai_stats(), vv_ai_index_stats(), vv_ai_runtime_stats(), vv_ai_index_meta()
// Models vv_ai_models_available(), vv_ai_models_loaded()
// Diagnosis vv_ai_health(), vv_ai_recent_logs()
// Memory vv_ai_memory_path(), vv_ai_memory_read(), vv_ai_memory_write()
// Retrieval vv_ai_retrieve()
// Jobs vv_ai_job_dir(), vv_ai_job_path(), vv_ai_job_read()
//
// CONFIGURATION
// master.conf AI_ENABLED, AI_INDEX_DB, AI_SEARCH_K, AI_SEARCH_PER_FILE,
// AI_REQUEST_TIMEOUT, AI_CONNECT_TIMEOUT
// host*.conf HOST*_OLLAMA_URL, HOST*_OLLAMA_MODEL, HOST*_OLLAMA_EMBED_MODEL
// ═══════════════════════════════════════════════════════════════════════════════════════════════
require_once __DIR__ . '/config.php';
// vv_conf_key_is_secret() — the same definition of "this key holds a credential" that decides
// what the conf audit log redacts. One list, so the two cannot disagree about what a secret is.
require_once __DIR__ . '/confform.php';
// VV_AI_JOB_DIR and VV_AI_TOKEN_CACHE_DIR are defined in config.php with every other cache path,
// read from master.conf so this layer and load_config.sh cannot disagree about where they are.
// Must match what AI/lib/chunk.js actually assigns, and what the AI tab's retrieval-source
// dropdown offers. All three drifted: the indexer has always produced 'ui' for
// Plugin/unraid/pages/readme/*.md and the dropdown has always offered it, but this whitelist
// omitted it — so choosing "WebUI — how the pages work" and asking anything returned
// "Unknown kind: ui" and no answer at all. The page help was indexed and unreachable through
// the one filter built to reach it.
const VV_AI_KINDS = ['header', 'readme', 'manual', 'template', 'doc', 'ui'];
// Profiles — what each one is, and what it is allowed to see and do. One table, in one file,
// read by everything: this endpoint, the worker, and the shared chat include every page renders.
// vv_ai_profile_can() and friends come from there.
require_once __DIR__ . '/ai_profiles.php';
// Whether a General Chat message is really about this installation. Shared by the deterministic
// backstop and the handoff, so both agree by construction: a question the backstop would have
// refused is exactly the one the handoff should escalate, and drift between the two would leave
// a class of question that is neither answered nor deferred.
function vv_ai_chat_needs_varaverk(string $question): bool {
return vv_ai_mentions_varaverk($question)
|| (bool)preg_match('/\b[\w.-]+\.sh\b|\b[A-Z][A-Z0-9]*(_[A-Z0-9]+)+\b/', $question)
// Naming something that has a log is as Varaverk-specific as naming a script file, and
// this is how an operator actually refers to them: "the daily orch", not
// "daily_sync_maintenance.sh". Without this, "how did the daily orch go" asked in General
// Chat was refused correctly and then went nowhere — the refusal is right, but the
// question was answerable one profile up and the handoff could not see that.
|| vv_ai_resolve_run_target($question) !== '';
}
// Is the operator asking about THEIR OWN machine, without naming anything on it?
//
// A different axis from vv_ai_chat_needs_varaverk(), and the reason that one cannot be extended
// to cover this. That detector recognises Varaverk-shaped tokens — a .sh file, a SCREAMING_CASE
// key, a run target — so its complement is every question that names none, which is unbounded.
// "why is my array so slow tonight", "how much space do I have left" and "is everything healthy"
// are all about this machine and contain nothing to detect. Enumerating that class topic by topic
// is endless; the operator referring to their own system is not, because it is grammar rather
// than subject matter. Possessives, this-machine deictics, state questions and questions about
// what happened here cover most of it in about a dozen patterns.
//
// Used to decide when NOT to relax the chat profile's caution. Wrong in the safe direction: a
// false positive costs a redirect to a profile that can actually see the machine, a false
// negative risks a general web page answered as though it described this installation.
function vv_ai_asks_about_this_system(string $question): bool {
$q = ' ' . mb_strtolower(trim($question)) . ' ';
$builtin = [
// Theirs, or the box in front of them.
'/\b(my|our|mine)\b/',
'/\bthis (box|server|machine|system|install(ation)?|setup|rig|host|array|pool|cache|node)\b/',
// State of something here. "is everything healthy", "are the containers up".
'/\b(is|are|was|were)\b[^.?!]{0,40}\b(health(y|ier)?|ok|okay|fine|up|down|running|stopped|'
. 'slow|fast|full|empty|broken|failing|stuck|degraded|offline|online)\b/',
// What happened here, and when.
'/\bwhat (happened|went wrong|is (going on|happening))\b/',
'/\b(last night|this morning|yesterday|today|tonight|right now|currently|at the moment)\b/',
// Capacity and load, which are only ever about the machine being asked from.
'/\bhow (much|many)\b[^.?!]{0,30}\b(space|room|ram|memory|disk|storage|free|left|used)\b/',
'/\bdo i have\b/',
// Ownership of running things.
'/\b(my|the) (containers?|dockers?|shares?|disks?|drives?|pools?|arrays?)\b/',
];
foreach ($builtin as $re) if (preg_match($re, $q)) return true;
// Operator-supplied phrases, for the ones grammar does not reach — a nickname for the box, a
// share name, whatever turns up in practice. Pipe-separated rather than a conf array because
// vv_conf_vars() reads scalars and hands back "(" for an array, and this needs to be readable
// at question time from a settings field the operator can edit.
//
// Matched as literal text, never as patterns. The field is editable from the settings UI, and
// a regex arriving from there would run against every question asked.
$extra = trim((string) (vv_conf_vars()['AI_CHAT_MY_SYSTEM_PHRASES'] ?? ''));
if ($extra !== '') {
foreach (explode('|', $extra) as $phrase) {
$phrase = mb_strtolower(trim($phrase));
if ($phrase !== '' && mb_strpos($q, $phrase) !== false) return true;
}
}
return false;
}
// Asking for a script to be written, as opposed to asking about one that exists. The verb is
// required: "what does arr_sync.sh do" names a script and wants documentation, "write me a script
// that does X" names none and wants code. Getting that backwards in either direction is the whole
// risk here, so the pattern is anchored on the request rather than on the word "script".
function vv_ai_wants_code(string $question): bool {
return (bool)preg_match(
'/\b(write|create|make|draft|generate|build|give\s+me|need|want)\b[^.?!]{0,40}'
. '\b(a\s+|an\s+|me\s+a\s+)?(bash\s+|shell\s+|sh\s+)?'
. '(script|one[- ]?liner|cron\s?job|command|snippet)\b/i',
$question
);
}
// Diagnostic phrasing — "why did X fail", "X is broken", "it never runs". Shared with the
// worker's own gate so the router and the gate cannot disagree about what counts as a diagnosis.
function vv_ai_is_diagnostic(string $question): bool {
return (bool)preg_match(
'/\b(why|fail(ed|ing|ure)?|error|broken?|not work|isn.t work|wrong|stuck|hang|'
. 'never runs?|didn.t|won.t|debug|troubleshoot|diagnos)/i',
$question
);
}
// Which profile should answer a General Chat question, or '' to leave it in chat.
//
// One router, ordered most specific first, because these overlap on purpose: "why did the script
// I asked you to write fail" is diagnostic AND mentions a script, and the operator means the
// first. The order is the policy.
//
// code asked for something to be written. Checked first — it is the only intent that
// is about a thing that does not exist yet, so nothing else can claim it.
// troubleshoot something is wrong. Requires diagnostic phrasing AND something Varaverk-shaped
// to diagnose; "why is the sky blue" is diagnostic phrasing about nothing here.
// varaverk about this installation, but not a fault and not a request for code.
//
// Deliberately conservative: an unrecognised question stays in chat, which is the profile that
// cannot invent claims about this system. Escalation adds capability, so a wrong escalation costs
// more than a missed one — and the worker reverts to chat anyway if retrieval comes back empty.
function vv_ai_route_from_chat(string $question): string {
if (vv_ai_wants_code($question)) return 'code';
if (vv_ai_is_diagnostic($question) && vv_ai_chat_needs_varaverk($question)) return 'troubleshoot';
if (vv_ai_chat_needs_varaverk($question)) return 'varaverk';
return '';
}
function vv_ai_config(): array {
static $cfg = null;
if ($cfg !== null) return $cfg;
$vars = vv_conf_vars();
// The node whose model this one uses — itself if it has one, the owner's otherwise. Was
// vv_detect_host(), which meant a node without a GPU had no AI rather than borrowing one.
$modelHost = vv_ai_model_host();
$host = strtoupper($modelHost);
// Borrowed models are reached by hostname, not by the owner's own loopback. HOST1_OLLAMA_URL
// is http://localhost:11434, which is true on host1 and points at nothing anywhere else, so a
// remote borrower substitutes the owner's Tailscale name — the mesh resolves those and there
// are no hardcoded IPs anywhere in this project.
//
// The owner keeps loopback for its own calls. Rewriting it for everyone would push host1's own
// traffic onto the tailnet interface and make local AI depend on Tailscale being up, which is
// a real dependency to add for no gain.
$rawUrl = rtrim(trim($vars["{$host}_OLLAMA_URL"] ?? ''), '/');
if ($modelHost !== vv_detect_host() && $rawUrl !== '') {
$ownerName = trim((string)($vars[strtoupper($modelHost)] ?? ''));
if ($ownerName !== '') {
$rawUrl = preg_replace('#://(localhost|127\.0\.0\.1)([:/]|$)#i',
'://' . $ownerName . '$2', $rawUrl);
}
}
$cfg = [
'enabled' => strtolower(trim($vars['AI_ENABLED'] ?? 'false')) === 'true',
'url' => $rawUrl,
'model_host' => $modelHost,
'borrowed' => $modelHost !== vv_detect_host(),
'model' => trim($vars["{$host}_OLLAMA_MODEL"] ?? ''),
'embed_model' => trim($vars["{$host}_OLLAMA_EMBED_MODEL"] ?? 'nomic-embed-text'),
'db' => trim($vars['AI_INDEX_DB'] ?? '') ?: AI_DATA_DIR . '/ai_index.db',
'k' => (int)($vars['AI_SEARCH_K'] ?? 8),
'per_file' => (int)($vars['AI_SEARCH_PER_FILE'] ?? 3),
'timeout' => (int)($vars['AI_REQUEST_TIMEOUT'] ?? 240),
'connect' => (int)($vars['AI_CONNECT_TIMEOUT'] ?? 5),
];
// AI_INDEX_DB is written as "$DATA_DIR/ai_index.db" in conf; the shell expands it, PHP does
// not. Left as a literal it would name a file that cannot exist.
$cfg['db'] = str_replace(['$DATA_DIR', '${DATA_DIR}'], DATA_DIR, $cfg['db']);
return $cfg;
}
function vv_ai_enabled(): bool {
return vv_ai_config()['enabled'];
}
// Index size, coverage and staleness. Staleness compares the newest git-tracked file against
// the last build: the indexer only ever ingests tracked content, so anything else would report
// permanent drift.
function vv_ai_index_stats(): array {
$cfg = vv_ai_config();
$out = ['exists' => false, 'chunks' => 0, 'files' => 0, 'size' => 0,
'built' => null, 'newest_source' => null, 'stale' => null, 'kinds' => []];
if (!file_exists($cfg['db'])) return $out;
$out['exists'] = true;
$out['size'] = (int)@filesize($cfg['db']);
$db = escapeshellarg($cfg['db']);
$q = fn(string $sql) => trim((string)@shell_exec('sqlite3 ' . $db . ' ' . escapeshellarg($sql) . ' 2>/dev/null'));
$out['chunks'] = (int)$q('SELECT COUNT(*) FROM vv_chunks;');
$out['files'] = (int)$q('SELECT COUNT(*) FROM vv_files;');
$built = (int)$q('SELECT MAX(indexed) FROM vv_files;');
$out['built'] = $built ?: null;
foreach (explode("\n", $q('SELECT kind, COUNT(*) FROM vv_chunks GROUP BY kind;')) as $line) {
if (!$line) continue;
[$k, $n] = array_pad(explode('|', $line, 2), 2, 0);
if ($k !== '') $out['kinds'][$k] = (int)$n;
}
$newest = trim((string)@shell_exec(
'cd ' . escapeshellarg(SCRIPTS_DIR) . ' && git ls-files -z 2>/dev/null'
. ' | xargs -0 stat -c %Y 2>/dev/null | sort -rn | head -1'
));
if ($newest !== '') {
$out['newest_source'] = (int)$newest;
if ($out['built'] !== null) $out['stale'] = (int)$newest > $out['built'];
}
return $out;
}
// Ollama reachability and — the number that matters on this hardware — whether the generation
// model is fully resident on the GPU. size_vram below size means layers are on the CPU, which
// costs roughly 4x throughput and is invisible everywhere else.
function vv_ai_runtime_stats(): array {
$cfg = vv_ai_config();
$out = ['reachable' => false, 'loaded' => null, 'offload_pct' => null,
'context' => null, 'vram_used' => null, 'vram_total' => null, 'gpu' => null];
if ($cfg['url'] === '') return $out;
$ctx = stream_context_create(['http' => [
'method' => 'GET', 'timeout' => max(2, $cfg['connect']), 'ignore_errors' => true,
]]);
$raw = @file_get_contents($cfg['url'] . '/api/ps', false, $ctx);
if ($raw === false) return $out;
$out['reachable'] = true;
$ps = json_decode($raw, true);
foreach ($ps['models'] ?? [] as $m) {
if (($m['name'] ?? '') !== $cfg['model']) continue;
$size = (int)($m['size'] ?? 0);
$vram = (int)($m['size_vram'] ?? 0);
$out['loaded'] = true;
$out['context'] = $m['context_length'] ?? null;
$out['vram_used'] = $vram;
$out['vram_total'] = $size;
$out['offload_pct'] = $size > 0 ? (int)round($vram / $size * 100) : null;
break;
}
if ($out['loaded'] === null) $out['loaded'] = false;
$gpu = trim((string)@shell_exec(
'nvidia-smi --query-gpu=name,memory.used,memory.total,utilization.gpu '
. '--format=csv,noheader,nounits 2>/dev/null | tail -1'
));
if ($gpu !== '') {
$p = array_map('trim', explode(',', $gpu));
if (count($p) >= 4) {
$out['gpu'] = ['name' => $p[0], 'mem_used' => (int)$p[1],
'mem_total' => (int)$p[2], 'util' => (int)$p[3]];
}
}
return $out;
}
// Models Ollama actually has on disk. The configured model naming a tag that does not exist is
// the single most likely misconfiguration here — it is what happens whenever a model is pulled,
// wired into conf, and later removed.
function vv_ai_models_available(): ?array {
$cfg = vv_ai_config();
if ($cfg['url'] === '') return null;
$ctx = stream_context_create(['http' => ['timeout' => max(2, $cfg['connect']), 'ignore_errors' => true]]);
$raw = @file_get_contents($cfg['url'] . '/api/tags', false, $ctx);
if ($raw === false) return null;
$d = json_decode($raw, true);
if (!isset($d['models'])) return null;
return array_map(fn($m) => [
'name' => $m['name'] ?? '',
'size' => (int)($m['size'] ?? 0),
], $d['models']);
}
// Currently resident models with their offload split.
function vv_ai_models_loaded(): ?array {
$cfg = vv_ai_config();
if ($cfg['url'] === '') return null;
$ctx = stream_context_create(['http' => ['timeout' => max(2, $cfg['connect']), 'ignore_errors' => true]]);
$raw = @file_get_contents($cfg['url'] . '/api/ps', false, $ctx);
if ($raw === false) return null;
$d = json_decode($raw, true);
if (!isset($d['models'])) return null;
$out = [];
foreach ($d['models'] as $m) {
$size = (int)($m['size'] ?? 0);
$vram = (int)($m['size_vram'] ?? 0);
$out[] = [
'name' => $m['name'] ?? '',
'context' => $m['context_length'] ?? null,
'size' => $size,
'vram' => $vram,
'offload' => $size > 0 ? (int)round($vram / $size * 100) : null,
];
}
return $out;
}
// vv_meta records the embed model and dimensionality the index was built with.
function vv_ai_index_meta(): array {
$cfg = vv_ai_config();
if (!file_exists($cfg['db'])) return [];
// Columns are k/v, not key/value. Named wrongly this returns nothing and the embedder-match
// check reports "unrecorded" — a silent pass on the one mismatch that returns confident
// nonsense rather than an error.
$raw = trim((string)@shell_exec(
'sqlite3 ' . escapeshellarg($cfg['db']) . ' ' . escapeshellarg('SELECT k, v FROM vv_meta;') . ' 2>/dev/null'
));
$out = [];
foreach (explode("\n", $raw) as $line) {
if ($line === '') continue;
[$k, $v] = array_pad(explode('|', $line, 2), 2, '');
if ($k !== '') $out[$k] = $v;
}
return $out;
}
// Config checked against reality. Each check is ok | warn | bad, with the remedy attached —
// the point is to name the setting that is wrong, not merely report that something failed.
function vv_ai_health(): array {
$cfg = vv_ai_config();
$checks = [];
$add = function (string $id, string $label, string $state, string $detail, string $fix = '')
use (&$checks) {
$checks[] = ['id' => $id, 'label' => $label, 'state' => $state,
'detail' => $detail, 'fix' => $fix];
};
$add('enabled', 'AI enabled', $cfg['enabled'] ? 'ok' : 'bad',
$cfg['enabled'] ? 'AI_ENABLED=true' : 'AI_ENABLED is false',
$cfg['enabled'] ? '' : 'Set AI_ENABLED=true in master.conf');
if (!command_exists_node()) {
$add('node', 'node runtime', 'bad', 'node not found on PATH',
'Retrieval shells to AI/lib/cli.js and cannot run without it');
}
if ($cfg['url'] === '') {
$add('url', 'Ollama URL', 'bad', 'not configured',
'Set ' . strtoupper(vv_detect_host()) . '_OLLAMA_URL in the host conf');
return $checks;
}
$avail = vv_ai_models_available();
if ($avail === null) {
$add('reach', 'Ollama reachable', 'bad', 'no response from ' . $cfg['url'],
'Check the Ollama container is running and the URL is correct');
return $checks;
}
$add('reach', 'Ollama reachable', 'ok', $cfg['url']);
$names = array_column($avail, 'name');
// The configured tag not existing is the failure this whole section is for.
if ($cfg['model'] === '') {
$add('gen', 'Generation model', 'bad', 'not configured',
'Set ' . strtoupper(vv_detect_host()) . '_OLLAMA_MODEL in the host conf');
} elseif (!vv_ai_model_installed($cfg['model'], $names)) {
$add('gen', 'Generation model', 'bad', $cfg['model'] . ' is not installed',
'Either `ollama pull` it, or point _OLLAMA_MODEL at one of: ' . implode(', ', array_slice($names, 0, 4)));
} else {
$add('gen', 'Generation model', 'ok', $cfg['model']);
}
if ($cfg['embed_model'] === '' || !vv_ai_model_installed($cfg['embed_model'], $names)) {
$add('embed', 'Embedding model', 'bad',
($cfg['embed_model'] ?: 'not configured') . ' is not installed',
'Retrieval cannot embed a query without it — `ollama pull ' . ($cfg['embed_model'] ?: 'nomic-embed-text') . '`');
} else {
$add('embed', 'Embedding model', 'ok', $cfg['embed_model']);
}
$ix = vv_ai_index_stats();
$meta = vv_ai_index_meta();
if (!$ix['exists']) {
$add('index', 'Index', 'bad', 'not built', 'Run AI/ai_index.sh');
} else {
$add('index', 'Index', 'ok', number_format($ix['chunks']) . ' chunks from ' . $ix['files'] . ' files');
// A vector built by one embedding model is meaningless to another. Changing the embed
// model without reindexing does not error — it silently returns nonsense, scored
// confidently, which is the hardest failure here to notice from the answers alone.
$builtWith = $meta['embed_model'] ?? '';
if ($builtWith !== '' && $cfg['embed_model'] !== ''
&& vv_ai_model_norm($builtWith) !== vv_ai_model_norm($cfg['embed_model'])) {
$add('embedmatch', 'Index / embedder match', 'bad',
'index built with ' . $builtWith . ', conf says ' . $cfg['embed_model'],
'Vectors from different models are not comparable — rerun AI/ai_index.sh --force');
} else {
$add('embedmatch', 'Index / embedder match', 'ok', $builtWith ?: 'unrecorded');
}
if ($ix['stale'] === true) {
$add('fresh', 'Index freshness', 'warn', 'tracked files are newer than the index',
'Answers may cite code that has changed — rerun AI/ai_index.sh');
} else {
$add('fresh', 'Index freshness', 'ok', 'current');
}
}
$loaded = vv_ai_models_loaded();
if ($loaded !== null && $cfg['model'] !== '') {
$hit = null;
foreach ($loaded as $m) if ($m['name'] === $cfg['model']) { $hit = $m; break; }
if ($hit === null) {
$add('resident', 'Model resident', 'warn', 'not loaded — first question will load it');
} elseif ($hit['offload'] !== null && $hit['offload'] < 100) {
$add('resident', 'GPU offload', 'bad', $hit['offload'] . '% on GPU — layers on CPU',
'Roughly 4x slower on this card. Lower num_ctx or use a smaller quant');
} else {
$add('resident', 'GPU offload', 'ok', '100% on GPU at ' . ($hit['context'] ?? '?') . ' ctx');
}
}
return $checks;
}
// Ollama reports tags fully qualified — nomic-embed-text:latest — while conf commonly carries
// the bare name, and both are valid references. Comparing literally reports an installed model
// as missing, which is exactly the false alarm that makes a health panel worth ignoring.
function vv_ai_model_norm(string $name): string {
$name = trim($name);
if ($name === '') return '';
// A colon in the final path segment is a tag; anything else is a registry path.
$last = substr($name, strrpos($name, '/') === false ? 0 : strrpos($name, '/') + 1);
return str_contains($last, ':') ? $name : $name . ':latest';
}
function vv_ai_model_installed(string $name, array $available): bool {
$n = vv_ai_model_norm($name);
foreach ($available as $m) {
if (vv_ai_model_norm($m) === $n) return true;
}
return false;
}
function command_exists_node(): bool {
static $has = null;
if ($has !== null) return $has;
return $has = trim((string)@shell_exec('command -v node 2>/dev/null')) !== '';
}
// Recent warnings and errors across the orchestrator logs. Read live rather than indexed:
// logs churn constantly, would dominate a 3341-chunk index, and embedding similarity retrieves
// them poorly compared with recency plus a severity filter.
function vv_ai_recent_logs(int $max = 40): array {
$dir = '/var/log/varaverk';
if (!is_dir($dir)) return [];
$cmd = 'grep -rhE "\[(ERROR|WARN|CRITICAL|FAILED)\]|✗" ' . escapeshellarg($dir)
. ' --include="*.log" 2>/dev/null | tail -' . max(1, min($max, 200));
$raw = (string)@shell_exec($cmd);
$out = [];
foreach (explode("\n", $raw) as $line) {
$line = trim(preg_replace('/\033\[[0-9;]*[mK]/', '', $line));
if ($line === '') continue;
$out[] = mb_substr($line, 0, 300);
}
return $out;
}
function vv_ai_stats(): array {
$cfg = vv_ai_config();
return [
'enabled' => $cfg['enabled'],
'model' => $cfg['model'],
'embed_model' => $cfg['embed_model'],
'url' => $cfg['url'],
'k' => $cfg['k'],
'index' => vv_ai_index_stats(),
'runtime' => vv_ai_runtime_stats(),
'loaded' => vv_ai_models_loaded(),
'health' => vv_ai_health(),
'ts' => time(),
];
}
// ── Shared collection ─────────────────────────────────────────────────────────
// vv_ai_stats() costs about a second on this host — vv_ai_runtime_stats() alone is 60-480ms
// depending on how quickly Ollama and nvidia-smi answer, and it was being paid by the AI tab
// every 30 seconds per open tab, plus again by anything else that wanted the same numbers.
//
// So it is collected once, by Tools/api_cache_writer.php, into the 'ai' cache; every surface
// reads that. This is the same arrangement the monitor and arrs payloads already use and for the
// same reason — polling faster cannot make the figures newer, it only decides how soon a page
// notices the writer's update.
//
// ?live=1 stays available for the one case that needs it: you changed something and want to see
// the result rather than a payload written before you changed it.
function vv_ai_stats_cached(bool $live = false): array {
if (!$live) {
$c = vv_cache_read('ai', 300);
if ($c !== null) return $c;
}
return vv_ai_stats();
}
// The Monitor tab's slice of the same collection. Derived rather than collected: taking the AI
// row's figures from a second call to vv_ai_runtime_stats() would pay the whole cost twice per
// cache write, and — worse — the dashboard and the AI tab could disagree about whether the model
// is resident, because they would have asked at different moments.
//
// Tokens are deliberately absent. This block once carried today's total for a single line on the
// AI card; that line is now a card of its own, which fetches the whole ledger from
// api/ai.php?action=tokens because it shows the week and all-time beside today. Leaving the field
// here would have run vv_ai_token_stats() on every cache write — once a minute, forever — and
// discarded all but one number of it that nothing reads.
function vv_ai_monitor_block(array $stats): array {
return [
'model' => $stats['model'] ?? '',
'runtime' => $stats['runtime'] ?? [],
'index' => $stats['index'] ?? [],
];
}
// Retrieval via AI/lib/cli.js. Returns ['ok'=>bool,'results'=>[],'intents'=>[],'error'=>?string].
function vv_ai_retrieve(string $query, string $kind = '', string $section = '', ?int $k = null): array {
$cfg = vv_ai_config();
if ($query === '') return ['ok' => false, 'error' => 'query is required', 'results' => []];
if ($cfg['url'] === '') return ['ok' => false, 'error' => 'Ollama URL is not configured', 'results' => []];
if (!file_exists($cfg['db'])) return ['ok' => false, 'error' => 'No index — run AI/ai_index.sh', 'results' => []];
if ($kind !== '' && !in_array($kind, VV_AI_KINDS, true)) {
return ['ok' => false, 'error' => 'Unknown kind: ' . $kind, 'results' => []];
}
$cli = SCRIPTS_DIR . '/AI/lib/cli.js';
if (!file_exists($cli)) return ['ok' => false, 'error' => 'AI/lib/cli.js not found', 'results' => []];
$k = max(1, min($k ?? $cfg['k'], 25));
$cmd = 'timeout ' . max(10, $cfg['connect'] * 6) . ' node --no-warnings '
. escapeshellarg($cli) . ' search'
. ' --db=' . escapeshellarg($cfg['db'])
. ' --url=' . escapeshellarg($cfg['url'])
. ' --model=' . escapeshellarg($cfg['embed_model'])
. ' --query=' . escapeshellarg($query)
. ' --k=' . $k
. ' --per-file=' . max(1, $cfg['per_file'])
. ($kind !== '' ? ' --kind=' . escapeshellarg($kind) : '')
. ($section !== '' ? ' --section=' . escapeshellarg($section) : '')
. ' --json 2>/dev/null';
$raw = trim((string)@shell_exec($cmd));
if ($raw === '') return ['ok' => false, 'error' => 'Retrieval produced no output', 'results' => []];
$d = json_decode($raw, true);
if (!is_array($d) || !isset($d['results'])) {
return ['ok' => false, 'error' => 'Retrieval returned unparseable output', 'results' => []];
}
return ['ok' => true, 'results' => $d['results'], 'intents' => $d['intents'] ?? [],
'scanned' => $d['scanned'] ?? 0];
}
// ── Question shape ───────────────────────────────────────────────────────────────────────────
//
// "Varaverk" is a coined word with no spell-check and an awkward keyboard shape, so it arrives
// misspelled often — varavrk, veraverk, varavek, varverk. Edit distance catches those without a
// hand-written pattern that would need extending every time a new typo appears.
//
// This is search, not identity. The rule against fuzzy hostname matching is about deciding which
// machine you are talking to, where a near-miss must never resolve; here a near-miss only widens
// a document search, and the cost of being wrong is a slightly odd set of passages.
function vv_ai_mentions_varaverk(string $q): bool {
foreach (preg_split('/[^a-z0-9]+/i', mb_strtolower($q), -1, PREG_SPLIT_NO_EMPTY) as $w) {
if (strlen($w) < 5 || strlen($w) > 12) continue; // bound the comparison window
if ($w === 'varaverk' || levenshtein($w, 'varaverk') <= 2) return true;
}
return false;
}
// "What is Varaverk", "explain varavrk", "whats this thing do". Intent routing boosts PURPOSE
// for these, which surfaces every script's one-line purpose and buries the top-level prose that
// actually defines the system — so the model answers that the context does not define it, while
// README.md sits in the index unread.
function vv_ai_is_definitional(string $q): bool {
if (!vv_ai_mentions_varaverk($q)) return false;
return (bool)preg_match(
'/\b(what\'?s?|whats|define|definition|describe|explain|overview|tell me about|'
. 'purpose of|point of|elevator|in a nutshell)\b/i', $q);
}
// ── Memory ───────────────────────────────────────────────────────────────────────────────────
//
// A small operator-maintained file handed to the model at the start of every conversation.
// Injected, never indexed: it changes constantly, and vector similarity is the wrong retrieval
// mechanism for "things I was told to remember". Living under DATA_DIR keeps it out of the
// repository and therefore out of the index by construction.
// Two memory slots, and the distinction between them is the whole point.
//
// assisted — written by the operator. Authoritative: the prompt tells the model to prefer it
// over retrieved passages, because a human asserting a fact about their own machine
// outranks a doc that may be stale.
// learned — proposed by the assistant and accepted by the operator. A hint only. Retrieval
// overrules it, and it is trimmed first when the budget bites. A model that can
// write memory the prompt ranks above the source code will restate its own mistakes
// forever, so it is deliberately given the weaker seat.
//
// Kept as two named slots rather than an arbitrary list: two is what the split is for, and a
// general N-file scheme would be machinery serving nobody.
function vv_ai_memory_path(string $kind = 'assisted'): string {
$vars = vv_conf_vars();
$key = $kind === 'learned' ? 'AI_MEMORY_LEARNED_FILE' : 'AI_MEMORY_ASSISTED_FILE';
$p = trim($vars[$key] ?? '');
// Back-compat. Installs upgraded from the single-file era have AI_MEMORY_FILE set and a
// populated ai_memory.md; losing that silently would drop everything the operator had
// written. The legacy path wins only while the new one does not exist.
if ($kind === 'assisted') {
$legacy = trim($vars['AI_MEMORY_FILE'] ?? '');
$legacy = $legacy !== '' ? $legacy : AI_DATA_DIR . '/ai_memory.md';
$legacy = str_replace(['$DATA_DIR', '${DATA_DIR}'], DATA_DIR, $legacy);
$newp = $p !== '' ? str_replace(['$DATA_DIR', '${DATA_DIR}'], DATA_DIR, $p)
: AI_DATA_DIR . '/mem_assisted.md';
if (!file_exists($newp) && file_exists($legacy)) return $legacy;
return $newp;
}
$p = str_replace(['$DATA_DIR', '${DATA_DIR}'], DATA_DIR, $p);
return $p !== '' ? $p : AI_DATA_DIR . '/mem_learned.md';
}
// A hard ceiling on the learned file, well under the total. Without it a store that grows on its
// own eventually occupies the whole budget and the operator's own memory is what gets truncated
// away — the exact inversion of which one matters.
function vv_ai_memory_learned_max(): int {
$vars = vv_conf_vars();
$n = (int)($vars['AI_MEMORY_LEARNED_MAX_CHARS'] ?? 1200);
return max(0, min($n, vv_ai_memory_max()));
}
// Which profiles a slot is given to. "*" means every profile. Narrowing it is how a General Chat
// question about bash syntax stops carrying this machine's PCIe topology.
function vv_ai_memory_applies(string $kind, string $profile): bool
{
$vars = vv_conf_vars();
// The learned slot answers to the same gate that fills it. Without this, switching
// AI_MEMORY_LEARN_ENABLED off stopped new proposals but left every line already kept in every
// future prompt — a gate that closes the tap and not the tank. Gated here rather than at the
// one call site so any future consumer of vv_ai_memory_assemble() inherits it, and deliberately
// NOT in vv_ai_memory_read(), which reports what is on disk for the AI tab to display.
if ($kind === 'learned'
&& strtolower(trim($vars['AI_MEMORY_LEARN_ENABLED'] ?? 'false')) !== 'true') return false;
$key = $kind === 'learned' ? 'AI_MEMORY_LEARNED_PROFILES' : 'AI_MEMORY_ASSISTED_PROFILES';
$spec = trim($vars[$key] ?? '*');
if ($spec === '' || $spec === '*') return true;
if ($profile === '') return true;
foreach (explode(',', $spec) as $one) {
if (strcasecmp(trim($one), $profile) === 0) return true;
}
return false;
}
function vv_ai_memory_max(): int {
$vars = vv_conf_vars();
$n = (int)($vars['AI_MEMORY_MAX_CHARS'] ?? 4000);
return max(200, min($n, 20000));
}
// Returns ['text'=>string,'chars'=>int,'truncated'=>bool,'exists'=>bool].
// Truncates rather than refusing: an over-long memory file should cost its own tail, not the
// whole conversation, and the notice tells the model its knowledge is incomplete rather than
// letting it assume it saw everything.
// Assembles what this profile actually gets, as two separately-labelled blocks so the caller can
// state a different precedence for each. Budget is applied to the pair: learned is capped by its
// own ceiling first, then trimmed against whatever the assisted file leaves — assisted is never
// shortened to make room for learned.
function vv_ai_memory_assemble(string $profile = ''): array
{
$max = vv_ai_memory_max();
$out = ['assisted' => '', 'learned' => '', 'chars' => 0,
'truncated' => false, 'files' => []];
$read = function (string $kind) use ($profile, &$out): string {
if (!vv_ai_memory_applies($kind, $profile)) return '';
$p = vv_ai_memory_path($kind);
if (!file_exists($p)) return '';
$t = trim((string)@file_get_contents($p));
if ($t === '') return '';
$out['files'][] = ['kind' => $kind, 'path' => $p, 'chars' => mb_strlen($t)];
return $t;
};
$assisted = $read('assisted');
$learned = $read('learned');
// The operator's file gets the budget first, in full. If it alone exceeds the cap that is a
// problem to report, not one to solve by dropping the other slot silently.
if (mb_strlen($assisted) > $max) {
$assisted = mb_substr($assisted, 0, $max) . "\n\n[assisted memory truncated at {$max} characters]";
$out['truncated'] = true;
$learned = '';
} else {
$room = min(vv_ai_memory_learned_max(), $max - mb_strlen($assisted));
if ($room <= 0) {
$learned = '';
} elseif (mb_strlen($learned) > $room) {
// Cut on a line boundary — half a sentence asserted as fact is worse than one fewer.
$learned = mb_substr($learned, 0, $room);
$cut = mb_strrpos($learned, "\n");
if ($cut !== false && $cut > 0) $learned = mb_substr($learned, 0, $cut);
$out['truncated'] = true;
}
}
$out['assisted'] = $assisted;
$out['learned'] = $learned;
$out['chars'] = mb_strlen($assisted) + mb_strlen($learned);
return $out;
}
function vv_ai_memory_read(string $kind = 'assisted'): array {
$p = vv_ai_memory_path($kind);
$max = $kind === 'learned' ? vv_ai_memory_learned_max() : vv_ai_memory_max();
if (!file_exists($p)) return ['text' => '', 'chars' => 0, 'truncated' => false, 'exists' => false];
$raw = (string)@file_get_contents($p);
$len = mb_strlen($raw);
if ($len <= $max) {
return ['text' => $raw, 'chars' => $len, 'truncated' => false, 'exists' => true];
}
return [
'text' => mb_substr($raw, 0, $max) . "\n\n[memory truncated at {$max} characters]",
'chars' => $len,
'truncated' => true,
'exists' => true,
];
}
// Atomic, and refuses to exceed the cap. The cap is a context budget shared with retrieval and
// reasoning on every turn, so it is enforced on write rather than silently trimmed on read.
function vv_ai_memory_write(string $text, string $kind = 'assisted'): array {
$p = vv_ai_memory_path($kind);
$max = $kind === 'learned' ? vv_ai_memory_learned_max() : vv_ai_memory_max();
$len = mb_strlen($text);
if ($len > $max) {
$what = $kind === 'learned' ? 'Learned memory' : 'Memory';
return ['ok' => false, 'error' => "{$what} is {$len} characters; the limit is {$max}. "
. "It is included in every prompt, so it competes with retrieval for context."];
}
$dir = dirname($p);
if (!is_dir($dir) && !@mkdir($dir, 0755, true)) {
return ['ok' => false, 'error' => 'Cannot create ' . $dir];
}
$tmp = $p . '.vv.tmp';
if (@file_put_contents($tmp, $text) === false) {
return ['ok' => false, 'error' => 'Cannot write ' . $tmp];
}
if (!@rename($tmp, $p)) {
@unlink($tmp);
return ['ok' => false, 'error' => 'Cannot install ' . $p];
}
return ['ok' => true, 'chars' => $len];
}
// ── Token accounting ─────────────────────────────────────────────────────────────────────────
// One appended row per completed turn: date|time|host|profile|source|prompt|completion|tok_s.
// Pipe-delimited under data/ to match arr_cleanup_stats.db and bandwidth_history.db rather than
// inventing a format for one feature.
//
// Recording is best-effort and never raises. A turn that answered correctly must not be reported
// as failed because its accounting row could not be written — the number is the cheap part here
// and the answer is the expensive one.
function vv_ai_token_db(): string {
$p = str_replace(['$DATA_DIR', '${DATA_DIR}'], DATA_DIR,
trim(vv_conf_vars()['AI_TOKEN_DB'] ?? ''));
return $p !== '' ? $p : AI_DATA_DIR . '/ai_token_history.db';
}
function vv_ai_token_retain(): int {
$n = (int)(vv_conf_vars()['AI_TOKEN_RETAIN_ROWS'] ?? 20000);
return max(500, min($n, 500000));
}
// LOCK_EX because two turns can finish together — the composer allows a second question while
// the first is still generating, and each runs in its own detached worker.
function vv_ai_token_record(string $profile, string $source, int $prompt, int $completion, ?float $tokS): void {
if ($prompt <= 0 && $completion <= 0) return; // nothing generated; not a turn worth a row
$db = vv_ai_token_db();
$dir = dirname($db);
if (!is_dir($dir) && !@mkdir($dir, 0755, true)) return;
$row = implode('|', [
date('Y-m-d'), date('H:i:s'),
vv_detect_host(),
preg_replace('/[^a-z0-9_-]/i', '', $profile) ?: 'unknown',
preg_replace('/[^a-z0-9_-]/i', '', $source) ?: 'unknown',
max(0, $prompt), max(0, $completion),
$tokS === null ? '' : number_format($tokS, 1, '.', ''),
]) . "\n";
@file_put_contents($db, $row, FILE_APPEND | LOCK_EX);
vv_ai_token_prune($db);
}
// Pruning is gated on filesize rather than done every turn: filesize() is a stat, whereas
// reading and rewriting the whole file is not something a chat turn should pay for per message.
// The threshold is derived from the row cap so raising the cap does not silently disable it.
function vv_ai_token_prune(string $db): void {
$keep = vv_ai_token_retain();
if ((int)@filesize($db) < $keep * 60) return; // ~60 bytes a row, generously
$rows = @file($db, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
if (!is_array($rows) || count($rows) <= $keep) return;
$tmp = $db . '.vv.tmp';
if (@file_put_contents($tmp, implode("\n", array_slice($rows, -$keep)) . "\n") === false) return;
@rename($tmp, $db);
}
// Partner ledgers pulled by AI/ai_token_sync.sh into tmpfs. Same trick conf_sync.sh uses for
// partner confs: the reader treats a missing file as "unknown", never as zero.
//
// The directory is AI_TOKEN_CACHE_DIR from master.conf, resolved in config.php. It used to be
// hardcoded here, on the reasoning that the shell side hardcoded it too and so there was no
// single value to drift away from — true when it was written, and no longer true.
// Aggregates the file into today / last 7 days / all time, both overall and per host.
//
// Every detected host is present in 'hosts' whether or not it has rows, with 'seen' telling the
// two apart. A host with no rows is not the same fact as a host with zero tokens, and the page
// must be able to say "nothing collected here" rather than draw a confident 0.
function vv_ai_token_stats(): array {
$db = vv_ai_token_db();
$today = date('Y-m-d');
$week = date('Y-m-d', strtotime('-6 days')); // 7 days inclusive of today
$blank = ['turns' => 0, 'prompt' => 0, 'completion' => 0, 'total' => 0];
$out = [
'exists' => file_exists($db),
'today' => $blank, 'week' => $blank, 'all' => $blank,
'hosts' => [], 'profiles' => [], 'sources' => [],
'first' => null, 'last' => null, 'best_tok_s' => null,
];
$me = vv_detect_host();
foreach (vv_known_hosts() as $id => $name) {
$out['hosts'][$id] = ['name' => $name, 'seen' => false, 'self' => $id === $me,
'synced' => null, 'today' => $blank, 'week' => $blank, 'all' => $blank];
}
// Ledgers to read: our own, plus whatever AI/ai_token_sync.sh has pulled from partners.
//
// Each entry carries the host slot it is allowed to contribute rows for. A partner file may
// only add rows whose host column matches its filename — a ledger copied into the wrong slot,
// or a partner that somehow cached ours, would otherwise be counted twice against a total
// that still looked plausible. Our own file is trusted for any host, because it is the only
// one written here and its host column is written by vv_detect_host().
$ledgers = [[$db, null]];
foreach ((array)@glob(VV_AI_TOKEN_CACHE_DIR . '/host*.tokens.db') as $partnerDb) {
if (!preg_match('/(host\d+)\.tokens\.db$/', $partnerDb, $m)) continue;
if ($m[1] === $me) continue; // our own ledger is already read above
$ledgers[] = [$partnerDb, $m[1]];
if (isset($out['hosts'][$m[1]])) $out['hosts'][$m[1]]['synced'] = @filemtime($partnerDb) ?: null;
}
$add = function (array $b, int $p, int $c): array {
$b['turns']++; $b['prompt'] += $p; $b['completion'] += $c; $b['total'] += $p + $c;
return $b;
};
foreach ($ledgers as [$path, $onlyHost]) {
$fh = @fopen($path, 'r');
if (!$fh) continue;
while (($line = fgets($fh)) !== false) {
$f = explode('|', rtrim($line, "\r\n"));
if (count($f) < 8) continue; // partial write, or a format change
[$date, , $host, $profile, $source, $p, $c, $tokS] = $f;
if (!preg_match('/^\d{4}-\d{2}-\d{2}$/', $date)) continue;
if ($onlyHost !== null && $host !== $onlyHost) continue;
$p = (int)$p; $c = (int)$c;
if (!isset($out['hosts'][$host])) {
// A host in the file that conf no longer lists — renamed, or a row copied in.
// Shown rather than dropped, so the totals reconcile with the per-host rows.
$out['hosts'][$host] = ['name' => $host, 'seen' => false, 'self' => false,
'synced' => null, 'today' => $blank, 'week' => $blank, 'all' => $blank];
}
$out['hosts'][$host]['seen'] = true;
foreach ([['all', true], ['week', $date >= $week], ['today', $date === $today]] as [$k, $hit]) {
if (!$hit) continue;
$out[$k] = $add($out[$k], $p, $c);
$out['hosts'][$host][$k] = $add($out['hosts'][$host][$k], $p, $c);
}
$out['profiles'][$profile] = ($out['profiles'][$profile] ?? 0) + $p + $c;
$out['sources'][$source] = ($out['sources'][$source] ?? 0) + $p + $c;
if ($tokS !== '' && (float)$tokS > (float)($out['best_tok_s'] ?? 0)) $out['best_tok_s'] = (float)$tokS;
// Min/max rather than first-row/last-row. Appends are chronological in practice, but
// a file hand-edited, merged or restored out of order would otherwise report a span
// that reads backwards — and the row cap means the oldest row is not permanent.
if ($out['first'] === null || $date < $out['first']) $out['first'] = $date;
if ($out['last'] === null || $date > $out['last']) $out['last'] = $date;
}
fclose($fh);
}
arsort($out['profiles']);
arsort($out['sources']);
$out['days'] = $out['first'] ? max(1, (int)((strtotime($today) - strtotime($out['first'])) / 86400) + 1) : 0;
return $out;
}
// ── Scoped lookups for the Scheduler's assistant ──────────────────────────────────────────────
// The tail of one named script's log, for the troubleshooting profile. vv_ai_recent_logs()
// answers "is anything wrong anywhere"; this answers "why did THIS fail", which is a different
// question and needs the ordinary lines too, not just WARN and ERROR — the last thing a script
// printed before stopping is usually not labelled as a warning.
//
// Contained to LOG_DIR by realpath. The id arrives from a request, and a log path that escaped
// would read arbitrary files into a model's context.
function vv_ai_scoped_log(string $id, int $lines = 120): array {
$rel = preg_replace('/\.sh$/', '', trim($id)) . '.log';
if ($rel === '' || str_contains($rel, "\0")) return ['ok' => false, 'error' => 'bad id'];
$base = realpath(LOG_DIR);
$path = realpath(LOG_DIR . '/' . $rel);
if ($base === false || $path === false) return ['ok' => false, 'error' => 'no log yet'];
if (!str_starts_with($path, $base . '/')) return ['ok' => false, 'error' => 'outside log dir'];
if (!is_file($path)) return ['ok' => false, 'error' => 'no log yet'];
$all = @file($path, FILE_IGNORE_NEW_LINES) ?: [];
return [
'ok' => true,
'path' => $rel,
'total' => count($all),
'mtime' => @filemtime($path) ?: null,
'tail' => array_slice($all, -max(10, min($lines, 400))),
];
}
// The run record a script's wrapper writes beside its log: how the last run ended, in four
// fields, without reading a line of the log. "How did it go" is answerable from this alone —
// status, exit code and the window it ran in — and the log tail then supplies the detail.
// Separate from vv_ai_scoped_log() because they fail independently: a script killed mid-run
// leaves a log and no record, and that difference is itself the answer.
function vv_ai_run_record(string $id): array {
$rel = preg_replace('/\.sh$/', '', trim($id)) . '.json';
if ($rel === '' || str_contains($rel, "\0")) return ['ok' => false];
$base = realpath(LOG_DIR);
$path = realpath(LOG_DIR . '/' . $rel);
if ($base === false || $path === false) return ['ok' => false];
if (!str_starts_with($path, $base . '/')) return ['ok' => false];
$r = json_decode((string)@file_get_contents($path), true);
if (!is_array($r) || !isset($r['start'])) return ['ok' => false];
$start = (int)$r['start'];
$end = isset($r['end']) ? (int)$r['end'] : 0;
return [
'ok' => true,
'status' => (string)($r['status'] ?? '?'),
'exit' => isset($r['exit']) ? (int)$r['exit'] : null,
'start' => $start,
'end' => $end ?: null,
'duration' => $end > $start ? $end - $start : null,
];
}
// Which script a run-outcome question is about, when the operator names it in prose rather than
// by opening its log. "How did the daily orch go" has to resolve to Orchestrators/daily_sync_
// maintenance before anything can be attached to the context.
//
// Aliases are an explicit table, and matching against real log ids is exact on the underscore
// tokens — no similarity scoring. The failure mode of a scored match here is attaching the wrong
// script's log and answering confidently about a run the operator never asked about, which is
// indistinguishable from a correct answer unless they already knew. Ambiguity returns nothing so
// the question falls through to ordinary retrieval, which is merely unhelpful rather than wrong.
function vv_ai_resolve_run_target(string $question): string {
$q = strtolower($question);
static $aliases = [
'Orchestrators/daily_sync_maintenance' => ['daily orch', 'daily orchestrator',
'daily sync', 'daily maintenance', 'daily run'],
'Orchestrators/weekly_sync_maintenance' => ['weekly orch', 'weekly orchestrator',
'weekly sync', 'weekly maintenance', 'weekly run'],
'Orchestrators/critical_sync_maintenance' => ['critical orch', 'critical sync', 'critical run'],
'Orchestrators/intermediate_sync_maintenance' => ['intermediate orch', 'intermediate sync'],
'Orchestrators/watchdog_orchestrator' => ['watchdog orch', 'watchdog orchestrator',
'watchdogs'],
'Orchestrators/transcode_management' => ['transcode orch', 'transcode management'],
'Orchestrators/array_started' => ['array start', 'array started'],
'Orchestrators/monthly_maintenance' => ['monthly orch', 'monthly maintenance'],
'Orchestrators/sunday_morning_coffee_report' => ['coffee report', 'sunday report',
'sunday morning coffee'],
];
foreach ($aliases as $id => $phrases) {
foreach ($phrases as $p) if (str_contains($q, $p)) return $id;
}
// Anything with a log but no alias — named outright, either as an id or a bare script name.
//
// Whole words only. A substring test matched the `ai` log inside the word "f-ai-l", so
// "why did this run fail" resolved to ai.log and would have attached it to a question about
// something else entirely. Short ids make that failure common rather than exotic: mail, again,
// available, maintenance all contain it.
$hits = [];
foreach (vv_ai_log_ids() as $id) {
foreach ([strtolower($id), strtolower(basename($id))] as $needle) {
if ($needle !== '' && preg_match('/\b' . preg_quote($needle, '/') . '\b/', $q)) {
$hits[] = $id;
break;
}
}
}
$hits = array_unique($hits);
if (count($hits) === 1) return reset($hits);
// Longest match wins only when one candidate contains every other — "daily_sync_maintenance"
// over "sync", not a coin toss between two unrelated scripts.
if (count($hits) > 1) {
usort($hits, fn($a, $b) => strlen($b) - strlen($a));
$longest = $hits[0];
foreach (array_slice($hits, 1) as $h) {
if (!str_contains(strtolower($longest), strtolower(basename($h)))) return '';
}
return $longest;
}
return '';
}
// Every script id that has a log, relative to LOG_DIR and without the extension. One level of
// nesting, which is how the log tree is actually laid out (Orchestrators/, Plugin/).
function vv_ai_log_ids(): array {
$base = realpath(LOG_DIR);
if ($base === false) return [];
$ids = [];
foreach (glob($base . '/*.log') ?: [] as $f) {
$ids[] = basename($f, '.log');
}
foreach (glob($base . '/*/*.log') ?: [] as $f) {
$ids[] = basename(dirname($f)) . '/' . basename($f, '.log');
}
return $ids;
}
// Where a conf key actually lives. Deterministic on purpose: the model should narrate this, not
// work it out. Searches the host's own conf and master, reports file, line and value.
//
// This is the "Jonny" case — someone is sure a setting is in master.conf and it is really in the
// host conf. Failing to find it is the unhelpful answer; finding it and saying where is the
// useful one, and neither should depend on the model guessing which file to trust.
function vv_ai_find_conf_key(string $key): array {
if (!preg_match('/^[A-Za-z][A-Za-z0-9_]{1,63}$/', $key)) return ['ok' => false];
$me = vv_detect_host();
$order = array_values(array_unique(array_filter([
$me !== 'unknown' ? $me . '.conf' : null,
'master.conf',
])));
foreach (vv_get_conf_files() as $f) if (!in_array($f, $order, true)) $order[] = $f;
// Two passes, active before commented, across every file — not first-match-wins per line.
// These confs document each setting in a comment block above it, and those blocks contain
// lines like "# RSYNC_ENABLED=false → ALL rsync stops everywhere". A single pass matched
// that prose at line 529 and reported the setting as commented out while the live value sat
// active at line 546. Reporting an enabled setting as disabled is the exact failure this
// lookup exists to prevent, so the active definition always wins.
$q = preg_quote($key, '/');
$files = [];
foreach ($order as $file) $files[$file] = @file(CONF_DIR . '/' . $file, FILE_IGNORE_NEW_LINES) ?: [];
foreach ($files as $file => $lines) {
foreach ($lines as $i => $line) {
if (preg_match('/^\s*' . $q . '\s*=/', $line)) {
return ['ok' => true, 'file' => $file, 'line' => $i + 1,
'text' => trim($line), 'commented' => false];
}
}
}
foreach ($files as $file => $lines) {
foreach ($lines as $i => $line) {
if (preg_match('/^\s*#\s*' . $q . '\s*=/', $line)) {
return ['ok' => true, 'file' => $file, 'line' => $i + 1,
'text' => trim($line), 'commented' => true];
}
}
}
return ['ok' => false];
}
// One definition of a valid scope, used by every caller. A scope names something in this page's
// own view state — a conf file, a script, a log. It reaches the model as text and, for the
// troubleshooting profile, composes a path under LOG_DIR, so `..` is refused outright rather
// than left for the containment check downstream to catch. That check stays: this is the first
// gate, not the only one.
function vv_ai_scope_ok(string $scope): bool {
if (!preg_match('#^[A-Za-z0-9 ._/-]{1,80}$#', $scope)) return false;
if (str_contains($scope, '..')) return false;
return true;
}
// ── AI-filed bug reports ─────────────────────────────────────────────────────────────────────
// When the troubleshooter concludes the evidence shows a defect in Varaverk itself — not a
// setting — it files a report here rather than only saying so in a chat window that closes.
//
// Filed automatically, with the care put into the guard rather than into a review queue: a
// report requires a component and a quoted line of evidence, and identical findings collapse
// onto one record with a seen count. Without that, asking the same question three times files
// three bugs and the pile becomes noise within a week.
//
// Under data/ and therefore gitignored: these quote this installation's logs.
function vv_ai_bugs_dir(): string {
$d = AI_DATA_DIR . '/ai_bugs';
if (!is_dir($d)) @mkdir($d, 0755, true);
return $d;
}
// Same component + same summary is the same finding. Hashing them means a recurring fault
// increments a counter instead of breeding files, and the count is itself the signal: seen 40
// times is a different problem from seen once.
function vv_ai_bug_file(array $r): string {
return vv_ai_bugs_dir() . '/' . substr(sha1(strtolower($r['component'] . '|' . $r['summary'])), 0, 12) . '.json';
}
// Does the quoted evidence actually appear in the log the model was shown?
//
// The prompt asks for a verbatim line. This checks it, because "manufactured a cause to have
// one" is the specific failure this profile was warned against and warnings are not guards —
// the same reason General Chat needed a regex behind its instruction, not just a firmer wording.
//
// Whitespace-normalised and matched on the longest quoted fragment: the model reliably keeps the
// text and unreliably keeps the indentation. A short fragment would match by coincidence, so
// anything under 25 characters is not treated as a match at all.
function vv_ai_evidence_in_log(string $evidence, array $tail): bool {
if (!$tail) return false;
$norm = fn(string $s) => preg_replace('/\s+/', ' ', trim($s));
$hay = ' ' . implode(' ⏎ ', array_map($norm, $tail)) . ' ';
$frags = array_filter(array_map($norm, preg_split('/\R/', $evidence)),
fn($l) => mb_strlen($l) >= 25);
usort($frags, fn($a, $b) => mb_strlen($b) <=> mb_strlen($a));
foreach (array_slice($frags, 0, 5) as $f) {
if (str_contains($hay, $f)) return true;
// Allow a trimmed tail of the fragment — models often drop a trailing clause.
$head = mb_substr($f, 0, max(25, (int)(mb_strlen($f) * 0.6)));
if (mb_strlen($head) >= 25 && str_contains($hay, $head)) return true;
}
return false;
}
// Is the named component a real file in this installation? A defect report against a path that
// does not exist is a confabulation, and it is free to check.
function vv_ai_component_exists(string $component): bool {
$c = trim($component);
if ($c === '' || !vv_ai_scope_ok($c)) return false;
$base = realpath(SCRIPTS_DIR);
$p = realpath(SCRIPTS_DIR . '/' . $c);
return $base !== false && $p !== false && str_starts_with($p, $base . '/') && is_file($p);
}
function vv_ai_bug_write(string $component, string $summary, string $evidence, array $ctx = []): array {
$component = trim($component);
$summary = trim($summary);
$evidence = trim($evidence);
// Evidence is mandatory. A report that cannot quote the line it is based on is an opinion,
// and this file exists to hold findings that can be checked.
if ($component === '' || $summary === '' || $evidence === '') return ['ok' => false];
if (!vv_ai_scope_ok($component)) return ['ok' => false];
// Redacted on the way in, not on the way out. A report is built from this record, shown on
// the AI page from this record, and pasted into an issue from this record — redacting at any
// one of those leaves the other two holding the secret, and the store keeps it forever.
//
// Counted so the report can say something was removed. Silent redaction reads as a short log,
// and a reader who cannot tell the difference cannot judge what they are looking at.
$summary = vv_ai_redact($summary);
$evidence = vv_ai_redact($evidence);
$redacted = substr_count($summary . "\0" . $evidence, VV_AI_REDACTED);
foreach ($ctx as $k => $v) {
if (is_string($v)) {
$ctx[$k] = vv_ai_redact($v);
$redacted += substr_count($ctx[$k], VV_AI_REDACTED);
}
}
// The commit as it was when the fault was seen, not when the report is read. "Is this already
// fixed" is the first question any maintainer asks, and a version captured later answers a
// different question. Slot id (host1/host2) rather than the hostname — this text is written
// to be pasted into a public issue.
$commit = '';
$head = @file_get_contents(SCRIPTS_DIR . '/.git/HEAD');
if (is_string($head) && preg_match('#^ref:\s*(\S+)#', trim($head), $hm)) {
$commit = substr(trim((string)@file_get_contents(SCRIPTS_DIR . '/.git/' . $hm[1])), 0, 12);
} elseif (is_string($head)) {
$commit = substr(trim($head), 0, 12);
}
$rec = [
'component' => mb_substr($component, 0, 120),
'summary' => mb_substr($summary, 0, 300),
'evidence' => mb_substr($evidence, 0, 2000),
'commit' => $commit,
'host' => vv_detect_host(),
'first' => time(),
'last' => time(),
'seen' => 1,
'open' => true,
'context' => array_slice($ctx, 0, 12),
'redacted' => $redacted,
// The shape this record was written in. A report format will change; being able to read
// an old record without guessing which fields it has is what makes that survivable.
'template' => 1,
];
$p = vv_ai_bug_file($rec);
if (is_file($p)) {
$old = json_decode((string)@file_get_contents($p), true);
if (is_array($old)) {
$rec['first'] = $old['first'] ?? $rec['first'];
$rec['seen'] = (int)($old['seen'] ?? 0) + 1;
$rec['open'] = $old['open'] ?? true; // dismissing it stays dismissed
}
}
$rec['id'] = basename($p, '.json');
if (@file_put_contents($p, json_encode($rec, JSON_PRETTY_PRINT)) === false) return ['ok' => false];
return ['ok' => true, 'id' => $rec['id'], 'seen' => $rec['seen']];
}
// Renders one stored bug as the report that gets pasted into an issue.
//
// The shape is built around one rule: observed fact and inference are separated and labelled.
// A report that blends "the log said X" with "this is probably a permissions problem" is one that
// the next reader — a person, or a model asked to look at it — will quote back as evidence. Every
// heading below is either quoted output or explicitly marked as a reading of it.
//
// Written to be pasted somewhere public, so it carries the host slot rather than the hostname and
// nothing that identifies the machine. What it does carry is the two facts a maintainer asks for
// first: which commit, and how many times.
function vv_ai_bug_report(array $b): string {
$g = fn(string $k, $d = '') => $b[$k] ?? $d;
$ctx = (array) $g('context', []);
$when = fn($t) => $t ? date('Y-m-d H:i', (int) $t) : '?';
$seen = (int) $g('seen', 1);
$out = "## Varaverk bug report\n\n";
$out .= "| | |\n|---|---|\n";
$out .= '| **Summary** | ' . str_replace('|', '\\|', (string) $g('summary')) . " |\n";
$out .= '| **Component** | `' . $g('component') . "` |\n";
$out .= '| **Commit** | `' . ($g('commit') ?: 'unknown') . "` |\n";
$out .= '| **Seen** | ' . $seen . '× · first ' . $when($g('first'))
. ' · last ' . $when($g('last')) . " |\n";
$out .= '| **Host slot** | ' . $g('host', '?') . " |\n";
$out .= '| **Report id** | `' . $g('id', '?') . "` |\n";
$out .= '| **Template** | ' . (int) $g('template', 1) . " |\n\n";
// Whether the quote below was checked against the log it claims to come from. The worker
// refuses to file a bug whose evidence it could not find, so this is normally true — but a
// report that asserts it without having checked is worth less than one that says it did not.
$verified = $ctx['verified'] ?? null;
$vLabel = $verified === true ? 'verified present in ' . ($ctx['log'] ?? 'the log')
: ($verified === false ? 'NOT found in the log — treat with suspicion'
: 'not checked against a log');
$out .= "### Evidence — verbatim, " . $vLabel . "\n\n```\n"
. rtrim((string) $g('evidence')) . "\n```\n\n";
// Deliberately after the evidence and deliberately labelled. This is the assistant's reading,
// and the heading has to survive being skim-read: a diagnosis printed next to a log quote
// gets remembered as a second log quote, and the next reader inherits a guess as a finding.
if (!empty($ctx['diagnosis'])) {
$out .= "### The assistant's reading — INFERENCE, not established\n\n"
. "> " . str_replace("\n", "\n> ", trim((string) $ctx['diagnosis'])) . "\n\n"
. "_Written by a 14B model from the evidence above. Worth checking, not worth "
. "trusting._\n\n";
}
if (!empty($ctx['asked'])) {
$out .= "### What was being asked when it was noticed\n\n> "
. str_replace("\n", "\n> ", trim((string) $ctx['asked'])) . "\n\n";
}
$out .= "### Environment\n\n";
$env = array_filter([
'Unraid ' . (vv_ai_bug_unraid_version() ?: '?'),
'PHP ' . PHP_VERSION,
!empty($ctx['profile']) ? 'assistant profile: ' . $ctx['profile'] : null,
!empty($ctx['scope']) ? 'scope: ' . $ctx['scope'] : null,
]);
$out .= '- ' . implode("\n- ", $env) . "\n\n";
// Stated even when nothing was removed, because "0 values" and no line at all read very
// differently to someone deciding whether the log looks suspiciously short.
$n = (int) $g('redacted', 0);
$out .= "### Redaction\n\n";
$out .= $n > 0
? $n . ' value' . ($n === 1 ? '' : 's') . " matched a known credential or a secret-shaped "
. "assignment and " . ($n === 1 ? 'was' : 'were') . " replaced with `" . VV_AI_REDACTED . "`.\n"
: "Nothing matched a known credential. Note that redaction catches secrets held in this "
. "install's conf, or written as an assignment — an unlabelled token printed by another "
. "program would not be caught, so read the evidence before sending.\n";
return $out;
}
// The machine as it is right now — hardware, containers, storage — summarised small enough to
// sit in a prompt.
//
// Read from the cache the WebGUI already writes every minute rather than measured here. A
// question must not cost a docker ps across fifty containers plus a disk sweep; the numbers move
// on their own schedule and one minute stale is the same answer. The age is stated so the model
// can say "a minute ago" rather than implying it looked just now.
//
// Strictly read-only, and there is no counterpart that changes any of it. Knowing a container is
// down is what lets an explanation be about this machine instead of about Unraid in general;
// restarting it is a decision that belongs to a person looking at the screen.
function vv_ai_system_state(): string {
$p = '/tmp/varaverk/api/monitor.json';
if (!is_readable($p)) return '';
$d = json_decode((string) @file_get_contents($p), true);
if (!is_array($d)) return '';
$age = time() - (int) @filemtime($p);
$s = "LIVE MACHINE STATE (read-only, measured " . ($age < 90 ? 'under a minute' : round($age / 60) . ' minutes')
. " ago — you cannot change any of it)\n";
$sys = $d['system'] ?? [];
if ($sys) {
$s .= '- host: ' . ($sys['cpu_model'] ?? '?') . ', ' . ($sys['cpu_threads'] ?? '?') . " threads\n";
}
if (!empty($d['cpu']['overall'])) $s .= '- cpu: ' . $d['cpu']['overall'] . "% overall\n";
$r = $d['resources'] ?? [];
if (!empty($r['ram_total_mb'])) {
$used = (int) $r['ram_total_mb'] - (int) ($r['ram_free_mb'] ?? 0);
$s .= sprintf("- memory: %d of %d MB used\n", $used, (int) $r['ram_total_mb']);
}
foreach ((array) ($d['gpus'] ?? []) as $g) {
$s .= sprintf("- gpu %d: %s, %d/%d MB, %d%% util, %d°C\n", $g['index'] ?? 0, $g['name'] ?? '?',
$g['memory_used'] ?? 0, $g['memory_total'] ?? 0, $g['utilization'] ?? 0,
$g['temperature'] ?? 0);
}
// Named individually only when something is wrong with them. Fifty healthy container names is
// the bulk of this block and none of it is ever the answer; the handful that are not are.
//
// Taken from the payload's own `stopped` list rather than derived. There is no state field on
// a container row — only `status`, which reads "Up 15 hours (healthy)" — so an earlier version
// of this looked for "run", found it nowhere, and reported all fifty as down. A block that
// confidently lists healthy containers as stopped is worse than no block: it is exactly the
// sort of thing the model would build a whole wrong explanation on.
$ctrs = (array) ($d['containers'] ?? []);
$stopped = (array) ($d['stopped'] ?? []);
$down = array_values(array_filter(array_map(
fn($c) => trim((string) (is_array($c) ? ($c['name'] ?? '') : $c)), $stopped)));
// Up, but failing its own healthcheck. Running is not the same as working, and this is the
// state that produces "it is started, so why is nothing happening".
$sick = [];
foreach ($ctrs as $c) {
if (stripos((string) ($c['status'] ?? ''), 'unhealthy') !== false) $sick[] = (string) ($c['name'] ?? '?');
}
$s .= '- containers: ' . count($ctrs) . ' running';
$s .= $down ? ', STOPPED: ' . implode(', ', array_slice($down, 0, 12)) : ', none stopped';
$s .= $sick ? '; UNHEALTHY: ' . implode(', ', array_slice($sick, 0, 8)) : '';
$s .= "\n";
foreach ((array) ($d['storage'] ?? []) as $pool) {
if (empty($pool['name'])) continue;
$s .= sprintf("- pool %s: %s%% used\n", $pool['name'], $pool['used_pct'] ?? $pool['pct'] ?? '?');
}
$par = $d['parity'] ?? [];
if ($par) {
$s .= '- array: ' . (($par['num_missing'] ?? 0) ? $par['num_missing'] . ' disks MISSING' : 'no missing disks')
. (($par['num_disabled'] ?? 0) ? ', ' . $par['num_disabled'] . ' disabled' : '')
. (!empty($par['in_progress']) ? ', parity operation in progress' : '') . "\n";
}
if (!empty($d['ups']['available'])) {
$s .= '- ups: ' . ($d['ups']['status'] ?? '?') . "\n";
}
return $s . "\n";
}
// Where a report can go from this install, resolved once so the page and the sender agree.
//
// Local is opt-in and off by default, because a default that files into the operator's own
// tracker means every report from every other install lands somewhere the maintainer never
// looks — a fallback chain is right for fetching code and wrong for sending a report.
function vv_ai_bug_targets(): array {
$v = vv_conf_vars();
$slot = strtoupper(vv_detect_host());
$on = strtolower(trim($v['BUG_REPORT_LOCAL_ENABLED'] ?? 'false')) === 'true';
$url = trim((string) ($v[$slot . '_BUG_REPORT_URL'] ?? ''));
$repo = trim((string) ($v[$slot . '_BUG_REPORT_REPO'] ?? ''));
$token = trim((string) ($v[$slot . '_BUG_REPORT_TOKEN'] ?? ''));
return [
// Configured is not the same as enabled: the switch is off but the details are filled in
// is a state worth showing, because it is what "why did this go to GitHub" looks like.
'local_enabled' => $on,
'local_configured' => $url !== '' && $repo !== '' && $token !== '',
'local_url' => $url,
'local_repo' => $repo,
'github_repo' => trim((string) ($v['BUG_REPORT_GITHUB_REPO'] ?? 'FailedProxy/Varaverk')),
];
}
// Files the report as an issue on the operator's own Gitea. Only ever reached when the switch is
// on and the details are present — never as a fallback from a failed GitHub send, because the two
// go to different people and quietly substituting one for the other is the whole failure mode
// this design exists to avoid.
function vv_ai_bug_send_local(string $title, string $body): array {
$t = vv_ai_bug_targets();
if (!$t['local_enabled']) return ['ok' => false, 'error' => 'local reporting is switched off'];
if (!$t['local_configured']) return ['ok' => false, 'error' => 'local reporting is on but URL, repo or token is blank'];
if (!function_exists('curl_init')) return ['ok' => false, 'error' => 'curl is unavailable'];
$v = vv_conf_vars();
$slot = strtoupper(vv_detect_host());
$token = trim((string) ($v[$slot . '_BUG_REPORT_TOKEN'] ?? ''));
$api = rtrim($t['local_url'], '/') . '/api/v1/repos/' . trim($t['local_repo'], '/') . '/issues';
$ch = curl_init($api);
curl_setopt_array($ch, [
CURLOPT_POST => true,
CURLOPT_HTTPHEADER => ['Content-Type: application/json',
'Authorization: token ' . $token],
CURLOPT_POSTFIELDS => json_encode(['title' => $title, 'body' => $body]),
CURLOPT_RETURNTRANSFER => true,
CURLOPT_TIMEOUT => 15,
CURLOPT_CONNECTTIMEOUT => 5,
CURLOPT_FOLLOWLOCATION => false,
]);
$resp = curl_exec($ch);
$code = (int) curl_getinfo($ch, CURLINFO_HTTP_CODE);
$err = curl_error($ch);
curl_close($ch);
if ($resp === false) return ['ok' => false, 'error' => 'could not reach Gitea: ' . $err];
if ($code === 401 || $code === 403) return ['ok' => false, 'error' => 'Gitea refused the token (HTTP ' . $code . ')'];
if ($code === 404) return ['ok' => false, 'error' => 'Gitea has no such repo (HTTP 404) — check the owner/repo'];
if ($code < 200 || $code >= 300) return ['ok' => false, 'error' => 'Gitea returned HTTP ' . $code];
$d = json_decode((string) $resp, true);
// The URL back is the whole point of sending server-side rather than opening a form: it is
// proof the issue exists, and somewhere to go and look at it.
return ['ok' => true, 'url' => $d['html_url'] ?? '', 'number' => $d['number'] ?? null];
}
// Best effort, and blank rather than wrong. /etc/unraid-version is a shell assignment; on
// anything that is not Unraid there is simply no file and the report says "?".
function vv_ai_bug_unraid_version(): string {
$raw = @file_get_contents('/etc/unraid-version');
if (!is_string($raw)) return '';
return preg_match('/"([^"]+)"/', $raw, $m) ? $m[1] : trim(explode('=', $raw, 2)[1] ?? '');
}
function vv_ai_bugs_list(bool $openOnly = true): array {
$out = [];
foreach ((array)@glob(vv_ai_bugs_dir() . '/*.json') as $f) {
$r = json_decode((string)@file_get_contents($f), true);
if (!is_array($r) || ($openOnly && empty($r['open']))) continue;
$out[] = $r;
}
usort($out, fn($a, $b) => ($b['last'] ?? 0) <=> ($a['last'] ?? 0));
return $out;
}
function vv_ai_bug_set_open(string $id, bool $open): bool {
if (!preg_match('/^[0-9a-f]{12}$/', $id)) return false;
$p = vv_ai_bugs_dir() . '/' . $id . '.json';
$r = json_decode((string)@file_get_contents($p), true);
if (!is_array($r)) return false;
$r['open'] = $open;
return @file_put_contents($p, json_encode($r, JSON_PRETTY_PRINT)) !== false;
}
// ── Incident journal ─────────────────────────────────────────────────────────────────────────
// "We have seen this before, and here is what it was." Appended as you work through logs, and
// fed back the next time the same script is being diagnosed.
//
// Deliberately NOT part of the AI index. The index reads git-tracked files only, and this lives
// under data/ which is gitignored — which is correct twice over: incident notes are about this
// installation and should not be pushed, and retrieval by similarity is the wrong lookup here.
// The right question is "what has gone wrong with THIS script before", which is an exact match
// on the scope, not a vector search. Same reasoning as the standing memory file.
//
// Markdown rather than a delimited .db because the useful part is prose — a fix is a sentence,
// not a field — and it stays hand-editable when a note turns out to be wrong.
function vv_ai_incidents_path(): string {
return AI_DATA_DIR . '/ai_incidents.md';
}
// One entry, appended. The symptom is captured from what was being asked; the fix is written by
// the operator. That split matters: the model's diagnosis is a hypothesis, and writing a
// hypothesis into institutional memory as fact is how a wrong answer outlives the incident.
function vv_ai_incident_add(string $scope, string $symptom, string $fix): array {
$scope = trim($scope);
$symptom = trim($symptom);
$fix = trim($fix);
if ($scope === '' || $fix === '') return ['ok' => false, 'error' => 'scope and fix are required'];
if (!vv_ai_scope_ok($scope)) return ['ok' => false, 'error' => 'bad scope'];
$entry = "\n## " . date('Y-m-d') . ' · ' . $scope . "\n"
. ($symptom !== '' ? '**Symptom:** ' . mb_substr($symptom, 0, 400) . "\n" : '')
. '**Fix:** ' . mb_substr($fix, 0, 1200) . "\n";
$p = vv_ai_incidents_path();
if (!is_dir(dirname($p)) && !@mkdir(dirname($p), 0755, true)) {
return ['ok' => false, 'error' => 'cannot create data dir'];
}
if (!file_exists($p)) {
@file_put_contents($p, "# Incident journal\n\nWhat went wrong, and what actually fixed it."
. " Written by the operator from the Scheduler assistant; fed back when the same"
. " script is diagnosed again.\n");
}
if (@file_put_contents($p, $entry, FILE_APPEND | LOCK_EX) === false) {
return ['ok' => false, 'error' => 'cannot write journal'];
}
return ['ok' => true];
}
// Past entries for one scope, newest first. Capped hard: this rides in the prompt alongside a
// 120-line log and retrieved passages, and the context budget is already the tight thing.
function vv_ai_incidents_for(string $scope, int $max = 4): array {
$p = vv_ai_incidents_path();
if ($scope === '' || !is_file($p)) return [];
$blocks = preg_split('/^## /m', (string)@file_get_contents($p));
$want = strtolower(trim($scope));
$base = strtolower(basename(trim($scope)));
$hits = [];
foreach ($blocks as $b) {
$b = trim($b);
if ($b === '' || !str_contains($b, "\n")) continue;
[$head] = explode("\n", $b, 2);
$head = strtolower($head);
// Match the scope as written, or its basename — the chip may carry a path where an
// older entry carried only the script name.
if (!str_contains($head, $want) && !str_contains($head, $base)) continue;
$hits[] = '## ' . $b;
}
return array_slice(array_reverse($hits), 0, max(1, $max));
}
// ── Stored conversations ──────────────────────────────────────────────────────
// Chats live in DATA_DIR, not /tmp, because the point of storing them is that they outlive a
// reboot — the job files above are deliberately the opposite. One file per conversation, same
// shape as ai_bugs: a directory of small JSON records is trivially prunable and a corrupt one
// costs a single chat rather than the whole history.
//
// The store is capped, and pruning happens on write rather than on a schedule. Nothing else
// runs often enough to be trusted with it, and an unbounded directory here would quietly grow
// for as long as the operator keeps talking to the assistant.
function vv_ai_chats_dir(): string {
$d = AI_DATA_DIR . '/ai_chats';
if (!is_dir($d)) @mkdir($d, 0755, true);
return $d;
}
// ── Secret redaction ─────────────────────────────────────────────────────────────────────────
// A conversation about settings is a conversation that contains credentials. Asking the
// assistant to change an API key means typing the key, and a stored transcript is replayed into
// a later prompt when the thread is reopened — so without this a secret would be written to
// disk in cleartext and then handed back to the model on every subsequent turn, indefinitely.
//
// Redaction happens on the way to disk and to the log, never to the live turn. The model needs
// the real value to carry out the change being asked for; what it does not need is that value
// still sitting in the transcript a week later.
const VV_AI_REDACTED = '[redacted]';
// The values this host actually holds, longest first so a key that contains another as a
// substring cannot be half-replaced. Only secret-shaped conf keys contribute, and only values
// long enough to be a credential — redacting every occurrence of a two-character setting would
// shred ordinary prose.
function vv_ai_known_secrets(): array {
static $cache = null;
if ($cache !== null) return $cache;
$vals = [];
foreach (vv_conf_vars() as $k => $v) {
$v = trim((string)$v);
if (strlen($v) < 8) continue;
if (!vv_conf_key_is_secret((string)$k)) continue;
// A value that is still a ${...} reference is a template, not a credential.
if (str_contains($v, '${')) continue;
$vals[] = $v;
}
$vals = array_values(array_unique($vals));
usort($vals, fn($a, $b) => strlen($b) <=> strlen($a));
return $cache = $vals;
}
// Two passes, because they catch different things.
//
// Known values catch a credential this host already holds, however it is worded — pasted bare,
// quoted, or buried mid-sentence. Exact string matching, so there are no false positives.
//
// Assignment shapes catch the credential that is not in the conf yet, which is precisely the
// case that matters here: "change the Emby API key to <new value>" is a secret arriving, and it
// will not match anything on disk until after the write it is asking for.
function vv_ai_redact(string $text): string {
if ($text === '') return $text;
foreach (vv_ai_known_secrets() as $secret) {
$text = str_replace($secret, VV_AI_REDACTED, $text);
}
// NAME=value / "api_key": value / api-key: value — anchored on a secret-shaped name so an
// ordinary "count=12" is untouched.
$text = preg_replace(
'/\b([A-Za-z0-9_\-]*(?:api[_\-]?key|password|passwd|secret|token|_pass|apikey)[A-Za-z0-9_\-]*)'
. '(\s*["\']?\s*[:=]\s*["\']?)([^\s"\',;]{6,})/i',
'$1$2' . VV_AI_REDACTED,
$text
) ?? $text;
// "set the api key to <value>", "password is <value>" — the same secret arriving as prose
// rather than as an assignment.
$text = preg_replace(
'/\b((?:api[ _\-]?key|password|passphrase|secret|token)\s+(?:to|is|as|=)\s+)["\']?([^\s"\',;]{6,})/i',
'$1' . VV_AI_REDACTED,
$text
) ?? $text;
return $text;
}
// Redacts every message body in place, leaving roles and any other metadata alone.
function vv_ai_redact_messages(array $messages): array {
foreach ($messages as &$m) {
if (isset($m['content']) && is_string($m['content'])) $m['content'] = vv_ai_redact($m['content']);
}
unset($m);
return $messages;
}
function vv_ai_chats_max(): int {
$n = (int)(vv_conf_vars()['AI_CHAT_HISTORY_MAX'] ?? 10);
return max(1, min(50, $n));
}
// Hex-only, for exactly the reason vv_ai_job_path() is: the id composes a path. Ids are minted
// server-side and the client only ever echoes one back, so anything else is a caller that has
// no business naming a file here.
function vv_ai_chat_path(string $id): ?string {
if (!preg_match('/^[0-9a-f]{32}$/', $id)) return null;
return vv_ai_chats_dir() . '/' . $id . '.json';
}
// The first thing asked, which is what the operator will recognise the conversation by. Falls
// back rather than returning empty: a blank row in the list is indistinguishable from a broken
// one.
function vv_ai_chat_title(array $messages): string {
foreach ($messages as $m) {
if (($m['role'] ?? '') !== 'user') continue;
$t = trim(preg_replace('/\s+/', ' ', (string)($m['content'] ?? '')));
if ($t !== '') return mb_substr($t, 0, 80);
}
return 'Untitled conversation';
}
// Newest first, metadata only. The list card renders ten rows and none of them need the
// transcript — sending every message of every stored chat to draw a sidebar would be the most
// expensive read on the monitor page.
function vv_ai_chats_list(): array {
$out = [];
foreach (glob(vv_ai_chats_dir() . '/*.json') ?: [] as $f) {
$d = json_decode(@file_get_contents($f) ?: '', true);
if (!is_array($d) || empty($d['id'])) continue;
$out[] = [
'id' => $d['id'],
'ts' => (int)($d['ts'] ?? 0),
// Last activity, as distinct from ts. The list is ordered by creation so the prune
// order and the visible order agree, but "which conversation was I last in" is a
// different question and needs the other timestamp to answer it.
'updated' => (int)($d['updated'] ?? $d['ts'] ?? 0),
'profile' => (string)($d['profile'] ?? 'chat'),
'scope' => (string)($d['scope'] ?? ''),
'title' => (string)($d['title'] ?? 'Untitled conversation'),
'turns' => (int)($d['turns'] ?? count($d['messages'] ?? [])),
];
}
usort($out, fn($a, $b) => $b['ts'] <=> $a['ts']);
return $out;
}
function vv_ai_chat_read(string $id): ?array {
$p = vv_ai_chat_path($id);
if ($p === null || !file_exists($p)) return null;
$d = json_decode(@file_get_contents($p) ?: '', true);
return is_array($d) ? $d : null;
}
function vv_ai_chat_delete(string $id): bool {
$p = vv_ai_chat_path($id);
if ($p === null || !file_exists($p)) return false;
return @unlink($p);
}
// Oldest go first, by stored timestamp rather than mtime — a chat that is reopened and continued
// is rewritten, and ordering on mtime would make "the one I have been using all week" look like
// the newest thing in the store while a genuinely older thread got dropped in its place.
function vv_ai_chats_prune(?int $max = null): int {
$max = $max ?? vv_ai_chats_max();
$list = vv_ai_chats_list();
$n = 0;
foreach (array_slice($list, $max) as $old) {
if (vv_ai_chat_delete($old['id'])) $n++;
}
return $n;
}
// Writes a whole conversation. An empty id mints one; a known id overwrites in place, which is
// what makes a continued conversation stay one row in the list instead of breeding a new one per
// turn. Written to a temp file and renamed, so a reader never sees half a transcript.
function vv_ai_chat_save(string $id, string $profile, array $messages, string $scope = ''): array {
if (!$messages) return ['ok' => false, 'error' => 'Nothing to save'];
if ($id === '') $id = bin2hex(random_bytes(16));
$p = vv_ai_chat_path($id);
if ($p === null) return ['ok' => false, 'error' => 'Invalid chat id'];
// Preserve the original creation time across rewrites. Ordering the list by last activity
// would be defensible, but it would also mean an old thread jumps the queue the moment it is
// reopened, and the prune above is written against creation order.
$prev = vv_ai_chat_read($id);
$created = (int)($prev['created'] ?? time());
// Redacted here rather than at the point the message was composed, because the live turn
// needs the real value to carry out what was asked. This is the boundary between "in flight"
// and "on disk", and it is the last place the cleartext exists. The title is derived after,
// so a credential cannot survive in the conversation list either.
$messages = vv_ai_redact_messages($messages);
// Scope travels with the conversation. A Scheduler thread is bound to what the operator had
// open — a script, a log, a conf key — and its turns carry log excerpts chosen for that
// thing. Storing the scope means reopening the thread anywhere restores the context it was
// reasoned in, rather than silently continuing a troubleshooting conversation against
// whatever happens to be on screen. The isolation the page enforces in memory becomes a
// property of the record instead of something lost the moment it is saved.
$rec = [
'id' => $id,
'created' => $created,
'ts' => $created,
'updated' => time(),
'profile' => $profile,
'scope' => $scope,
'title' => vv_ai_chat_title($messages),
'turns' => (int)floor(count($messages) / 2),
'messages' => $messages,
];
$tmp = $p . '.tmp';
if (@file_put_contents($tmp, json_encode($rec, JSON_PRETTY_PRINT)) === false
|| !@rename($tmp, $p)) {
@unlink($tmp);
return ['ok' => false, 'error' => 'Could not write ' . $p];
}
vv_ai_chats_prune();
return ['ok' => true, 'id' => $id, 'title' => $rec['title']];
}
function vv_ai_job_dir(): string {
if (!is_dir(VV_AI_JOB_DIR)) @mkdir(VV_AI_JOB_DIR, 0700, true);
return VV_AI_JOB_DIR;
}
// Hex-only, fixed length. The token composes a path, so the pattern is what confines it to the
// job directory.
function vv_ai_job_path(string $token): ?string {
if (!preg_match('/^[0-9a-f]{32}$/', $token)) return null;
return VV_AI_JOB_DIR . '/' . $token . '.json';
}
function vv_ai_job_read(string $token): ?array {
$p = vv_ai_job_path($token);
if ($p === null || !file_exists($p)) return null;
$d = json_decode(@file_get_contents($p) ?: '', true);
return is_array($d) ? $d : null;
}
Reference in New Issue View Git Blame Copy Permalink