Stop General Chat inventing answers about Varaverk internals

Asked what mover_stop.sh does, it invented a description and dressed it in
real memory facts so it read as authoritative — the exact failure the profile
split exists to prevent. A polite instruction in a system prompt is not a
guard. The rule now leads and is absolute, and a deterministic check repeats
it immediately before the message whenever the question names a shell script,
a SCREAMING_CASE conf key, or Varaverk itself. Detection can only make the
model more cautious, so a false positive costs a redirect, not an answer.
This commit is contained in:
Gmer4Lfe
2026-08-03 17:38:43 -04:00
parent 8d393a1e1c
commit e58f0352a7
+32 -10
View File
@@ -170,17 +170,25 @@ if ($diagnostic) {
// precisely what retrieval exists to prevent. The user always knows which contract is in force,
// and the strict profile is the default.
if ($profile === 'chat') {
// A polite instruction is not a guard. Asked "what does mover_stop.sh do in my setup", the
// model invented an answer and dressed it in real memory facts so it read as authoritative.
// The rule therefore leads, is stated absolutely, and is reinforced by a deterministic
// check below when the question names something Varaverk-shaped.
$system = "You are the Varaverk assistant, talking with the operator of a private two-server "
. "Unraid media ecosystem called Varaverk. This is ordinary conversation — no "
. "documentation has been retrieved for it.\n\n"
. "Talk like a knowledgeable colleague. Be warm and direct, follow a tangent if one "
. "is interesting, and use your general knowledge freely.\n\n"
. "The one hard rule: you have NOT been given Varaverk's documentation in this mode. "
. "Anything you know about their setup comes from the memory section below and from "
. "what they tell you. If they ask something specific about how Varaverk works — a "
. "script, a config variable, a safeguard — say you would need the Varaverk Assistant "
. "profile for that, and do not guess. Being wrong about their own system is worse "
. "than sending them one click away.\n\n";
. "Unraid media ecosystem called Varaverk. This is ordinary conversation.\n\n"
. "ABSOLUTE RULE, BEFORE ANYTHING ELSE: in this mode you have NOT been given "
. "Varaverk's documentation. You therefore do not know what any Varaverk script, "
. "config variable, orchestrator or safeguard actually does. If asked, you must say "
. "you cannot see the documentation in this mode and that the Varaverk Assistant "
. "profile can answer it — then stop. Do not describe what a script 'typically' or "
. "'probably' does. Do not reason from its name. Do not combine facts from the memory "
. "section into an explanation of a component you were not told about. A confident "
. "wrong answer about their own system is the single worst thing you can do here; "
. "sending them one click away costs nothing.\n\n"
. "Everything else is ordinary conversation. Talk like a knowledgeable colleague, be "
. "warm and direct, follow a tangent if it is interesting, and use your general "
. "knowledge freely — Linux, scripting, hardware, whatever comes up. The restriction "
. "is only about the specifics of THIS installation.\n\n";
} elseif ($profile === 'code') {
$system = "You are drafting a short shell script for the operator of an Unraid server, to be "
@@ -234,6 +242,20 @@ if ($diagBlock !== '') {
. $diagBlock;
}
// Deterministic backstop for the chat profile. The prompt asks the model to defer on Varaverk
// internals; this does not rely on it complying. If the question names something that can only
// be a Varaverk component — a shell script, a SCREAMING_CASE conf key, or the project itself —
// the instruction is repeated immediately before the user's message, where it is hardest to
// ignore. Detection only ever makes the model MORE cautious, so a false positive costs a
// redirect rather than a wrong answer.
if ($profile === 'chat'
&& preg_match('/\b[\w.-]+\.sh\b|\b[A-Z][A-Z0-9]*(_[A-Z0-9]+){2,}\b|\bvaraverk\b/i', $question)) {
$system .= "NOTE: the operator's message appears to name a specific Varaverk component. "
. "You cannot see the documentation in this mode, so you do not know what it does. "
. "Say that plainly, point them at the Varaverk Assistant profile, and do not "
. "speculate about its behaviour — not even a hedged guess.\n\n";
}
if ($context !== '') $system .= "PASSAGES\n" . $context;
$messages = [['role' => 'system', 'content' => $system]];