87 lines
6.9 KiB
Plaintext
87 lines
6.9 KiB
Plaintext
# ═══════════════════════════════════════════════════════════════════════════════════════════════
|
|
# Expected routing for the AI assistant, one case per line, checked by ai_explain_check.sh.
|
|
#
|
|
# Every line here is a bug that was found by reading an answer and noticing it was wrong. That is
|
|
# the expensive way to find them and it does not scale: the operator is the test suite, and the
|
|
# operator is busy. Written down, each one costs a second to re-check forever.
|
|
#
|
|
# These assert ROUTING, never wording. What a profile is allowed, which script a question names,
|
|
# which gates fire and what evidence gets attached are all decided before the model is involved,
|
|
# so they are identical every run. The prose is not and is deliberately not asserted — a test that
|
|
# depends on how the model phrases something fails for reasons nobody wants to read about.
|
|
#
|
|
# FORMAT
|
|
# question | profile | scope | kind | assertions
|
|
#
|
|
# Blank fields are allowed. Assertions are space-separated:
|
|
# profile=X the profile that ends up answering, after any handoff
|
|
# target=X resolved run target, or 'none'
|
|
# run=yes|no the run-outcome gate
|
|
# diag=yes|no the diagnostic gate
|
|
# caps=none the profile holds no capabilities at all
|
|
# has=a,b every one of these must be attached
|
|
# hasnt=a,b none of these may be attached
|
|
#
|
|
# Attachment keys: health warnings run_record log_tail log_missing incidents conf_keys
|
|
# ═══════════════════════════════════════════════════════════════════════════════════════════════
|
|
|
|
# ── Run-outcome questions must arrive with the run, not with directions to the log panel ───────
|
|
# The phrasing that started it. "went last" matched; "the run went" did not, one word order apart.
|
|
lets check the daily orch log and see how the run went | varaverk | Scheduler | | target=Orchestrators/daily_sync_maintenance run=yes has=run_record,log_tail
|
|
lets look at daily orch log and see how it went last run | varaverk | Scheduler | | run=yes has=run_record,log_tail
|
|
# Confirmed good in the WebGUI dock. Plural "logs", and "how it went" rather than "how the run
|
|
# went" — neither exact string was covered, and both are how the question actually gets typed.
|
|
lets check the daily orch logs and see how it went | varaverk | Scheduler | | target=Orchestrators/daily_sync_maintenance run=yes has=run_record,log_tail
|
|
how did the daily orch go | varaverk | | | target=Orchestrators/daily_sync_maintenance run=yes has=run_record,log_tail
|
|
how did the weekly orch go last run | varaverk | | | target=Orchestrators/weekly_sync_maintenance run=yes has=run_record
|
|
did the watchdog orchestrator run | varaverk | | | target=Orchestrators/watchdog_orchestrator run=yes
|
|
give me a rundown of the daily orch | varaverk | | | run=yes has=run_record
|
|
how long did the daily orch take | varaverk | | | run=yes has=run_record
|
|
|
|
# Works from the AI tab, which sends no scope at all — resolution is from the question, not the page.
|
|
how did the daily orch go last run | varaverk | | | target=Orchestrators/daily_sync_maintenance has=run_record
|
|
|
|
# ── A clean run earns the record and the log, and nothing else ─────────────────────────────────
|
|
# The health sweep and the recent-warning lines are ~660 tokens that answer nothing when the run
|
|
# exited 0, taken straight out of the passages' budget. They come back the moment it did not.
|
|
lets check the daily orch logs and see how it went | varaverk | Scheduler | | diag=no hasnt=health,warnings has=run_record,log_tail
|
|
how did the daily orch go | varaverk | | | diag=no hasnt=health,warnings
|
|
why did the daily orch fail | varaverk | | | diag=yes has=health,warnings,log_tail
|
|
why did the weekly orch fail | varaverk | | | diag=yes has=health,warnings
|
|
|
|
# ── Definitional questions must NOT be answered with last night's log ──────────────────────────
|
|
what does the daily orchestrator do | varaverk | | | target=Orchestrators/daily_sync_maintenance run=no hasnt=run_record,log_tail
|
|
how does the daily orch work | varaverk | | | run=no hasnt=run_record,log_tail
|
|
what is the daily orchestrator | varaverk | | | run=no hasnt=run_record
|
|
|
|
# ── Ambiguity resolves to nothing rather than guessing a script ────────────────────────────────
|
|
# A scored match here attaches the wrong log and answers confidently about a run nobody asked
|
|
# about, which is indistinguishable from a right answer unless you already knew.
|
|
how did sync go | varaverk | | | target=none run=no
|
|
what does RSYNC_ENABLED do | varaverk | | | target=none run=no has=conf_keys
|
|
# Whole words only. "fail" contains "ai", and ai.log is a real log id — a substring match here
|
|
# attached the AI's own log to questions about everything else.
|
|
why did this run fail | varaverk | | | target=none
|
|
is the mail relay working again | varaverk | | | target=none
|
|
|
|
# ── The Recent Activity buttons: one click from a row to an answer about that run ──────────────
|
|
# Both open the log first, so the dock is scoped to troubleshoot against that script's log id.
|
|
Why did this run fail? | troubleshoot | Orchestrators/daily_sync_maintenance | | diag=yes has=log_tail,run_record
|
|
How did this run go? | troubleshoot | Orchestrators/weekly_sync_maintenance | | has=log_tail,run_record
|
|
|
|
# ── General Chat holds nothing, and hands Varaverk questions up rather than deferring ──────────
|
|
how was your day | chat | | | profile=chat caps=none hasnt=health,log_tail,incidents,conf_keys
|
|
what does arr_sync.sh do | chat | | | profile=varaverk
|
|
is RSYNC_ENABLED on right now | chat | | | profile=varaverk has=conf_keys
|
|
how did the daily orch go | chat | | | profile=varaverk run=yes has=run_record
|
|
|
|
# Chat opened against a script must not be handed the operator's own incident notes about it.
|
|
how was your day | chat | Orchestrators/daily_sync_maintenance | | profile=chat hasnt=incidents
|
|
|
|
# ── Troubleshoot gets the log for whatever is open, plus the right to file a bug ───────────────
|
|
why did the weekly orch fail | troubleshoot | Orchestrators/weekly_sync_maintenance | | diag=yes has=log_tail,run_record
|
|
what is going on here | troubleshoot | Orchestrators/daily_sync_maintenance | | diag=yes has=log_tail
|
|
|
|
# ── The code profile answers from the model alone: no passages, no live state ──────────────────
|
|
write me a script that copies a folder | code | | | hasnt=health,log_tail,incidents,conf_keys
|