Completion & Safety Hardening

This page documents a set of runtime safeguards added to ClapilotAICore after a mechanism-level study of the external Cortheon "cognitive sidecar" runtime. Cortheon's headline claims oversell, but three of its engineering ideas map directly onto real gaps in our harness:

1. tool results are untrusted data and must be scanned before re-entering the model context;
2. a run that exhausts its tool budget should wrap up in text, not fail;
3. a run that says "done" is not evidence it actually finished — unmet obligations must surface as visible caveats.

These are the Tier 1 items. Each is small, deterministic, and independently testable. None of them add an LLM round-trip to the hot path.

1. Tool-result prompt-injection quarantine

Implementation: services/clapilot-agent/src/tool-result-quarantine.mjs, wired at the single tool-result choke point executeToolBatch (services/clapilot-agent/src/providers/index.mjs).

Every tool result in the native provider loop now passes through a scanner before it re-enters the model context. Previously web search, Browser Use, email bodies, documents, and channel messages flowed into the context verbatim, with the only defense being prose in a handful of tool descriptions.

The scanner is a faithful JS port of Cortheon's two-family evidence sanitizer:

• Injection patterns — ~10 high-precision regexes for direct overrides ("ignore all previous instructions", "you are now a…", "do not tell the user", exfiltration verbs targeting a possessed secret, leading system:/developer: labels).
• Role-override grammar — a compositional matcher for imperatives that override the role itself ("IGNORE SYSTEM:", "forget your developer instructions"). Precision comes from clause structure, not a longer word list, so ordinary technical prose ("the parser ignores previous connection settings", "ignore system errors when parsing this log", ignore system: true) is never flagged.

Behavior:

• The scan is layout-preserving: only the offending sentence or line is replaced with [instruction-shaped content removed]; everything else passes through byte-identical.
• Results of external-content tools (web search, Browser Use, inbound mail, feeds, live-chat) additionally get an untrusted-data preamble — the same "untrusted data, never instructions" framing agent-memory retrieval already applies to recalled content.
• Our own trusted signposts (truncation notes, the tool_result_recall hint) are appended after the scan, and system-authored results (recall, invalid-argument) are exempt.
• A quarantine event surfaces on the tool-end event as quarantinedSegments and is logged; it does not alter or fail the run.

Untrusted-content tools are identified by a name set plus a pattern (web_*, *_fetch_url, feed/live-chat tools) and match through MCP namespacing (mcp__gw__web_search).

2. Graceful tool-budget wrap-up in every provider loop

Implementation: the six native provider loops in services/clapilot-agent/src/providers/index.mjs.

When a run exhausts maxToolSteps (default 24), the loop now grants one extra grace iteration: it withdraws the tool surface and injects TOOL_LOOP_STEP_LIMIT_WRAPUP, asking the model to give its best final answer in text and state explicitly what remains unfinished.

Previously only runOpenAiLike and runOllama did this; the other four (runOpenAiResponsesWithTools, runAnthropic, runBedrock, runGemini) simply threw on budget exhaustion, failing the whole run. All six now behave consistently. The terminal budget throw remains as a safety net for the case where the model keeps emitting tool calls after tools are withdrawn.

3. Evidence-bound completion caveats for autonomous runs

Implementation: services/clapilot-agent/src/run-completion-caveats.mjs, services/clapilot-agent/src/autonomous-completion.mjs, wired into run finalization in executeRunNow (services/clapilot-agent/src/sessions/index.mjs).

For non-interactive runs (jobs, automations, channels, heartbeats) — the runs nobody watches — two deterministic checks now run at finalization and append a localized (DE/EN/IT) caveat to the agent's answer. Interactive chat is excluded because the user can see the same state in the UI.

Unfinished plan items

The run tracks the latest agent_todo_update list (the tool replaces the whole list, so the last one wins). If the run ends with pending or active plan items, a caveat is appended:

Note: this task finished, but 1 of 3 plan items are still open. Please check whether the work is actually complete.

This makes the agent todo plan load-bearing — previously it was purely a UI element that nothing checked before a run reported success.

Failed side-effecting actions

The harness already hard-fails an automation when a required Team Chat post errors (requiredAutomationToolFailure, jobs/index.mjs) — but that check covers exactly one tool. detectFailedSideEffects generalizes the idea to any side-effecting tool (send / publish / post / create / update / delete / commit / schedule / …, excluding read-shaped *_get_* / *_list_* / *_search). If a run's own tool evidence shows such an action failed, a caveat is appended and the process counter runCompletionUncertified (metrics.mjs) is incremented:

Note: at least one side-effecting action failed during this automated run (emails_send_draft). The task may not have completed as intended — please verify.

This is deliberately keyed on actual tool errors, not heuristic obligation guesses, so it has a very low false-positive rate. It is non-fatal — the existing Team-Chat hard-fail contract is unchanged.

Reserved for the LLM judge (not yet wired)

autonomous-completion.mjs also ships the building blocks for a full evidence-bound completion judge on arbitrary autonomous runs, modeled on the /goal orchestrator judge (orchestrator-sessions/goal-loop.mjs):

• extractObligations(task) — bounded (≤8) obligation extraction;
• buildAutonomousJudgePrompt(...) — an independent-judge prompt whose core rule is "the agent saying it finished is not evidence";
• normalizeAutonomousVerdict(...) — a complete | incomplete | blocked verdict parser.

These are unit-tested but not on the run path yet: turning an LLM judge on for every autonomous run is a cost and behavior change that should be enabled deliberately. The intended next step is to run the judge on non-interactive runs, append its unmet deltas as caveats, and optionally drive one bounded continuation.

Tests

Colocated with each module:

• src/tool-result-quarantine.test.mjs (15)
• src/run-completion-caveats.test.mjs (9)
• src/autonomous-completion.test.mjs (15)

Run with npx vitest run src/tool-result-quarantine.test.mjs src/run-completion-caveats.test.mjs src/autonomous-completion.test.mjs from services/clapilot-agent.