Repository navigation
fix(llm-client): keep response text out of JSON error diagnostics - #1016
Conversation
) When completeJson() got a response that was not JSON, failed to parse, or failed heuristic repair, the diagnostic sent to the log callback (wired to api.logger.debug) and returned by getLastError() copied response text: via preview/jsonPreview fields and via raw JSON.parse messages, which quote their input. That text can be the conversation or memory being extracted. All three transports (host, direct API, OAuth) now build these diagnostics with one shared formatter that keeps the stage label, a category (non-json / parse-failure / repair-failure), the response length (chars), the extracted candidate length (jsonChars) and the parse error class plus a numeric position when the engine reports one. No raw preview opt-in is added. Valid JSON responses and the repair-success log line are unchanged. test/llm-json-error-redaction.test.mjs drives the real createLlmClient() through each transport (fake runtimeLlmComplete, loopback HTTP server, fake OAuth file + intercepted SSE fetch) with a synthetic marker and fails 9/12 against the previous implementation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019whQZvXdU3jHYnAnTxvvPC
|
CI status on head
Generated by Claude Code |
rwmjhb
left a comment
There was a problem hiding this comment.
Reviewed 1c7dd6b through the orchestrator. Approved as an improvement over the existing raw-response previews and parser messages; full-suite verification passed. Automated build verification was skipped.
Please address the remaining bounded redaction gap: describeJsonParseError() searches the entire parser message with an unanchored position-number regex. On Node 24, malformed fenced output containing "position 9876543210" can have those response-derived digits reported as a parser offset in logs and getLastError(), including the repair-error path. This does not retain the whole response, but it means redaction is not complete. Omit offsets or extract only proven engine metadata outside quoted input.
Add numeric-marker regressions across the three transports, plus wrapped/fenced responses where raw and extracted JSON lengths differ; assert chars and jsonChars separately. Existing alphabetic-marker exclusions do not catch this numeric leak.
Closes #1014
What changed
When
createLlmClient().completeJson()got a response that was not JSON, could not be parsed, or could not be repaired, the diagnostic sent to thelogcallback (wired toapi.logger.debug) and returned bygetLastError()copied response text. It got in two ways:preview=/jsonPreview=fields, andJSON.parseerror messages, which quote their input (V8:Unexpected token 'S', "{"note": SYNTHETIC_"... is not valid JSON).All three transports (host callback, direct API, OAuth) had the same three error sites. In
src/llm-client.ts, all nine now use one sharedformatJsonFailure()formatter, backed bydescribeJsonParseError(), which keeps only the error class and a numeric position. The unusedpreviewText()helper is removed. No opt-in raw preview is added: the client has no existing debug flag that fits.Unchanged:
no JSON object found/JSON.parse failedmessage prefixesdist/src/llm-client.jsis rebuilt (it is the onlydistfile that changed), and there is a CHANGELOG entry.Diagnostic fields kept
[label]host-transport/OAuth/ none (direct API)category=non-json | parse-failure | repair-failurechars=response lengthjsonChars=length of the extracted JSON candidate (parse and repair failures)error=/repairError=: parse error class (e.g.SyntaxError), plusat position Nwhen the engine message has a numeric positionExample (host transport):
Tests
test/llm-json-error-redaction.test.mjsis registered innpm testand in thellm-clients-and-authCI group. It calls the realcreateLlmClient()through each transport:runtimeLlmCompletefetchthat returns SSEEach transport runs four cases:
The tests check that neither log lines nor
getLastError()contain the marker, apreview=field, or the rawJSON.parsemessage. They also check that the label, category, length and error class are still there, and that the valid control parses with no diagnostic.Against the previous
src/llm-client.ts, 9 of 12 fail (all error cases) and the 3 valid controls pass.Out of scope
index.tshas separate ingress/auto-capture debug lines withpreview=. Those are not part of this LLM-client path and are not touched.Generated by Claude Code