fix: read the answer from the message_end event pi 0.87.x emits - #79
Conversation
pi is installed unpinned in the agent-pi image and its event vocabulary has
already changed once. A v0.4.0 agent-pi pod produced
{"type":"message_end","message":{...,"role":"assistant",...}} followed
by {"type":"agent_settled"}, and extractEventText's two-case switch matched
neither — so a run that had answered correctly came back as "no result found
in pi CLI output".
The symptom points at the model; the defect is in the parser. Only a real
prompt through the runner exercises the path at all, which is how it
survived: the fleet's working agents run older images, and the
recently-built ones had never been prompted.
The message_end case carries a role guard, because that event fires for
every role — without it a user or system message would be returned as the
answer, and a run that failed would look like a successful echo of its own
prompt.
Both vocabularies are kept: agent_end/message_update still cover the older
pi that pi-agent's pinned v0.1.7 image runs, which is exactly why that
agent kept working while the freshly-built one did not.
There was a problem hiding this comment.
I have completed the review. Here's my assessment:
Step 5: Consolidated Report
Must Fix (Critical)
None.
Should Fix (Important)
None.
Nice to Have (Optional)
None.
Notes
-
pi/pi-runner_test.go— The ast-grep finding (go-testing/suite-test-file-required) is a false positive.pi_suite_test.goexists in the same directory and is a proper Ginkgo suite (TestPi+RunSpecs). The newvar _ = Describe("piRunner event vocabulary", ...)block is correctly placed in the same package (pi_test) and discovered by that suite. -
Role guard correctness — Verified at
pi-runner.go:246-248: themessage_endcase returns""for non-assistantroles. This is load-bearing and well-documented in both code comments and test comments. -
Design trade-off acknowledged — The CHANGELOG entry and function docstring (lines 225-235) explicitly acknowledge that a future vocabulary change would silently produce "no result found". This is an intentional, documented trade-off.
-
Test quality — Three specs cover: older
agent_endvocabulary, newmessage_endvocabulary (withthinkingblock + text block), and the role-guard failure case. The shim pattern is clean and consistent with the existing tests in the file.
Selector Mode Traceability
| Finding | Rule | Disposition |
|---|---|---|
| Suite file required | go-testing/suite-test-file-required |
False positive — pi_suite_test.go exists and is proper |
Verdict
{
"verdict": "approve",
"summary": "Bugfix adds `message_end` handling to `extractEventText` with a load-bearing role guard. Three new Ginkgo specs cover the old vocabulary, the new vocabulary, and the role-guard failure case. The ast-grep suite-file finding is a false positive — `pi_suite_test.go` already exists.",
"comments": [],
"concerns_addressed": [
{
"concern": "correctness: role guard on message_end is load-bearing",
"disposition": "addressed",
"detail": "Verified at pi-runner.go:246-248 — returns '' for non-assistant roles, preventing system/user messages from being returned as answers. Well-documented in code comments and test comments."
},
{
"concern": "correctness: future pi releases with new event types will silently return 'no result found'",
"disposition": "not-an-issue",
"detail": "This is a documented, intentional trade-off. Function docstring (lines 225-235) and CHANGELOG entry explicitly acknowledge it. Not a defect."
}
]
}
The defect
A
v0.4.0agent-pi pod answered a prompt correctly and the runner reported failure:pi was fine. Its stream ended with:
{"type":"message_end","message":{...,"role":"assistant","content":[ {"type":"thinking",...},{"type":"text","text":"ACK"}]}} {"type":"agent_settled"}extractEventTextswitches on exactly two event types —agent_endandmessage_update— and matched neither, soresultTextstayed empty. The symptom points at the model; the defect is in the parser.Why it survived
pi is installed unpinned in the agent-pi image:
so its event vocabulary drifts with every rebuild — and the only path that exercises the parser is a real prompt through the runner. The fleet's working agents run older images (
pi-agentis pinned atv0.1.7), and the recently-built ones had never been prompted. The drift is invisible precisely because the agents that work are the ones nobody rebuilt.This is therefore fleet-wide latent, not service-mode-specific: any agent-pi image rebuilt today would break the same way on its next real prompt.
The fix
extractEventTextgains amessage_endcase.piEvent.Messagealready carriesRoleandContent, so it is small — but the role guard is load-bearing:message_endfires for every role, so without it a run producing no assistant text would return the system prompt as its answer. That is the worst available failure shape — a broken run that looks like a successful one — so it has its own spec.Both vocabularies are kept rather than replaced:
agent_end/message_updatestill cover the older pi thatpi-agent's pinned image runs, which is exactly why that agent kept working.Tests
7 of 7specs in thepipackage, up from 4. Three new, using the file's existingpi-shim pattern so they run through the publicRun:agent_endvocabulary still readsmessage_endstream reads —thinkingblock thentext, exactly as observed livemake precommitPASS —go test -raceall 8 packages, golangci-lint 0 issues, gosec clean, trivy 0 vulns / 0 secrets, osv clean.Still to come
Pinning pi in
agent-pi's Dockerfile to0.87.1— the version thev0.4.0image actually runs, confirmed against npm — which is the half that stops the recurrence. This PR makes it work; that pin is what keeps it working.