fix(anthropic): keep the messages cache readable past ambient status

[andybons]

The problem

Every user turn re-read the whole conversation at the cache-write price, silently.

withAmbientStatus (engine/process.go) appends an *EngineContext part to the newest user message on the per-request copy only — durable history never sees it. So the message that carried the block on turn N is re-rendered without it on turn N+1. The messages-tier breakpoint (provider/anthropic/transcode.go) sat after that block, so the entry each turn wrote was never a prefix of the next turn's request and could never be read back: only the system entry hit, and the conversation was rewritten every turn. identityStatusSegment is present on every request once version and start time are configured (engine/identity_status.go), so in serve this was the normal path, not an edge case.

The design

markAmbientBoundary adds one breakpoint: the last block strictly before the request's first ambient block. That is where the bytes stop changing across the turn boundary, so the entry it writes is exactly the prefix the next turn re-sends unchanged. Steady state becomes the healthy signature — read everything through the previous turn, write only the newest turn's delta.

The tail breakpoint stays. It is what lets the steps within one turn read each other, and a second entry costs nothing because a write bills only the delta past the highest hit. Three markers stays under the API's limit of four, and a request with no ambient block gets no boundary marker (its tail entry is already reusable).

An ambient block is identified by the engine-context sentinel, which message.NeutralizeEngineContextSentinel guarantees only a genuine *EngineContext part can emit — so this needs no signature or interface change.

Verification

provider/anthropic/cache_ambient_boundary_test.go drives the real transcoder over a turn pair and measures the longest prefix that both ends on a breakpoint and still matches the next turn's request: 0 blocks before this change, 3 after. Red-verified by disabling the call — both boundary tests report reusable prefix = 0 blocks. Cases cover mid-turn requests, the no-ambient control, the four-breakpoint budget, and a leading ambient block.

The delegated claude-code lane cannot have this defect — it sends the CLI only the newest user message and resumes the child's own session, so it never re-renders an earlier message. What it can change is the system prompt it passes on every turn, so engine/claude_code_cache_stability_test.go pins that two consecutive delegated turns pass byte-identical --append-system-prompt text; red-verified by appending a per-turn marker in runClaudeCodeTurn.

go test -race ./... green (22 packages), go vet clean.

Measuring it in the fleet

cache_read_tokens / cache_write_tokens are already journaled per turn (engine/turn_metrics.go). On a native-lane session, cache_write_tokens per user turn should drop from roughly the conversation size to the last turn's delta.

[Copilot]

🟢 Approval recommended

The implementation and added tests align with the stated design and verification, with only a minor documentation grammar nit noted.

Pull request overview

This PR fixes Anthropic Messages prompt-cache reuse across turns when the engine injects per-request ambient status (*message.EngineContext) into the newest user message, ensuring the cacheable prefix remains byte-stable from turn to turn.

Changes:

• Add an “ambient boundary” cache breakpoint (before the first ambient block) so cross-turn cache entries remain readable.
• Add regression tests for cross-turn cache prefix reuse (Anthropic native lane) and stable --append-system-prompt bytes across delegated Claude Code turns.
• Update documentation to describe the new breakpoint behavior and rationale.

File summaries

File	Description
provider/anthropic/transcode.go	Injects a new cache breakpoint at the last block before the first ambient EngineContext block via markAmbientBoundary.
provider/anthropic/cache_ambient_boundary_test.go	New tests validate the longest reusable cached prefix across a turn boundary and enforce the breakpoint budget.
engine/claude_code_cache_stability_test.go	New test ensures delegated Claude Code turns pass byte-identical --append-system-prompt text across turns.
docs/models-and-providers.md	Documents the ambient boundary breakpoint and its effect on cache economics.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.