[austinchennn]

Description:

Fixes #63, Fixes #64

This PR is the playback half (L2/L3) of the follow-ups to #65 and #66, stacked on #67 for its BoundedMap. It is based on provider-errors-usage, and will be retargeted to main once #67 merges.

Interruption guarantee 3, "the transcript matches what was heard", depended on five structures inside AgentSession that changed at different times and were never cleaned up (#64). It also depended on edge reports that skipped audio the user never heard at all (#63).

Behaviours Completed

• One ledger for what was heard. PlaybackLedger (packages/core/src/playback.ts) replaces #audioItems, #trailingAudio, #played, #interruptedItems and #untruncated with one record per audio item.
  • Each record holds an explicit phase (live, trailing, cut, interrupted), the received and played milliseconds, and the original text.
  • received(), report(), interrupt(), endTurn() and userTookFloor() only return truncation decisions. AgentSession applies them to the model and the transcript, the same split as ToolStreams.
  • This commit is a pure refactor: every existing test passes unchanged.
• fix(core): 打断相关的记录在长会话中只增不减 #64: bounded.
  • The ledger keeps at most PLAYBACK_ITEMS_KEPT (32) items, oldest first, whatever their phase. Because an item's report, phase and text are one record, forgetting it forgets all of it.
  • The separate trailing-audio cap is gone.
  • A late report still truncates within the window. A report for an item already forgotten, or never received, is ignored.
• fix(client): 打断时未开始播放的音频 item 不上报 playback，provider 对话不会被截断 #63: audio nobody heard is truncated too.
  • Client: PlaybackRing.flush() returns 0 for items still queued that never started, and the session now sends those reports.
  • Server: an item whose audio.start was dropped on an interruption, or whose audio arrived while no socket was attached, is reported to the session as played for 0 ms.
• Spec 1.4. §8.4 now says a client reports never-started items with playedMs 0, and a server treats audio it could not deliver as not heard. This is a SHOULD-level clarification; the playback schema already allows 0.
• Docs. Guarantee 3 in docs/ARCHITECTURE.md describes the ledger and the edge contract. The interruption page (English and Chinese) and the client and SDK playback bullets are updated to match.

Files Changed

• Core: new packages/core/src/playback.ts; packages/core/src/session.ts; new packages/core/test/playback.test.ts; packages/core/test/session.test.ts.
• Client: packages/client/src/{ring,audio,session}.ts; packages/client/test/{audio-dsp,session}.test.ts.
• Server: packages/server/src/index.ts, packages/server/test/server.test.ts.
• Spec and docs: spec/protocol.md (§8.4 and Appendix B), spec/README.md, docs/ARCHITECTURE.md, website/src/content/docs/{concepts,zh-cn/concepts}/interruption.md.

Brief Explanation

• Single responsibility. AgentSession orchestrates; the ledger owns the reconciliation as a small explicit state machine:

  • live → trailing at turn end;
  • trailing → cut when the user takes the floor;
  • cut or live → interrupted when cut short;
  • heard in full → forgotten.

  The ledger has no model, emitter or clock, so its transitions are unit-tested directly. session.ts loses about 40 lines.

• The edge owns the truth about what was heard. The client knows what it played. The server is also an edge: it knows what it never delivered (a dropped audio.start, no socket attached). Both report through the same playback() entry point, so core needs no special case.

• Cost. received() runs on the audio hot path with one map lookup and no allocation. Core is 5.72 KB of its 15 KB budget, client 5.28 KB of 20 KB.

Testing

• The fix(core): 打断相关的记录在长会话中只增不减 #64 bounding test failed against the unbounded ledger. The fix(client): 打断时未开始播放的音频 item 不上报 playback，provider 对话不会被截断 #63 tests (ring reports 0, client sends 0, server reports dropped audio, server reports detached audio) failed before the fix.
• New ledger tests: refining on later reports, only the closed responses interrupted, clamping to received audio, trailing audio cut after the user takes the floor (report before or after), audio heard in full forgotten, unknown items ignored, original text kept, bounded eviction.
• New session-level bounding test.
• npm test (31 files, 520 tests), npm run typecheck, npm run check:budgets, npm run check:spec, npm run test:e2e (7), npm run test:conformance (169), npm run test:workers (2) and the docs build: all green.

Unsure About

• Untested against Boson. Whether Boson accepts conversation.item.truncate with audio_end_ms: 0 has not been checked against the live API. OpenAI accepts it, and Boson documents a 0 ack for text-only sessions.
• Detached audio timing. Audio produced while detached is reported unheard once per item. If the client reconnects mid-item, its later report overrides the 0 and counts only what played after the reconnect, which errs toward truncating more.
• Window size. PLAYBACK_ITEMS_KEPT is 32. A turn with more than 32 audio items would forget its oldest live items early. No current adapter produces more than about two per response.
• Custom sinks. An AudioSink that does not return zeros keeps working; it just misses this correction.