You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Team state injected via experimental.chat.system.transform invalidates the provider prefix cache on every state change #53
Team state is pushed through experimental.chat.system.transform on every LLM call (src/index.ts:382, output.system.push(prompt) at src/index.ts:391). The pushed block carries member statuses, task counts, in-progress/recent tasks and undelivered messages (src/system-prompt.ts:24buildLeadSystemPrompt, src/system-prompt.ts:169buildTeammateSystemPrompt), and it mutates whenever a teammate transitions busy↔ready, a task is claimed/completed, or a message is delivered (markDelivered at src/system-prompt.ts:97 and :194 flips the pending list between turns).
Providers cache by prefix, so a 1-character change anywhere in the system prompt invalidates the entire conversation for that request: everything after the first changed byte is re-billed at full input price. The lead is only woken when state changed, so nearly every lead turn pays this; teammates pay it too via buildTeammateSystemPrompt.
Measured on a local proxy that logs provider usage (same mechanism, different plugin mutating the system prompt — the invalidation is a property of prefix caching, not of the proxy):
request
usage
cache hit
steady state
input=94427 cached=88832
94–100%
first request after the system prompt changed
input=91313 cached=24576
27%
worst observed
input=96833 cached=0
0%
At typical cache-read pricing (~0.1× input), that is ~7× the input cost of a hot turn, on every churned turn. It recovers on the next request only if the block stops changing.
Related upstream work in the same class: anomalyco/opencode #58485 ("Moving plan/build reminders invalidate the prompt cache and increase API costs"), #54935 ("Parallel sub-agents start a fresh session with their own cache key and no concurrency cap, so they cost more than one agent"), #50777, #56422.
Suggestion
The wake path is already cache-safe: src/index.ts:288-291 sends the tickle as a trailing promptAsync text part ([System: N new team message(s) available]), with wakeLeadTimestamps / WAKE_LEAD_COOLDOWN_MS dedupe. The message payload and the state summary could ride the same way — appending at the tail only costs the new bytes, everything before stays cached.
The reason the payload sits in the system prompt looks like compaction survival (README:303 "the lead's system prompt carries member statuses and task counts, so it stays aware across turns including after compaction"), and that constraint is already covered by the hook you implement at src/index.ts:395 (experimental.session.compacting → buildTeamCompactionContext): re-inject the state there after compaction. The Plan→Build agent flip in #37 is orthogonal (fixed by 42a29ec / 5919ee0 via explicit agent+model on promptAsync).
Minimal alternative if the current placement is intentional: a config knob (systemPromptTeamState: full | state-only | off, or a switch that moves message payloads to the trailing wake message) so cache-sensitive setups can opt out.
Activity