Repository navigation
Anthropic prompt caching: cache tokens not emitted on the Anthropic paths — is session total_tokens systematically low? #1445
Description
A semantics question found while auditing token accounting across agent-telemetry SDKs.
The observation
On the Anthropic paths, usage extraction reads only usage.input_tokens / usage.output_tokens and computes total = input + output:
- non-streaming:
agentops/instrumentation/providers/anthropic/attributes/message.py(~L319-332) - streaming:
.../anthropic/stream_wrapper.py(~L167-177 and ~L406-418)
Anthropic's cache_read_input_tokens / cache_creation_input_tokens are separate fields (not included in input_tokens), and I don't find them read anywhere on these paths — LLM_USAGE_CACHE_READ_INPUT_TOKENS / CACHE_CREATION_INPUT_TOKENS are defined in agentops/semconv/span_attributes.py but have no setter calls in the Anthropic instrumentation.
Meanwhile the OpenAI Responses path (providers/openai/attributes/response.py, ~L296-307) does emit cached_tokens → cache_read and reasoning_tokens — so the asymmetry is within one SDK.
Why it matters
The OTel GenAI semantic conventions for Anthropic (semantic-conventions-genai, docs/gen-ai/anthropic.md) are explicit: cache read/write tokens MUST be added to the native input_tokens to compute gen_ai.usage.input_tokens. If they're dropped instead, a prompt-caching Claude session reports total_tokens systematically below what the provider actually meters (cache reads are billed, just at a lower rate — they're not free tokens).
The question
Is this asymmetry intentional (e.g. a deliberate "native usage only" convention on the Anthropic path)? If not: for sessions with cache_control enabled, does any downstream token/cost figure derived from these spans undercount? Happy to send a minimal repro — same prompt called twice with caching on, asserting the cache attributes are absent and the total vs the Anthropic console's billed tokens diverge — or a small PR adding the cache fields per the semconv rule.
Context: I maintain AgentMeasure, a measurement-conformance project running a public 30-projects/30-days check of agent-telemetry semantics; this is the finding worth asking about directly. Not claiming a bug without your read on the intended convention.
Activity
jstar0 commentedon Sep 8, 2026
Independently confirmed on current main that Anthropic usage extraction in both the non-streaming and streaming paths reads only input_tokens and output_tokens; the existing cache-token semantic constants are not set on these paths. A synthetic usage object with input_tokens=100, output_tokens=20, cache_read_input_tokens=500, and cache_creation_input_tokens=50 would therefore expose only prompt=100, completion=20, and total=120, with no cache breakdown.
This looks like a focused observability-correctness fix, but it changes the emitted usage contract. I can prepare a small PR covering the message and stream paths plus fixture-based regressions after maintainers confirm whether the intended transition is: (a) add cache breakdown attributes and include cache tokens in input/total, or (b) preserve native input/output and expose cache fields separately. I will wait for that direction before starting code.
roy-tong commentedon Sep 8, 2026
Thanks for the independent confirmation — and for the (a)/(b) framing, which is exactly the decision point.
From the issue-filer side I'd recommend (b): preserve the native input/output totals as today and expose cache-read / cache-creation as separate attributes. Two reasons: (1) it matches the gen-ai semconv shape — cache fields are distinct attributes and the consumer chooses its accounting basis — and (2) the alternative (folding cache tokens into input while also emitting breakdowns) silently recreates the double-count class this issue is about unless every downstream consumer knows to subtract, which most won't.
What would help either way is a fixture pinned to the chosen contract: synthetic usage (input 100 / output 20 / cache_read 500 / cache_creation 50) asserting exactly which attributes are emitted and that no consumer-visible aggregate adds cache tokens into input totals. AgentMeasure keeps a corpus of exactly these shapes (cache-accounting + reasoning-subset invariants); happy to supply the fixture or review the PR against it — either is fine.