Skip to content

fix(langchain): Record reasoning tokens as metric - #2393

Open
Dylan Pulver (dylanpulver) wants to merge 1 commit into
braintrustdata:mainfrom
dylanpulver:fix/langchain-reasoning-tokens
Open

fix(langchain): Record reasoning tokens as metric#2393
Dylan Pulver (dylanpulver) wants to merge 1 commit into
braintrustdata:mainfrom
dylanpulver:fix/langchain-reasoning-tokens

Conversation

@dylanpulver

@dylanpulver Dylan Pulver (dylanpulver) commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

LangChain's UsageMetadata carries output_token_details.reasoning, and nothing in js/src or
integrations read it. completion_tokens covers the spend, so what was lost is the breakdown:
no way to see how much of a reasoning model's output went to thinking, on any LangChain or
LangGraph span.

Mapped reasoning only. js/src/wrappers/mastra.ts:164-172 settles the scope, picking up the
non-modality detail fields (cacheRead, cacheWrite, reasoning) and leaving the modality ones
alone. OutputTokenDetails also carries audio, image, video, document and text through
ModalitiesTokenDetails; the input side drops its copies too, so those are a separate change.

Zero is recorded, not dropped, matching prompt_cached_tokens: 0 in the existing snapshots.

js/src/instrumentation/plugins/langsmith-plugin.ts:456 has the same gap. Left alone because
#2386 is moving that tree.

Snapshot churn is 30 added lines across 10 files, every one completion_reasoning_tokens, both
sides of each span-tree pair regenerated, no deletions.

Ran pnpm test in js/ (1648 passed), check:typings, lint, and test:e2e:update then
test:e2e once each, not twice. On that assert pass scenarios/git-metadata timed out at 30s
under parallel load. It passes in isolation here in 1.19s and on main in 1.20s, and the two
scenarios this change touches pass, so I read it as local contention. Worth confirming on CI.

Both new unit tests fail against main on assertions.

@dylanpulver Dylan Pulver (dylanpulver) changed the title fix(langchain): record reasoning tokens as completion_reasoning_tokens fix(langchain): Record reasoning tokens as metric Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant