fix(langchain): Record reasoning tokens as metric - #2393
Open
Dylan Pulver (dylanpulver) wants to merge 1 commit into
Open
fix(langchain): Record reasoning tokens as metric#2393Dylan Pulver (dylanpulver) wants to merge 1 commit into
Dylan Pulver (dylanpulver) wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LangChain's
UsageMetadatacarriesoutput_token_details.reasoning, and nothing injs/srcorintegrationsread it.completion_tokenscovers the spend, so what was lost is the breakdown:no way to see how much of a reasoning model's output went to thinking, on any LangChain or
LangGraph span.
Mapped
reasoningonly.js/src/wrappers/mastra.ts:164-172settles the scope, picking up thenon-modality detail fields (
cacheRead,cacheWrite,reasoning) and leaving the modality onesalone.
OutputTokenDetailsalso carriesaudio,image,video,documentandtextthroughModalitiesTokenDetails; the input side drops its copies too, so those are a separate change.Zero is recorded, not dropped, matching
prompt_cached_tokens: 0in the existing snapshots.js/src/instrumentation/plugins/langsmith-plugin.ts:456has the same gap. Left alone because#2386 is moving that tree.
Snapshot churn is 30 added lines across 10 files, every one
completion_reasoning_tokens, bothsides of each span-tree pair regenerated, no deletions.
Ran
pnpm testinjs/(1648 passed),check:typings,lint, andtest:e2e:updatethentest:e2eonce each, not twice. On that assert passscenarios/git-metadatatimed out at 30sunder parallel load. It passes in isolation here in 1.19s and on
mainin 1.20s, and the twoscenarios this change touches pass, so I read it as local contention. Worth confirming on CI.
Both new unit tests fail against
mainon assertions.