fix(bedrock): fold cache tokens into non-streaming converse usage - #13
Open
teriyakichild wants to merge 1 commit into
Open
teriyakichild wants to merge 1 commit into
teriyakichild wants to merge 1 commit into
Conversation
The non-streaming Converse conversion copied inputTokens raw, dropping cacheReadInputTokens / cacheWriteInputTokens even though the internal TokenUsage parses them — AWS defines inputTokens as exclusive of cached tokens, so prompt usage was understated whenever caching was active. Fold them in, matching the streaming path.
Greptile SummaryAligns non-streaming Amazon Bedrock Converse token accounting with the streaming path.
Confidence Score: 5/5The PR appears safe to merge with no actionable correctness or security issues identified. The new non-streaming conversion matches the existing streaming arithmetic, handles absent cache counters as zero, preserves provider-reported output and total usage, and has regression coverage for cache-active responses.
|
| Filename | Overview |
|---|---|
| rig-integrations/rig-bedrock/src/types/assistant_content.rs | Correctly folds optional cache token counts into non-streaming input usage and verifies the normalized totals with a focused regression test. |
Reviews (1): Last reviewed commit: "fix(bedrock): fold cache tokens into non..." | Re-trigger Greptile
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #12, surfaced in downstream adversarial review of mezmo/aura#631 (finding 3): the non-streaming Converse path copied
inputTokensraw and droppedcacheReadInputTokens/cacheWriteInputTokens, even though the internalTokenUsageparses them. AWS definesinputTokensas exclusive of cached tokens, so non-streaming prompt usage was understated whenever caching was active. The streaming path already folds correctly — this aligns the non-streaming conversion, with a regression test.aura is streaming-only, so no aura behavior changes; the pin can move on its next routine bump.