Repository navigation
M2: atomic reserve-before-work quota check - #25
Merged
Merged
Conversation
Part of M2. The old quota check was check-then-later-decrement: quota_remaining() read the counter before forwarding to RAG, consume_quota() only decremented it after a successful response. Any number of concurrent requests could all observe "quota available" before any of them decremented, and all of them would succeed -- overrunning a near-zero quota by however many were in flight at once. Replaces that pair with reserve_quota() (atomic Redis DECR, compensated back if it would go negative -- only as many concurrent reservations as there are units left can ever succeed) called before forwarding to RAG, and release_quota() (refund) called if the forwarded request then fails upstream, since the reservation assumed success before knowing the outcome. Both are wired into the one shared _handle_billable_literature_call path /literature/answers and /literature/search both already go through, so this closes the race for both at once. Tests: 355 passed, including new coverage for concurrent-reservation exhaustion (exactly one of two requests over a 1-unit quota succeeds, never both) and quota refund on upstream failure. ruff clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part of M2 (make quota enforcement real and race-free). The old quota check was check-then-later-decrement:
quota_remaining()read the counter before forwarding to RAG,consume_quota()only decremented it after a successful response came back. Any number of concurrent requests could all observe "quota available" before any of them decremented, and all of them would succeed — overrunning a near-zero quota by however many were in flight at once.reserve_quota()— atomic RedisDECR, compensated back (INCR) if it would go negative. Only as many concurrent reservations as there are units left can ever succeed; unmetered orgs (no quota key) are unaffected.release_quota()— refunds a reservation when the forwarded request then fails upstream, since the reservation optimistically assumed success before knowing the outcome._handle_billable_literature_callpath/literature/answersand/literature/searchalready go through, so this closes the race for both endpoints at once.Test plan
pytest— 355 passed, including new coverage for concurrent-reservation exhaustion (exactly one of two requests over a 1-unit quota succeeds, never both) and quota refund on upstream failureruff check .— clean🤖 Generated with Claude Code