Repository navigation
feat: BYOK provider routing, token metering, and the model catalog - #31
Merged
Merged
Conversation
Closes design audit gap #4 in full (paired with omnibioai-auth's key-reveal endpoint and omnibioai-rag's Claude/OpenAI client routing): - build_rag_query accepts model="claude"/"openai" only together with use_own_key: true -- there is no platform-wide key for either provider, only an organization's own BYOK key. - _handle_billable_literature_call reveals the organization's decrypted key from omnibioai-auth immediately before forwarding a BYOK-routed call to RAG (never stored, never logged, attached to that one request body only), resolved before the concurrency slot and quota are touched so a request that can't get a key never consumes either. - A BYOK-routed call that gets real token counts back from RAG also emits llm.tokens.input/llm.tokens.output usage events (V1Store. emit_usage gains quantity/unit params for this) -- never fabricated for the default path, where these stay absent exactly as before. model_source/token counts in the public response now relay whatever RAG actually reports, rather than the old hardcoded "omnibioai_gpu"/ None/None. - GET /v1/models lists claude/openai alongside the default, price: null for all three (BYOK means an org pays its own provider directly; this platform still charges nothing on top). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes design audit gap #4 in full, as three paired PRs across omnibioai-auth (key-reveal endpoint), omnibioai-rag (actual Claude/OpenAI client routing), and this repo (request validation, the reveal-and-forward step, token-event emission, and the model catalog).
build_rag_querynow acceptsmodel="claude"/"openai"-- only together withuse_own_key: true, since there is no platform-wide key for either provider, only an organization's own BYOK key. Either alone is rejected with a clearfieldpointing at whichever one is missing._handle_billable_literature_callreveals the organization's decrypted key from omnibioai-auth's new internal endpoint immediately before forwarding a BYOK-routed call -- resolved before the concurrency slot and quota are touched, so a request that can't get a key never consumes either. The key is attached to the one RAG request body only; never stored, logged, or visible in the headers forwarded downstream.llm.tokens.input/llm.tokens.outputusage events (V1Store.emit_usagegainedquantity/unitparams for this, defaulting to the unchanged1/"requests") -- never fabricated for the default path, where these stay absent exactly as before.model_source/token counts in the public/v1/literature/answersresponse now relay whatever RAG actually reports, instead of the old hardcoded"omnibioai_gpu"/None/None.GET /v1/modelslistsclaude/openaialongsidedefault,price: nullfor all three -- BYOK means an org pays its own provider directly; this platform still charges nothing on top (same "don't invent a dollar figure" rule as everywhere else this gap has touched pricing).Test plan
pytest tests/test_literature_contract.py-- 57 passed, including the full BYOK validation matrix (model without use_own_key, use_own_key without a BYOK model, both together accepted and passed through, default path's RAG body has nomodelkey at all)pytest tests/test_v1_literature.py-- 78 passed (71 existing + 7 new: reveal call + RAG forward with the key attached, 503/400/502 error mapping for reveal failures, token-usage event emission alongside the answer event, confirming the default path never emits token events, and confirming/searchstill ignoresmodel/use_own_keyentirely)🤖 Generated with Claude Code