feat(llm): add OrcaRouter as a named LLM provider - #2180
Open
XiaoHuo888-hue wants to merge 1 commit into
Open
Conversation
Add OrcaRouter (https://www.orcarouter.ai) as a named provider for LLM-based extraction. OrcaRouter is an OpenAI-compatible gateway at https://api.orcarouter.ai/v1; model ids use the orcarouter/<model> prefix (e.g. orcarouter/auto, orcarouter/free). The pinned unclecode-litellm build has no native orcarouter/ provider prefix, so route orcarouter/<model> through the openai provider while keeping the full model id intact and pointing base_url at the gateway: - config.py: register orcarouter in PROVIDER_MODELS_PREFIXES (auto-resolves ORCAROUTER_API_KEY), add ORCAROUTER_BASE_URL and orcarouter_litellm_params() - utils.py: route orcarouter/ providers in perform_completion_with_backoff, aperform_completion_with_backoff, extract_blocks_batch, get_text_embeddings - async_configs.py: LLMConfig defaults base_url to the gateway for orcarouter/ - cli.py: route orcarouter/ in the -q streaming path - docs: document OrcaRouter usage in llm-strategies.md and README - tests: test_orcarouter_provider.py covers routing + auto key/base_url Verified locally: 9 new unit tests pass; chat completion and embeddings live-tested against the OrcaRouter gateway (HTTP 200). Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: jinhao.song <jinhao.song@myflashcloud.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add OrcaRouter as a named LLM provider for LLM-based extraction. OrcaRouter is an OpenAI-compatible gateway at
https://api.orcarouter.ai/v1; model ids use theorcarouter/<model>prefix (e.g.orcarouter/auto,orcarouter/free). It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.The pinned
unclecode-litellmbuild has no nativeorcarouter/provider prefix, so Crawl4AI routesorcarouter/<model>through theopenaiprovider while keeping the full model id intact and pointingbase_urlat the gateway.base_urldefaults tohttps://api.orcarouter.ai/v1andORCAROUTER_API_KEYis auto-resolved, so users can just passprovider="orcarouter/auto".List of files changed and why
crawl4ai/config.py— registerorcarouterinPROVIDER_MODELS_PREFIXES(auto-resolvesORCAROUTER_API_KEY), addORCAROUTER_BASE_URLandorcarouter_litellm_params()helper.crawl4ai/utils.py— routeorcarouter/providers inperform_completion_with_backoff,aperform_completion_with_backoff,extract_blocks_batch, andget_text_embeddings.crawl4ai/async_configs.py—LLMConfigdefaultsbase_urlto the gateway fororcarouter/providers.crawl4ai/cli.py— routeorcarouter/in the-qstreaming path and mention it in the provider hint.docs/md_v2/extraction/llm-strategies.md— document OrcaRouter usage.README.md— add an OrcaRouter comment to the LLM extraction example.tests/test_orcarouter_provider.py— new unit tests for routing + auto key/base_url resolution.How Has This Been Tested?
tests/test_orcarouter_provider.py, 9 tests) pass.perform_completion_with_backoffwithprovider="orcarouter/auto"): HTTP 200, returnsORCA-LIVE-OK.get_text_embeddingsthrough OrcaRouter: HTTP 200.python -m py_compileon all modified modules passes.Checklist:
Disclosure: I'm an engineer on the OrcaRouter team.