Skip to content

feat(llm): add OrcaRouter as a named LLM provider - #2180

Open
XiaoHuo888-hue wants to merge 1 commit into
unclecode:developfrom
XiaoHuo888-hue:add-orcarouter-provider
Open

feat(llm): add OrcaRouter as a named LLM provider#2180
XiaoHuo888-hue wants to merge 1 commit into
unclecode:developfrom
XiaoHuo888-hue:add-orcarouter-provider

Conversation

@XiaoHuo888-hue

Copy link
Copy Markdown

Summary

Add OrcaRouter as a named LLM provider for LLM-based extraction. OrcaRouter is an OpenAI-compatible gateway at https://api.orcarouter.ai/v1; model ids use the orcarouter/<model> prefix (e.g. orcarouter/auto, orcarouter/free). It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.

The pinned unclecode-litellm build has no native orcarouter/ provider prefix, so Crawl4AI routes orcarouter/<model> through the openai provider while keeping the full model id intact and pointing base_url at the gateway.

llm_config = LLMConfig(
    provider="orcarouter/auto",
    api_token=os.getenv("ORCAROUTER_API_KEY"),
)

base_url defaults to https://api.orcarouter.ai/v1 and ORCAROUTER_API_KEY is auto-resolved, so users can just pass provider="orcarouter/auto".

List of files changed and why

  • crawl4ai/config.py — register orcarouter in PROVIDER_MODELS_PREFIXES (auto-resolves ORCAROUTER_API_KEY), add ORCAROUTER_BASE_URL and orcarouter_litellm_params() helper.
  • crawl4ai/utils.py — route orcarouter/ providers in perform_completion_with_backoff, aperform_completion_with_backoff, extract_blocks_batch, and get_text_embeddings.
  • crawl4ai/async_configs.pyLLMConfig defaults base_url to the gateway for orcarouter/ providers.
  • crawl4ai/cli.py — route orcarouter/ in the -q streaming path and mention it in the provider hint.
  • docs/md_v2/extraction/llm-strategies.md — document OrcaRouter usage.
  • README.md — add an OrcaRouter comment to the LLM extraction example.
  • tests/test_orcarouter_provider.py — new unit tests for routing + auto key/base_url resolution.

How Has This Been Tested?

  • New unit tests (tests/test_orcarouter_provider.py, 9 tests) pass.
  • Live-tested against the OrcaRouter gateway through the actual Crawl4AI completion path (perform_completion_with_backoff with provider="orcarouter/auto"): HTTP 200, returns ORCA-LIVE-OK.
  • Live-tested embeddings via get_text_embeddings through OrcaRouter: HTTP 200.
  • python -m py_compile on all modified modules passes.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added/updated unit tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Disclosure: I'm an engineer on the OrcaRouter team.

Add OrcaRouter (https://www.orcarouter.ai) as a named provider for LLM-based
extraction. OrcaRouter is an OpenAI-compatible gateway at
https://api.orcarouter.ai/v1; model ids use the orcarouter/<model> prefix
(e.g. orcarouter/auto, orcarouter/free).

The pinned unclecode-litellm build has no native orcarouter/ provider prefix,
so route orcarouter/<model> through the openai provider while keeping the full
model id intact and pointing base_url at the gateway:

- config.py: register orcarouter in PROVIDER_MODELS_PREFIXES (auto-resolves
  ORCAROUTER_API_KEY), add ORCAROUTER_BASE_URL and orcarouter_litellm_params()
- utils.py: route orcarouter/ providers in perform_completion_with_backoff,
  aperform_completion_with_backoff, extract_blocks_batch, get_text_embeddings
- async_configs.py: LLMConfig defaults base_url to the gateway for orcarouter/
- cli.py: route orcarouter/ in the -q streaming path
- docs: document OrcaRouter usage in llm-strategies.md and README
- tests: test_orcarouter_provider.py covers routing + auto key/base_url

Verified locally: 9 new unit tests pass; chat completion and embeddings
live-tested against the OrcaRouter gateway (HTTP 200).

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: jinhao.song <jinhao.song@myflashcloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant