Repository navigation
feat(llm): add Cheaper Inference provider - #303
Closed
aiapienthusiast wants to merge 1 commit into
Closed
aiapienthusiast wants to merge 1 commit into
aiapienthusiast wants to merge 1 commit into
Conversation
Owner
|
Thanks for the spam. I'm going to close this one. Cheaper Inference is an OpenAI-compatible gateway, so anyone who wants to use it can already point the OpenAI provider at its endpoint without any code change. I'd rather not add dedicated providers, README placement and allowlist entries for resellers, given the maintenance cost across all the file paths and locales. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds Cheaper Inference as a cloud provider for the web UI and the CLI.
It follows the provider pattern of #121 (NVIDIA NIM) and #247.
Cheaper Inference is an OpenAI-compatible LLM gateway.
One API key gives access to models from several labs.
Usage:
python translate.py -i book.txt --provider cheaperinference \ --cheaperinference_api_key YOUR_KEY -m gpt-5.4-mini -tl FrenchIn the web UI, select "Cheaper Inference" and enter the key.
You can also set
CHEAPERINFERENCE_API_KEYin.env.Details
Backend
CheaperInferenceProvider(src/core/llm/providers/cheaperinference.py), based onOpenAICompatibleProvider.llm_client, config:CHEAPERINFERENCE_API_KEY,CHEAPERINFERENCE_MODEL,CHEAPERINFERENCE_API_ENDPOINT..env, like DeepSeek. The provider does not read the request endpoint./api/modelslists only rows withtype: text. Image and video models are skipped._KEY_PROVIDERS(resume) and in the endpoint allowlist.Frontend
CLI and docs
--provider cheaperinferenceand--cheaperinference_api_key..env.example.Tests
tests/unit/test_cheaperinference_provider.py.pytest tests/unit tests/test_frontend_i18n.py: no new failures againstmain.isort --profile black --check-onlyis clean on the new files.Live check
gpt-5.4-minireturned a reply throughcreate_llm_provider("cheaperinference").translate.py --provider cheaperinferencetranslated a one-line TXT file./api/modelsreturned the text model list.