Skip to content

feat(llm): add Cheaper Inference provider - #303

Closed
aiapienthusiast wants to merge 1 commit into
hydropix:mainfrom
aiapienthusiast:cheaperinference-provider
Closed

aiapienthusiast wants to merge 1 commit into
hydropix:mainfrom
aiapienthusiast:cheaperinference-provider

Conversation

@aiapienthusiast

Copy link
Copy Markdown

This PR adds Cheaper Inference as a cloud provider for the web UI and the CLI.
It follows the provider pattern of #121 (NVIDIA NIM) and #247.

Cheaper Inference is an OpenAI-compatible LLM gateway.
One API key gives access to models from several labs.

Usage:

python translate.py -i book.txt --provider cheaperinference \
    --cheaperinference_api_key YOUR_KEY -m gpt-5.4-mini -tl French

In the web UI, select "Cheaper Inference" and enter the key.
You can also set CHEAPERINFERENCE_API_KEY in .env.

Details

Backend

  • New CheaperInferenceProvider (src/core/llm/providers/cheaperinference.py), based on OpenAICompatibleProvider.
  • Factory, llm_client, config: CHEAPERINFERENCE_API_KEY, CHEAPERINFERENCE_MODEL, CHEAPERINFERENCE_API_ENDPOINT.
  • The endpoint comes from .env, like DeepSeek. The provider does not read the request endpoint.
  • /api/models lists only rows with type: text. Image and video models are skipped.
  • The per-job key goes through every path: EPUB, TXT/SRT, DOCX, PDF, refine and auto-prep.
  • The provider is in _KEY_PROVIDERS (resume) and in the endpoint allowlist.

Frontend

  • Provider entry, logo, key field, model loading and key status.
  • New strings are in all 7 locales.

CLI and docs

  • --provider cheaperinference and --cheaperinference_api_key.
  • README (logo, provider table, CLI example) and .env.example.

Tests

  • New tests/unit/test_cheaperinference_provider.py.
  • The key lists in the existing key tests include the new key.
  • pytest tests/unit tests/test_frontend_i18n.py: no new failures against main.
  • isort --profile black --check-only is clean on the new files.

Live check

  • gpt-5.4-mini returned a reply through create_llm_provider("cheaperinference").
  • translate.py --provider cheaperinference translated a one-line TXT file.
  • /api/models returned the text model list.

@hydropix

Copy link
Copy Markdown
Owner

Thanks for the spam. I'm going to close this one. Cheaper Inference is an OpenAI-compatible gateway, so anyone who wants to use it can already point the OpenAI provider at its endpoint without any code change. I'd rather not add dedicated providers, README placement and allowlist entries for resellers, given the maintenance cost across all the file paths and locales.

@hydropix hydropix closed this Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants