Skip to content

security(ai): FLUID_LLM_API_KEY only for a named provider; saved keys stay off project hosts - #853

Merged
fas89 merged 8 commits into
mainfrom
fix/llm-api-key-scope
Nov 12, 2026
Merged

fas89 merged 8 commits into
mainfrom
fix/llm-api-key-scope

Conversation

@fas89

@fas89 fas89 commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Two fixes for API keys going to the wrong host. Both came out of the security review of #852 and were decided by the founder on 2026-11-12. There is one commit for each.

1. FLUID_LLM_API_KEY goes only to a provider you name.

The key does not say which vendor it is for. Even so, forge sent it to whatever provider it picked, including one it only inferred:

  • a running Ollama, or the one OLLAMA_HOST names, as Authorization: Bearer (in cleartext when the host uses http);
  • the provider of the one key saved in the keyring;
  • the provider of the one key variable set.

So an OpenAI key could reach a local or remote Ollama, or Anthropic.

The key is now used only when --llm-provider or FLUID_LLM_PROVIDER names the provider the run uses. A provider forge picks itself gets its own key variable, then its keyring key, and a warning says the generic key was not used.

  • Interactive fluid forge and fluid init: setup replaces args.llm_provider, so each first records what the operator named (remember_named_llm_provider). Setup's own pick therefore counts as picked, not named.
  • fluid ai test uses the key only with --llm-provider.
  • fluid doctor counts it only with FLUID_LLM_PROVIDER.
  • A named Ollama still gets the key (ollama.com, a token proxy).

Because forge can pick OpenAI, Anthropic, Gemini and GitHub without a name, a project .env can no longer pair their endpoints with FLUID_LLM_API_KEY (credentials/protected_env.py). On such a run the operator's keyring key, not the project's key, would go to the project's host. Endpoints of providers forge reaches only by name (groq, deepseek, Azure, Bedrock, Vertex, and the unlisted litellm names) still pair with it.

2. Interactive setup keeps your saved key off a project's host.

A project .env may point a provider at its own host (OPENAI_BASE_URL …) next to a key of its own. That pairing is allowed because the project's key is what goes there.

Before this change:

  • Interactive setup loaded the provider that fluid ai setup saved and copied its keyring key over the project's key (set_session_env). For Gemini it copied the key into GOOGLE_API_KEY, which forge reads first. Your key went to the project's host.
  • fluid ai setup checked a typed or imported key with one litellm call, which reads that endpoint from the environment.

Now, while a project endpoint of that provider is in effect:

  • setup leaves the variable alone and asks for no key;
  • it skips a key it would import;
  • _validate_api_key sends nothing;
  • a message says why.

This reuses dotenv_store.project_endpoints_in_effect, the check the Snowflake readers already use.

Command Center is unaffected. fluid_cli_service.forge_from_intent always passes --llm-provider plus the provider's own key variable, and FLUID_LLM_API_KEY appears nowhere in CC (checked at its main, 695f4f9a).

No issue: these are follow-ups from the #852 security review.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would break existing functionality)
    • FLUID_LLM_API_KEY no longer reaches a provider forge inferred. Name it with --llm-provider <name> or FLUID_LLM_PROVIDER=<name>.
    • A project .env OpenAI, Anthropic, Gemini or GitHub endpoint needs that provider's own key variable.
  • Documentation update
  • Refactor / code cleanup
  • New provider or provider enhancement

Documentation

  • Docs PR linked:

    • Docs PR:
  • No docs needed — this change does not affect user-facing behaviour

    • Justification:
  • Docs TODO — I know docs are needed and will create a PR in forge_docs before this is merged

    Pages in forge_docs:

    • docs/advanced/environment-variables.md: FLUID_LLM_API_KEY applies only to a named provider.
    • docs/advanced/llm-providers.md: the key resolution order.
    • docs/advanced/project-dotenv-trust.md: the pairing table (OpenAI, Anthropic, Gemini and GitHub endpoints need the provider's own key), and setup never copies a saved key over a project's.
    • docs/advanced/production-troubleshooting.md: the "FLUID_LLM_API_KEY is set but not used" warning, and setup's "Not asking for a … key".
    • docs/cli/ai.md: fluid ai setup inside a project with its own LLM endpoint, and fluid ai test using the generic key only with --llm-provider.
    • docs/cli/doctor.md: readiness counts the generic key only with FLUID_LLM_PROVIDER.
    • docs/upgrading.md: the behaviour change, and how to name the provider.

    This repo's legacy docs/ENVIRONMENT_VARIABLES.md is now stale: the OpenAI, Gemini and GitHub Models rows of its pairing table, and the resolve order on line 194. I left it untouched, because docs live in forge_docs.

Checklist

  • I have read the Contributing Guide
  • My code follows the project's coding standards
  • I have added tests that prove my fix/feature works
  • All new and existing tests pass (pytest). Locally, only the shared venv's baseline fails, and it fails the same way on main; see Testing.
  • I have run ruff check and black with no errors
  • New maintained Python files include license headers (python scripts/add_license_headers.py; examples/** exempt)

Testing

New tests.

  • tests/test_llm_api_key_scope.py: 22 tests, hermetic (the keyring, the Ollama probe, the saved fluid ai setup config and HOME are patched).
    • On main (3900c45), 17 of them fail on their assertions. The key reaches Ollama, the keyring's provider and the key variable's provider.
    • In one of them a loopback stand-in for Ollama records the key in POST /api/generate.
    • The 5 named-provider guards pass on both.
  • tests/test_dotenv_pairing_matrix.py:
    • forge's key resolution is driven as a named provider and, for providers forge can pick, as a picked one;
    • a new refusal test covers the four groups;
    • 7 new tests cover interactive setup and key checks. Without the second commit, 6 of them fail; the control passes. The saved key reaches the project's listener for OpenAI (/chat/completions), Anthropic (/v1/messages) and Gemini (generateContent); a typed key reaches it for OpenAI and groq; and setup asks three times for a key it cannot check.

Full suite with CI's flags (-n auto --dist loadscope --ignore=tests/perf), on head 266ca78: 34,550 passed, 8 failed, 2 errors. The failures match main in the same venv: the same 8 tests fail there, and the 2 errors are No module named 'bashlex'.

CodeQL fix (bcc5eb5). CodeQL raised 3 py/clear-text-logging-sensitive-data alerts (#345-#347) on the two warnings this branch adds:

  • resolve_llm_config's note was unpacked from the same tuple as the key, returned by a call whose name reads as a password source. It is now built by _unnamed_provider_note from the provider names and the named flag only.
  • set_session_env's warning logged the provider string, which setup can get by classifying a typed key. It now logs the endpoint group, a constant from _PROVIDER_ENDPOINT_SERVICES picked by comparison, and the endpoint variable names.

After the fix the lint gates and 2,415 related tests pass; the only failures are the known main ones.

Merge of main. main (#851, the Agent Skills pack) was merged in as 36ac284. The only conflict was the CHANGELOG: it keeps main's [Unreleased] order and adds this branch's Security section after it. On the merged tree I re-ran all the lint gates, the related suites and #851's tests: 2,415 passed, and the only failures are the known main ones. Perf budgets: 41 passed and 7 xfailed, with the same module counts as main.

Lint:

  • ruff, the dead-code sweep and black 24.10.0;
  • detect-secrets 1.5.0 over the changed files;
  • license headers;
  • lint-imports (6 contracts kept);
  • the product guardrail.

Live, through the CLI. fluid forge data-model from-intent intent.yaml --llm-model llama3.2, with OLLAMA_HOST at a loopback stub that records headers and FLUID_LLM_API_KEY set. The run used env -i, a temp HOME, the null keyring, and other egress proxied to a dead port.

Run Requests to the stub Key seen Warning
This branch 2 0 printed
main 2 1 —
This branch with --llm-provider ollama 2 1 (a named Ollama still gets it) —

Reviews.

  • Security review (Opus) of the full diff, and again of the amended setup change: no findings.
  • Defect scan (diff mode), two Low findings:
    • the stale in-repo doc above;
    • setup asking three times for a key it could not check. Fixed in the second commit.

Screenshots / Output

== final head, --llm-model only (provider picked from OLLAMA_HOST)
  requests=2 key_seen=0
FLUID_LLM_API_KEY is set but not used: it goes only to a provider named with --llm-provider or FLUID_LLM_PROVIDER, and forge picked ollama itself. Pass --llm-provider ollama to use the key with it.
== main 3900c45f, --llm-model only
  requests=2 key_seen=1
== final head, --llm-provider ollama (named)
  requests=2 key_seen=1

Older problems, not in this PR:

  • test_ai_setup.py::TestModelCatalogIntegrity::test_class_defaults_match_catalog_flagship fails when it runs after test_dotenv_pairing_matrix.py in one process, on main too.
  • Setup's env-key offer (_offer_import_env_api_key) can save a key that came from a project .env into the operator's keyring.
  • Interactive fluid forge ignores --llm-provider / FLUID_LLM_PROVIDER when setup finds a provider.
  • Interactive setup infers github from GITHUB_API_KEY.
  • fluid ai test ignores FLUID_LLM_PROVIDER.

FLUID_LLM_API_KEY does not say which vendor it is for, yet forge sent it to
whatever provider it picked, including one it only inferred: a running Ollama
or the one OLLAMA_HOST names (as a bearer token, in cleartext over http), the
provider of the one keyring key, or the provider of the one key variable set.
So an OpenAI key could reach a local or remote Ollama, or Anthropic.

The key is now used only when --llm-provider or FLUID_LLM_PROVIDER names the
provider the run uses (founder decision, 2026-11-12). A provider forge picks
itself gets its own key variable, then its keyring key, and a warning says the
generic key was not used. Interactive fluid forge and fluid init record what
the operator named before their setup replaces args.llm_provider, so setup's
pick counts as picked. fluid ai test reads the key only with --llm-provider,
fluid doctor only with FLUID_LLM_PROVIDER.

A project .env can no longer pair an OpenAI, Anthropic, Gemini or GitHub
endpoint with FLUID_LLM_API_KEY: forge can pick those providers without a name,
and then the operator's keyring key, not the project's key, would go out.
…host

A project .env may point a provider at its own host (OPENAI_BASE_URL,
ANTHROPIC_BASE_URL, GEMINI_API_BASE, GITHUB_API_BASE) next to a key of its
own; credentials/protected_env pairs it because that key is what the readers
send. Interactive fluid forge and fluid init then loaded the provider that
fluid ai setup saved and copied its keyring key into the provider's key
variable with set_session_env, over the project's key (for Gemini into
GOOGLE_API_KEY, which forge reads before GEMINI_API_KEY). The operator's key
went to the project's host.

set_session_env now leaves the variable alone while a project endpoint of
that provider is in effect (dotenv_store.project_endpoints_in_effect, as the
Snowflake readers already do), so the project's own key goes there, and logs
that the operator's key was not used.

fluid ai setup checked a typed or imported key with one litellm call, which
reads the endpoint from the environment, so the check sent that key to the
project's host too. While such an endpoint is in effect, setup now asks for
no key, skips a key it would import, and _validate_api_key sends nothing.
For a provider without a mapped group, any LLM endpoint a project file set
counts.
@github-actions github-actions Bot added security Security-related changes or reports cli Changes to the CLI surface or implementation tests Test coverage or test infrastructure changes labels Nov 12, 2026
Comment thread fluid_build/llm/providers.py Fixed
Comment thread tests/test_llm_api_key_scope.py Fixed
CHANGELOG: keep main's [Unreleased] order (Added, Changed, Fixed) and add this branch's Security section after it.
Comment thread fluid_build/cli/ai_setup.py Fixed
Comment thread fluid_build/cli/ai_setup.py Fixed
Comment thread fluid_build/llm/providers.py Fixed
CodeQL (py/clear-text-logging-sensitive-data) flagged the two warnings this
branch adds:

- resolve_llm_config logged a note unpacked from the same tuple as the API
  key, returned by _api_key_for_run, a call whose name reads as a password
  source. The note now comes from _unnamed_provider_note, from the provider
  names and the named flag alone; the key comes from _resolve_api_key on
  its own.
- set_session_env's warning logged the provider string, which setup can get
  by classifying a typed key (_classify_key_shape -> detect_provider_from_
  api_key). The notice now names the endpoint group from
  _PROVIDER_ENDPOINT_SERVICES, picked by comparison so it is the table's own
  constant, and the endpoint variables; the provider string is never logged.
…ope tests

Addresses the two code-quality review threads on #853:

- the copilot_missing_llm_api_key suggestion this branch added was two
  adjacent string literals in a list (implicit concatenation, read as a
  possibly missing comma); it is one literal now;
- tests/test_llm_api_key_scope.py imported fluid_build.llm.providers both
  as a module and with from-imports; it uses the module import only, and the
  new interactive-setup matrix test does the same.
#854 walks _provider_name_from and the saved fluid ai setup choice in check_llm_readiness. There FLUID_LLM_API_KEY counts only for a named provider (_provider_is_named(None, env, ...)), never for the saved choice. resolve_llm_config keeps #854's 'not saved' auto-pick condition and this branch's unnamed-provider note.
Runs without a TTY use the provider fluid ai setup saved (#854). That provider is picked, not named, so it gets its own keyring key and the generic key is unused; without a key of its own it is not ready and resolve_llm_config raises copilot_missing_llm_api_key. Both tests fail on main 3b7e8c1.
…airing matrix

The file imported the module as llm_providers in two places and with 'from ... import LlmConfig' in _run_codex (github-code-quality on #853). _run_codex now uses llm_providers.LlmConfig as well.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli Changes to the CLI surface or implementation security Security-related changes or reports tests Test coverage or test infrastructure changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants