Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/advanced/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,14 +214,14 @@ See [cost tracking](./cost-tracking.md).

## LLM provider and routing

See [LLM providers](./llm-providers.md) and the [LiteLLM backend](./litellm-backend.md). With nothing here or in a provider key set, a non-interactive run picks a provider itself; see [When nothing is configured](./llm-providers.md#when-nothing-is-configured). *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).*
See [LLM providers](./llm-providers.md) and the [LiteLLM backend](./litellm-backend.md). *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* With nothing here or in a provider key set, a non-interactive run picks a provider itself; see [When nothing is configured](./llm-providers.md#when-nothing-is-configured). *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* When nothing names a provider and none is found, forge stops with `copilot_no_llm_provider`.

| Variable | Effect |
|---|---|
| `FLUID_LLM_PROVIDER` | Default provider. |
| `FLUID_LLM_MODEL` | Default model. |
| `FLUID_LLM_API_KEY` | API key used when no provider-specific key is set. |
| `FLUID_LLM_ENDPOINT` | Overrides the model endpoint. The routing model's calls use it too, unless `FLUID_LLM_ROUTING_ENDPOINT` names another. *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* |
| `FLUID_LLM_API_KEY` | API key for the selected provider. It wins over the provider's own key variable, such as `OPENAI_API_KEY`. It does not choose the provider: name one with `--llm-provider`, `FLUID_LLM_PROVIDER` or the key of exactly one provider. *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* With none named, forge stops with `copilot_no_llm_provider`; before #852 it sent the key to `gemini`. |
| `FLUID_LLM_ENDPOINT` | Overrides the model endpoint. *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* The routing model's calls use it too, unless `FLUID_LLM_ROUTING_ENDPOINT` names another. *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* For Ollama, forge's model pick and model check ask it too, and each `--tiered` stage calls it. |
| `FLUID_LLM_TIMEOUT_SECONDS` | Per-call timeout. Default `120`. |
| `FLUID_LLM_STREAMING` | Default on; `0`, `false`, `no` or `off` disables streaming. |
| `FLUID_LLM_ROUTING_ENDPOINT`, `FLUID_LLM_ROUTING_MODEL` | Optional routing endpoint and model. |
Expand Down Expand Up @@ -297,7 +297,7 @@ Read by `fluid mcp output-port`; see [MCP](./mcp.md).
| `FLUID_STORE_VECTOR_BACKING` | Backing store of the `vector` backend: `file` (default), `sqlite` or `postgres`. |
| `FLUID_SECRETS_INMEMORY` | `1` keeps `fluid secrets` values in memory instead of the OS keychain. Tests only. |
| `FLUID_ENCRYPTION_KEY`, `FLUID_ENCRYPTION_PASSPHRASE` | Fernet key, or a passphrase it is derived from, for the encrypted credential store. |
| `FLUID_ALLOW_PLAINTEXT_AI_SECRETS` | Legacy plaintext fallback for AI provider secrets. |
| `FLUID_ALLOW_PLAINTEXT_AI_SECRETS` | Legacy plaintext fallback for AI provider secrets. *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* It also lets `fluid ai setup` save a `user:password@` in an address, such as an Ollama host, to `~/.fluid/ai_config.json`; without it the address is saved without them. |
| `FLUID_ALLOW_PLAINTEXT_SOURCE_SECRETS` | Legacy plaintext secrets in `sources.yaml`; the file must also be mode 600. |

## Marketplace
Expand Down
9 changes: 9 additions & 0 deletions docs/advanced/error-codes.md
Original file line number Diff line number Diff line change
Expand Up @@ -378,6 +378,15 @@ The `fluid generate iac` and `fluid apply` runs print the refusal and exit befor
- For Ollama, start the local server and pull the requested model
- Pass --llm-model with a known-available model

### copilot_no_llm_provider

*On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* `ERR_COPILOT_NO_LLM_PROVIDER`. The documentation link lands on [Production troubleshooting](./production-troubleshooting.md), where [No LLM provider is selected](./production-troubleshooting.md#no-llm-provider-is-selected) explains each message.

- Pass --llm-provider, for example anthropic, openai, gemini or ollama
- Or set FLUID_LLM_PROVIDER, or the API key of exactly one provider
- Or run 'fluid ai setup' to choose and save a provider
- Run 'fluid doctor' to see what forge would select

### model_not_found

`ERR_MODEL_NOT_FOUND`. The documentation link lands on [Production troubleshooting](./production-troubleshooting.md).
Expand Down
2 changes: 1 addition & 1 deletion docs/advanced/litellm-backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ If you point LiteLLM at a model the catalog doesn't know, the run-start banner s
## Caveats

- Tool-use behaviour matches LiteLLM's wrapper. Provider failures surface as the typed errors described in [Typed Errors](./typed-errors.md) (`RateLimitError`, `ContextOverflowError` and the others).
- Forge's Ollama calls go to `OLLAMA_HOST` whatever its host; only the probe for a running Ollama, model discovery and the interactive Ollama setup stay on `localhost`. See [The Ollama endpoint](./llm-providers.md#the-ollama-endpoint). *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).*
- *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* Forge's Ollama calls go to `OLLAMA_HOST` whatever its host. See [The Ollama endpoint](./llm-providers.md#the-ollama-endpoint). *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* The probe for a running Ollama, model discovery and the interactive Ollama setup follow it too; before #852 they stayed on `localhost`.

## See also

Expand Down
29 changes: 23 additions & 6 deletions docs/advanced/llm-providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ When it picks an agent, forge logs a warning that names it:
No LLM provider is configured; using claude-code, the first coding agent found on PATH. Override with --llm-provider.
```

With neither, the run falls back to `gemini`, which needs a Gemini key and the `litellm` extra. Through 0.28.0 a run with no Ollama running always fell back to `gemini`, even with a signed-in Claude Code on `PATH`.
With neither, the run stops with `copilot_no_llm_provider` and says why; see [No LLM provider is selected](./production-troubleshooting.md#no-llm-provider-is-selected). *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* Before #852 it fell back to `gemini`, which needs a Gemini key and the `litellm` extra. Through 0.28.0 it did so whenever no Ollama was running, even with a signed-in Claude Code on `PATH`.

Any of these configures an LLM, and then no agent is picked:

Expand All @@ -70,12 +70,14 @@ Any of these configures an LLM, and then no agent is picked:
- `FLUID_LLM_API_KEY`, `FLUID_LLM_ENDPOINT`, `FLUID_LLM_MODEL`, `FLUID_LLM_ROUTING_ENDPOINT`, `FLUID_LLM_ROUTING_MODEL` or `OLLAMA_HOST`.
- A provider key: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, `GOOGLE_API_KEY` or `GITHUB_API_KEY`.
- An OpenAI, Anthropic or Gemini key in the OS keyring.
- A provider saved with `fluid ai setup`. A non-interactive run does not use the saved provider itself, so name it with `--llm-provider` or `FLUID_LLM_PROVIDER`.
- A provider saved with `fluid ai setup`. A run with `--non-interactive`, which `--agent` sets, does not use the saved provider itself, so name it with `--llm-provider` or `FLUID_LLM_PROVIDER`.
- The agent loop: `--agent-loop`, or `FLUID_COPILOT_AGENT_LOOP` set to `1`, `true`, `yes` or `on`. The loop needs tool use, which these agents do not offer.

*On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* Only some of these name a provider: `--llm-provider`, `FLUID_LLM_PROVIDER` and `FLUID_FORGE_AGENT`; the key of exactly one of OpenAI, Anthropic, Gemini (`GEMINI_API_KEY` or `GOOGLE_API_KEY`) and Ollama (`OLLAMA_HOST`); a keyring key for exactly one provider; and a provider saved with `fluid ai setup`, in a run without `--non-interactive`. The others, `GITHUB_API_KEY` among them, only stop forge from picking an agent: an Ollama that answers at `OLLAMA_HOST` or `localhost:11434` is still used. When nothing names a provider and no Ollama answers, forge stops with `copilot_no_llm_provider`. So pass `--llm-provider` together with `--llm-endpoint`, `--llm-model` or `FLUID_LLM_API_KEY`.

The pick works on the base install, because the agent CLIs need no `litellm` extra. A running local Ollama still comes first there, so that run asks for the extra instead of sending the prompt to an agent.

An interactive run does not pick an agent: `fluid forge` in a terminal offers Claude Code in its inline setup instead. With nothing configured, [`fluid doctor`](../cli/doctor.md) run without a terminal reports the provider this pick chooses. When the pick finds nothing, its Provider row says `Not selected`.
An interactive run does not pick an agent: `fluid forge` in a terminal offers Claude Code in its inline setup instead. With nothing configured, [`fluid doctor`](../cli/doctor.md) run without a terminal reports the provider this pick chooses. When the pick finds nothing, its Provider row says `Not selected`. *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* `fluid doctor --verbose` then shows, in its Message row, the reason forge would stop with.

## Claude Code

Expand Down Expand Up @@ -129,12 +131,27 @@ Forge reads `OLLAMA_HOST` the way Ollama's own Python client does ([`_parse_host
| `https://gpu-box` | `https://gpu-box:443` |
| `https://gpu-box/ollama` | `https://gpu-box:443/ollama` |

The host can be another machine. Model discovery still looks only at a local Ollama: with no model named, forge takes the largest model pulled on `localhost`, else `gemma4:latest`. With a remote `OLLAMA_HOST`, name the model with `--llm-model` or `FLUID_LLM_MODEL`. Forge's local-only checks print `OLLAMA_HOST points to a non-localhost address (...), ignoring.` for such a host. The call still goes to `OLLAMA_HOST`.

A stated endpoint is used for the routing model's calls too, unless `--llm-routing-endpoint` or `FLUID_LLM_ROUTING_ENDPOINT` names another. A project's `.env` cannot set `OLLAMA_HOST`; see [Project dotenv trust](./project-dotenv-trust.md).
The host can be another machine. A stated endpoint is used for the routing model's calls too, unless `--llm-routing-endpoint` or `FLUID_LLM_ROUTING_ENDPOINT` names another. A project's `.env` cannot set `OLLAMA_HOST`; see [Project dotenv trust](./project-dotenv-trust.md).

Through 0.28.0 these calls went to a `litellm://ollama/<model>` label unless an endpoint was stated, so they never reached the daemon.

### One address for every Ollama read

*On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).*

Every Ollama read takes any host, as Ollama's own client does:

- With no `--llm-model`, forge asks the daemon the call goes to, and uses its largest model, else `gemma4:latest`.
- The model check that `--require-llm`, `--tiered` and `FLUID_LLM_MODEL_PREFLIGHT` turn on asks that daemon too. The model check of `fluid ai test` asks its `--endpoint`, else `OLLAMA_HOST`, else `localhost:11434`.
- Each `--tiered` stage calls a stated endpoint, as the routing model does.
- The probe that finds a running Ollama, and `fluid ai setup`, read `OLLAMA_HOST`, else `localhost:11434`. A stated endpoint does not reach them.

Forge reads `--llm-endpoint` and `FLUID_LLM_ENDPOINT` the same way as `OLLAMA_HOST`: `--llm-endpoint gpu-box` means `http://gpu-box:11434`, and the scheme may be in upper or lower case. `fluid ai test --endpoint` takes only an absolute `http` or `https` URL. A stated endpoint does not choose Ollama, so pass `--llm-provider ollama` with it. `OLLAMA_HOST` on its own does choose it.

An address may carry `user:password@`, for an Ollama behind a proxy that asks for basic auth, and forge sends it as basic auth, to the call and to the model lookups. `fluid ai test --endpoint` refuses one. `fluid doctor` and `fluid ai setup` show it as `***:***@`. `fluid ai setup` saves the address to `~/.fluid/ai_config.json` without it, as it does for API keys, unless `FLUID_ALLOW_PLAINTEXT_AI_SECRETS` is set, and logs a warning to keep the credentials in the environment. An `OLLAMA_HOST` in the environment wins over the saved address, so keep them there.

Before #852, forge's default-model pick, the probe and `fluid ai setup` looked only at `localhost`. They ignored a non-local `OLLAMA_HOST` with the warning `OLLAMA_HOST points to a non-localhost address (...), ignoring.`, so the model could come from one daemon while the call went to another. The model checks read `OLLAMA_HOST`, never a stated endpoint, and a `--tiered` stage called the provider's default address. A stated endpoint was used exactly as given, so its `user:password@` reached the call but no lookup, and `OLLAMA_HOST` lost its `user:password@`.

## Tiered mode

`--tiered` chooses different models within the same provider, never across providers. A typical layout is:
Expand Down
6 changes: 4 additions & 2 deletions docs/advanced/network-safety.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ This page describes the user-visible behaviour: which flags opt in, which enviro
| `fluid odps import <url>` | Followed `http(s)` `contractId` references | Local only; remote references are rejected | `--allow-remote` |
| `BitolOdpsProvider().import_contract(...)` (Python) | `allow_remote=True` | `allow_remote=False` | `allow_remote=True` (keyword) |
| `ContractResolver(...)` (Python) | `allow_remote=True` | `allow_remote=False` | `allow_remote=True` (keyword) |
| Ollama endpoint | Configurable | Localhost only | None, by design. *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* Calls now go to `OLLAMA_HOST` whatever the host; see [Ollama and `OLLAMA_HOST`](#ollama-and-ollama-host). |
| Ollama endpoint | Configurable | Localhost only | None, by design. *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* Calls now go to `OLLAMA_HOST` whatever the host. *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* So do the probe, model discovery and the interactive setup; see [Ollama and `OLLAMA_HOST`](#ollama-and-ollama-host). |
| HTTP redirects on a safe-HTTP client | Followed | Not followed | `follow_redirects=True` (per client) |

The CLI flags `--no-remote` and `--seed-no-remote` remain as hidden aliases that do nothing, so older scripts keep working.
Expand Down Expand Up @@ -70,7 +70,9 @@ As of 0.18.1, nothing in the CLI reads `FLUID_SAFE_MODE` or acts on the global `

## Ollama and `OLLAMA_HOST`

*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* Forge's Ollama calls go to `--llm-endpoint` or `FLUID_LLM_ENDPOINT`, else to `OLLAMA_HOST` whatever its host, else to `http://localhost:11434`; see [The Ollama endpoint](./llm-providers.md#the-ollama-endpoint). Three things still look only at `localhost`, and ignore a non-local `OLLAMA_HOST` with a warning: the probe that finds a running Ollama, model discovery, and the interactive Ollama setup. A project's `.env` cannot set `OLLAMA_HOST` ([Project dotenv trust](./project-dotenv-trust.md)).
*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* Forge's Ollama calls go to `--llm-endpoint` or `FLUID_LLM_ENDPOINT`, else to `OLLAMA_HOST` whatever its host, else to `http://localhost:11434`; see [The Ollama endpoint](./llm-providers.md#the-ollama-endpoint). A project's `.env` cannot set `OLLAMA_HOST` ([Project dotenv trust](./project-dotenv-trust.md)).

*On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* No Ollama read is held to `localhost` any more. The probe that finds a running Ollama, model discovery and the interactive Ollama setup follow `OLLAMA_HOST` too, and `fluid ai test --endpoint` takes an Ollama address on any host, over `http` or `https`; it still refuses a `user:password@` in the URL. Before #852 those three looked only at `localhost` and ignored a non-local `OLLAMA_HOST` with a warning, and `fluid ai test` refused an Ollama endpoint that was not `http` on `localhost`. [One address for every Ollama read](./llm-providers.md#one-address-for-every-ollama-read) also covers a `user:password@` in forge's own Ollama address.

Through 0.28.0 the calls went to a `litellm://ollama/<model>` label unless an endpoint was stated, so no `OLLAMA_HOST`, local or remote, reached a daemon.

Expand Down
Loading
Loading