Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/advanced/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,14 +214,14 @@ See [cost tracking](./cost-tracking.md).

## LLM provider and routing

See [LLM providers](./llm-providers.md) and the [LiteLLM backend](./litellm-backend.md).
See [LLM providers](./llm-providers.md) and the [LiteLLM backend](./litellm-backend.md). With nothing here or in a provider key set, a non-interactive run picks a provider itself; see [When nothing is configured](./llm-providers.md#when-nothing-is-configured). *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).*

| Variable | Effect |
|---|---|
| `FLUID_LLM_PROVIDER` | Default provider. |
| `FLUID_LLM_MODEL` | Default model. |
| `FLUID_LLM_API_KEY` | API key used when no provider-specific key is set. |
| `FLUID_LLM_ENDPOINT` | Overrides the model endpoint. |
| `FLUID_LLM_ENDPOINT` | Overrides the model endpoint. The routing model's calls use it too, unless `FLUID_LLM_ROUTING_ENDPOINT` names another. *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* |
| `FLUID_LLM_TIMEOUT_SECONDS` | Per-call timeout. Default `120`. |
| `FLUID_LLM_STREAMING` | Default on; `0`, `false`, `no` or `off` disables streaming. |
| `FLUID_LLM_ROUTING_ENDPOINT`, `FLUID_LLM_ROUTING_MODEL` | Optional routing endpoint and model. |
Expand Down Expand Up @@ -258,7 +258,7 @@ Provider-specific keys (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`,
| `FLUID_FORGE_TOOL_SEARCH` | Enables lazy tool-search deferral in the agent loop. |
| `FLUID_GITHUB_MCP`, `FLUID_SNOWFLAKE_MCP`, `FLUID_DBT_MCP` | Enable delegation to the hosted GitHub, Snowflake and dbt MCP servers. |
| `FLUID_HOSTED_MCP_TIMEOUT_SECONDS` | Per-operation timeout for hosted MCP calls. |
| `FLUID_COPILOT_AGENT_LOOP` | `1`, `true`, `yes` or `on` selects the multi-turn agent loop. Same as `--agent-loop`. |
| `FLUID_COPILOT_AGENT_LOOP` | `1`, `true`, `yes` or `on` selects the multi-turn agent loop. Same as `--agent-loop`. The loop also keeps a non-interactive run from picking a coding agent; see [When nothing is configured](./llm-providers.md#when-nothing-is-configured). *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* |
| `FLUID_COPILOT_JUDGE`, `FLUID_COPILOT_ENRICHMENT`, `FLUID_JUDGE_SELF_CRITIQUE`, `FLUID_COPILOT_SELF_EVAL`, `FLUID_COPILOT_CHECKPOINT`, `FLUID_COPILOT_PII_CLASSIFIER` | On by default; `0`, `false`, `no` or `off` skips that stage. |
| `FLUID_COPILOT_POST_SYNTHESIS`, `FLUID_COPILOT_EPISODIC_MEMORY` | On by default; only the value `0` turns them off. |
| `FLUID_COPILOT_SEMANTIC_MEMORY` | Opts in to semantic-memory writes and lookups. |
Expand Down
6 changes: 3 additions & 3 deletions docs/advanced/litellm-backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ LiteLLM was a hard dependency of core `data-product-forge` (`litellm>=1.83.14,<2
| `openai` | `openai` | `gpt-4.1-mini` |
| `anthropic`, `claude` | `anthropic` | `claude-haiku-4-5` |
| `gemini` | `gemini` | `gemini-2.5-flash` |
| `ollama` | `ollama` (localhost only) | `gemma3:4b` |
| `ollama` | `ollama` | `gemma3:4b` |

A forge run normally gets its model from `fluid ai models`, which is `claude-sonnet-4-6` for Anthropic, `gemini-2.5-pro` for Gemini and `gemma4:latest` for Ollama; the fallback column applies only when nothing names a model.

Expand Down Expand Up @@ -43,7 +43,7 @@ LiteLLM reports the cost of each call, and Forge folds it into `.fluid/agents/<r
| `FLUID_LLM_MODEL` | Model name. |
| `FLUID_LITELLM_MODEL_PREFIX` | Replace the LiteLLM prefix Forge puts in front of the model name (`<prefix>/<model>`). |
| `OPENAI_API_KEY` / `ANTHROPIC_API_KEY` / `GEMINI_API_KEY` | Provider keys. LiteLLM reads the same names the underlying SDKs use. |
| `OLLAMA_HOST` | Ollama endpoint. Local addresses only. |
| `OLLAMA_HOST` | Ollama's address. See [The Ollama endpoint](./llm-providers.md#the-ollama-endpoint). |
| `LITELLM_*` | Any LiteLLM-specific variable. LiteLLM reads these directly; Forge does not filter them. |

See the [environment variables index](./environment-variables.md) for everything else.
Expand All @@ -63,7 +63,7 @@ If you point LiteLLM at a model the catalog doesn't know, the run-start banner s
## Caveats

- Tool-use behaviour matches LiteLLM's wrapper. Provider failures surface as the typed errors described in [Typed Errors](./typed-errors.md) (`RateLimitError`, `ContextOverflowError` and the others).
- Ollama is restricted to `localhost` (`127.0.0.1` / `::1`) by the SSRF guard. See [network safety](./network-safety.md).
- Forge's Ollama calls go to `OLLAMA_HOST` whatever its host; only the probe for a running Ollama, model discovery and the interactive Ollama setup stay on `localhost`. See [The Ollama endpoint](./llm-providers.md#the-ollama-endpoint). *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).*

## See also

Expand Down
97 changes: 94 additions & 3 deletions docs/advanced/llm-providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ FLUID_LLM_MODEL=gemma4:latest \
fluid forge data-model from-intent intent.yaml -o customer_orders.fluid.yaml
```

A run that names no provider and cannot ask a question picks one itself; see [When nothing is configured](#when-nothing-is-configured). *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).*

## Supported providers

The `openai`, `anthropic`, `gemini` and `ollama` providers run through LiteLLM and need `pip install 'data-product-forge[litellm]'` (alias `[ai]`). *(since 0.28.0, [forge-cli #837](https://github.com/Agenticstiger/forge-cli/pull/837))* `mcp-sampling` works only inside `fluid mcp serve`, which needs `[mcp]`. The four local agent CLIs need neither extra; only Claude Code needs no setup of its own, since the others reuse their own CLI login or key.
Expand All @@ -23,9 +25,9 @@ The `openai`, `anthropic`, `gemini` and `ollama` providers run through LiteLLM a
| Anthropic | `anthropic` (alias `claude`) | `claude-sonnet-4-6` | Tool-forced structured output and provider-native prompt caching. Streamed runs report token usage in cost summaries. |
| OpenAI | `openai` | `gpt-4.1-mini` | Strict JSON Schema output where available. Tiered runs use `gpt-4.1` for deep logical modeling. Set `FLUID_OPENAI_STRICT_SCHEMA=1` to harden the response-format schema for `gpt-4o`, `gpt-4.1` and o-series models that reject permissive nested objects. |
| Gemini | `gemini` | `gemini-2.5-pro` | Uses Gemini response schema where suitable and validator repair when needed. |
| Ollama | `ollama` | `gemma4:latest` | Local-only; JSON mode is model-gated. The capability and token-budget catalogs cover `gemma` 1 to 4, `qwen3-coder`, `qwen3`, `qwen2.5`, `llama3.1`, `llama3.2`, `llama3.3`, `mistral`, `mixtral`, `deepseek` and `phi`. See [Capability Warnings](capability-warnings.md) for tool-use accuracy notes per family. |
| Ollama | `ollama` | `gemma4:latest` | Calls go to the [Ollama endpoint](#the-ollama-endpoint). JSON mode is model-gated. The capability and token-budget catalogs cover `gemma` 1 to 4, `qwen3-coder`, `qwen3`, `qwen2.5`, `llama3.1`, `llama3.2`, `llama3.3`, `mistral`, `mixtral`, `deepseek` and `phi`. See [Capability Warnings](capability-warnings.md) for tool-use accuracy notes per family. |
| MCP sampling | `mcp-sampling` | none | No key of its own. When forge runs inside an MCP client, the model request goes back through the connection and the client's own LLM answers it. |
| Local agent CLIs | `claude-code`, `codex`, `cursor`, `kiro` | the agent's own | Forge shells out to a coding-agent CLI you already have installed. Claude Code uses your subscription login; the others reuse `CODEX_API_KEY`, `CURSOR_API_KEY` and `KIRO_API_KEY`. |
| Local agent CLIs | `claude-code`, `codex`, `cursor`, `kiro` | the agent's own | Forge shells out to a coding-agent CLI you already have installed. Claude Code uses your subscription login; the others reuse `CODEX_API_KEY`, `CURSOR_API_KEY` and `KIRO_API_KEY`. Forge runs `claude -p` apart from your own Claude Code setup; see [Claude Code](#claude-code). |

Those are the values `fluid forge --llm-provider` and `fluid forge data-model ... --llm-provider` accept. Other names LiteLLM understands, such as `bedrock`, `vertex` and `azure`, are rejected by the argument parser as of 0.18.1:

Expand All @@ -44,6 +46,95 @@ fluid ai models --json # the same plan as JSON, one object per provider

`fluid ai models` prints one plan per provider and cannot be narrowed to one. As of 0.18.1 its `--provider` flag does not work: `fluid ai models --provider gemini` exits 2 with `Unknown provider 'gemini' — installed providers: aws, datamesh_manager, gcp, local, redshift, snowflake`, because the CLI checks the value against the infrastructure providers. Filter the JSON instead, for example with `jq .gemini`. `fluid ai test` and `fluid ai setup` take `--provider` and are not affected.

## When nothing is configured

*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).*

A run that cannot ask a question picks an LLM itself when nothing names one. That is a run whose stdin is not a terminal, or one with `--non-interactive`, which `fluid forge --agent` sets when it runs AI mode. Forge uses the first of these it finds:

1. A local Ollama answering on `http://localhost:11434`, so local data stays on the machine.
2. The first coding-agent CLI on `PATH`, in this order: `claude` (`claude-code`), `codex`, `cursor-agent` (`cursor`) and `kiro-cli` (`kiro`).

When it picks an agent, forge logs a warning that names it:

```text
No LLM provider is configured; using claude-code, the first coding agent found on PATH. Override with --llm-provider.
```

With neither, the run falls back to `gemini`, which needs a Gemini key and the `litellm` extra. Through 0.28.0 a run with no Ollama running always fell back to `gemini`, even with a signed-in Claude Code on `PATH`.

Any of these configures an LLM, and then no agent is picked:

- `--llm-provider`, `FLUID_LLM_PROVIDER` or `FLUID_FORGE_AGENT`. The provider they name is used.
- `--llm-endpoint`, `--llm-model`, `--llm-routing-endpoint` or `--llm-routing-model`.
- `FLUID_LLM_API_KEY`, `FLUID_LLM_ENDPOINT`, `FLUID_LLM_MODEL`, `FLUID_LLM_ROUTING_ENDPOINT`, `FLUID_LLM_ROUTING_MODEL` or `OLLAMA_HOST`.
- A provider key: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, `GOOGLE_API_KEY` or `GITHUB_API_KEY`.
- An OpenAI, Anthropic or Gemini key in the OS keyring.
- A provider saved with `fluid ai setup`. A non-interactive run does not use the saved provider itself, so name it with `--llm-provider` or `FLUID_LLM_PROVIDER`.
- The agent loop: `--agent-loop`, or `FLUID_COPILOT_AGENT_LOOP` set to `1`, `true`, `yes` or `on`. The loop needs tool use, which these agents do not offer.

The pick works on the base install, because the agent CLIs need no `litellm` extra. A running local Ollama still comes first there, so that run asks for the extra instead of sending the prompt to an agent.

An interactive run does not pick an agent: `fluid forge` in a terminal offers Claude Code in its inline setup instead. With nothing configured, [`fluid doctor`](../cli/doctor.md) run without a terminal reports the provider this pick chooses. When the pick finds nothing, its Provider row says `Not selected`.

## Claude Code

*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).*

`--llm-provider claude-code` runs `claude -p` on your machine. Forge keeps each call apart from your own Claude Code setup, as Anthropic's [Agent SDK](https://github.com/anthropics/claude-agent-sdk-python) does:

| Flag or variable | Effect |
| --- | --- |
| `--system-prompt` | Forge's prompt replaces Claude Code's own system prompt. |
| `--tools ""` | No built-in tools. |
| `--strict-mcp-config` | No MCP servers. Forge passes no `--mcp-config`. |
| `--setting-sources=` | None of your user, project or local settings: no hooks, plugins or `CLAUDE.md`. |
| `--no-session-persistence` | The call is not saved as a session. |
| `CLAUDECODE` | Removed from the environment `claude` starts in. Claude Code sets it in the shells it starts. |

Forge passes each flag only when the installed `claude --help` lists it, so an older Claude Code still runs. Without `--system-prompt`, forge appends its prompt with `--append-system-prompt`, as it did through 0.28.0. The [Claude Code CLI reference](https://code.claude.com/docs/en/cli-reference) describes each flag.

### Signing in

A subscription login (run `claude` once and sign in) and an `ANTHROPIC_API_KEY` in the environment work as before. Authentication kept in your own Claude Code settings does not reach forge's calls: an `apiKeyHelper`, or `CLAUDE_CODE_USE_BEDROCK`, `CLAUDE_CODE_USE_VERTEX` and their variables in the `env` block of `settings.json`. Export those variables in the shell that runs forge, or set `ANTHROPIC_API_KEY`. Claude Code's [authentication](https://code.claude.com/docs/en/authentication) page lists the variables each method needs.

### When the call fails

`claude -p` reports a failure as a JSON result with `is_error: true` on stdout, and leaves stderr empty. Forge reads that result. A login problem reads:

```text
claude-code is installed but not authenticated.
```

The hint under it says ``Run `claude` once and sign in (subscription), or set ANTHROPIC_API_KEY.`` Any other failure shows Claude Code's own message. A result with `is_error: true` fails the call even when `claude` exits `0`. Through 0.28.0 forge read only stderr, and showed `claude-code exited with code 1:` with no reason.

## The Ollama endpoint

*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).*

Forge's Ollama calls go to the first of these that is set:

1. `--llm-endpoint`, or `FLUID_LLM_ENDPOINT`.
2. `OLLAMA_HOST`.
3. `http://localhost:11434`, litellm's default.

Forge reads `OLLAMA_HOST` the way Ollama's own Python client does ([`_parse_host`](https://github.com/ollama/ollama-python/blob/35d9667e84ebbd9d792826868d050027f75c1cc1/ollama/_client.py)), so a value Ollama accepts works here too:

| `OLLAMA_HOST` | Calls go to |
| --- | --- |
| `gpu-box` | `http://gpu-box:11434` |
| `gpu-box:8080` | `http://gpu-box:8080` |
| `:8080` | `http://127.0.0.1:8080` |
| `http://gpu-box` | `http://gpu-box:80` |
| `https://gpu-box` | `https://gpu-box:443` |
| `https://gpu-box/ollama` | `https://gpu-box:443/ollama` |

The host can be another machine. Model discovery still looks only at a local Ollama: with no model named, forge takes the largest model pulled on `localhost`, else `gemma4:latest`. With a remote `OLLAMA_HOST`, name the model with `--llm-model` or `FLUID_LLM_MODEL`. Forge's local-only checks print `OLLAMA_HOST points to a non-localhost address (...), ignoring.` for such a host. The call still goes to `OLLAMA_HOST`.

A stated endpoint is used for the routing model's calls too, unless `--llm-routing-endpoint` or `FLUID_LLM_ROUTING_ENDPOINT` names another. A project's `.env` cannot set `OLLAMA_HOST`; see [Project dotenv trust](./project-dotenv-trust.md).

Through 0.28.0 these calls went to a `litellm://ollama/<model>` label unless an endpoint was stated, so they never reached the daemon.

## Tiered mode

`--tiered` chooses different models within the same provider, never across providers. A typical layout is:
Expand Down Expand Up @@ -102,7 +193,7 @@ fluid forge data-model from-intent intent.yaml \
| `OPENAI_API_KEY` | OpenAI key |
| `ANTHROPIC_API_KEY` | Anthropic key |
| `GOOGLE_API_KEY` or `GEMINI_API_KEY` | Gemini key |
| `OLLAMA_HOST` | Ollama endpoint; local addresses only |
| `OLLAMA_HOST` | Ollama's address; see [The Ollama endpoint](#the-ollama-endpoint) |

### Agent-loop tuning

Expand Down
8 changes: 5 additions & 3 deletions docs/advanced/network-safety.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ This page describes the user-visible behaviour: which flags opt in, which enviro
| `fluid odps import <url>` | Followed `http(s)` `contractId` references | Local only; remote references are rejected | `--allow-remote` |
| `BitolOdpsProvider().import_contract(...)` (Python) | `allow_remote=True` | `allow_remote=False` | `allow_remote=True` (keyword) |
| `ContractResolver(...)` (Python) | `allow_remote=True` | `allow_remote=False` | `allow_remote=True` (keyword) |
| Ollama endpoint | Configurable | Localhost only | None, by design |
| Ollama endpoint | Configurable | Localhost only | None, by design. *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* Calls now go to `OLLAMA_HOST` whatever the host; see [Ollama and `OLLAMA_HOST`](#ollama-and-ollama-host). |
| HTTP redirects on a safe-HTTP client | Followed | Not followed | `follow_redirects=True` (per client) |

The CLI flags `--no-remote` and `--seed-no-remote` remain as hidden aliases that do nothing, so older scripts keep working.
Expand Down Expand Up @@ -68,9 +68,11 @@ An IPv4-mapped address (`::ffff:...`) or a NAT64 address (`64:ff9b::/96`) that c

As of 0.18.1, nothing in the CLI reads `FLUID_SAFE_MODE` or acts on the global `--safe-mode` flag to refuse network calls.

## Ollama is localhost only
## Ollama and `OLLAMA_HOST`

The Ollama provider uses `http://localhost:11434` unless `OLLAMA_HOST` names another localhost URL. An `OLLAMA_HOST` that points elsewhere is ignored with a warning, and the default is used. You cannot point `fluid forge --llm-provider ollama` at a remote Ollama instance this way. For a remote model, use a hosted provider (OpenAI, Anthropic, Gemini, Bedrock or Vertex).
*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* Forge's Ollama calls go to `--llm-endpoint` or `FLUID_LLM_ENDPOINT`, else to `OLLAMA_HOST` whatever its host, else to `http://localhost:11434`; see [The Ollama endpoint](./llm-providers.md#the-ollama-endpoint). Three things still look only at `localhost`, and ignore a non-local `OLLAMA_HOST` with a warning: the probe that finds a running Ollama, model discovery, and the interactive Ollama setup. A project's `.env` cannot set `OLLAMA_HOST` ([Project dotenv trust](./project-dotenv-trust.md)).

Through 0.28.0 the calls went to a `litellm://ollama/<model>` label unless an endpoint was stated, so no `OLLAMA_HOST`, local or remote, reached a daemon.

## Contract SQL and `$ref`

Expand Down
4 changes: 4 additions & 0 deletions docs/advanced/production-troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,10 @@ Update the key at the source `fluid ai status` reports, the provider environment

[`fluid ai test`](../cli/ai.md#ai-test) is the connectivity check. As of 0.18.1, `fluid ai test` with no provider configured and no `--provider` fails with `❌ Unexpected error: name 'detect_ollama_available' is not defined` and exit code 2. Name the provider, or run `fluid ai setup` first. `fluid ai test --provider <name> --json` prints a report with `ok`, `exit_code` and, on failure, an `error` object with `code`, `message` and `suggestions`.

### A coding agent that is not signed in

*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* With `--llm-provider claude-code`, a login problem reads `claude-code is installed but not authenticated.` Run `claude` once and sign in, or set `ANTHROPIC_API_KEY`. Authentication kept in a Claude Code setting (an `apiKeyHelper`, or Bedrock or Vertex variables in `settings.json`) does not reach forge's calls: export those variables in the shell. See [Claude Code](./llm-providers.md#claude-code).

## An optional package is not installed

**Symptom:** a command exits 1, with no traceback, and prints a message naming the feature, the package and the extra, followed by a short error code in brackets:
Expand Down
Loading
Loading