diff --git a/README.md b/README.md index aacf0f1..f998731 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ The site is organised the way a reader uses it (get started, tutorials, how-to g ## Current Versioning -- Current CLI release documented here: `0.28.0` +- Current CLI release documented here: `0.29.0` - Contract schema: `fluidVersion: 0.7.6` is the stable default; `0.7.7` is an opt-in preview Those are different on purpose. `fluid version` reports the installed CLI release, while `fluidVersion` inside a contract selects the contract schema version. diff --git a/docs/.vuepress/cli-version.json b/docs/.vuepress/cli-version.json index 55f9109..cdcbea3 100644 --- a/docs/.vuepress/cli-version.json +++ b/docs/.vuepress/cli-version.json @@ -1,6 +1,6 @@ { - "supportedCliVersion": "0.28.0", - "supportedCliInstallSpec": "data-product-forge==0.28.0", + "supportedCliVersion": "0.29.0", + "supportedCliInstallSpec": "data-product-forge==0.29.0", "supportedFluidContractVersion": "0.7.6", "quickstartScaffoldVersion": "0.7.5", "supportedSdkVersion": "0.10.0", diff --git a/docs/advanced/environment-variables.md b/docs/advanced/environment-variables.md index 0974871..f2c5bf1 100644 --- a/docs/advanced/environment-variables.md +++ b/docs/advanced/environment-variables.md @@ -214,14 +214,14 @@ See [cost tracking](./cost-tracking.md). ## LLM provider and routing -See [LLM providers](./llm-providers.md) and the [LiteLLM backend](./litellm-backend.md). *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* With nothing here or in a provider key set, a non-interactive run picks a provider itself; see [When nothing is configured](./llm-providers.md#when-nothing-is-configured). *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* When nothing names a provider and none is found, forge stops with `copilot_no_llm_provider`. +See [LLM providers](./llm-providers.md) and the [LiteLLM backend](./litellm-backend.md). *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* With nothing here or in a provider key set, a non-interactive run picks a provider itself; see [When nothing is configured](./llm-providers.md#when-nothing-is-configured). *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* When nothing selects a provider and none is found, forge stops with `copilot_no_llm_provider`. *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* A provider saved with `fluid ai setup` comes before that pick, in a run without a terminal too; see [Which provider a run uses](./llm-providers.md#which-provider-a-run-uses). | Variable | Effect | |---|---| | `FLUID_LLM_PROVIDER` | Default provider. | -| `FLUID_LLM_MODEL` | Default model. | -| `FLUID_LLM_API_KEY` | API key for the selected provider. It wins over the provider's own key variable, such as `OPENAI_API_KEY`. It does not choose the provider: name one with `--llm-provider`, `FLUID_LLM_PROVIDER` or the key of exactly one provider. *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* With none named, forge stops with `copilot_no_llm_provider`; before #852 it sent the key to `gemini`. | -| `FLUID_LLM_ENDPOINT` | Overrides the model endpoint. *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* The routing model's calls use it too, unless `FLUID_LLM_ROUTING_ENDPOINT` names another. *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* For Ollama, forge's model pick and model check ask it too, and each `--tiered` stage calls it. | +| `FLUID_LLM_MODEL` | Default model. *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* Like `--llm-model`, it passes over a coding agent, whether forge would pick it or `fluid ai setup` saved it. | +| `FLUID_LLM_API_KEY` | *(since 0.29.0, [forge-cli #853](https://github.com/Agenticstiger/forge-cli/pull/853))* API key for a provider named with `--llm-provider`, `FLUID_LLM_PROVIDER` or `FLUID_FORGE_AGENT`; there it wins over the provider's own key variable, such as `OPENAI_API_KEY`. A provider forge picks itself gets its own key variable, then its keyring key, and a warning says `FLUID_LLM_API_KEY` was not used; see [Which key a provider gets](./llm-providers.md#which-key-a-provider-gets). Through 0.28.0 the key went to whatever provider the run used, ahead of its own key variable. The key does not choose the provider. *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* With none selected, forge stops with `copilot_no_llm_provider`; through 0.28.0 it sent the key to `gemini`. | +| `FLUID_LLM_ENDPOINT` | Overrides the model endpoint. *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* The routing model's calls use it too, unless `FLUID_LLM_ROUTING_ENDPOINT` names another. *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* For Ollama, forge's model pick and model check ask it too, and each `--tiered` stage calls it. *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* It also passes over a coding agent that `fluid ai setup` saved. | | `FLUID_LLM_TIMEOUT_SECONDS` | Per-call timeout. Default `120`. | | `FLUID_LLM_STREAMING` | Default on; `0`, `false`, `no` or `off` disables streaming. | | `FLUID_LLM_ROUTING_ENDPOINT`, `FLUID_LLM_ROUTING_MODEL` | Optional routing endpoint and model. | @@ -258,7 +258,7 @@ Provider-specific keys (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, | `FLUID_FORGE_TOOL_SEARCH` | Enables lazy tool-search deferral in the agent loop. | | `FLUID_GITHUB_MCP`, `FLUID_SNOWFLAKE_MCP`, `FLUID_DBT_MCP` | Enable delegation to the hosted GitHub, Snowflake and dbt MCP servers. | | `FLUID_HOSTED_MCP_TIMEOUT_SECONDS` | Per-operation timeout for hosted MCP calls. | -| `FLUID_COPILOT_AGENT_LOOP` | `1`, `true`, `yes` or `on` selects the multi-turn agent loop. Same as `--agent-loop`. The loop also keeps a non-interactive run from picking a coding agent; see [When nothing is configured](./llm-providers.md#when-nothing-is-configured). *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* | +| `FLUID_COPILOT_AGENT_LOOP` | `1`, `true`, `yes` or `on` selects the multi-turn agent loop. Same as `--agent-loop`. *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* The loop also keeps a non-interactive run from picking a coding agent; see [When nothing is configured](./llm-providers.md#when-nothing-is-configured). *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* It passes over a coding agent that `fluid ai setup` saved, too. | | `FLUID_COPILOT_JUDGE`, `FLUID_COPILOT_ENRICHMENT`, `FLUID_JUDGE_SELF_CRITIQUE`, `FLUID_COPILOT_SELF_EVAL`, `FLUID_COPILOT_CHECKPOINT`, `FLUID_COPILOT_PII_CLASSIFIER` | On by default; `0`, `false`, `no` or `off` skips that stage. | | `FLUID_COPILOT_POST_SYNTHESIS`, `FLUID_COPILOT_EPISODIC_MEMORY` | On by default; only the value `0` turns them off. | | `FLUID_COPILOT_SEMANTIC_MEMORY` | Opts in to semantic-memory writes and lookups. | @@ -297,7 +297,7 @@ Read by `fluid mcp output-port`; see [MCP](./mcp.md). | `FLUID_STORE_VECTOR_BACKING` | Backing store of the `vector` backend: `file` (default), `sqlite` or `postgres`. | | `FLUID_SECRETS_INMEMORY` | `1` keeps `fluid secrets` values in memory instead of the OS keychain. Tests only. | | `FLUID_ENCRYPTION_KEY`, `FLUID_ENCRYPTION_PASSPHRASE` | Fernet key, or a passphrase it is derived from, for the encrypted credential store. | -| `FLUID_ALLOW_PLAINTEXT_AI_SECRETS` | Legacy plaintext fallback for AI provider secrets. *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* It also lets `fluid ai setup` save a `user:password@` in an address, such as an Ollama host, to `~/.fluid/ai_config.json`; without it the address is saved without them. | +| `FLUID_ALLOW_PLAINTEXT_AI_SECRETS` | Legacy plaintext fallback for AI provider secrets. *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* It also lets `fluid ai setup` save a `user:password@` in an address, such as an Ollama host, to `~/.fluid/ai_config.json`; without it the address is saved without them. | | `FLUID_ALLOW_PLAINTEXT_SOURCE_SECRETS` | Legacy plaintext secrets in `sources.yaml`; the file must also be mode 600. | ## Marketplace diff --git a/docs/advanced/error-codes.md b/docs/advanced/error-codes.md index da2f801..277ca67 100644 --- a/docs/advanced/error-codes.md +++ b/docs/advanced/error-codes.md @@ -380,7 +380,7 @@ The `fluid generate iac` and `fluid apply` runs print the refusal and exit befor ### copilot_no_llm_provider -*On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* `ERR_COPILOT_NO_LLM_PROVIDER`. The documentation link lands on [Production troubleshooting](./production-troubleshooting.md), where [No LLM provider is selected](./production-troubleshooting.md#no-llm-provider-is-selected) explains each message. +*(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* `ERR_COPILOT_NO_LLM_PROVIDER`. The documentation link lands on [Production troubleshooting](./production-troubleshooting.md), where [No LLM provider is selected](./production-troubleshooting.md#no-llm-provider-is-selected) explains each message. - Pass --llm-provider, for example anthropic, openai, gemini or ollama - Or set FLUID_LLM_PROVIDER, or the API key of exactly one provider diff --git a/docs/advanced/litellm-backend.md b/docs/advanced/litellm-backend.md index 950714b..91339b0 100644 --- a/docs/advanced/litellm-backend.md +++ b/docs/advanced/litellm-backend.md @@ -63,7 +63,7 @@ If you point LiteLLM at a model the catalog doesn't know, the run-start banner s ## Caveats - Tool-use behaviour matches LiteLLM's wrapper. Provider failures surface as the typed errors described in [Typed Errors](./typed-errors.md) (`RateLimitError`, `ContextOverflowError` and the others). -- *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* Forge's Ollama calls go to `OLLAMA_HOST` whatever its host. See [The Ollama endpoint](./llm-providers.md#the-ollama-endpoint). *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* The probe for a running Ollama, model discovery and the interactive Ollama setup follow it too; before #852 they stayed on `localhost`. +- *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* Forge's Ollama calls go to `OLLAMA_HOST` whatever its host. See [The Ollama endpoint](./llm-providers.md#the-ollama-endpoint). *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* The probe for a running Ollama, model discovery and the interactive Ollama setup follow it too; through 0.28.0 they stayed on `localhost`. ## See also diff --git a/docs/advanced/llm-providers.md b/docs/advanced/llm-providers.md index 945ff71..bb88b8e 100644 --- a/docs/advanced/llm-providers.md +++ b/docs/advanced/llm-providers.md @@ -14,7 +14,7 @@ FLUID_LLM_MODEL=gemma4:latest \ fluid forge data-model from-intent intent.yaml -o customer_orders.fluid.yaml ``` -A run that names no provider and cannot ask a question picks one itself; see [When nothing is configured](#when-nothing-is-configured). *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* +A run that names no provider and cannot ask a question picks one itself; see [When nothing is configured](#when-nothing-is-configured). *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* ## Supported providers @@ -46,9 +46,46 @@ fluid ai models --json # the same plan as JSON, one object per provider `fluid ai models` prints one plan per provider and cannot be narrowed to one. As of 0.18.1 its `--provider` flag does not work: `fluid ai models --provider gemini` exits 2 with `Unknown provider 'gemini' — installed providers: aws, datamesh_manager, gcp, local, redshift, snowflake`, because the CLI checks the value against the infrastructure providers. Filter the JSON instead, for example with `jq .gemini`. `fluid ai test` and `fluid ai setup` take `--provider` and are not affected. +## Which provider a run uses + +*(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* + +Forge takes the provider from the first of these that answers. [`fluid doctor`](../cli/doctor.md) walks the same ladder, so its Provider row names what a run uses. It reads only the environment, not a run's flags, so it picks an agent at rung 5 only without a terminal. + +1. A provider you name: `--llm-provider`, `FLUID_LLM_PROVIDER` or `FLUID_FORGE_AGENT`. +2. The key variable of exactly one provider: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY` or `GOOGLE_API_KEY`, or `OLLAMA_HOST`. +3. The provider `fluid ai setup` saved, with its model and its Ollama host. +4. The key of exactly one provider in the OS keyring. +5. A running Ollama, then, in a run without a terminal or with `--non-interactive` that configures nothing, the first coding-agent CLI on `PATH`; see [When nothing is configured](#when-nothing-is-configured). + +A run without a terminal uses the saved choice too. Through 0.28.0, `fluid forge` with `--non-interactive` and `fluid forge data-model` skipped it. The interactive setup of `fluid forge` and `fluid init` read it, and so did `fluid doctor` and `fluid ai test`. + +The saved choice fills in only what the run leaves out, and only for the saved provider: + +- A named provider takes nothing from it. Without `--llm-model`, a named provider gets its catalog model. +- A saved coding agent, or `mcp-sampling`, is passed over when the run states an endpoint or a model (`--llm-endpoint`, `--llm-model` and their routing twins, or `FLUID_LLM_ENDPOINT`, `FLUID_LLM_MODEL` and theirs) or runs the agent loop. The ladder then goes on to rung 4. +- `claude` in the saved file means `anthropic`, and `claude_code` means `claude-code`. +- The saved Ollama host applies when `OLLAMA_HOST` is not set. A saved `endpoint` field is not applied. + +`fluid mission run` always runs the agent loop, so it never picks a coding agent; see [`fluid mission`](../cli/mission.md). + +### Which key a provider gets + +*(since 0.29.0, [forge-cli #853](https://github.com/Agenticstiger/forge-cli/pull/853))* + +`FLUID_LLM_API_KEY` does not say which vendor it is for, so forge sends it only to a provider named with `--llm-provider`, `FLUID_LLM_PROVIDER` or `FLUID_FORGE_AGENT`. There it wins over the provider's own key variable. A named Ollama gets it too, for ollama.com or a proxy that takes a token. + +A provider forge picks itself never gets `FLUID_LLM_API_KEY`: rungs 2 to 5 above, and the provider that interactive setup finds in `fluid forge` and `fluid init`. It gets its own key variable, then its keyring key, and forge warns: + +```text +FLUID_LLM_API_KEY is set but not used: it goes only to a provider named with --llm-provider or FLUID_LLM_PROVIDER, and forge picked ollama itself. Pass --llm-provider ollama to use the key with it. +``` + +`fluid ai test` sends the key only to the provider its `--provider` names, and `fluid doctor` counts it only when `FLUID_LLM_PROVIDER` or `FLUID_FORGE_AGENT` names the provider. Through 0.28.0 forge sent the key to whatever provider it used, so an OpenAI key could reach another vendor, such as the provider of the one key in the OS keyring. + ## When nothing is configured -*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* +*(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* A run that cannot ask a question picks an LLM itself when nothing names one. That is a run whose stdin is not a terminal, or one with `--non-interactive`, which `fluid forge --agent` sets when it runs AI mode. Forge uses the first of these it finds: @@ -61,7 +98,7 @@ When it picks an agent, forge logs a warning that names it: No LLM provider is configured; using claude-code, the first coding agent found on PATH. Override with --llm-provider. ``` -With neither, the run stops with `copilot_no_llm_provider` and says why; see [No LLM provider is selected](./production-troubleshooting.md#no-llm-provider-is-selected). *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* Before #852 it fell back to `gemini`, which needs a Gemini key and the `litellm` extra. Through 0.28.0 it did so whenever no Ollama was running, even with a signed-in Claude Code on `PATH`. +*(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* With neither, the run stops with `copilot_no_llm_provider` and says why; see [No LLM provider is selected](./production-troubleshooting.md#no-llm-provider-is-selected). Through 0.28.0 it fell back to `gemini`, which needs a Gemini key and the `litellm` extra, whenever no Ollama was running, even with a signed-in Claude Code on `PATH`. Any of these configures an LLM, and then no agent is picked: @@ -70,18 +107,18 @@ Any of these configures an LLM, and then no agent is picked: - `FLUID_LLM_API_KEY`, `FLUID_LLM_ENDPOINT`, `FLUID_LLM_MODEL`, `FLUID_LLM_ROUTING_ENDPOINT`, `FLUID_LLM_ROUTING_MODEL` or `OLLAMA_HOST`. - A provider key: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, `GOOGLE_API_KEY` or `GITHUB_API_KEY`. - An OpenAI, Anthropic or Gemini key in the OS keyring. -- A provider saved with `fluid ai setup`. A run with `--non-interactive`, which `--agent` sets, does not use the saved provider itself, so name it with `--llm-provider` or `FLUID_LLM_PROVIDER`. +- A provider saved with `fluid ai setup`. *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* Runs use it as rung 3 of [Which provider a run uses](#which-provider-a-run-uses), except a saved coding agent that the run passes over there. Through 0.28.0 a run with `--non-interactive` did not use it. - The agent loop: `--agent-loop`, or `FLUID_COPILOT_AGENT_LOOP` set to `1`, `true`, `yes` or `on`. The loop needs tool use, which these agents do not offer. -*On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* Only some of these name a provider: `--llm-provider`, `FLUID_LLM_PROVIDER` and `FLUID_FORGE_AGENT`; the key of exactly one of OpenAI, Anthropic, Gemini (`GEMINI_API_KEY` or `GOOGLE_API_KEY`) and Ollama (`OLLAMA_HOST`); a keyring key for exactly one provider; and a provider saved with `fluid ai setup`, in a run without `--non-interactive`. The others, `GITHUB_API_KEY` among them, only stop forge from picking an agent: an Ollama that answers at `OLLAMA_HOST` or `localhost:11434` is still used. When nothing names a provider and no Ollama answers, forge stops with `copilot_no_llm_provider`. So pass `--llm-provider` together with `--llm-endpoint`, `--llm-model` or `FLUID_LLM_API_KEY`. +*(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* Only some of these select a provider: the rungs of [Which provider a run uses](#which-provider-a-run-uses). The others, `GITHUB_API_KEY` among them, only stop forge from picking an agent: an Ollama that answers at `OLLAMA_HOST` or `localhost:11434` is still used. When nothing selects a provider and no Ollama answers, forge stops with `copilot_no_llm_provider`. So pass `--llm-provider` together with `--llm-endpoint`, `--llm-model` or `FLUID_LLM_API_KEY`. The pick works on the base install, because the agent CLIs need no `litellm` extra. A running local Ollama still comes first there, so that run asks for the extra instead of sending the prompt to an agent. -An interactive run does not pick an agent: `fluid forge` in a terminal offers Claude Code in its inline setup instead. With nothing configured, [`fluid doctor`](../cli/doctor.md) run without a terminal reports the provider this pick chooses. When the pick finds nothing, its Provider row says `Not selected`. *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* `fluid doctor --verbose` then shows, in its Message row, the reason forge would stop with. +An interactive run does not pick an agent: `fluid forge` in a terminal offers Claude Code in its inline setup instead. With nothing configured, [`fluid doctor`](../cli/doctor.md) run without a terminal reports the provider this pick chooses. When the pick finds nothing, its Provider row says `Not selected`. *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* `fluid doctor --verbose` then shows, in its Message row, the reason forge would stop with. ## Claude Code -*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* +*(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* `--llm-provider claude-code` runs `claude -p` on your machine. Forge keeps each call apart from your own Claude Code setup, as Anthropic's [Agent SDK](https://github.com/anthropics/claude-agent-sdk-python) does: @@ -112,7 +149,7 @@ The hint under it says ``Run `claude` once and sign in (subscription), or set AN ## The Ollama endpoint -*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* +*(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* Forge's Ollama calls go to the first of these that is set: @@ -137,7 +174,7 @@ Through 0.28.0 these calls went to a `litellm://ollama/` label unless an ### One address for every Ollama read -*On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* +*(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* Every Ollama read takes any host, as Ollama's own client does: @@ -150,7 +187,7 @@ Forge reads `--llm-endpoint` and `FLUID_LLM_ENDPOINT` the same way as `OLLAMA_HO An address may carry `user:password@`, for an Ollama behind a proxy that asks for basic auth, and forge sends it as basic auth, to the call and to the model lookups. `fluid ai test --endpoint` refuses one. `fluid doctor` and `fluid ai setup` show it as `***:***@`. `fluid ai setup` saves the address to `~/.fluid/ai_config.json` without it, as it does for API keys, unless `FLUID_ALLOW_PLAINTEXT_AI_SECRETS` is set, and logs a warning to keep the credentials in the environment. An `OLLAMA_HOST` in the environment wins over the saved address, so keep them there. -Before #852, forge's default-model pick, the probe and `fluid ai setup` looked only at `localhost`. They ignored a non-local `OLLAMA_HOST` with the warning `OLLAMA_HOST points to a non-localhost address (...), ignoring.`, so the model could come from one daemon while the call went to another. The model checks read `OLLAMA_HOST`, never a stated endpoint, and a `--tiered` stage called the provider's default address. A stated endpoint was used exactly as given, so its `user:password@` reached the call but no lookup, and `OLLAMA_HOST` lost its `user:password@`. +Through 0.28.0, forge's default-model pick, the probe and `fluid ai setup` looked only at `localhost`, and ignored a non-local `OLLAMA_HOST` with the warning `OLLAMA_HOST points to a non-localhost address (...), ignoring.` The model checks read `OLLAMA_HOST`, never a stated endpoint. ## Tiered mode diff --git a/docs/advanced/network-safety.md b/docs/advanced/network-safety.md index 81045a6..41c627b 100644 --- a/docs/advanced/network-safety.md +++ b/docs/advanced/network-safety.md @@ -13,7 +13,7 @@ This page describes the user-visible behaviour: which flags opt in, which enviro | `fluid odps import ` | Followed `http(s)` `contractId` references | Local only; remote references are rejected | `--allow-remote` | | `BitolOdpsProvider().import_contract(...)` (Python) | `allow_remote=True` | `allow_remote=False` | `allow_remote=True` (keyword) | | `ContractResolver(...)` (Python) | `allow_remote=True` | `allow_remote=False` | `allow_remote=True` (keyword) | -| Ollama endpoint | Configurable | Localhost only | None, by design. *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* Calls now go to `OLLAMA_HOST` whatever the host. *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* So do the probe, model discovery and the interactive setup; see [Ollama and `OLLAMA_HOST`](#ollama-and-ollama-host). | +| Ollama endpoint | Configurable | Localhost only | None, by design. *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* Calls now go to `OLLAMA_HOST` whatever the host. *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* So do the probe, model discovery and the interactive setup; see [Ollama and `OLLAMA_HOST`](#ollama-and-ollama-host). | | HTTP redirects on a safe-HTTP client | Followed | Not followed | `follow_redirects=True` (per client) | The CLI flags `--no-remote` and `--seed-no-remote` remain as hidden aliases that do nothing, so older scripts keep working. @@ -70,9 +70,9 @@ As of 0.18.1, nothing in the CLI reads `FLUID_SAFE_MODE` or acts on the global ` ## Ollama and `OLLAMA_HOST` -*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* Forge's Ollama calls go to `--llm-endpoint` or `FLUID_LLM_ENDPOINT`, else to `OLLAMA_HOST` whatever its host, else to `http://localhost:11434`; see [The Ollama endpoint](./llm-providers.md#the-ollama-endpoint). A project's `.env` cannot set `OLLAMA_HOST` ([Project dotenv trust](./project-dotenv-trust.md)). +*(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* Forge's Ollama calls go to `--llm-endpoint` or `FLUID_LLM_ENDPOINT`, else to `OLLAMA_HOST` whatever its host, else to `http://localhost:11434`; see [The Ollama endpoint](./llm-providers.md#the-ollama-endpoint). A project's `.env` cannot set `OLLAMA_HOST` ([Project dotenv trust](./project-dotenv-trust.md)). -*On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* No Ollama read is held to `localhost` any more. The probe that finds a running Ollama, model discovery and the interactive Ollama setup follow `OLLAMA_HOST` too, and `fluid ai test --endpoint` takes an Ollama address on any host, over `http` or `https`; it still refuses a `user:password@` in the URL. Before #852 those three looked only at `localhost` and ignored a non-local `OLLAMA_HOST` with a warning, and `fluid ai test` refused an Ollama endpoint that was not `http` on `localhost`. [One address for every Ollama read](./llm-providers.md#one-address-for-every-ollama-read) also covers a `user:password@` in forge's own Ollama address. +*(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* No Ollama read is held to `localhost` any more. The probe that finds a running Ollama, model discovery and the interactive Ollama setup follow `OLLAMA_HOST` too, and `fluid ai test --endpoint` takes an Ollama address on any host, over `http` or `https`; it still refuses a `user:password@` in the URL. Through 0.28.0 those three looked only at `localhost` and ignored a non-local `OLLAMA_HOST` with a warning, and `fluid ai test` refused an Ollama endpoint that was not `http` on `localhost`. [One address for every Ollama read](./llm-providers.md#one-address-for-every-ollama-read) also covers a `user:password@` in forge's own Ollama address. Through 0.28.0 the calls went to a `litellm://ollama/` label unless an endpoint was stated, so no `OLLAMA_HOST`, local or remote, reached a daemon. diff --git a/docs/advanced/production-troubleshooting.md b/docs/advanced/production-troubleshooting.md index bb248f1..fb4df84 100644 --- a/docs/advanced/production-troubleshooting.md +++ b/docs/advanced/production-troubleshooting.md @@ -256,14 +256,14 @@ Update the key at the source `fluid ai status` reports, the provider environment ### No LLM provider is selected -*On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* `copilot_no_llm_provider` means that nothing named an LLM provider and no Ollama answered at `OLLAMA_HOST` or `localhost:11434`, so forge stopped instead of guessing one. Its first line says why: +*(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* `copilot_no_llm_provider` means that nothing selected an LLM provider and no Ollama answered at `OLLAMA_HOST` or `localhost:11434`, so forge stopped instead of guessing one. Its first line says why: | Message | Cause | | --- | --- | -| `No LLM provider configured. Run 'fluid ai setup' or set an API key env var.` | Nothing names a provider. A run without a terminal, or with `--non-interactive`, also found no coding-agent CLI on `PATH`, or a setting that names none, such as `--llm-model` or `--llm-endpoint`, kept it from looking for one. Keys in the OS keyring for more than one provider give this message too. See [When nothing is configured](./llm-providers.md#when-nothing-is-configured). | +| `No LLM provider configured. Run 'fluid ai setup' or set an API key env var.` | Nothing selects a provider ([Which provider a run uses](./llm-providers.md#which-provider-a-run-uses)). A run without a terminal, or with `--non-interactive`, also found no coding-agent CLI on `PATH`, or a setting such as `--llm-model` or `--llm-endpoint`, which selects no provider, kept it from looking for an agent. Keys in the OS keyring for more than one provider give this message too. See [When nothing is configured](./llm-providers.md#when-nothing-is-configured). | | `No LLM provider is selected: OPENAI_API_KEY, ANTHROPIC_API_KEY are all set, so none of them is chosen. Pick one with --llm-provider or FLUID_LLM_PROVIDER.` | More than one provider's key is set in the environment. `OLLAMA_HOST` counts as Ollama's. | -| `No LLM provider is selected: FLUID_LLM_API_KEY does not say which provider the key is for. Pick one with --llm-provider or FLUID_LLM_PROVIDER.` | `FLUID_LLM_API_KEY` is set, but nothing names a provider. | -| `No LLM provider is selected for this run: 'fluid ai setup' saved anthropic, but this command does not read the saved choice. Pass --llm-provider anthropic.` | The command does not read the provider `fluid ai setup` saved: `fluid forge data-model`, or `fluid forge` with `--non-interactive`, which `--agent` sets. | +| `No LLM provider is selected: FLUID_LLM_API_KEY does not say which provider the key is for. Pick one with --llm-provider or FLUID_LLM_PROVIDER.` | `FLUID_LLM_API_KEY` is set, but nothing selects a provider. | +| `No LLM provider is selected: 'fluid ai setup' saved claude-code, and an agent is not used when the run states an endpoint or a model, or runs the agent loop. Name a provider with --llm-provider.` | *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* `fluid ai setup` saved a coding agent or `mcp-sampling`, and the run states an endpoint or a model or runs the agent loop, so forge passes the agent over. | The suggestions under it are the same in each case: @@ -274,11 +274,19 @@ The suggestions under it are the same in each case: • Run 'fluid doctor' to see what forge would select ``` -`fluid doctor --verbose` shows the same reason in its Message row, except for a saved provider, which `fluid doctor` reads. Before #852 forge used `gemini` in each case: it sent `FLUID_LLM_API_KEY` to Google whatever key it was, and with two provider keys set it asked for a Gemini key. +`fluid doctor --verbose` shows the same reason in its Message row. *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* It walks the same ladder, but reads only the environment: a run's `--llm-endpoint`, `--llm-model` or `--agent-loop` flag does not reach it. Through 0.28.0 forge used `gemini` in each case: it sent `FLUID_LLM_API_KEY` to Google whatever key it was, and with two provider keys set it asked for a Gemini key. + +### A key forge did not use + +*(since 0.29.0, [forge-cli #853](https://github.com/Agenticstiger/forge-cli/pull/853))* These warnings say that forge kept a key from a provider: + +- `FLUID_LLM_API_KEY is set but not used: it goes only to a provider named with --llm-provider or FLUID_LLM_PROVIDER, and forge picked ollama itself.` The provider came from a key variable, the saved choice, the keyring or a running Ollama, so it gets its own key variable or keyring key instead. Pass `--llm-provider ` to send it the generic key. See [Which key a provider gets](./llm-providers.md#which-key-a-provider-gets). +- `FLUID_LLM_API_KEY is set but not used: it is for anthropic, the provider you named, and this run uses openai.` You named a provider, and the interactive setup of `fluid forge` or `fluid init` chose another, so the key is not sent to the provider setup chose. +- `Your key is not used: a project dotenv file points OpenAI at its own host (OPENAI_BASE_URL), so your key is not sent there; run 'fluid ai setup' outside this project.` Interactive `fluid forge` or `fluid init` found a project `.env` endpoint for the provider it set up, so it left that provider's key variable to the project's own key. See [LLM endpoints in a project file](./project-dotenv-trust.md#llm-endpoints-in-a-project-file). ### A coding agent that is not signed in -*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* With `--llm-provider claude-code`, a login problem reads `claude-code is installed but not authenticated.` Run `claude` once and sign in, or set `ANTHROPIC_API_KEY`. Authentication kept in a Claude Code setting (an `apiKeyHelper`, or Bedrock or Vertex variables in `settings.json`) does not reach forge's calls: export those variables in the shell. See [Claude Code](./llm-providers.md#claude-code). +*(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* With `--llm-provider claude-code`, a login problem reads `claude-code is installed but not authenticated.` Run `claude` once and sign in, or set `ANTHROPIC_API_KEY`. Authentication kept in a Claude Code setting (an `apiKeyHelper`, or Bedrock or Vertex variables in `settings.json`) does not reach forge's calls: export those variables in the shell. See [Claude Code](./llm-providers.md#claude-code). ## An optional package is not installed diff --git a/docs/advanced/project-dotenv-trust.md b/docs/advanced/project-dotenv-trust.md index ed32c2f..6f793d9 100644 --- a/docs/advanced/project-dotenv-trust.md +++ b/docs/advanced/project-dotenv-trust.md @@ -160,6 +160,12 @@ while a `_*` credential-looking name of the same prefix is set in your r environment. This is how npm binds an auth token to the registry host it was issued for, and how git matches a stored credential to a URL's host. +### LLM endpoints in a project file + +*(since 0.29.0, [forge-cli #853](https://github.com/Agenticstiger/forge-cli/pull/853))* For OpenAI, Anthropic, Gemini and GitHub Models, only the provider's own key in the project file pairs an endpoint such as `OPENAI_BASE_URL`, `ANTHROPIC_BASE_URL`, `GEMINI_API_BASE` or `GITHUB_API_BASE`: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY` or `GOOGLE_API_KEY`, and `GITHUB_API_KEY`. `FLUID_LLM_API_KEY` no longer pairs them; for Anthropic it did only beside `ANTHROPIC_AUTH_TOKEN`. Forge can pick these providers without a name (GitHub Models only in interactive setup), and a provider it picked gets your keyring key, not the project's `FLUID_LLM_API_KEY`. The endpoints of providers forge reaches only by name, such as `GROQ_API_BASE` and `DEEPSEEK_API_BASE`, still pair with `FLUID_LLM_API_KEY`. + +While a project endpoint of a provider is in effect, interactive `fluid forge` and `fluid init` leave that provider's key variable alone, so the project's own key is the one sent, and they warn that your key was not used. `fluid ai setup` asks for no key for that provider, skips a key it would import, and checks none, because the check would send the key to the project's host; run it outside the project. For a provider other than these four, any LLM endpoint a project file sets counts. Through 0.28.0, setup copied the key it had saved over the project's key (for Gemini into `GOOGLE_API_KEY`, which forge reads first), so your key went to the project's host. + ### A Snowflake account from the project pins every Snowflake reader While a Snowflake account from the project's files is in effect, every Snowflake reader — the diff --git a/docs/cli/README.md b/docs/cli/README.md index 8186073..f197907 100644 --- a/docs/cli/README.md +++ b/docs/cli/README.md @@ -1,6 +1,6 @@ # CLI Reference -This section tracks the command surface of `forge-cli` `0.28.0`, the release `pip install data-product-forge` installs. `fluid --help` shows the promoted commands. +This section tracks the command surface of `forge-cli` `0.29.0`, the release `pip install data-product-forge` installs. `fluid --help` shows the promoted commands. ::: tip Looking up *how to do something* (not *what a command does*)? The page you actually want is [**CLI by task →**](./tasks/) — narrative walkthroughs organized around things you might be trying to accomplish, like "deploy to a new cloud", "add quality rules", "debug a failed run", or "add agent governance". Each task page links to the relevant commands. **This** page is the command index, grouped by purpose — better for lookup, worse for onboarding. diff --git a/docs/cli/ai.md b/docs/cli/ai.md index 13d10c4..c7836a9 100644 --- a/docs/cli/ai.md +++ b/docs/cli/ai.md @@ -67,13 +67,13 @@ i No metadata-source catalogs configured yet. Run: fluid ai setup --source Sends a small request to the configured provider and reports whether it answered. Use it before a long forge run, or in CI to check a key. -*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* For `openai`, `anthropic`, `gemini` and `ollama`, the request is the call `fluid forge` makes, through litellm. With no endpoint set, the report's `endpoint` is `null` and the table says `provider default`. Through 0.28.0 the probe went to a `litellm:///` label, so without `--endpoint` it failed with `AI test network error ... (UnsupportedProtocol)`. +*(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* For `openai`, `anthropic`, `gemini` and `ollama`, the request is the call `fluid forge` makes, through litellm. With no endpoint set, the report's `endpoint` is `null` and the table says `provider default`. Through 0.28.0 the probe went to a `litellm:///` label, so without `--endpoint` it failed with `AI test network error ... (UnsupportedProtocol)`. | Option | Description | | --- | --- | -| `--provider {gemini,openai,anthropic,claude,ollama}` | Test this provider instead of the configured one. | +| `--provider {gemini,openai,anthropic,claude,ollama}` | Test this provider instead of the configured one. *(since 0.29.0, [forge-cli #853](https://github.com/Agenticstiger/forge-cli/pull/853))* Only this provider gets `FLUID_LLM_API_KEY`; the configured one gets its own key variable or its saved key. | | `--model ID` | Override the model for this test. | -| `--endpoint URL` | Override the endpoint: an absolute `http` or `https` URL without `user:password@`. A cloud endpoint must be HTTPS, so an API key is never sent in plaintext. An Ollama endpoint may be on any host, over `http` or `https`. *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* Before #852 an Ollama endpoint had to be `http` on `localhost`. | +| `--endpoint URL` | Override the endpoint: an absolute `http` or `https` URL without `user:password@`. A cloud endpoint must be HTTPS, so an API key is never sent in plaintext. An Ollama endpoint may be on any host, over `http` or `https`. *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* Through 0.28.0 an Ollama endpoint had to be `http` on `localhost`. | | `--timeout-seconds N` | HTTP timeout for the request. | | `--json` | Print a stable JSON report instead of the table. | @@ -108,7 +108,7 @@ fluid ai test --provider ollama --endpoint http://127.0.0.1:9 --timeout-seconds } ``` -*On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* The model check before the call asks the address the call goes to: `--endpoint`, else `OLLAMA_HOST`, else `localhost:11434`. An address that does not answer gives no model list, so the example above goes on to the call and exits `5`. Before #852 the check ignored `--endpoint`: with a local Ollama running, the example exited `4` with `ai_test_model_unavailable` when that Ollama did not have the model. +*(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* The model check before the call asks the address the call goes to: `--endpoint`, else `OLLAMA_HOST`, else `localhost:11434`. An address that does not answer gives no model list, so the example above goes on to the call and exits `5`. The process exit code equals `exit_code`: @@ -120,7 +120,7 @@ The process exit code equals `exit_code`: | `4` | The request failed in another way, for example an unknown model. | | `5` | A network failure, a `429` or a `5xx`. | -*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* A `401` or `403` from the call exits `3`. A model the provider does not have exits `4`: it is missing from the provider's model list (`ai_test_model_unavailable`), or the call returns a `404`. A reply without `FLUID_OK` exits `4` too (`ai_test_unexpected_response`). Any other failure of the call, such as a refused connection, a timeout, a `429` or a `5xx`, exits `5` with `"code": "copilot_litellm_request_failed"` and a message that starts `litellm request failed:`. The failed run above reads that way on `main`, in place of `ai_test_network_error`. +*(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* A `401` or `403` from the call exits `3`. A model the provider does not have exits `4`: it is missing from the provider's model list (`ai_test_model_unavailable`), or the call returns a `404`. A reply without `FLUID_OK` exits `4` too (`ai_test_unexpected_response`). Any other failure of the call, such as a refused connection, a timeout, a `429` or a `5xx`, exits `5` with `"code": "copilot_litellm_request_failed"` and a message that starts `litellm request failed:`. From 0.29.0 the failed run above reads that way, in place of `ai_test_network_error`. With no provider configured and no key in the environment, `fluid ai test` exits `2` with `ai_test_no_provider` and the message `No AI provider configured. Run 'fluid ai setup' or set a provider API key.` *(since 0.28.0, [forge-cli #839](https://github.com/Agenticstiger/forge-cli/pull/839))* Through 0.27.0, `fluid ai test` and `fluid ai test --json` stopped with `Unexpected error: name 'detect_ollama_available' is not defined` instead, also with exit `2`. @@ -147,8 +147,9 @@ fluid ai models --json | jq '.gemini.tier_models' - `setup` is interactive and requires a TTY with `rich` installed. It walks through provider choice (Google Gemini, OpenAI, Anthropic, Ollama, or skip), validates the API key with a lightweight call, and persists settings. - Provider config is written to `~/.fluid/ai_config.json` (chmod 600). API keys are stored in the OS keyring when available and are not written to the JSON file by default. - If no keyring backend is available, the API key is kept only for the current process. Set `FLUID_ALLOW_PLAINTEXT_AI_SECRETS=1` only when you deliberately want plaintext local persistence. -- *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* The interactive Ollama setup uses `OLLAMA_HOST` whatever its host, else `http://localhost:11434`. It shows a `user:password@` in the address as `***:***@`, and saves the address without it unless `FLUID_ALLOW_PLAINTEXT_AI_SECRETS` is set. Before #852 it replaced a non-local `OLLAMA_HOST` with `http://localhost:11434`, with a warning. See [One address for every Ollama read](../advanced/llm-providers.md#one-address-for-every-ollama-read). *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* Forge's own Ollama calls, and `fluid ai test`, go to `OLLAMA_HOST` whatever the host; see [The Ollama endpoint](../advanced/llm-providers.md#the-ollama-endpoint). -- The same setup flow is invoked inline on first use of [`fluid forge`](./forge.md) in a terminal when no provider is configured. *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* A run without a terminal picks a provider itself instead; see [When nothing is configured](../advanced/llm-providers.md#when-nothing-is-configured). *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* With no provider named or found, such a run stops with `copilot_no_llm_provider`. +- *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* The interactive Ollama setup uses `OLLAMA_HOST` whatever its host, else `http://localhost:11434`. It shows a `user:password@` in the address as `***:***@`, and saves the address without it unless `FLUID_ALLOW_PLAINTEXT_AI_SECRETS` is set. Through 0.28.0 it replaced a non-local `OLLAMA_HOST` with `http://localhost:11434`, with a warning. See [One address for every Ollama read](../advanced/llm-providers.md#one-address-for-every-ollama-read). *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* Forge's own Ollama calls, and `fluid ai test`, go to `OLLAMA_HOST` whatever the host; see [The Ollama endpoint](../advanced/llm-providers.md#the-ollama-endpoint). +- The same setup flow is invoked inline on first use of [`fluid forge`](./forge.md) in a terminal when no provider is configured. *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* A run without a terminal picks a provider itself instead; see [When nothing is configured](../advanced/llm-providers.md#when-nothing-is-configured). *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* With no provider named or found, such a run stops with `copilot_no_llm_provider`. *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* A run without a terminal uses the provider this setup saved. +- *(since 0.29.0, [forge-cli #853](https://github.com/Agenticstiger/forge-cli/pull/853))* While a project's `.env` points a provider at its own host, for example with `OPENAI_BASE_URL` and `OPENAI_API_KEY`, setup asks for no key for that provider, skips a key it would import, and checks none, because the check would send the key there. Run `fluid ai setup` outside the project. See [LLM endpoints in a project file](../advanced/project-dotenv-trust.md#llm-endpoints-in-a-project-file). - Provider environment variables (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, etc.) are respected as a fallback. - For strict provider validation, run `fluid forge data-model from-intent ... --require-llm` so provider failures do not silently fall back to heuristics. - For tiered per-stage model selection, pass `--tiered` on the forge command. Tiering stays within the active provider and collapses to single-model mode when the provider has no distinct tier catalog. diff --git a/docs/cli/contract-tests.md b/docs/cli/contract-tests.md index 7fa4640..186fbd4 100644 --- a/docs/cli/contract-tests.md +++ b/docs/cli/contract-tests.md @@ -87,7 +87,7 @@ The contract is loaded the way `fluid plan` loads it, with `$ref` fragments and ```yaml - name: Schema gate run: | - pip install data-product-forge==0.28.0 + pip install data-product-forge==0.29.0 fluid contract-tests contract.fluid.yaml --baseline baseline.schema.json ``` diff --git a/docs/cli/doctor.md b/docs/cli/doctor.md index cead616..7e116c8 100644 --- a/docs/cli/doctor.md +++ b/docs/cli/doctor.md @@ -64,7 +64,7 @@ fluid doctor --env `fluid doctor` with neither `--scope` nor `--env` runs the default checks: -- **Forge copilot readiness**: whether an LLM provider and model are configured, and whether credentials are present. *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* Its Endpoint row says `Provider default` when no endpoint is set and the provider picks its own address; through 0.28.0 it showed a `litellm:///` label. A ready coding agent's row says `coding-agent://`. With `FLUID_LLM_PROVIDER` set to `claude-code`, `codex`, `cursor` or `kiro`, the copilot is ready when the agent's CLI (`claude`, `codex`, `cursor-agent` or `kiro-cli`) is on `PATH`; the CLI's own sign-in is not checked. Without the CLI, the table says Setup needed and the panel names it: `The 'cursor-agent' CLI for cursor is not on PATH.` Through 0.28.0 these providers read "No API key found". With nothing configured and no terminal, readiness reports the provider a non-interactive forge run picks, or `Not selected` when that pick finds nothing; see [When nothing is configured](../advanced/llm-providers.md#when-nothing-is-configured). *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* With no provider selected, the Message row that `--verbose` shows gives the same reason as forge's `copilot_no_llm_provider`, for example `No LLM provider is selected: OPENAI_API_KEY, ANTHROPIC_API_KEY are all set, so none of them is chosen. Pick one with --llm-provider or FLUID_LLM_PROVIDER.` Before #852 the row always read `No LLM provider configured. Run 'fluid ai setup' or set an API key env var.`, and forge fell back to `gemini` where readiness said `Not selected`. A `user:password@` in the endpoint shows as `***:***@`. +- **Forge copilot readiness**: whether an LLM provider and model are configured, and whether credentials are present. *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* Its Endpoint row says `Provider default` when no endpoint is set and the provider picks its own address; through 0.28.0 it showed a `litellm:///` label. A ready coding agent's row says `coding-agent://`. With `FLUID_LLM_PROVIDER` set to `claude-code`, `codex`, `cursor` or `kiro`, the copilot is ready when the agent's CLI (`claude`, `codex`, `cursor-agent` or `kiro-cli`) is on `PATH`; the CLI's own sign-in is not checked. Without the CLI, the table says Setup needed and the panel names it: `The 'cursor-agent' CLI for cursor is not on PATH.` Through 0.28.0 these providers read "No API key found". With nothing configured and no terminal, readiness reports the provider a non-interactive forge run picks, or `Not selected` when that pick finds nothing; see [When nothing is configured](../advanced/llm-providers.md#when-nothing-is-configured). *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* With no provider selected, the Message row that `--verbose` shows gives the same reason as forge's `copilot_no_llm_provider`, for example `No LLM provider is selected: OPENAI_API_KEY, ANTHROPIC_API_KEY are all set, so none of them is chosen. Pick one with --llm-provider or FLUID_LLM_PROVIDER.` Through 0.28.0 the row always read `No LLM provider configured. Run 'fluid ai setup' or set an API key env var.`, and forge fell back to `gemini` where readiness said `Not selected`. A `user:password@` in the endpoint shows as `***:***@`. *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* Readiness now walks the ladder a run walks, so the Provider row names what a run uses: `FLUID_FORGE_AGENT` counts as naming the provider, and a provider that `FLUID_LLM_PROVIDER` names reports its catalog model, not the one `fluid ai setup` saved; see [Which provider a run uses](../advanced/llm-providers.md#which-provider-a-run-uses). *(since 0.29.0, [forge-cli #853](https://github.com/Agenticstiger/forge-cli/pull/853))* The Auth row counts `FLUID_LLM_API_KEY` only when `FLUID_LLM_PROVIDER` or `FLUID_FORGE_AGENT` names the provider. - **Memory store backend**: which backend holds copilot memory (`file`, `sqlite`, `postgres` or `vector`), where it lives, and whether it is readable and writable. - **Provider readiness**: under `--verbose`, a **Databricks Provider** row reports `Not configured` when no `DATABRICKS_*` workspace variable is set, and otherwise whether the credential resolves under the same rules `fluid apply` uses, whether `DATABRICKS_WAREHOUSE_ID` is set and whether `databricks-sdk` is installed. It never opens a connection and never changes the exit code. See [Databricks](../providers/databricks.md#fluid-doctor-and-the-welcome-scan). - **Feature availability**: a table of CLI components (the schema manager, the sovereignty and agent-policy validators, the provider action parsers for GCP, AWS and Snowflake, the dbt engine, the copilot, the LiteLLM backend and the agent web tools), each marked available, not available or not installed. `--features-only` prints just this table. diff --git a/docs/cli/forge.md b/docs/cli/forge.md index 6f3fe8d..fe548d6 100644 --- a/docs/cli/forge.md +++ b/docs/cli/forge.md @@ -2,7 +2,7 @@ Author a data product contract with an AI copilot, an offline guided interview, or an empty skeleton. By default forge writes `contract.fluid.yaml` and `.fluid/forge-receipt.json`; a scaffold (`--scaffold`), a CI pipeline (`--ci`) and engine artifacts add files. -AI forge with an API-key provider needs the `litellm` extra (alias `ai`): `pip install 'data-product-forge[litellm]'`. *(since 0.28.0, [forge-cli #837](https://github.com/Agenticstiger/forge-cli/pull/837))* Without it, an explicit `--llm-provider` that names an API-key provider (`openai`, `anthropic`, `gemini`, `ollama`), on `fluid forge`, on `fluid forge data-model` or with `--agent-loop`, prints that line and exits 1 with `[ERR_COPILOT_LITELLM_UNAVAILABLE]`, before the interview reads any context or memory. `fluid forge data-model` with no `--llm-provider`, `--llm-model` or `--llm-endpoint` runs offline and needs no extra (`--llm-model` alone does not read the provider saved with `fluid ai setup`: it uses a local Ollama that answers, else `gemini`, which needs a Gemini key. *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* With no Ollama answering, it stops with `copilot_no_llm_provider` instead.). The forge message names two ways to author without it: `--llm-provider claude-code` (your Claude Code login) and `--template starter`; the `--agent-loop` message names only `--template starter`, because the agent CLIs lack tool use. `codex`, `cursor` and `kiro` also work without litellm, but they reuse their own CLI login or key. `--blank` and the other templates need no AI. `--llm-provider mcp-sampling` works only inside `fluid mcp serve`, which needs the `mcp` extra. Bare interactive `fluid forge` with no provider uses Claude Code when it is installed, and otherwise prints the hint and offers guided mode. +AI forge with an API-key provider needs the `litellm` extra (alias `ai`): `pip install 'data-product-forge[litellm]'`. *(since 0.28.0, [forge-cli #837](https://github.com/Agenticstiger/forge-cli/pull/837))* Without it, an explicit `--llm-provider` that names an API-key provider (`openai`, `anthropic`, `gemini`, `ollama`), on `fluid forge`, on `fluid forge data-model` or with `--agent-loop`, prints that line and exits 1 with `[ERR_COPILOT_LITELLM_UNAVAILABLE]`, before the interview reads any context or memory. `fluid forge data-model` with no `--llm-provider`, `--llm-model` or `--llm-endpoint` runs offline and needs no extra (`--llm-model` alone names no provider. *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* The run takes one from [Which provider a run uses](../advanced/llm-providers.md#which-provider-a-run-uses), the provider `fluid ai setup` saved included, and passes over a saved coding agent. Through 0.28.0 the command ignored the saved provider and, with no other signal, used a local Ollama that answered, else `gemini`, which needs a Gemini key. *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* With nothing found, it now stops with `copilot_no_llm_provider`.). The forge message names two ways to author without it: `--llm-provider claude-code` (your Claude Code login) and `--template starter`; the `--agent-loop` message names only `--template starter`, because the agent CLIs lack tool use. `codex`, `cursor` and `kiro` also work without litellm, but they reuse their own CLI login or key. `--blank` and the other templates need no AI. `--llm-provider mcp-sampling` works only inside `fluid mcp serve`, which needs the `mcp` extra. Bare interactive `fluid forge` with no provider uses Claude Code when it is installed, and otherwise prints the hint and offers guided mode. ## Syntax @@ -81,7 +81,7 @@ Both apply on the default AI-copilot path only. See [Single file or fragments?]( | --- | --- | | `--llm-provider NAME` | `openai`, `anthropic`, `claude`, `gemini` or `ollama`; or a keyless provider: `mcp-sampling` (the calling IDE's model) or `claude-code`, `codex`, `cursor`, `kiro` (a local agent CLI). Providers installed through the `fluid_build.llm_providers` entry point are also accepted. | | `--llm-model NAME` | Model identifier. | -| `--llm-endpoint URL` | Exact HTTP endpoint for the selected provider. It does not choose the provider. *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* The routing model's calls use it too, unless `--llm-routing-endpoint` names another. *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* For Ollama it is read like `OLLAMA_HOST`, so `gpu-box` means `http://gpu-box:11434`, and forge's model pick, its model check and each `--tiered` stage use it too; see [One address for every Ollama read](../advanced/llm-providers.md#one-address-for-every-ollama-read). | +| `--llm-endpoint URL` | Exact HTTP endpoint for the selected provider. It does not choose the provider. *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* The routing model's calls use it too, unless `--llm-routing-endpoint` names another. *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* For Ollama it is read like `OLLAMA_HOST`, so `gpu-box` means `http://gpu-box:11434`, and forge's model pick, its model check and each `--tiered` stage use it too; see [One address for every Ollama read](../advanced/llm-providers.md#one-address-for-every-ollama-read). | | `--forge-agent-mode MODE` | For the local agent CLIs: `envelope` (default; the agent returns the contract as JSON on stdout) or `agentic` (the agent writes `contract.fluid.yaml` into the workspace). | | `--llm-routing-model NAME` | A fast, cheap model for interview clarification and AI self-evaluation. | | `--llm-routing-endpoint URL` | Endpoint override for the routing model. | @@ -93,7 +93,7 @@ Both apply on the default AI-copilot path only. See [Single file or fragments?]( Show the model plan per provider with [`fluid ai models`](./ai.md), and check connectivity with [`fluid ai test`](./ai.md#ai-test). -*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* With no AI config, a run without a terminal picks a provider itself; see [When nothing is configured](../advanced/llm-providers.md#when-nothing-is-configured). *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* When it finds none, the run stops with `copilot_no_llm_provider`. +*(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* With no AI config, a run without a terminal picks a provider itself; see [When nothing is configured](../advanced/llm-providers.md#when-nothing-is-configured). *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* When it finds none, the run stops with `copilot_no_llm_provider`. *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* A provider saved with `fluid ai setup` is AI config here too: a run without a terminal uses it. ### Prompt profiles and overlays @@ -409,9 +409,9 @@ fluid forge --agent fluid forge --agent --emit-plan ``` -- It bundles `--yes` and `FLUID_FORGE_NO_*=1`, and emits **JSON-Lines progress events** on stdout so the calling agent can stream status. +- It bundles `--yes` and `FLUID_FORGE_NO_*=1`, and emits **JSON-Lines progress events** on stdout so the calling agent can stream status. *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* Stdout carries only those events: panels, other output and the message about a missing extra go to stderr. Through 0.28.0 they shared stdout with the events. - With no mode flag and no AI flag, it writes the blank contract (`--blank`), so it can never drop into the interactive mode picker. -- Any of `--llm-provider`, `--llm-model`, `--llm-endpoint` or `--context` selects AI mode instead. Forge's AI authors the contract without asking anything. The contract it writes is reported in a `forge.contract_written` event. A failed generation ends with a non-zero `exit_code` in `forge.done` and a non-zero exit status. With nothing configured, the provider is picked as in [When nothing is configured](../advanced/llm-providers.md#when-nothing-is-configured). *On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* Through 0.28.0 these flags were ignored: the run wrote the TODO placeholder contract and reported `exit_code` `0`. *On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* The event, and forge's receipt, cover a contract written to any `--target-dir`: one outside the directory you run forge in, one more than two levels below it, or one given with a literal `~`. A contract outside the current directory is reported by its absolute path. Before #852, AI mode reported only a contract within two levels of the current directory, and for another target it emitted only `forge.start` and `forge.done`. +- Any of `--llm-provider`, `--llm-model`, `--llm-endpoint` or `--context` selects AI mode instead. Forge's AI authors the contract without asking anything. The contract it writes is reported in a `forge.contract_written` event. A failed generation ends with a non-zero `exit_code` in `forge.done` and a non-zero exit status. With nothing configured, the provider is picked as in [When nothing is configured](../advanced/llm-providers.md#when-nothing-is-configured). *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* Through 0.28.0 these flags were ignored: the run wrote the TODO placeholder contract and reported `exit_code` `0`. *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* The event, and forge's receipt, cover a contract written to any `--target-dir`: one outside the directory you run forge in, one more than two levels below it, or one given with a literal `~`. A contract outside the current directory is reported by its absolute path. - `--emit-plan` makes the run emit a single deterministic `forge.plan` event — the per-product-type (SDP / ADP / CDP) field checklist for the agent to fill in — instead of authoring the contract itself. For example, to have a signed-in Claude Code author from an intent file: diff --git a/docs/cli/mission.md b/docs/cli/mission.md index d075fe2..e3ed0f2 100644 --- a/docs/cli/mission.md +++ b/docs/cli/mission.md @@ -51,7 +51,7 @@ deterministic checks pass or a ceiling fires. Needs a configured LLM provider. | `[contract]` | Contract to work on. Default `contract.fluid.yaml`. | | `--resume` | Re-enter the newest unfinished run for this mission. VERIFY is idempotent, so resuming just re-verifies what is on disk — there is no replay. | | `--run-id ` | Target a specific mission run id. | -| `--llm-provider ` | LLM provider (default: your configured provider). | +| `--llm-provider ` | LLM provider (default: your configured provider). *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* A mission always runs the agent loop, so it never picks a coding agent itself, saved with `fluid ai setup` or found on `PATH`. A coding agent named with `--llm-provider` still runs, but it has no tools for the loop, and the mission stalls. | | `--llm-model ` | LLM model id. | | `--workspace ` | Workspace root for receipts and tool confinement (default: auto-detected). | | `--json` | Emit the outcome as JSON. | @@ -249,7 +249,7 @@ cosmetics ([`0.13.0` security fixes](../RELEASE_NOTES_0.13.0.md#security)). ```yaml - name: Mission gate run: | - pip install data-product-forge==0.28.0 + pip install data-product-forge==0.29.0 fluid mission check quality-coverage contract.fluid.yaml ``` diff --git a/docs/cli/scaffold-ci.md b/docs/cli/scaffold-ci.md index ee5a04b..47b82b2 100644 --- a/docs/cli/scaffold-ci.md +++ b/docs/cli/scaffold-ci.md @@ -41,7 +41,7 @@ The generated file installs `data-product-forge` from PyPI (`pip install --quiet | publish | On `main` with `DMM_API_URL` set: `fluid publish $CONTRACT --catalog $CATALOG`. | ::: warning Pin the version in a pipeline that holds credentials -An unpinned `pip install data-product-forge` takes whatever release is latest when the job runs, and the same job holds the credentials the apply stage uses. After generating, edit the install line to pin the version you tested, for example `pip install --quiet "data-product-forge==0.28.0"`, and move the pin deliberately when you upgrade. [`fluid generate ci`](./generate.md#fluid-generate-ci) pins the install to the version that generated the file. +An unpinned `pip install data-product-forge` takes whatever release is latest when the job runs, and the same job holds the credentials the apply stage uses. After generating, edit the install line to pin the version you tested, for example `pip install --quiet "data-product-forge==0.29.0"`, and move the pin deliberately when you upgrade. [`fluid generate ci`](./generate.md#fluid-generate-ci) pins the install to the version that generated the file. ::: The settings are `CONTRACT` (the path you passed), `PROVIDER` (default `default`), `BUILD_ID` (default empty; set it to a `builds[].id` to run that build), `CATALOG` (default `datamesh-manager`), `AIRFLOW_DAGS_DEST` and `DMM_API_URL`. GitLab reads them as CI/CD variables, GitHub Actions as `env`, repository `vars` and `secrets`, Jenkins from the `environment` block. diff --git a/docs/cli/version.md b/docs/cli/version.md index bb66151..3e1b3f7 100644 --- a/docs/cli/version.md +++ b/docs/cli/version.md @@ -69,7 +69,7 @@ Python: 3.11.15 (darwin) System: Darwin arm64 ``` -`fluid --version` is the top-level flag and prints one line, `FLUID Forge CLI v0.28.0`. +`fluid --version` is the top-level flag and prints one line, `FLUID Forge CLI v0.29.0`. ## Options diff --git a/docs/faq/README.md b/docs/faq/README.md index 5c6ab4e..662eef5 100644 --- a/docs/faq/README.md +++ b/docs/faq/README.md @@ -58,7 +58,7 @@ pip install --upgrade "data-product-forge[local]" fluid version ``` -Then follow the [upgrade guide](../upgrading.md). It has a checklist for each version you cross, because some upgrades need action: `0.17.0`, for example, moves remote state to a per-provider key for a contract that uses a per-contract state key, and needs one `fluid schedule-sync` to retire old Airflow DAGs if you sync Airflow DAGs, and `0.16.3` changed some results on purpose. Pin an exact version in CI (`data-product-forge==0.28.0`), not a range. +Then follow the [upgrade guide](../upgrading.md). It has a checklist for each version you cross, because some upgrades need action: `0.17.0`, for example, moves remote state to a per-provider key for a contract that uses a per-contract state key, and needs one `fluid schedule-sync` to retire old Airflow DAGs if you sync Airflow DAGs, and `0.16.3` changed some results on purpose. Pin an exact version in CI (`data-product-forge==0.29.0`), not a range. Your contracts do not need a new `fluidVersion`: `0.27.0` reads `0.7.1` to `0.7.6`, and `0.7.7` as a preview. diff --git a/docs/providers/README.md b/docs/providers/README.md index 04508f1..fa3e65f 100644 --- a/docs/providers/README.md +++ b/docs/providers/README.md @@ -8,7 +8,7 @@ Fluid Forge uses one contract format across local and provider-backed execution ## Docs baseline -- CLI release covered by the primary docs: `0.28.0` +- CLI release covered by the primary docs: `0.29.0` - Contract schema: `0.7.6` is the current stable version, and the default since `0.26.0`; `0.7.7` is a preview you opt into with `fluidVersion: "0.7.7"`. The GCP retention, encryption and `binding.principals` fields need `0.7.6`. - Which `fluidVersion` each scaffolding command writes is listed in [`fluid init`](../cli/init.md#which-fluidversion-each-path-writes). diff --git a/docs/recipes/per-environment-overlays.md b/docs/recipes/per-environment-overlays.md index b1a9c1a..c1d421b 100644 --- a/docs/recipes/per-environment-overlays.md +++ b/docs/recipes/per-environment-overlays.md @@ -250,7 +250,7 @@ jobs: matrix: env: [dev, staging, prod] steps: - - run: pip install "data-product-forge==0.28.0" + - run: pip install "data-product-forge==0.29.0" - run: fluid validate contract.fluid.yaml --env ${{ matrix.env }} - run: fluid plan contract.fluid.yaml --env ${{ matrix.env }} ``` diff --git a/docs/reference/contract-0.7.6.md b/docs/reference/contract-0.7.6.md index a938e9e..a5f3057 100644 --- a/docs/reference/contract-0.7.6.md +++ b/docs/reference/contract-0.7.6.md @@ -9,7 +9,7 @@ description: "The fields of the FLUID 0.7.6 contract schema (stable): path, type Status: **stable**. Schema 0.7.6 is the newest stable schema bundled with the CLI. The newer 0.7.7 schema is a preview; see the [0.7.7 preview delta](./contract-0.7.7-preview.md). -Generated from `fluid_build/schemas/fluid-schema-0.7.6.json` in the `data-product-forge` 0.28.0 package (sha256 `61ca91a63087b89b`). It is the schema `fluid validate` checks a contract against, rendered without edits. To regenerate: `python scripts/gen_contract_reference.py`. +Generated from `fluid_build/schemas/fluid-schema-0.7.6.json` in the `data-product-forge` 0.29.0 package (sha256 `61ca91a63087b89b`). It is the schema `fluid validate` checks a contract against, rendered without edits. To regenerate: `python scripts/gen_contract_reference.py`. Each table row is one field, addressed by its path from the contract root: `.` descends into an object, `[]` marks the items of a list, and `` stands for any key of a map. **Required** means required inside its parent object; the parent can itself be optional. **Allowed values** lists the enum, pattern and range the schema enforces. An object typed plainly `object` rejects keys the schema does not list; `object (open)` accepts extra keys and `object (free-form)` has no listed keys. A row that begins "Only when ..." exists only under that condition. See [how to read this reference](./README.md#how-to-read-a-row). diff --git a/docs/reference/contract-0.7.7-preview.md b/docs/reference/contract-0.7.7-preview.md index e21911f..0e7b87f 100644 --- a/docs/reference/contract-0.7.7-preview.md +++ b/docs/reference/contract-0.7.7-preview.md @@ -9,7 +9,7 @@ description: "What the FLUID 0.7.7 preview schema adds to, and changes in, the s Status: **preview**. A contract opts in with `fluidVersion: "0.7.7"`; a contract that declares 0.7.6 is unaffected. This page lists the difference between the 0.7.6 schema and the 0.7.7 schema as bundled with the CLI: added fields, fields whose definition changed, and fields removed. A field not listed here has the same type, required flag, allowed values, default and description in both; for those, use the [0.7.6 reference](./contract-0.7.6.md). A definition used at several paths is listed once, at its first path. -Generated from `fluid_build/schemas/fluid-schema-0.7.7.json` in the `data-product-forge` 0.28.0 package (sha256 `0f11fb44bc9dafbc`). It is the schema `fluid validate` checks a contract against, rendered without edits. To regenerate: `python scripts/gen_contract_reference.py`. +Generated from `fluid_build/schemas/fluid-schema-0.7.7.json` in the `data-product-forge` 0.29.0 package (sha256 `0f11fb44bc9dafbc`). It is the schema `fluid validate` checks a contract against, rendered without edits. To regenerate: `python scripts/gen_contract_reference.py`. Each table row is one field, addressed by its path from the contract root: `.` descends into an object, `[]` marks the items of a list, and `` stands for any key of a map. **Required** means required inside its parent object; the parent can itself be optional. **Allowed values** lists the enum, pattern and range the schema enforces. An object typed plainly `object` rejects keys the schema does not list; `object (open)` accepts extra keys and `object (free-form)` has no listed keys. A row that begins "Only when ..." exists only under that condition. See [how to read this reference](./README.md#how-to-read-a-row). diff --git a/docs/sdk-and-plugins/journeys/custom-validator.md b/docs/sdk-and-plugins/journeys/custom-validator.md index bbb7349..4bf56be 100644 --- a/docs/sdk-and-plugins/journeys/custom-validator.md +++ b/docs/sdk-and-plugins/journeys/custom-validator.md @@ -459,7 +459,7 @@ Three places this typically gets installed: ``` 3. **CI** — the bundle from [your-own-ci](./your-own-ci.md) already has a `validate` stage. Add `my-org-validators` to the `pip install` line: ```yaml - - run: pip install --index-url "" "data-product-forge==0.28.0" "my-org-validators==" + - run: pip install --index-url "" "data-product-forge==0.29.0" "my-org-validators==" - run: fluid validate contract.fluid.yaml --strict env: FLUID_PLUGINS_ALLOWLIST: "steward-required,cost-center-required,classification-from-vocab,local" diff --git a/docs/sdk-and-plugins/journeys/your-own-scaffolding.md b/docs/sdk-and-plugins/journeys/your-own-scaffolding.md index d7ac6d1..83b6404 100644 --- a/docs/sdk-and-plugins/journeys/your-own-scaffolding.md +++ b/docs/sdk-and-plugins/journeys/your-own-scaffolding.md @@ -176,7 +176,7 @@ keywords = [ dependencies = [ "pydantic>=2.0", - "data-product-forge==0.28.0", + "data-product-forge==0.29.0", ] [project.optional-dependencies] diff --git a/docs/sdk-and-plugins/reference/companion-packages.md b/docs/sdk-and-plugins/reference/companion-packages.md index bbad991..e06a63d 100644 --- a/docs/sdk-and-plugins/reference/companion-packages.md +++ b/docs/sdk-and-plugins/reference/companion-packages.md @@ -48,7 +48,7 @@ The PyPI name reflects the product brand (`data-product-forge`). The import path ```toml dependencies = [ - "data-product-forge==0.28.0", # pin exact for reproducibility + "data-product-forge==0.29.0", # pin exact for reproducibility ] ``` diff --git a/docs/sdk-and-plugins/reference/trust-model.md b/docs/sdk-and-plugins/reference/trust-model.md index 8613562..007db0c 100644 --- a/docs/sdk-and-plugins/reference/trust-model.md +++ b/docs/sdk-and-plugins/reference/trust-model.md @@ -177,7 +177,7 @@ Plugins are tested, scanned, and approved by your platform team before showing u ```toml # In your deploy environment's pyproject.toml or requirements: -data-product-forge==0.28.0 +data-product-forge==0.29.0 data-product-forge-sdk==0.10.0 data-product-forge-custom-scaffold==0.4.1 my-org-validators==1.2.0 diff --git a/docs/upgrading.md b/docs/upgrading.md index 4545fc4..2008db0 100644 --- a/docs/upgrading.md +++ b/docs/upgrading.md @@ -822,25 +822,32 @@ pip install "data-product-forge[standard]" # litellm and mcp: the pre-0.28.0 AI - *(since 0.28.0, [forge-cli #832](https://github.com/Agenticstiger/forge-cli/pull/832))* Nine internal modules that nothing imported are removed: `fluid_build.logging_utils`, `fluid_build.cli.init_samples`, `fluid_build.copilot.schemas.table_metadata`, `fluid_build.providers.local.ducksql`, `.mocks` and `.samples`, and `fluid_build.runtimes.dataform_gcp`, `.dbt_bigquery` and `.dbt_duckdb`. No command or contract behaves differently. It matters only to code that imported one of them directly. -## On forge-cli `main` since 0.28.0 +## Changes in 0.29.0 -*On forge-cli `main`, not yet in a release ([forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850)).* These changes to AI authoring can change what a script, a CI job or an IDE agent sees: +These changes to AI authoring can change what a script, a CI job or an IDE agent sees. -- **`fluid forge --agent` honours the AI flags.** With `--llm-provider`, `--llm-model`, `--llm-endpoint` or `--context`, it authors with forge's AI instead of writing the blank TODO contract, and a failed generation exits non-zero. See [Headless agent mode](./cli/forge.md#headless-agent-mode). -- **A run with nothing configured and no terminal picks an LLM.** It uses a running local Ollama, else the first coding-agent CLI on `PATH`, where it used to fall back to `gemini`. Name the provider to keep a job from picking one. See [When nothing is configured](./advanced/llm-providers.md#when-nothing-is-configured). -- **`claude -p` no longer reads your Claude Code settings.** Authentication kept there (an `apiKeyHelper`, or Bedrock or Vertex variables in `settings.json`) must come from the shell environment. See [Claude Code](./advanced/llm-providers.md#claude-code). -- **Ollama calls reach the daemon.** They go to `--llm-endpoint`, else `OLLAMA_HOST`, else `localhost:11434`, and a remote `OLLAMA_HOST` is honoured. See [The Ollama endpoint](./advanced/llm-providers.md#the-ollama-endpoint). -- **`fluid ai test` probes the real call.** A rejected key exits `3`, a model the provider does not have exits `4`, and other failed calls exit `5`. See [`ai test`](./cli/ai.md#ai-test). +### Which provider and which key -### Follow-ups to these changes +- *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* **A run with nothing configured and no terminal picks an LLM.** It uses a running local Ollama, else the first coding-agent CLI on `PATH`, where it used to fall back to `gemini`. Name the provider to keep a job from picking one. See [When nothing is configured](./advanced/llm-providers.md#when-nothing-is-configured). +- *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* **No `gemini` fallback.** When nothing selects a provider, forge stops with `copilot_no_llm_provider` and says why. It used to fall back to `gemini`, and sent `FLUID_LLM_API_KEY` there whatever key it was. Pass `--llm-provider`, set `FLUID_LLM_PROVIDER` or the key of exactly one provider, or save a provider with `fluid ai setup`. See [No LLM provider is selected](./advanced/production-troubleshooting.md#no-llm-provider-is-selected). +- *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* **Runs without a terminal use the provider `fluid ai setup` saved**, with its model and Ollama host, and `fluid doctor` walks the same ladder. A saved coding agent is passed over when a run states an endpoint or a model, or runs the agent loop. See [Which provider a run uses](./advanced/llm-providers.md#which-provider-a-run-uses). +- *(since 0.29.0, [forge-cli #853](https://github.com/Agenticstiger/forge-cli/pull/853))* **`FLUID_LLM_API_KEY` goes only to a provider you name.** A job that used it with a provider forge picked, such as a running Ollama or the provider of the one key variable set, now sends that provider's own key, or none, and forge warns. Name the provider: `--llm-provider ` or `FLUID_LLM_PROVIDER=`. See [Which key a provider gets](./advanced/llm-providers.md#which-key-a-provider-gets). +- *(since 0.29.0, [forge-cli #853](https://github.com/Agenticstiger/forge-cli/pull/853))* **A project `.env` endpoint for OpenAI, Anthropic, Gemini or GitHub Models needs the provider's own key beside it.** `FLUID_LLM_API_KEY` no longer pairs it, and interactive setup no longer copies your saved key over the project's key. See [LLM endpoints in a project file](./advanced/project-dotenv-trust.md#llm-endpoints-in-a-project-file). +- *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* **`claude -p` no longer reads your Claude Code settings.** Authentication kept there (an `apiKeyHelper`, or Bedrock or Vertex variables in `settings.json`) must come from the shell environment. See [Claude Code](./advanced/llm-providers.md#claude-code). -*On forge-cli `main`, not yet in a release ([forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852)).* These follow-ups to the changes above can change what a script, a CI job or an IDE agent sees: +### Ollama -- **No `gemini` fallback.** When nothing names a provider, forge stops with `copilot_no_llm_provider` and says why. It used to fall back to `gemini`, and sent `FLUID_LLM_API_KEY` there whatever key it was. Pass `--llm-provider`, or set `FLUID_LLM_PROVIDER` or the key of exactly one provider. See [No LLM provider is selected](./advanced/production-troubleshooting.md#no-llm-provider-is-selected). -- **`OLLAMA_HOST` may name any machine everywhere.** The probe for a running Ollama, the default-model pick and `fluid ai setup` no longer swap a non-local host for `localhost`, and `fluid ai test --endpoint` takes an Ollama address on any host. -- **Forge looks for Ollama models where it sends the call.** With `--llm-endpoint` or `FLUID_LLM_ENDPOINT`, the default-model pick and the model check ask that daemon, and each `--tiered` stage calls it. A `user:password@` in the address now goes as basic auth to the model lookups too, and `OLLAMA_HOST` no longer drops it. See [One address for every Ollama read](./advanced/llm-providers.md#one-address-for-every-ollama-read). -- **`fluid forge --agent` reports a contract written to any `--target-dir`.** See [Headless agent mode](./cli/forge.md#headless-agent-mode). -- **`LlmProvider.list_available_models` takes an `endpoint` keyword**, the address the call will go to. Forge passes it only when an endpoint is stated, so code that subclasses `LlmProvider` and overrides the method with two arguments keeps working until then. With an endpoint stated and the model check on (`--require-llm`, `--tiered` or `FLUID_LLM_MODEL_PREFLIGHT`), the check fails with `copilot_llm_model_preflight_failed`; add the keyword to avoid it. +- *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* **Ollama calls reach the daemon.** They go to `--llm-endpoint`, else `OLLAMA_HOST`, else `localhost:11434`, and a remote `OLLAMA_HOST` is honoured. See [The Ollama endpoint](./advanced/llm-providers.md#the-ollama-endpoint). +- *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* **`OLLAMA_HOST` may name any machine everywhere.** The probe for a running Ollama, the default-model pick and `fluid ai setup` no longer swap a non-local host for `localhost`, and `fluid ai test --endpoint` takes an Ollama address on any host. +- *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* **Forge looks for Ollama models where it sends the call.** With `--llm-endpoint` or `FLUID_LLM_ENDPOINT`, the default-model pick and the model check ask that daemon, and each `--tiered` stage calls it. A `user:password@` in the address goes as basic auth to the call and to the model lookups. See [One address for every Ollama read](./advanced/llm-providers.md#one-address-for-every-ollama-read). + +### `fluid forge --agent`, `fluid ai test` and provider code + +- *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* **`fluid forge --agent` honours the AI flags.** With `--llm-provider`, `--llm-model`, `--llm-endpoint` or `--context`, it authors with forge's AI instead of writing the blank TODO contract, and a failed generation exits non-zero. See [Headless agent mode](./cli/forge.md#headless-agent-mode). +- *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* **In AI mode, `fluid forge --agent` reports the contract for any `--target-dir`.** See [Headless agent mode](./cli/forge.md#headless-agent-mode). +- *(since 0.29.0, [forge-cli #854](https://github.com/Agenticstiger/forge-cli/pull/854))* **`fluid forge --agent` stdout is JSON Lines only.** Panels, other output and the missing-extra message go to stderr. +- *(since 0.29.0, [forge-cli #850](https://github.com/Agenticstiger/forge-cli/pull/850))* **`fluid ai test` probes the real call.** A rejected key exits `3`, a model the provider does not have exits `4`, and other failed calls exit `5`. See [`ai test`](./cli/ai.md#ai-test). +- *(since 0.29.0, [forge-cli #852](https://github.com/Agenticstiger/forge-cli/pull/852))* **`LlmProvider.list_available_models` takes an `endpoint` keyword**, the address the call will go to. Forge passes it only when an endpoint is stated, so code that subclasses `LlmProvider` and overrides the method with two arguments keeps working until then. With an endpoint stated and the model check on (`--require-llm`, `--tiered` or `FLUID_LLM_MODEL_PREFLIGHT`), the check fails with `copilot_llm_model_preflight_failed`; add the keyword to avoid it. ## DuckDB 1.x to 2.x diff --git a/docs/vision.md b/docs/vision.md index c4c09b7..2e16011 100644 --- a/docs/vision.md +++ b/docs/vision.md @@ -147,7 +147,7 @@ fluid market --search "customer analytics" ## Versioning in the docs -- Current CLI release baseline: `0.28.0` +- Current CLI release baseline: `0.29.0` - Current scaffolded contract examples: `fluidVersion: 0.7.5` That split is intentional. The CLI release and the contract schema version move on related but different timelines. diff --git a/scripts/forge-cli-releases.json b/scripts/forge-cli-releases.json index 73fea1f..b2853f1 100644 --- a/scripts/forge-cli-releases.json +++ b/scripts/forge-cli-releases.json @@ -21,7 +21,7 @@ ], "repo": "Agenticstiger/forge-cli", "changelogUrl": "https://raw.githubusercontent.com/Agenticstiger/forge-cli/main/CHANGELOG.md", - "reviewedThroughRelease": "0.28.0", + "reviewedThroughRelease": "0.29.0", "releaseOfPr": { "707": "0.20.0", "709": "0.20.0",