Skip to content

fix(ai,receipts): ai models --provider takes the LLM registry; run ids never repeat - #856

Merged
fas89 merged 5 commits into
mainfrom
claude/small-fixes-runid-74c1d5
Nov 12, 2026
Merged

fas89 merged 5 commits into
mainfrom
claude/small-fixes-runid-74c1d5

Conversation

@fas89

@fas89 fas89 commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Two small fixes found on 2026-11-12.

fluid ai models --provider works. The flag stored its value in args.provider, the slot of
the global infrastructure --provider. _validate_global_args then read every LLM name as an
infrastructure provider: fluid ai models --provider gemini exited 2 with "Unknown provider
'gemini' — installed providers: aws, databricks, gcp, local, snowflake (see fluid providers)".
forge_docs has described this since 0.18.1.

  • The flag now uses dest="llm_provider", as ai setup and ai test already do.
  • Its choices are LlmProviderChoices, the container fluid forge --llm-provider uses: the
    built-ins (listed in --help and in the invalid-choice error) plus installed
    fluid_build.llm_providers plugins. --help scans no entry points.
  • --provider claude-code, mcp-sampling or a plugin name prints that provider's model plan.
    The unfiltered fluid ai models output is unchanged.

Receipt run ids never repeat in one process. generate_run_id (the run_id of
.fluid/init-receipt.json and forge-receipt.json) paired a millisecond timestamp with 32 random
bits. Ids made in one millisecond collided by the birthday bound, and test_run_ids_are_unique
failed once on CI with assert 999 == 1000.

  • The suffix is now monotonic, as in a ULID or a UUIDv7 (RFC 9562, section 6.2). Each new
    millisecond starts it at a random 31-bit value, and each later id in that millisecond adds one.
  • The last millisecond is kept when the clock steps back, and a used-up suffix moves to the next
    millisecond.
  • A lock guards the state, and os.register_at_fork resets it in a forked child.
  • The format is unchanged (<11 hex digits>-<8 hex digits>), and nothing in forge parses it. The
    .fluid/agents/ run directories and fluid stats use _preview_panel.new_run_id.

Commits: e19b3d1 (ai models), 7ee05c6 (run ids), e5bf382 and 1105c7b (test fixes from the PR gate), and a merge of main (the v0.29.0 cut, #855)
that moves both CHANGELOG entries into the new empty [Unreleased].

Design note

  • Journey. Someone choosing an LLM runs fluid ai models --provider <name> to see the
    primary, routing and tier models forge would use, then fluid forge --llm-provider <name>.
    Both commands take the same names.
  • States for ai models --provider:
    • a valid name prints that provider's plan (a table, or --json with one key);
    • claude still shows as anthropic;
    • a keyless agent or mcp-sampling shows its placeholder plan, because the agent or the IDE
      picks the model;
    • an installed plugin shows its default model;
    • an unknown name exits 2 and lists the built-in names, as fluid forge --llm-provider does;
    • with no flag, the output is unchanged (the four catalog providers).
  • Errors. The misleading infrastructure list ("installed providers: aws, …") no longer
    appears for an LLM flag.
  • Consistency. dest="llm_provider", as in ai setup and ai test. The choices
    container is shared with forge --llm-provider, so --help still scans no entry points.
  • Run ids. The shape stays <11 hex>-<8 hex>, so receipt readers see no change.
    Uniqueness within one process is now guaranteed instead of likely, ids keep their time order
    across clock steps, and a fork is safe.

Borrow before build

  • Inside the repo: reused LlmProviderChoices and get_llm_provider. I checked
    _preview_panel.new_run_id, _acquisition_common.generate_run_id and
    copilot/store/audit_trail._next_audit_suffix. None fits: each has a different format or
    meaning.
  • Outside sources:
    • RFC 9562 §6.2: monotonic counters, and a
      counter overflow may move the timestamp ahead.
    • CPython 3.14 uuid.uuid7(): a module-level last timestamp and counter.
      cpython#138862 shows the risk of
      running ahead.
    • ulid/spec#11: monotonicity and overflow.
  • Decision: reimplement the documented algorithm in about 15 stdlib lines.
    • uuid.uuid7 needs Python 3.14 (forge supports 3.10) and gives a 36-character id.
    • The ULID and uuid7 packages on PyPI would be new base dependencies.

Reviews (/ship gate)

  • security-review: no findings at confidence 8 or above, before and after the fix round.
  • world-class-inspection (diff mode): PASS. The delivery review (one auditor running upstream ln-52) found no P0–P2 issue. The UX walk of the changed CLI surface scores 9/10 and found one P3, left as a follow-up: for keyless providers the table shows the agent's own name as the model. That name is a placeholder for the model the agent is configured with.
  • defect-scan (diff mode): no product-code defect. One Medium test defect: the tests
    depended on $FLUID_PROVIDER. It is fixed in e5bf382, and a re-check auditor confirmed the
    fix.
  • ripwire --quality-delta, run on a fresh main worktree with the branch applied: exit 0
    after 1105c7b. Before that commit it gated on the run-id test class (95 lines, bar 60).
  • Rounds: 1.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would break existing functionality)
  • Documentation update
  • Refactor / code cleanup
  • New provider or provider enhancement

Documentation

  • Docs PR linked:
    • Docs PR:
  • No docs needed — this change does not affect user-facing behaviour
    • Justification:
  • Docs TODO — I know docs are needed and will create a PR in forge_docs before this is merged

docs/advanced/llm-providers.md (line 43) and the "fluid ai models --provider <name> fails in
0.18.1" warning in docs/walkthrough/ai-forge-data-model.md describe the exit 2 and a jq
workaround. Both need updating.

Checklist

  • I have read the Contributing Guide
  • My code follows the project's coding standards
  • I have added tests that prove my fix/feature works
  • All new and existing tests pass (pytest)
  • I have run ruff check and black with no errors
  • New maintained Python files include license headers (python scripts/add_license_headers.py; examples/** exempt)

Testing

  • New tests fail on main (3332651) and pass here. Examples:
    • With the clock stopped and the random source fixed, 1000 ids: assert 1 == 1000 on main.
    • main(["ai", "models", "--provider", "gemini", "--json"]) returned 2 on main.
    • The invalid-choice list was the old five names.
  • Tests stay green whatever the shell sets. The new and changed tests pass with
    FLUID_PROVIDER unset, gcp and bogus. The defect scan reproduced two env-dependent
    failures, fixed in e5bf382.
  • The fork test discriminates. It forks for real in a fresh interpreter. A mutation run with
    os.register_at_fork stubbed out makes the child repeat the parent's next id.
  • Gates, run as CI runs them:
    • ruff over the whole tree and the dead-code sweep;
    • black 24.10.0 over the whole tree;
    • the product guardrail, with its self-test;
    • scripts/check_license_headers.py and tests/test_license_headers.py;
    • lint-imports (6 contracts kept) and scripts/mypy_strict.py;
    • detect-secrets-hook 1.5.0 over the changed files through xargs (rc 0; the baseline is
      unchanged).
  • Related suites: 450 passed (ai_setup*, LLM provider plugins, global provider flag, init and
    forge receipts, agent_skills, acquisition_common).
  • Cold path: build_parser() loads 1028 modules on main and on this branch, and none of mcp,
    httpx, jsonschema or litellm.
  • Live, through the fluid CLI:
    • fluid ai models --provider <name> --json for all 10 registry names, plus Gemini, exits 0
      with one key. claude maps to anthropic. No plan carries endpoint or api_key.
    • fluid ai models --json without a filter still lists anthropic, gemini, ollama and openai.
    • An unknown name exits 2 with the 10 names.
    • fluid --provider aws ai models --provider gemini --json exits 0.
    • A real fluid_build.llm_providers entry point (a *.dist-info on PYTHONPATH) is accepted
      and shown; without it the name is refused.
    • A real os.fork() child seeds its own suffix.
    • fluid init demo --blank --yes writes both receipts with well-formed ids.
  • PR gate: see the Reviews section above.

Known, not changed here:

  • fluid generate-pipeline --provider github_actions exits 2 for the same reason
    (cli/pipeline_generator.py:227).
  • test_global_provider_flag.py fails 2 tests when FLUID_PROVIDER is set to an empty string,
    on main too.

Screenshots / Output

$ fluid ai models --provider gemini --json        # main 33326517
⚠️ Unknown provider 'gemini' — installed providers: aws, databricks, gcp, local, snowflake (see `fluid providers`)
exit 2

$ fluid ai models --provider gemini --json        # this branch
{
  "gemini": {
    "primary_model": "gemini-2.5-pro",
    "routing_model": "gemini-2.5-flash",
    ...
exit 0

$ fluid ai models --provider not-a-provider
fluid ai models: error: argument --provider: invalid choice: 'not-a-provider' (choose from openai, anthropic, claude, gemini, ollama, mcp-sampling, claude-code, codex, cursor, kiro)

The flag stored its value in args.provider, the slot of the global
infrastructure --provider, so _validate_global_args read every LLM name as an
infrastructure provider: `fluid ai models --provider gemini` exited 2 with
"Unknown provider 'gemini' — installed providers: aws, databricks, gcp,
local, snowflake". Its own choices were five hard-coded names.

The flag now uses dest=llm_provider, as `ai setup` and `ai test` do, and
LlmProviderChoices, the container `fluid forge --llm-provider` uses: the
built-ins (listed in --help and in an invalid-choice error) plus installed
fluid_build.llm_providers plugins, with no entry-point scan on --help.
show_ai_models shows any of those names, and _provider_model_plan resolves
mcp-sampling and plugins through get_llm_provider. The unfiltered
`fluid ai models` output is unchanged.

tests/test_cli_extension_commands.py asserted the value on args.provider,
which pinned the bug; it now reads args.llm_provider.
The suffix was 32 random bits beside a millisecond timestamp, so ids made in
one millisecond collided by the birthday bound. 1000 ids in a tight loop
repeated one in roughly 1 run of 10,000, which made test_run_ids_are_unique
fail once on CI with `assert 999 == 1000`.

The suffix is now monotonic, as in a ULID or a UUIDv7 (RFC 9562, section
6.2): a random 31-bit start in each new millisecond, plus one for each later
id in it. The last millisecond is kept when the clock steps back, a used-up
suffix moves to the next millisecond, a lock guards the state, and
os.register_at_fork resets it in a forked child. The format stays
<11 hex digits>-<8 hex digits>, and nothing in forge parses it: the
.fluid/agents run directories and fluid stats use _preview_panel.new_run_id.

The new tests stop the clock and fix the random source, so a random-only
suffix fails them every time. The flaky test itself is unchanged.
The defect scan of this branch reproduced two failures with FLUID_PROVIDER
set. The global --provider defaults to it, so the module-scoped parser saw
'gcp' in args.provider, and with FLUID_PROVIDER=bogus main() stopped on the
global check before ai models ran. Both tests now clear the variable.

The fork test called _reset_run_id_state directly, so deleting the
os.register_at_fork line would still have passed. It now forks for real in a
fresh interpreter. With the hook stubbed out, the child repeats the parent's
next id and the test fails.
The only conflict is CHANGELOG.md: both fix entries move into the new empty
[Unreleased] above [0.29.0]. The ai models entry now says the bug was still
in 0.29.0, which shipped without this fix.
@github-actions github-actions Bot added cli Changes to the CLI surface or implementation tests Test coverage or test infrastructure changes labels Nov 12, 2026
ripwire --quality-delta gated on TestGenerateRunId growing from 12 to 95
lines (bar 60). Format and uniqueness stay in TestGenerateRunId. The clock,
used-up suffix and fork cases move to TestRunIdSequence, and the fork
test's child script becomes the module constant _FORK_SCRIPT.
@fas89
fas89 merged commit 065ebc3 into main Nov 12, 2026
38 checks passed
fas89 added a commit that referenced this pull request Nov 12, 2026
The only conflict was CHANGELOG.md, where both sides added [Unreleased]
entries. Every entry is kept: this branch's Changed entry, then one Fixed list
with #856's two entries and this branch's three.
fas89 added a commit that referenced this pull request Nov 12, 2026
CHANGELOG [Unreleased] Fixed: keep both sides' entries, #856's two first, then this branch's three. fluid_build/cli/ai_setup.py merged cleanly.
fas89 added a commit that referenced this pull request Nov 12, 2026
The only conflict was CHANGELOG.md, where both sides added [Unreleased]
Fixed entries. Every entry is kept: #856's two, #858's three, then this
branch's three, under this branch's Changed entry.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli Changes to the CLI surface or implementation tests Test coverage or test infrastructure changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant