Skip to content

feat(templates): model-provider coverage — wizard model step, A2A/AG-UI overrides, --api-base, TypeScript providers - #2507

Open
shawnxli wants to merge 7 commits into
refactorfrom
model-provider-coverage
Open

shawnxli wants to merge 7 commits into
refactorfrom
model-provider-coverage

Conversation

@shawnxli

@shawnxli shawnxli commented Oct 2, 2026 •

Copy link
Copy Markdown

Description

Eight commits, each self-contained and green; please read them in order.

1. feat(tui): the code-based create / add-runtime wizards ask the model provider.
Today both wizards scaffold every model-aware template with its Bedrock default; provider, model
id and API key are flag-only. This adds a model provider step, shown exactly for templates whose
shortcut has supportsModelProviderOverride, mirroring the harness wizard's model step: provider
radio → model id (prefilled from the shared default table) → API key file (file://<path>,
resolved at submit through the same SourceResolver as --api-key). The answer is handed to
resolveRuntimeTemplateShortcut as the same overrides the flags supply, so the wizard produces
the flag path's ScaffoldRuntimeInput byte for byte (golden parity tests). In a China region the
step starts on LiteLLM with no model id and annotates the blocked providers; the existing gates
remain the enforcement, and the create handler's China memory strip moves into a helper shared
with the wizard. DEFAULT_MODEL_IDS moves next to MODEL_PROVIDERS in projectSchemas/runtime.

2. feat(templates): a2a-python-strands and agui-python-strands take the overrides.
Their model/load.py was the strands Bedrock branch with the id inlined. They get the strands
model module and per-provider strands-agents[extra] block, and supportsModelProviderOverride.
The A2A resolver passed no modelProvider/modelId to the renderer and the AG-UI resolver did
not wire the model scaffold at all; both now mirror the HTTP strands resolver. Bedrock output is
unchanged. Both resolvers also keep memory/ when the scaffold has no memory resource (the China
path), as #2426 did for agent-python-strands: each entrypoint imports it unconditionally and it
degrades to no memory without its env var.

3. feat(templates): --api-base for OpenAI-compatible endpoints; TypeScript strands providers.
--api-base <url>, valid with --model-provider open_ai, points the OpenAI client at any
OpenAI-compatible endpoint (Python client_args.base_url, TS clientConfig.baseURL) and is
persisted as the runtime's modelApiBase for the deploy gate. Other providers reject it. In China
regions an OpenAI runtime with a base URL passes create/add/deploy, one without is refused
(api.openai.com is unreachable); the three gates now share one rule
(chinaModelProviderRestriction). agent-typescript-strands gets provider branches
(@strands-agents/sdk models/anthropic|openai|google|bedrock), an identity-backed
model/apiKey.ts (bedrock-agentcore/identity withApiKey, LOCAL_DEV reads .env.local),
and conditional peer deps. LiteLLM is Python-only, so lite_llm is refused for TypeScript
templates with a pointer to open_ai --api-base; the wizard does not offer it there.

4. fix(templates): bump aws-opentelemetry-distro to ~= 0.21.0 in every Python template.
0.18/0.19 build the default OTLP endpoints with a hardcoded .amazonaws.com suffix, so agents in
cn-north-1/cn-northwest-1 log a NameResolutionError on every export and never deliver
logs or traces. 0.21.0 (2026-10-01) is partition-aware. Resolution verified with uv lock for the
strands and langchain templates.

5. fix(templates): keep FastAPI from double-exporting AG-UI telemetry.
FastAPI 0.142 configures OTLP export from the OTEL_* environment at startup and adds its own
exporters next to the OpenTelemetry distro's. On the AgentCore Runtime the distro signs with SigV4
and FastAPI's exporters do not, so every batch was sent twice and the unsigned copy failed with
403 MissingAuthenticationTokenException after each invocation (all partitions; no telemetry was
lost). create_strands_app takes no FastAPI options, so the AG-UI template assembles the same app
with telemetry={"auto_configure": False} (CORS, /invocations, /ping) and pins
fastapi >= 0.142. Verified live: 0 export errors, traces delivered.

6. fix(tui): stop the model step scrolling forever when an input line wraps.
Found by this PR's own CI: on macOS and Windows the temp path typed into the API-key field
wraps at 100 columns, the field grows by a row, and the scroll-into-view callback and the
viewport re-measure fed each other (the measured viewport grows by exactly the offset scrolled)
until React stopped it and the wizard went blank. The callback now scrolls back toward its
anchor only if the field fits there after discounting those rows, stays put when the field is
fully visible, and scrolls forward as before; fields revealed before the viewport is measured
still re-anchor. The harness wizard's model field shares the callback and gets the same fix.
The two file:// tests compare the review row with whitespace removed, and a new test types a
path that wraps.

7. test(tui): cover the model step's key handling. Codecov flagged the runtime model step's
esc-on-the-list, arrow moves between fields and return-to-a-missing-field branches as untested;
one wizard test walks all three (RuntimeModelField.tsx now fully covered).

8. style(templates): tidy the rendered scaffolds. Diffing every template's output against the
pre-PR CLI (all 11 templates with defaults plus the pre-existing provider/China flag paths, 17
scaffolds) showed three rendering artifacts beside the intended changes: the TypeScript model
module had switched to double quotes (new jsStr helper renders single-quoted literals), the
Python OpenAI load.py had a stray space before } (Handlebars reads }}} as an unescaped close,
so {{/if~}} }), and the strands-agents extras conditional rendered a whitespace-only line
(pre-existing for agent-python-strands, copied to A2A/AG-UI; now one line). After this the
default scaffolds differ from the base CLI only by the distro pin, the AG-UI FastAPI change and the
async TypeScript model loader.

Design notes: the API key enters the TUI as a file:// source (no new secret surface); the
default model id is never sent as an override; open_ai + --api-base was chosen over a new
openai_compatible provider or mapping lite_llm onto the OpenAI client for TypeScript, which
would have mislabelled the generated code.

Related Issue

Closes #2506

Documentation PR

Included here: README.md (create section), docs/china-regions.md, command.md (regenerated),
src/assets/templates/agent-typescript-strands/README.md.

Type of Change

  • New feature
  • Documentation update

Testing

  • I ran bun test — 4017 pass, 0 fail (34 new tests: wizard screens, create/add handlers,
    manager China gates, template scaffolds, AG-UI app assembly, wrapped-input scroll, renderer
    helpers); also green
    with a macOS-length TMPDIR, which is how CI first caught commit 6's bug
  • bun run test:e2e not run; instead every China path was verified live in cn-north-1 from
    wizard scaffold through agentcore deploy and agentcore invoke: Python strands (LiteLLM),
    TypeScript strands (open_ai --api-base), A2A and AG-UI (LiteLLM) — each answered, 0 runtime
    errors, traces delivered; the deploy-gate negatives (Bedrock, missing --api-base) refused
    before any CloudFormation call. Commercial wizard flows checked by hand for scaffold parity.
  • I ran bun run typecheck
  • I ran bun run lint:check
  • I ran bun run format:check
  • I ran bun run build
  • src/assets/ modified; snapshots unaffected (manifests and runtime specs unchanged for
    Bedrock scaffolds), full suite green without --update-snapshots

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

The `agentcore create` and `agentcore add runtime` wizards scaffolded every
model-aware template with its Bedrock default: provider, model id and API
key were flag-only (--model-provider / --model-id / --api-key). Add a
`model provider` step, shown exactly for templates whose shortcut has
supportsModelProviderOverride, mirroring the harness wizard's model step:
provider radio → model id (prefilled from the shared default table) → API
key file for the providers that need one.

The answer is handed to resolveRuntimeTemplateShortcut as the same
overrides the flags supply, so the wizard converges on the flag path's
ScaffoldRuntimeInput (the default model id is not sent as an override).
The API key is taken as a `file://` source and read at submit through the
same SourceResolver as --api-key; a screen has no stdin, so '-' and inline
values are refused with the resolver's messages.

In a China region the step starts on LiteLLM with no model id (the LiteLLM
default routes to Amazon Bedrock) and annotates the four blocked providers;
the existing create/add gates remain the enforcement. The create handler's
China memory strip moves into a shared helper so the wizard drops the
default memory the same way the CLI does.

DEFAULT_MODEL_IDS moves next to MODEL_PROVIDERS in projectSchemas/runtime
so the wizard and the template renderer read one table.
…ds templates

a2a-python-strands and agui-python-strands hardcoded the Bedrock model: their
model/load.py was agent-python-strands' Bedrock branch with the id inlined,
pyproject pinned plain strands-agents, and the shortcuts declared no
override, so --model-provider/--model-id/--api-key (and now the wizard's
model step) were refused. In a China region that left them unusable — the
gates treat an override-less model template as Bedrock.

Give both templates the strands model module (provider branches, safeJson
model id, requires_api_key identity decorator) and the per-provider
strands-agents[extra] dependency block, and flip supportsModelProviderOverride.
The A2A resolver already wired the API-key credential scaffold but did not
pass modelProvider/modelId to the renderer; the AG-UI resolver did neither —
both now render and merge the model scaffold exactly like the HTTP strands
template. Bedrock scaffolds are unchanged byte for byte.

`--model-provider lite_llm --model-id deepseek/deepseek-chat` is now the
China route for A2A and AG-UI agents, same as for agent-python-strands.
…pt strands providers

Add `--api-base <url>` to `agentcore create` and `agentcore add runtime`,
valid with `--model-provider open_ai`: it points the OpenAI client at any
OpenAI-compatible endpoint instead of api.openai.com (Python: client_args
base_url; TypeScript: clientConfig.baseURL). The URL is persisted as the
runtime's `modelApiBase` in agentcore.json, next to modelProvider/modelId,
so the deploy-time China gate can classify it. Other providers reject the
flag — they keep their own APIs, and one escape hatch (plus LiteLLM) covers
every OpenAI-format vendor.

China (aws-cn): an OpenAI runtime pointed at a base URL passes the create,
add and deploy gates; one without is refused (api.openai.com is not
reachable). The three gates now share one rule, chinaModelProviderRestriction,
so they cannot drift.

Enable model-provider overrides on agent-typescript-strands. model/load.ts
becomes a provider-branched template (Bedrock, Anthropic, OpenAI, Gemini via
@strands-agents/sdk models/*), the API key is fetched through
bedrock-agentcore/identity withApiKey in a new model/apiKey.ts (rendered
only for API-key providers; LOCAL_DEV reads .env.local like the Python
templates), and the provider's SDK peer dependency is added to package.json
conditionally. LiteLLM is a Python library, so `lite_llm` is refused for
TypeScript templates with a pointer to open_ai + --api-base; the wizard does
not offer it there.

Wizard: the model step gains an "API base URL" field for openai (optional,
required in China regions); in China a TypeScript template starts on openai
with an empty model id, a Python template on litellm as before.
… export telemetry

Every pinned Python template carried `aws-opentelemetry-distro ~= 0.18.0`
(langchain `~= 0.19.0`). Those releases build the default OTLP endpoints
as `https://logs.<region>.amazonaws.com/v1/logs` (and the xray equivalent)
whenever the runtime injects none — the AgentCore Runtime only injects
endpoints on its local-collector path — so an agent in cn-north-1 or
cn-northwest-1 logs a NameResolutionError on every export and delivers
no logs or traces. 0.21.0 (2026-10-01) derives the partition suffix from
the region (`amazonaws.com.cn` for `cn-*`) and accepts it in its endpoint
patterns.

Move all seven pinned templates to `~= 0.21.0`. Resolution verified with
`uv lock` for the strands (LiteLLM) and langchain templates
(opentelemetry-api 1.44.0, opentelemetry-instrumentation-langchain 0.62.4).
bedrock-managed-agents was already unpinned.
FastAPI 0.142 configures OTLP export from the OTEL_* environment at
ASGI lifespan startup and adds its own exporters to the providers the
OpenTelemetry distro already set up. On the AgentCore Runtime the
distro signs its requests with SigV4; FastAPI's exporters do not, so
every trace and log batch was sent twice and the unsigned copy failed
with 403 MissingAuthenticationTokenException after each invocation.

ag_ui_strands.create_strands_app takes no FastAPI options, so the
AG-UI template now assembles the same app itself with
telemetry={"auto_configure": False}: the CORS middleware, the AG-UI
endpoint at /invocations, and /ping. fastapi is pinned to >= 0.142,
where the telemetry argument exists. The other templates are Starlette
apps and are unaffected.
@github-actions github-actions Bot added the size/xl PR size: XL label Oct 2, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added agentcore-harness-reviewing AgentCore Harness review in progress claude-security-reviewing Claude Code /security-review in progress labels Oct 2, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Oct 2, 2026

@agentcore-devx-automation agentcore-devx-automation Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AgentCore Harness Review

Verdict: Looks good

Reviewed the five commits as a unit. The changes hang together: supportsModelProviderOverride is extended consistently (A2A, AG-UI, TypeScript strands), the shortcut resolver, scaffold schema, flag handlers, both wizards, the AG-UI app assembly fix, the OTel distro bump, and the China gates all move in lockstep. A few things I specifically looked for and liked:

  • Gates share one rule. chinaModelProviderRestriction is used by validateCreateRegionSupport, FsProjectManager.addResource, and the deploy gate uses the same classification against modelApiBase persisted in agentcore.json. No chance of the three places drifting.
  • Secret handling. The TUI never holds the API key — only the file:// source; the key is resolved via SourceResolver at submit, so wizard output matches the --api-key flag path. The review surface asserts the raw key is not rendered.
  • Scaffold parity. Both wizards hand the answer to resolveRuntimeTemplateShortcut, and tests assert byte-for-byte equivalence with the flag path (inputs deep-equals the shortcut result; default model id intentionally not sent as an override).
  • AG-UI fix is targeted. Replacing create_strands_app with explicit FastAPI(..., telemetry={"auto_configure": False}) + add_strands_fastapi_endpoint / add_ping + fastapi >= 0.142 is the smallest change that stops the double-export, and the test asserts both the pin and the absence of create_strands_app(.
  • Memory module is always rendered now (previously filtered out when the scaffold had no memory). I confirmed a2a-python-strands/memory/session.py and agent-typescript-strands/memory/memory.ts both return None / null when the env var is missing, so the China path is unchanged behaviorally and the comment is accurate.
  • Tests use real temp dirs and real file I/O, no excessive mocking at module boundaries; the new tests (API-key file resolution, provider switching, China deploy gate for OpenAI with/without modelApiBase, AG-UI assembly) exercise the actual scaffolder and project manager.

No telemetry instrumentation gap: this repo records command paths centrally in router/router.tsx (recordCommandPath), not per-handler, so there is nothing new to wire in the create/add handlers.

Nothing blocking from me. 🚢

@agentcore-devx-automation agentcore-devx-automation Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Oct 2, 2026
@codecov-commenter

codecov-commenter commented Oct 2, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.43%. Comparing base (70a932c) to head (10c36e0).
⚠️ Report is 2 commits behind head on refactor.

Additional details and impacted files
@@             Coverage Diff              @@
##           refactor    #2507      +/-   ##
============================================
+ Coverage     97.40%   97.43%   +0.03%     
============================================
  Files           642      643       +1     
  Lines         46865    47335     +470     
============================================
+ Hits          45647    46122     +475     
+ Misses         1218     1213       -5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

When a value typed into a model-step field wraps onto a second row, the
field grows and the content overflows the viewport by that row. The
scroll-into-view callback then scrolled down, the measured viewport
grew by exactly the offset scrolled, the callback re-derived its target
from the anchor and scrolled back, the viewport shrank again — until
React stopped it with "Maximum update depth exceeded" and the wizard
went blank. CI hit it on macOS and Windows, whose temp paths are long
enough for the file:// API-key path to wrap at 100 columns.

The callback now scrolls back toward the anchor only if the field fits
there after discounting the rows the viewport gained from the offset,
otherwise stays put when the field is fully visible, and scrolls
forward by the usual amount. Fields revealed before the viewport was
measured still re-anchor once it has grown. Same change in the harness
wizard's model field, which shares the callback. The two file:// tests
compare the review row with whitespace removed, since the long path
wraps there too, and a new test types a path that wraps.
@github-actions github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Oct 2, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Oct 2, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Oct 2, 2026
Codecov on #2507 flagged the runtime model step's esc-on-the-list,
arrow moves between fields, and the return-to-a-missing-field branch
as untested. One wizard test walks all three.
@github-actions github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Oct 2, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Oct 3, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Oct 3, 2026
@github-actions github-actions Bot removed the size/xl PR size: XL label Oct 3, 2026
@github-actions github-actions Bot added the size/xl PR size: XL label Oct 3, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xl PR size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants