Skip to content

copilot: route ug copilot through Microsoft Foundry provider services - #1267

Open
xi-db wants to merge 8 commits into
mainfrom
xi-db/stack/copilot-foundry
Open

xi-db wants to merge 8 commits into
mainfrom
xi-db/stack/copilot-foundry

Conversation

@xi-db

@xi-db xi-db commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

🥞 Stacked PR

Use this link to review incremental changes.


What did you change, and why?

Change: ug copilot --provider <catalog>.<schema>.<name> now accepts Microsoft Foundry (microsoft_foundry) Model Provider Services, as Codex already does. Before, Copilot refused them ("can't route to ... microsoft_foundry").

  • Wire per target. Foundry hosts more than OpenAI models (Claude, DeepSeek, GPT), and its targets declare only openai/v1/chat/completions, including the Claude ones.
    • Claude targets use the Anthropic wire (dbx-anthropic on /ai-gateway/anthropic), whatever they declare. Foundry answers Claude on Anthropic Messages and Responses but returns 404 on Chat Completions. On the Anthropic wire Copilot also sends its Claude request settings, and limits are capped at 200K as for other provider services.
    • GPT 5+ and o-series targets use Responses, and everything else uses Chat Completions, on /ai-gateway/openai/v1, as for OpenAI and Azure OpenAI services.
  • Selection. ug configure now offers Foundry services for Copilot under External Models, and a managed model_provider_service of this type applies to Copilot.

How do you know it works?

Testing:

  • Unit:
    • classifier: a Foundry Claude target on Anthropic, gpt-5-nano on Responses, DeepSeek on Chat Completions;
    • catalog: Foundry targets that declare only Chat Completions keep Claude on Anthropic;
    • service_usable_for_tool and supports_provider_service accept microsoft_foundry for Copilot.
    • just lint is clean.
  • Integration (tests/integration/test_ug_copilot_providers.py, non-blocking Copilot lane): a file task through the existing Foundry MPS fixture on gpt-5-nano (the one Codex's Foundry journey uses), on dbx-responses, with the answering model checked from Copilot's session events.
  • Live (ug from this branch, three Foundry services on a Databricks workspace, API-key and Entra auth):
Foundry target Copilot wire Result
claude-opus-4-8, and the default start model Anthropic ✅ OK
deepseek Chat Completions ✅ OK
gpt-5.4-nano Responses ✅ OK

A direct request per gateway route for the same targets: Claude answered on Anthropic Messages and Responses, and 404'd on Chat Completions; DeepSeek answered on Chat Completions and Responses, and 404'd on Anthropic Messages.

Stacked on #1256.

This pull request and its description were written by Isaac.

xi-db and others added 4 commits October 23, 2026 14:49
ug copilot --model-location <catalog>.<schema> registers the schema's model
services in the session registry, from the gateway's Anthropic model listing
as for Claude Code. Services are sent by full name; the gateway routes them
as model services. Services named like Claude or GPT models use their usual
wire (GPT 5+ on Responses, since GPT 5+ rejects the temperature Copilot sends
on the Anthropic wire); the rest use the Anthropic wire, which the gateway
translates for other models. The saved Custom Model Location and the managed
unity_catalog_location apply to Copilot like Claude Code and Codex.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Under a managed unity_catalog_location or model_provider_service, Copilot
dropped the budget recommendation and the managed default_model. README now
lists Copilot for the saved Custom Model Location.

Co-authored-by: Isaac <no-reply@databricks.com>
…Code and Codex

Unit tests for the saved location and its precedence with --provider, a managed location or
provider, the configure picker, and workspace and single-agent configure. Single-agent
`ug configure --agent copilot` now applies the managed model source like the other location
agents. The launch branch per agent uses the AGENT_* constants.

Co-authored-by: Isaac <no-reply@databricks.com>
@xi-db xi-db added the ug-review Run the automated UG review label Oct 23, 2026
Comment thread src/ucode/agents/copilot.py Outdated
Comment thread src/ucode/agents/copilot.py
Comment thread src/ucode/cli.py Outdated
Comment thread tests/test_cli.py
… for ug agents add

The location catalog strips gateway aliases with smart routing's
unwrap_anthropic_gateway_model instead of its own pattern. The single-agent AI-tools install
forces a managed-config refresh unless one already ran, so ug agents add copilot keeps it.

Co-authored-by: Isaac <no-reply@databricks.com>
@xi-db
xi-db force-pushed the xi-db/stack/copilot-foundry branch from f1c6dd0 to c925d9e Compare October 23, 2026 15:37
@github-actions

Copy link
Copy Markdown
Contributor

Polly AI Review

Blocking issues

  • P2 — first-run location launch fails without global models. can we forward the selected schema through Copilot bootstrap? a valid custom location currently fails global availability before discovery.
    Evidence: src/ucode/cli.py:4147 introduces the location handoff, but bootstrap at src/ucode/cli.py:3279 forwards only provider. With fresh state and no global Copilot models, configuration rejects the workspace before scoped discovery. The shared bootstrap gap predates this PR, but this Copilot entry point is new.
    Regression: add tests/test_cli.py::TestAutoConfigureOnFirstRun.test_first_run_configures_through_launch_model_location, retaining real bootstrap and availability checks with an empty global catalog.

Security vulnerabilities

None identified in the changed code. This is a static assessment, not live security validation.

Non-blocking notes

  • can we parameterize test_copilot_provider_and_model_location_are_mutually_exclusive with its existing Claude/Codex counterparts, retaining both assertions?
    tests/test_cli.py:1259
  • is Copilot model-location support intentionally included from the parent stack? it adds configure, persistence, and launch behavior beyond Foundry-provider acceptance.
    src/ucode/cli.py:4133, src/ucode/agents/copilot.py:388

Approach

The Foundry approach is consistent with existing provider-support and wire-classification patterns. No materially simpler implementation is apparent. The location feature also reuses existing discovery; its bootstrap handoff needs completion.

Summary

Diff size (computed): 14 files; +498 / -55 lines (553 changed text lines); 0 binary files.

Category Files Added Deleted Share of changed text lines
Tests and test support 7 371 12 69.3%
Documentation 1 8 5 2.4%
Dependencies and lockfiles 0 0 0 0.0%
Implementation / other 6 119 38 28.4%

Shares use additions + deletions, including generated text and lockfiles. Binary files have no line count. Tests include fixtures/helpers under test paths and colocated test/spec files; categories are path-based, not coverage measurements.

Changes

The intended outcome is to allow Copilot to use Microsoft Foundry provider services, selecting Anthropic for Claude, Responses for GPT-5+/o-series, and Chat Completions otherwise.

The diff contains implementation, unit/component tests, live integration scenarios, and documentation. It also includes stacked Copilot model-location support: schema discovery, saved and managed source selection, and transient launch state. No dependency or generated-file changes are included.

The Foundry changes are proportional to the outcome. The location work explains much of the additional breadth, subject to the scope question above. The concrete reduction opportunity is shared CLI conflict-test setup rather than removing distinct lifecycle coverage.

Tests

Test-by-test assessment
Test / case Behavior protected Layer Needed? Action / rationale
tests/integration/README.md — journey descriptions, inventory, fixture entry, and count updates Documents execution inputs and coverage Documentation keep Supports the added journeys; collected-case totals were not independently verified.
tests/integration/test_ug_copilot_model_location.py::test_ug_copilot_runs_a_task_on_a_model_location Fresh CLI launch, selected full model name, inference, cleanup Live process/transport keep Exercises the real discovery-to-Copilot boundary using existing primitives. Its workspace does not isolate absence of global models.
tests/integration/test_ug_copilot_providers.py::test_ug_copilot_runs_a_task_through_a_microsoft_foundry_provider Foundry GPT inference through Responses, model correlation, cleanup Live process/transport keep Establishes the external Foundry transport boundary, beyond mocked classification.
tests/test_agent_copilot.py::_stub_location, nested anthropic_catalog, and AnthropicModelCatalog import Explicit catalog inputs and schema observation Unit support keep Small reusable discovery seam; no additional global fixture machinery.
tests/test_agent_copilot.py::TestClassifyProviderModel.test_provider_type_and_id_decide_the_wire — Foundry Claude, GPT-5-nano, DeepSeek parameters Anthropic, Responses, and Chat Completions classification Unit keep Places the provider/model matrix at the lowest reliable layer.
tests/test_agent_copilot.py::TestBuildProviderCatalog.test_foundry_claude_uses_anthropic_whatever_it_declares Chat-only declarations cannot misroute Foundry Claude; mixed catalog remains usable Unit keep Protects the specific declaration-override regression beyond name-only classification.
tests/test_agent_copilot.py::TestLocationCatalog.test_registers_the_schemas_services_by_full_name Alias removal, full names, wire selection, filtering, schema forwarding Unit keep Covers discovery transformation not supplied by provider-catalog tests.
tests/test_agent_copilot.py::TestLocationCatalog.test_an_empty_schema_says_so Empty discovery produces a location-specific error Unit keep Covers a material failure path without live setup.
tests/test_agent_copilot.py::TestLaunch.test_model_location_launch_registers_the_schemas_services Registry composition, stale-model override, no provider-service header, token cap, start model, marker consumption Component keep Protects launch composition beyond isolated catalog conversion.
tests/test_cli.py::TestSubcommandRouting.test_copilot_launch_uses_saved_model_location Saved location reaches launch CLI component keep Covers persistence-to-launch handoff.
tests/test_cli.py::TestSubcommandRouting.test_copilot_explicit_provider_overrides_saved_location Explicit provider suppresses saved location CLI component keep Checks source precedence and absence of a competing marker.
tests/test_cli.py::TestSubcommandRouting.test_copilot_explicit_model_location_overrides_saved_provider Explicit location suppresses saved provider CLI component keep Protects the opposite source transition.
tests/test_cli.py::TestSubcommandRouting.test_copilot_saved_model_location_is_ignored_under_managed_config Developer state cannot override managed configuration CLI component keep Covers administrator/developer configuration ownership.
tests/test_cli.py::TestSubcommandRouting.test_copilot_provider_and_model_location_are_mutually_exclusive Conflicting source flags fail with actionable output CLI component consolidate Parameterize with the existing Claude/Codex cases, preserving exit-code and error assertions.
tests/test_cli.py::TestSubcommandRouting.test_invalid_model_location_is_rejected[copilot] Malformed schema is rejected CLI component keep Extends the existing input matrix without new setup.
tests/test_cli.py::TestClaudeModelFlag.test_model_location_sets_transient_copilot_launch_marker Explicit location reaches launch without configuring a global model CLI component keep Checks option/configure plumbing, but mocked setup cannot expose the bootstrap failure.
tests/test_cli.py::TestCopilotLaunchModel.test_managed_default_is_the_launch_model_under_a_managed_source — both source parameters Managed default survives provider-service and location selection CLI component keep Both named-source branches need the launch-model handoff.
tests/test_cli.py::TestCopilotLaunchModel.test_managed_source_overrides_saved_provider[mps] and [uc-parent] Managed sources supersede saved sources and bypass global discovery CLI component keep Retains distinct source-specific marker and ownership assertions.
tests/test_cli.py::TestConfigureAgentsSelection.test_picker_offers_location_for_copilot Picker offers location and clears the competing provider Component keep Covers the newly enabled configure entry point.
tests/test_cli.py::TestConfigureAgentsSelection.test_managed_copilot_parent_is_passed_to_generic_configure Multi-agent managed configuration forwards the schema CLI component keep Protects the generic configure entry point against global-availability dependence.
tests/test_cli.py::TestConfigureAgentsSelection.test_single_copilot_agent_passes_managed_parent_directly Single-agent configuration applies managed schema and refresh ownership CLI component keep Separate public entry point with additional refresh and displayed-source assertions.
tests/test_databricks.py::TestServiceUsableForTool.test_copilot_routes_anthropic_and_openai_services[microsoft_foundry] Shared filtering accepts Foundry for Copilot Unit keep Protects discoverability and validation independently of wire classification.
tests/test_databricks.py::TestServiceUsableForTool.test_copilot_skips_other_provider_types — removed Foundry negative parameter Remaining unsupported providers stay rejected Unit keep Foundry moves to positive coverage; Gemini Enterprise and custom-provider rejection remain.
tests/test_managed_setup.py::TestProviderServiceSupport.test_copilot_supports_anthropic_openai_bedrock_and_foundry — renamed existing case Managed setup accepts Foundry and still rejects unsupported providers Unit keep Extends the existing scenario; replaces the obsolete Foundry rejection with Gemini Enterprise.

Existing unchanged tests cover provider headers, native endpoints, bearer handling, Claude token limits, catalog failures, model fallback, and launch cleanup. These provide useful coverage without duplicating every assertion in live tests.

The concrete missing regression is the first-run location path identified above. Existing first-run tests mock configuration and therefore miss its availability failure. A Copilot-only component test with real bootstrap is sufficient; another live scenario is not required for that bug.

The new Foundry live scenario exercises GPT/Responses, not Claude or DeepSeek transport. Classification and declaration-override assertions cover those routing decisions locally; the reported manual transport checks were not independently repeated.

The frozen snapshot checksum and both required policy files were verified. No tests, collection, lint, or typecheck were executed; no passing result is claimed.

To confirm the bootstrap fix manually, use an isolated fresh UG profile and a workspace with no global Copilot models but a usable service in the selected schema:

ug copilot --workspace <workspace> --model-location <catalog.schema> --model <service-full-name> -p "reply ok"

The selected schema should permit startup instead of producing the global availability error.

Scope

Foundry acceptance, classification, selection, and associated documentation support the stated outcome. Copilot model-location support is coherent with the parent-stack context but is not explicitly included in the Foundry description; its inclusion needs clarification. No clearly unrelated changes were established.


Automated review by Polly · workflow run

xi-db and others added 2 commits October 23, 2026 15:54
…gents

Co-authored-by: Isaac <no-reply@databricks.com>
Foundry's Claude targets use the Anthropic wire whatever they declare (Foundry serves Claude on
Anthropic Messages, not Chat Completions); its GPT 5+ and o-series targets use Responses and the
rest Chat Completions, as for OpenAI and Azure OpenAI services.

Co-authored-by: Isaac <no-reply@databricks.com>
@xi-db
xi-db force-pushed the xi-db/stack/copilot-foundry branch from c925d9e to 2c81f2a Compare October 23, 2026 15:54
@xi-db

xi-db commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for the review.

  • P2, first-run --model-location without global models: leaving this as is. _auto_configure_tool is shared, so ug claude --model-location and ug codex --model-location hit the same availability check on a fresh setup today; Copilot matches them. Fixing the bootstrap for all three agents is better as its own change than inside the Foundry PR.
  • Parametrize the --provider + --model-location test: done in e9f4e4e (on copilot: run ug copilot on a Unity Catalog model location #1256): one test now covers Claude, Codex and Copilot.
  • Model-location scope: that comes from the parent PR, copilot: run ug copilot on a Unity Catalog model location #1256. This PR is stacked on it and targets main, so the diff includes it; the "Files changed" link in the stack section shows only the Foundry change.

Co-authored-by: Isaac <no-reply@databricks.com>
@xi-db
xi-db marked this pull request as ready for review October 23, 2026 16:04
@xi-db
xi-db requested a review from masonc08 October 23, 2026 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ug-review Run the automated UG review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants