Show the smart-routing savings estimate in the Claude mod by JJ27 · Pull Request #850 · databricks/unity-gateway

JJ27 · 2026-09-25T21:14:36Z

Summary

Adds a smart-routing savings estimate to ug's Claude Code mod, drawn as a row above the prompt. It's opt-in: SMART_ROUTER_CONFIG_VERSION=subagent_orch_savings_v0 turns it on with subagent routing and orchestration, or set ENABLE_SMART_ROUTING_SAVINGS=1 with routing enabled. Every existing preset sets the flag to 0.

💰 Est. saved with smart routing  $0.18  (48%) — Unity Gateway Smart Router v1.4.0
Smart routing cost  $0.04  more (10%) — Unity Gateway Smart Router v1.4.0

The row is styled like the status band from Hide the Smart Router band after one turn; show on/off in the prompt hint row #1138: a bold label, the amount in a green (saved) or yellow (cost more) pill, and a dim italic trailer.
It appears only once there is an estimate: a request has run on a model other than its baseline, and every model involved has a rate. Until then nothing is drawn, since the status band and prompt hint already show routing state.
On the first turn the row sits under the status band. After the band hides, the row stays on its own.
The trailer's version comes from skills/smart-router-orchestrator/SKILL.claude.md (metadata.version) and is shown only when ENABLE_SMART_ROUTER_ORCHESTRATOR=1.
/smart-router off stops tracking and hides the row; /smart-router on restores both.

This replaces the earlier transcript-statusline approach. It's built on the mods scaffolding in main and merged with #1138 and #1154.

Screenshots

Pending. Prod endpoint-rates still returns 404, so the screenshots use seeded list-price rates, not live endpoint rates.

How it works

Accounting (mod, hooks/register.ts): on each turn.step, for the main agent and subagents, the mod adds the request's token usage to totals grouped by agent, baseline, served model, and whether the request ran before a user model switch.
- Served is the model service the request named.
- The main agent's baseline is its own served model. A subagent's baseline is the main model in effect at the time.
- Claude Code's side queries (titles, compaction) aren't turn steps, so they never count.
- The savings hooks share register.ts with the routing controls, because Claude allows one hooks module per plugin.
Pricing (Python, ucode.smart_routing.savings): a hooks module has no Node APIs, and the price table lives in Python. So after each turn the mod writes its totals to a per-launch usage file and runs the pricer command on it. The pricer prints the estimate's parts ({"savings": {"saved", "amount", "percent"} | null}) as one JSON line. It computes:
- saved = Σ tokens × price(baseline) − Σ tokens × price(served), per token class: input, output, cache read, and 5-minute and 1-hour cache writes.
- Cache writes without the 5m/1h split are priced at the 1-hour rate, since ug launches Claude with 1-hour caching.
Coexisting with the status band: the band holds turn.complete's one unmatched hook, so the savings hook matches every turn with { isAborted: [false, true] }. The band's file is unchanged from main.
First-prompt routing: requests before a user switch are priced against the session's start model. After a /model, requests are priced against their own baseline.
Plugin options: ug passes the pricer argv, the usage file path, and the orchestrator version to the mod as pluginConfigs options. These are declared with defaults in the generated plugin's userConfig. The pricer is a multiple: true list, so paths with spaces and unicode reach the mod intact.
Hidden rather than undercounted: if any entry is unpriced or malformed, or there are no prices yet, there is no estimate and no row.

Price source

Each savings-enabled routed launch refreshes prices on a background thread from GET /api/ai-gateway/v2/endpoint-rates:batchGet, with repeated databricks_hosted_model_services params covering the router's and main agent's model services. Results are cached per workspace at ~/.ucode/model-prices-<host hash>.json.

The client reads the USD cost group: per-million-token rates for input, output, cache reads, and 5m/1h cache writes.
A failed refresh keeps the existing cache.
Served ids are matched back to system.ai names. This covers Bedrock-style ids, -colo-on-* deployments, and dotted versions.
This was validated end-to-end against the staging endpoint. Prod still returns 404 ("Pay-per-token endpoint rate lookup is not enabled"), so no real launch shows an estimate until that route is enabled. No client change is needed then.

Testing

703 focused Python tests passed. Retained coverage includes pricing calculations, cache/rate parsing, real pricer subprocess output, preset precedence and restoration, and packaging/integration contracts. The preset matrix covers unset, enabled, disabled, and conflicting inherited flags without the redundant full Cartesian product.
Claude Code 2.1.290: all 8 native suite checks passed. They validate the mod, option round-tripping, launch-local settings, and usage-file behavior.
Ruff check, Ruff format check, and Ty passed.
Prior manual checks with seeded rates covered the first-turn band and savings row, subsequent turns, routing off/on, the cost-more row, and the no-prices state. These were not rerun for this test cleanup.
Live token accumulation, savings-row rendering, endpoint-refresh failures, and the orchestrator-version display have no automated coverage. The mocked UI/launch/refresh cases are no longer part of this PR. Live endpoint rates remain subject to the production availability noted above.

Also in this PR: tracing CI fix

The Claude and Codex tracing journeys were timing out (SQL statement did not finish within 180 seconds). tests/integration/utils/sql.py picked the first warehouse when none was running, then counted warehouse startup against the query's 180s budget. The fix:

prefer a serverless warehouse;
allow up to 10 minutes in PENDING before the 180s execution limit starts;
report which phase timed out.

This can move to its own PR if reviewers prefer.

This pull request and its description were written by Isaac.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Carries PR #850's statusline, pricing, and endpoint-rates client onto the Claude mods stack. Conflicts with the mods branch are resolved by keeping both sides; ENABLE_SMART_ROUTING_SAVINGS stays in v2 since the routing flags moved to ucode.constants. Co-authored-by: Isaac <no-reply@databricks.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

The launch-scoped ug-smart-router mod now draws a dim row above the prompt with the estimated savings from subagent smart routing and, when Smart Router Orchestrator is active, the skill's version from SKILL.claude.md. The mod sums each turn request's tokens by served and baseline model and runs a stdlib-only pricer against cached endpoint rates, which each routed launch refreshes in the background. The pricer, usage file and version reach the mod as plugin options. ENABLE_SMART_ROUTING_SAVINGS=0 opts a launch out. Co-authored-by: Isaac <no-reply@databricks.com>

lilly-luo

can you add ENABLE_SMART_ROUTING_SAVINGS to config.py

unity-gateway/src/ucode/smart_routing/config.py

Line 36 in f05ef1e

_BASE_VERSIONS = {

, make it 0 for all the configurations, then add 1 more configuration that makes this 1?

# Conflicts: # src/ucode/agents/claude_mods/register.ts # src/ucode/smart_routing/v2.py # tests/README.md # tests/test_claude_mod.py

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Co-authored-by: Isaac <no-reply@databricks.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Co-authored-by: Isaac <no-reply@databricks.com>

Label the version as Unity Gateway Smart Router, show the amount in a green (saved) or yellow (cost more) pill, and render the row after the other AbovePrompt handlers so it sits under the band in either order. The pricer now prints the estimate's parts instead of preformatted text. Co-authored-by: Isaac <no-reply@databricks.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Claude allows one unmatched hook per event, and the status band already has the turn.complete one, so the savings hook matches every turn on isAborted instead of the band exporting its hide logic to register.ts. Co-authored-by: Isaac <no-reply@databricks.com>

The status band and prompt hint row already show the routing state, so a version-only row just repeated the band. The orchestrator version now rides along with the estimate. Adds a mod test for both states. Co-authored-by: Isaac <no-reply@databricks.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

JJ27 marked this pull request as ready for review September 29, 2026 23:28

Copilot AI balanced review requested due to automatic review settings September 29, 2026 23:28

Copilot AI reviewed Sep 29, 2026

Copilot AI balanced review requested due to automatic review settings September 29, 2026 23:33

Copilot AI reviewed Sep 29, 2026

lilly-luo reviewed Oct 1, 2026

View reviewed changes

Comment thread src/ucode/smart_routing/claude_statusline.py Outdated

JJ27 force-pushed the josh-joseph/smart-routing-savings-statusline branch from e547da8 to 6f8dc9c Compare October 6, 2026 05:20

Copilot AI balanced review requested due to automatic review settings October 6, 2026 05:20

JJ27 changed the base branch from main to claude-mods-subagent-routing October 6, 2026 05:20

Copilot AI reviewed Oct 6, 2026

JJ27 force-pushed the josh-joseph/smart-routing-savings-statusline branch from 63aa3f9 to da5c0b9 Compare October 12, 2026 20:22

JJ27 requested review from AarushiShah-db and rohita5l as code owners October 12, 2026 20:22

JJ27 changed the title ~~Add Claude Code statusline estimating smart-routing savings (client side)~~ Show the smart-routing savings estimate in the Claude mod Oct 12, 2026

JJ27 changed the base branch from claude-mods-subagent-routing to main October 12, 2026 20:23

lilly-luo added the ug-review Run the automated UG review label Oct 13, 2026

lilly-luo reviewed Oct 13, 2026

View reviewed changes

lilly-luo added 2 commits October 13, 2026 21:59

Merge branch 'main' into josh-joseph/smart-routing-savings-statusline

9f98a6f

# Conflicts: # src/ucode/agents/claude_mods/register.ts # src/ucode/smart_routing/v2.py # tests/README.md # tests/test_claude_mod.py

Gate smart routing savings behind explicit opt-in

0ce99ee

Copilot AI balanced review requested due to automatic review settings October 13, 2026 22:17

Copilot AI reviewed Oct 13, 2026

JJ27 and others added 2 commits October 13, 2026 22:24

Redraw the savings row after /smart-router on|off

a5a3294

Co-authored-by: Isaac <no-reply@databricks.com>

Expect the savings flag in the Codex smart-router off snapshot

58b4e47

Co-authored-by: Isaac <no-reply@databricks.com>

Copilot AI balanced review requested due to automatic review settings October 14, 2026 00:08

Copilot AI reviewed Oct 14, 2026

JJ27 and others added 2 commits October 14, 2026 17:14

Merge branch 'main' into josh-joseph/smart-routing-savings-statusline

22172bc

Co-authored-by: Isaac <no-reply@databricks.com>

Copilot AI balanced review requested due to automatic review settings October 14, 2026 17:22

Copilot AI reviewed Oct 14, 2026

Merge main into Smart Router savings

2e2550c

Copilot AI balanced review requested due to automatic review settings October 16, 2026 22:56

Copilot AI reviewed Oct 16, 2026

JJ27 and others added 3 commits October 16, 2026 23:18

Merge branch 'main' into josh-joseph/smart-routing-savings-statusline

6c4c8a4

Copilot AI balanced review requested due to automatic review settings October 18, 2026 00:34

Copilot AI reviewed Oct 18, 2026

Trim mocked and redundant savings tests

0f4b3e1

Copilot AI balanced review requested due to automatic review settings October 19, 2026 04:02

Copilot AI reviewed Oct 19, 2026

Navigation Menu

Repository navigation

Show the smart-routing savings estimate in the Claude mod - #850

Show the smart-routing savings estimate in the Claude mod#850
JJ27 wants to merge 12 commits into
mainfrom
josh-joseph/smart-routing-savings-statusline

JJ27 commented Sep 25, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

lilly-luo left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

JJ27 commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Screenshots

How it works

Price source

Testing

Also in this PR: tracing CI fix

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

lilly-luo left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

JJ27 commented Sep 25, 2026 •

edited

Loading