Skip to content

Charge xAI's real rate for the slugs it retired on 2026-05-15 - #166

Merged
adambalogh merged 1 commit into
mainfrom
og/nice-shannon-1g40i8
Sep 21, 2026
Merged

adambalogh merged 1 commit into
mainfrom
og/nice-shannon-1g40i8

Conversation

@adambalogh

Copy link
Copy Markdown
Contributor

Found while syncing the SDK's TEE_LLM enum against this registry (OpenGradient/OpenGradient-SDK#309).

The bug

xAI retired grok-4, grok-4-fast, grok-4-1-fast, grok-4-1-fast-non-reasoning, grok-code-fast-1 and grok-3 on 2026-05-15 (retirement notice). It still accepts every one of those slugs — each is silently redirected to a live model (grok-4.3, or grok-build-0.1 for grok-code-fast-1) and, crucially, billed at grok-4.3's rate of $1.25 / $2.50 per MTok regardless of which name was sent.

The registry still carried their pre-retirement prices, so the gateway was charging callers less than xAI charges us:

Model Gateway charged (in/out per MTok) xAI bills Effect
grok-4-fast $0.20 / $0.50 $1.25 / $2.50 6.25x under on input, 5x on output
grok-4-1-fast $0.20 / $0.50 $1.25 / $2.50 6.25x / 5x under
grok-4-1-fast-non-reasoning $0.20 / $0.50 $1.25 / $2.50 6.25x / 5x under
grok-code-fast-1 $0.20 / $1.50 $1.25 / $2.50 6.25x / 1.7x under
grok-3-mini $0.30 / $0.50 $1.25 / $2.50 under
grok-4 $3.00 / $15.00 $1.25 / $2.50 overcharged callers
grok-3 $3.00 / $15.00 $1.25 / $2.50 overcharged callers

The fix

All seven now price at _XAI_RETIRED_INPUT_USD / _XAI_RETIRED_OUTPUT_USD, i.e. grok-4.3's rate.

They stay registered rather than being removed, so older SDK versions keep working — the SDK has dropped them from TEE_LLM in the companion PR — and they're marked for removal once no client sends them. grok-3-mini isn't named in xAI's notice but is gone from their models page, so it's priced the same way rather than left at its old rate.

Also in this pass

Audited every provider's deprecation list while I was here:

Corrected the Gemini note. It claimed gemini-2.5-flash/-pro/-flash-lite were "scheduled for deprecation on June 17, 2026 (flash-lite: July 22, 2026)". Those dates have passed and Google's list doesn't carry these models — they're still served. The only Gemini retirement that matters here is gemini-3.1-flash-image-preview, and the registry already calls the GA gemini-3.1-flash-image.

Recorded the announced OpenAI shutdowns next to the affected entries (deprecations):

  • gpt-4.1-nano and o4-mini stop resolving 2026-10-23. OpenAI names those exact bare slugs, not dated snapshots, so they hard-fail that day. Replacements gpt-5.6-luna / gpt-5.6-terra are already registered.
  • o3, gpt-5, gpt-5-mini are undated aliases of snapshots retiring 2026-12-11. OpenAI does not document whether such an alias is repointed to a successor or retired with its snapshot. If it's repointed, these become mispriced exactly the way the xAI slugs were — worth a calendar reminder.

Fixed the model lists in CLAUDE.md and README.md, which advertised models the registry doesn't accept: claude-sonnet-4-0, claude-3-7-sonnet, claude-3-5-haiku, grok-2 and gemini-3-pro-preview are all absent from _MODEL_LOOKUP, and the README's completion example posted "claude-3.7-sonnet". Both lists are now checked against the registry, and the retired-but-registered models are documented in one place.

For the record, no Anthropic model in the registry is retired — all 11 resolve to models Anthropic lists as Active, checked at the api_name level.

Testing

  • 457 passed, 6 skipped on the CI selection (tee_gateway/test/, tests/test_pricing.py, tests/test_opengradient_field.py)
  • make lint clean (ruff + mypy, 61 source files)
  • The five pricing tests that pinned the old Grok rates now pin the new ones, plus a new test_retired_xai_slugs_priced_at_grok_4_3_rate asserting the whole set matches grok-4.3. test_grok_4_fast_cheaper_than_grok_4 became vacuous (both retired, now identical) and is replaced by the same comparison over the live tiers, grok-4.3 vs grok-4.6.

PCR note

This touches model_registry.py, which is baked into the image, so PCRs change. measurements.txt is not updated here — matching every prior model-registry PR (e.g. #160, #153), which touch this same set of four files and leave it alone.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QmnaiLePKAQEkNfR8VPfja


Generated by Claude Code

xAI retired grok-4, grok-4-fast, grok-4-1-fast,
grok-4-1-fast-non-reasoning, grok-code-fast-1 and grok-3 on 2026-05-15
(docs.x.ai/developers/migration/may-15-retirement). It still ACCEPTS every
one of those slugs — each is silently redirected to a live model (grok-4.3,
or grok-build-0.1 for grok-code-fast-1) and, crucially, billed at grok-4.3's
rate of $1.25/$2.50 per MTok no matter which name was sent.

The registry still carried their pre-retirement prices, so the gateway was
charging callers less than xAI charges us:

  grok-4-fast                  $0.20/$0.50  ->  6.25x under on input, 5x on output
  grok-4-1-fast                $0.20/$0.50  ->  same
  grok-4-1-fast-non-reasoning  $0.20/$0.50  ->  same
  grok-code-fast-1             $0.20/$1.50  ->  6.25x / 1.7x under
  grok-3-mini                  $0.30/$0.50  ->  under
  grok-4, grok-3               $3.00/$15.00 ->  overcharged callers instead

All seven now price at _XAI_RETIRED_INPUT_USD / _XAI_RETIRED_OUTPUT_USD,
which is grok-4.3's rate. They stay registered so older SDK versions keep
working — the SDK has dropped them — and are marked for removal once no
client sends them. grok-3-mini is not named in xAI's notice but is gone from
their models page, so it is priced the same way rather than at its old rate.

Also in this pass, from auditing every provider's deprecation list:

- Corrected the Gemini note. It claimed gemini-2.5-flash/pro/flash-lite were
  "scheduled for deprecation on June 17, 2026 (flash-lite: July 22, 2026)".
  Those dates have passed and Google's deprecation list does not carry these
  models — they are still served. The only Gemini retirement that matters
  here is gemini-3.1-flash-image-preview, and the registry already calls the
  GA gemini-3.1-flash-image.

- Recorded the announced OpenAI shutdowns next to the affected entries:
  gpt-4.1-nano and o4-mini stop resolving on 2026-10-23 (OpenAI names those
  exact slugs, not dated snapshots), and o3/gpt-5/gpt-5-mini are undated
  aliases of snapshots retiring 2026-12-11. OpenAI does not document whether
  such an alias is repointed or retired with its snapshot; if repointed,
  they become mispriced exactly the way the xAI slugs were.

- Fixed the model lists in CLAUDE.md and README.md, which advertised models
  the registry does not accept: claude-sonnet-4-0, claude-3-7-sonnet,
  claude-3-5-haiku, grok-2 and gemini-3-pro-preview are not in
  _MODEL_LOOKUP, and the README's completion example posted
  "claude-3.7-sonnet". Both lists are now checked against the registry, and
  the retired-but-registered models are documented in one place.

tests/test_pricing.py: the five cases that pinned the old Grok prices now
pin the new ones, plus a new test asserting every retired slug bills at
grok-4.3's rate. test_grok_4_fast_cheaper_than_grok_4 became vacuous (both
are retired and now identical) and is replaced by the same comparison over
the live tiers, grok-4.3 vs grok-4.6.

457 passed, 6 skipped; make lint clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QmnaiLePKAQEkNfR8VPfja
@adambalogh
adambalogh marked this pull request as ready for review September 21, 2026 14:22
@adambalogh
adambalogh merged commit ca02747 into main Sep 21, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants