From d63768855e5cfb5ab9c1ef882fc02ba62a0924f2 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 21 Sep 2026 14:12:10 +0000 Subject: [PATCH] Charge xAI's real rate for the slugs it retired on 2026-05-15 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit xAI retired grok-4, grok-4-fast, grok-4-1-fast, grok-4-1-fast-non-reasoning, grok-code-fast-1 and grok-3 on 2026-05-15 (docs.x.ai/developers/migration/may-15-retirement). It still ACCEPTS every one of those slugs — each is silently redirected to a live model (grok-4.3, or grok-build-0.1 for grok-code-fast-1) and, crucially, billed at grok-4.3's rate of $1.25/$2.50 per MTok no matter which name was sent. The registry still carried their pre-retirement prices, so the gateway was charging callers less than xAI charges us: grok-4-fast $0.20/$0.50 -> 6.25x under on input, 5x on output grok-4-1-fast $0.20/$0.50 -> same grok-4-1-fast-non-reasoning $0.20/$0.50 -> same grok-code-fast-1 $0.20/$1.50 -> 6.25x / 1.7x under grok-3-mini $0.30/$0.50 -> under grok-4, grok-3 $3.00/$15.00 -> overcharged callers instead All seven now price at _XAI_RETIRED_INPUT_USD / _XAI_RETIRED_OUTPUT_USD, which is grok-4.3's rate. They stay registered so older SDK versions keep working — the SDK has dropped them — and are marked for removal once no client sends them. grok-3-mini is not named in xAI's notice but is gone from their models page, so it is priced the same way rather than at its old rate. Also in this pass, from auditing every provider's deprecation list: - Corrected the Gemini note. It claimed gemini-2.5-flash/pro/flash-lite were "scheduled for deprecation on June 17, 2026 (flash-lite: July 22, 2026)". Those dates have passed and Google's deprecation list does not carry these models — they are still served. The only Gemini retirement that matters here is gemini-3.1-flash-image-preview, and the registry already calls the GA gemini-3.1-flash-image. - Recorded the announced OpenAI shutdowns next to the affected entries: gpt-4.1-nano and o4-mini stop resolving on 2026-10-23 (OpenAI names those exact slugs, not dated snapshots), and o3/gpt-5/gpt-5-mini are undated aliases of snapshots retiring 2026-12-11. OpenAI does not document whether such an alias is repointed or retired with its snapshot; if repointed, they become mispriced exactly the way the xAI slugs were. - Fixed the model lists in CLAUDE.md and README.md, which advertised models the registry does not accept: claude-sonnet-4-0, claude-3-7-sonnet, claude-3-5-haiku, grok-2 and gemini-3-pro-preview are not in _MODEL_LOOKUP, and the README's completion example posted "claude-3.7-sonnet". Both lists are now checked against the registry, and the retired-but-registered models are documented in one place. tests/test_pricing.py: the five cases that pinned the old Grok prices now pin the new ones, plus a new test asserting every retired slug bills at grok-4.3's rate. test_grok_4_fast_cheaper_than_grok_4 became vacuous (both are retired and now identical) and is replaced by the same comparison over the live tiers, grok-4.3 vs grok-4.6. 457 passed, 6 skipped; make lint clean. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01QmnaiLePKAQEkNfR8VPfja --- CLAUDE.md | 26 +++++++++--- README.md | 21 +++++++--- tee_gateway/model_registry.py | 75 +++++++++++++++++++++++++++-------- tests/test_pricing.py | 55 +++++++++++++++++++------ 4 files changed, 137 insertions(+), 40 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index afbc585..45f4e71 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -124,15 +124,31 @@ Server configuration: ### Supported Providers -Model name prefixes determine routing: -- **OpenAI**: gpt-6-astra, gpt-4.1, gpt-5, gpt-5-mini, gpt-5.2, gpt-5.6-sol/terra/luna, o4-mini; image generation: gpt-image-2.5-flare, gpt-image-2.5-sunburst, gpt-image-2 -- **Anthropic**: claude-sonnet-4-0/4-5/4-6, claude-sonnet-5, claude-haiku-4-5, claude-opus-4-5/4-6/4-7/4-8, claude-opus-5, claude-fable-5, claude-fable-5-1, claude-3-7-sonnet, claude-3-5-haiku -- **Google**: gemini-3.8-flash, gemini-3.7-flash, gemini-3.6-flash, gemini-3.5-flash-lite, gemini-2.5-flash, gemini-2.5-flash-lite, gemini-2.5-pro, gemini-3-pro-preview, gemini-3-flash-preview, gemini-3.1-pro-preview, gemini-3.5-flash; image generation: gemini-2.5-flash-image, gemini-3.1-flash-image -- **xAI**: grok-2, grok-3, grok-3-mini, grok-4, grok-4.3, grok-4.5, grok-4.6, grok-4-fast, grok-4-1-fast; image generation: grok-2-image, grok-imagine-image-2.0 +`model_registry.py` is the source of truth; this list mirrors it. Model name +prefixes determine routing: +- **OpenAI**: gpt-6-astra, gpt-4.1, gpt-4.1-mini, gpt-5, gpt-5-mini, gpt-5.2, gpt-5.4, gpt-5.4-mini, gpt-5.4-nano, gpt-5.5, gpt-5.6-sol/terra/luna, o3; image generation: gpt-image-2.5-flare, gpt-image-2.5-sunburst, gpt-image-2 +- **Anthropic**: claude-sonnet-4-5/4-6, claude-sonnet-5, claude-haiku-4-5, claude-opus-4-5/4-6/4-7/4-8, claude-opus-5, claude-fable-5, claude-fable-5-1 +- **Google**: gemini-3.8-flash, gemini-3.7-flash, gemini-3.6-flash, gemini-3.5-flash, gemini-3.5-flash-lite, gemini-2.5-flash, gemini-2.5-flash-lite, gemini-2.5-pro, gemini-3-flash-preview, gemini-3.1-pro-preview; image generation: gemini-2.5-flash-image, gemini-3.1-flash-image +- **xAI**: grok-4.6, grok-4.5, grok-4.3, grok-4.20-reasoning, grok-4.20-non-reasoning; image generation: grok-2-image, grok-imagine-image-2.0 - **ByteDance** (BytePlus ModelArk, OpenAI-compatible, ap-southeast): seed-1.6, seed-1.8, seed-2.0-lite, deepseek-v4-flash, deepseek-v4-pro, glm-5.2 (Z.ai's model served via a ModelArk deployment endpoint); image generation: seedream-4.0, seedream-5.0-lite, seedance-4.5, seedance-5.0 - **OpenRouter** (OpenAI-compatible): hy4-preview, hermes-4-405b, hermes-4-70b, hy3 - **Z.ai** (Model API, OpenAI-compatible): image generation: glm-image (glm-5.2 chat is routed through BytePlus ModelArk, see ByteDance above) +Models kept registered but no longer offered to new clients — each still +resolves so older SDK versions keep working, and each is priced at what the +provider actually bills, not at its pre-retirement rate: +- **Retired by xAI on 2026-05-15**: grok-4, grok-4-fast, grok-4-1-fast, + grok-4-1-fast-non-reasoning, grok-code-fast-1, grok-3, grok-3-mini. xAI still + accepts these slugs but silently redirects them (to grok-4.3, or + grok-build-0.1 for grok-code-fast-1) and bills every one at grok-4.3's rate. +- **Shutting down at OpenAI on 2026-10-23**: gpt-4.1-nano, o4-mini. OpenAI + names those exact slugs, so they stop resolving that day; replacements are + gpt-5.6-luna and gpt-5.6-terra. +- **Watch 2026-12-11**: o3, gpt-5 and gpt-5-mini are undated aliases of + snapshots retiring then. OpenAI does not document whether such an alias is + repointed or retired with its snapshot; if repointed, they become mispriced + the way the xAI slugs were. + Image generation via OpenAI (gpt-image-2.5-flare, gpt-image-2.5-sunburst, gpt-image-2), xAI (grok-2-image, grok-imagine-image-2.0), ByteDance (seedream-4.0, seedream-5.0-lite, seedance-4.5, seedance-5.0), and Z.ai (glm-image) is served through a provider `/images/generations` endpoint rather than the chat path (see diff --git a/README.md b/README.md index 7aee78a..25fffd1 100644 --- a/README.md +++ b/README.md @@ -33,13 +33,22 @@ The gateway solves this by running inside a hardware-isolated Nitro Enclave wher | Provider | Models | |----------|--------| -| OpenAI | gpt-6-astra, gpt-4.1, gpt-5, gpt-5-mini, gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna, o4-mini | -| Anthropic | claude-fable-5-1, claude-sonnet-4-5, claude-sonnet-4-6, claude-haiku-4-5, claude-opus-4-5, claude-opus-4-6 | -| Google | gemini-3.8-flash, gemini-3.7-flash, gemini-3.6-flash, gemini-3.5-flash-lite, gemini-2.5-flash, gemini-2.5-flash-lite, gemini-2.5-pro, gemini-3-pro-preview, gemini-3-flash-preview | -| xAI | grok-4.6, grok-4.5, grok-4.3, grok-4, grok-4-fast, grok-4-1-fast, grok-4-1-fast-non-reasoning | -| ByteDance | seed-1.6, seed-1.8, seed-2.0-lite, deepseek-v4-flash, deepseek-v4-pro | +| OpenAI | gpt-6-astra, gpt-4.1, gpt-4.1-mini, gpt-5, gpt-5-mini, gpt-5.2, gpt-5.4, gpt-5.5, gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna, o3 | +| Anthropic | claude-fable-5-1, claude-fable-5, claude-opus-5, claude-opus-4-8, claude-opus-4-7, claude-opus-4-6, claude-opus-4-5, claude-sonnet-5, claude-sonnet-4-6, claude-sonnet-4-5, claude-haiku-4-5 | +| Google | gemini-3.8-flash, gemini-3.7-flash, gemini-3.6-flash, gemini-3.5-flash, gemini-3.5-flash-lite, gemini-2.5-flash, gemini-2.5-flash-lite, gemini-2.5-pro, gemini-3-flash-preview, gemini-3.1-pro-preview | +| xAI | grok-4.6, grok-4.5, grok-4.3, grok-4.20-reasoning, grok-4.20-non-reasoning | +| ByteDance | seed-1.6, seed-1.8, seed-2.0-lite, dola-seed-2.0-mini, deepseek-v4-flash, deepseek-v4-pro, glm-5.2 | | OpenRouter | hy4-preview, hermes-4-405b, hermes-4-70b, hy3 | +Image generation: gpt-image-2, gpt-image-2.5-flare, gpt-image-2.5-sunburst, +gemini-2.5-flash-image, gemini-3.1-flash-image, grok-2-image, +grok-imagine-image-2.0, seedream-4.0, seedream-5.0-lite, seedance-4.5, +seedance-5.0, glm-image. + +Some retired models stay registered so older SDK versions keep working; see +"Supported Providers" in CLAUDE.md. `tee_gateway/model_registry.py` is the +source of truth for all of the above. + ## Quick Start ### Local Development (without TEE) @@ -89,7 +98,7 @@ curl -X POST http://127.0.0.1:8000/v1/chat/completions \ curl -X POST http://127.0.0.1:8000/v1/completions \ -H "Content-Type: application/json" \ -d '{ - "model": "claude-3.7-sonnet", + "model": "claude-sonnet-4-6", "prompt": "Explain quantum computing in one sentence" }' diff --git a/tee_gateway/model_registry.py b/tee_gateway/model_registry.py index 2cb4596..cec3cfc 100644 --- a/tee_gateway/model_registry.py +++ b/tee_gateway/model_registry.py @@ -169,6 +169,12 @@ def image_tier(self, resolution: Optional[str]) -> Optional[ImageResolutionTier] # settled (see WebSearchOutcome.billable). WEB_SEARCH_PRICE_USD: Decimal = Decimal("0.015") +# What xAI charges for any slug it retired on 2026-05-15: every such request is +# redirected to a live model and billed at grok-4.3's rate ($1.25 in / $2.50 out +# per MTok), whatever name was sent. +_XAI_RETIRED_INPUT_USD: Decimal = Decimal("0.00000125") +_XAI_RETIRED_OUTPUT_USD: Decimal = Decimal("0.0000025") + # ByteDance ModelArk image *deployment* endpoints (api_name "ep-…", e.g. Seedance # 4.5, Seedream 5.0 Lite) return the URL response format and require these extra # params. The gateway fetches the returned URL and inlines the bytes, so the @@ -302,12 +308,21 @@ class SupportedModel(Enum): input_price_usd=Decimal("0.0000004"), output_price_usd=Decimal("0.0000016"), ) + # Shuts down 2026-10-23 (OpenAI names this exact slug, not a dated + # snapshot, so it stops resolving that day). Replacement: gpt-5.6-luna. + # Dropped from the SDK already; kept here for older SDK versions until the + # date. Same for o4-mini below (replacement: gpt-5.6-terra). GPT_4_1_NANO = ModelConfig( provider="openai", api_name="gpt-4.1-nano", input_price_usd=Decimal("0.0000001"), output_price_usd=Decimal("0.0000004"), ) + # o3, gpt-5 and gpt-5-mini are undated aliases of snapshots that shut down + # on 2026-12-11 (o3-2025-04-16, gpt-5-2025-08-07, gpt-5-mini-2025-08-07). + # OpenAI does not document whether such an alias is repointed to a + # successor or retired with its snapshot. If it is repointed, these become + # mispriced the way the retired xAI slugs were, so revisit before then. O3 = ModelConfig( provider="openai", api_name="o3", @@ -558,9 +573,14 @@ class SupportedModel(Enum): ) # ── Google Gemini ─────────────────────────────────────────────────── - # Note: gemini-2.5-flash, gemini-2.5-pro, and gemini-2.5-flash-lite are scheduled - # for deprecation on June 17, 2026 (flash-lite: July 22, 2026). Use the Gemini 3 - # replacements below for new integrations. + # Note: gemini-2.5-flash, gemini-2.5-pro and gemini-2.5-flash-lite are + # still served. A previous note here said they were "scheduled for + # deprecation on June 17, 2026 (flash-lite: July 22, 2026)"; those dates + # passed and Google's deprecation list does not carry them, so the note was + # wrong. The Gemini 3 models below are still the better default for new + # integrations. (Google did retire gemini-3.1-flash-image-preview on + # 2026-06-25 — the registry calls the GA gemini-3.1-flash-image, not that + # preview.) GEMINI_2_5_FLASH = ModelConfig( provider="google", api_name="gemini-2.5-flash", @@ -685,29 +705,43 @@ class SupportedModel(Enum): input_price_usd=Decimal("0.00000125"), output_price_usd=Decimal("0.0000025"), ) + # ── xAI models retired on 2026-05-15 ──────────────────────────────── + # xAI retired these slugs (docs.x.ai/developers/migration/may-15-retirement) + # but still ACCEPTS them: each is silently redirected to a live model, and + # every redirected request is billed at grok-4.3's rate regardless of the + # name sent. They are kept registered so older SDK versions keep working + # (the current SDK no longer offers them), but they must be priced at what + # xAI actually charges — $1.25/$2.50 per MTok — or the gateway eats the + # difference. They were previously priced at their pre-retirement rates, + # which undercharged grok-4-fast by 6.25x on input and 5x on output. + # Remove once no client sends them. + # -> grok-4.3 (low reasoning effort) GROK_4 = ModelConfig( provider="x-ai", api_name="grok-4", - input_price_usd=Decimal("0.000003"), - output_price_usd=Decimal("0.000015"), + input_price_usd=_XAI_RETIRED_INPUT_USD, + output_price_usd=_XAI_RETIRED_OUTPUT_USD, ) + # -> grok-4.3 (low reasoning effort) GROK_4_FAST = ModelConfig( provider="x-ai", api_name="grok-4-fast", - input_price_usd=Decimal("0.0000002"), - output_price_usd=Decimal("0.0000005"), + input_price_usd=_XAI_RETIRED_INPUT_USD, + output_price_usd=_XAI_RETIRED_OUTPUT_USD, ) + # -> grok-4.3 (low reasoning effort) GROK_4_1_FAST = ModelConfig( provider="x-ai", api_name="grok-4-1-fast", - input_price_usd=Decimal("0.0000002"), - output_price_usd=Decimal("0.0000005"), + input_price_usd=_XAI_RETIRED_INPUT_USD, + output_price_usd=_XAI_RETIRED_OUTPUT_USD, ) + # -> grok-4.3 (reasoning effort "none") GROK_4_1_FAST_NON_REASONING = ModelConfig( provider="x-ai", api_name="grok-4-1-fast-non-reasoning", - input_price_usd=Decimal("0.0000002"), - output_price_usd=Decimal("0.0000005"), + input_price_usd=_XAI_RETIRED_INPUT_USD, + output_price_usd=_XAI_RETIRED_OUTPUT_USD, ) GROK_4_20_REASONING = ModelConfig( provider="x-ai", @@ -721,11 +755,13 @@ class SupportedModel(Enum): input_price_usd=Decimal("0.000002"), output_price_usd=Decimal("0.000006"), ) + # Retired 2026-05-15 -> grok-build-0.1, billed at grok-4.3's rate. See the + # retired-slug note above. GROK_CODE_FAST_1 = ModelConfig( provider="x-ai", api_name="grok-code-fast-1", - input_price_usd=Decimal("0.0000002"), - output_price_usd=Decimal("0.0000015"), + input_price_usd=_XAI_RETIRED_INPUT_USD, + output_price_usd=_XAI_RETIRED_OUTPUT_USD, ) # Image generation via xAI's OpenAI-compatible /images/generations endpoint. # grok-2-image-1212 was retired in February 2026; grok-imagine-image is its @@ -959,17 +995,22 @@ class SupportedModel(Enum): ) # ── Legacy models (not in current SDK — retained for older SDK versions) ── + # grok-3 was retired on 2026-05-15 alongside the slugs above and is + # redirected to grok-4.3 (reasoning effort "none"), billed at grok-4.3's + # rate. grok-3-mini is not named in that notice but is no longer offered on + # xAI's models page, so it is priced the same way rather than at its old + # rate. See the retired-slug note above. GROK_3_MINI = ModelConfig( provider="x-ai", api_name="grok-3-mini", - input_price_usd=Decimal("0.0000003"), - output_price_usd=Decimal("0.0000005"), + input_price_usd=_XAI_RETIRED_INPUT_USD, + output_price_usd=_XAI_RETIRED_OUTPUT_USD, ) GROK_3 = ModelConfig( provider="x-ai", api_name="grok-3-latest", - input_price_usd=Decimal("0.000003"), - output_price_usd=Decimal("0.000015"), + input_price_usd=_XAI_RETIRED_INPUT_USD, + output_price_usd=_XAI_RETIRED_OUTPUT_USD, ) diff --git a/tests/test_pricing.py b/tests/test_pricing.py index c4c4b24..05f3b82 100644 --- a/tests/test_pricing.py +++ b/tests/test_pricing.py @@ -413,8 +413,33 @@ def test_grok_4_20_non_reasoning_resolves(self): def test_grok_code_fast_1_resolves(self): cfg = get_model_config("grok-code-fast-1") self.assertEqual(cfg.provider, "x-ai") - self.assertEqual(cfg.input_price_usd, Decimal("0.0000002")) - self.assertEqual(cfg.output_price_usd, Decimal("0.0000015")) + # Retired 2026-05-15 and redirected to grok-build-0.1; xAI bills it at + # grok-4.3's rate, so that is what we charge. + self.assertEqual(cfg.input_price_usd, Decimal("0.00000125")) + self.assertEqual(cfg.output_price_usd, Decimal("0.0000025")) + + def test_retired_xai_slugs_priced_at_grok_4_3_rate(self): + """Every slug xAI retired on 2026-05-15 bills at grok-4.3's rate. + + xAI redirects each of these to a live model and bills the request at + grok-4.3's price whatever name was sent, so pricing them at their old + pre-retirement rates made the gateway eat the difference (grok-4-fast + was 6.25x under on input, 5x on output). + """ + live = get_model_config("grok-4.3") + for name in ( + "grok-4", + "grok-4-fast", + "grok-4-1-fast", + "grok-4-1-fast-non-reasoning", + "grok-code-fast-1", + "grok-3", + "grok-3-mini", + ): + with self.subTest(model=name): + cfg = get_model_config(name) + self.assertEqual(cfg.input_price_usd, live.input_price_usd) + self.assertEqual(cfg.output_price_usd, live.output_price_usd) def test_grok_imagine_image_2_0_resolves(self): cfg = get_model_config("grok-imagine-image-2.0") @@ -873,15 +898,17 @@ def test_grok_4_cost(self): cost = self._calc("grok-4", 1000, 500) expected = _expected_cost_opg("grok-4", 1000, 500) self.assertEqual(cost, expected) - # Same pricing tier as claude-sonnet-4-5 - self.assertEqual(cost, 10_500_000_000_000_000) + # Retired slug, billed by xAI at grok-4.3's rate: + # 1000*0.00000125 + 500*0.0000025 = 0.00125 + 0.00125 = 0.0025 USD + self.assertEqual(cost, 2_500_000_000_000_000) def test_grok_4_fast_cost(self): cost = self._calc("grok-4-fast", 1000, 500) expected = _expected_cost_opg("grok-4-fast", 1000, 500) self.assertEqual(cost, expected) - # 1000*0.0000002 + 500*0.0000005 = 0.0002 + 0.00025 = 0.00045 USD - self.assertEqual(cost, 450_000_000_000_000) + # Retired slug, billed by xAI at grok-4.3's rate: + # 1000*0.00000125 + 500*0.0000025 = 0.00125 + 0.00125 = 0.0025 USD + self.assertEqual(cost, 2_500_000_000_000_000) def test_grok_4_1_fast_cost(self): cost = self._calc("grok-4-1-fast", 1000, 500) @@ -902,8 +929,9 @@ def test_grok_code_fast_1_cost(self): cost = self._calc("grok-code-fast-1", 1000, 500) expected = _expected_cost_opg("grok-code-fast-1", 1000, 500) self.assertEqual(cost, expected) - # 1000*0.0000002 + 500*0.0000015 = 0.0002 + 0.00075 = 0.00095 USD = 9.5e14 wei - self.assertEqual(cost, 950_000_000_000_000) + # Retired slug, billed by xAI at grok-4.3's rate: + # 1000*0.00000125 + 500*0.0000025 = 0.0025 USD = 2.5e15 wei + self.assertEqual(cost, 2_500_000_000_000_000) def test_grok_3_mini_cost(self): cost = self._calc("grok-3-mini", 1000, 500) @@ -981,10 +1009,13 @@ def test_gemini_flash_lite_cheaper_than_flash(self): flash = self._calc("gemini-2.5-flash", 1000, 1000) self.assertLess(lite, flash) - def test_grok_4_fast_cheaper_than_grok_4(self): - fast = self._calc("grok-4-fast", 1000, 1000) - full = self._calc("grok-4", 1000, 1000) - self.assertLess(fast, full) + def test_grok_4_3_cheaper_than_grok_4_6(self): + # grok-4 / grok-4-fast were retired on 2026-05-15 and both now bill at + # grok-4.3's rate, so the old fast-vs-full comparison is vacuous. The + # live tiers still hold this ordering. + cheap = self._calc("grok-4.3", 1000, 1000) + flagship = self._calc("grok-4.6", 1000, 1000) + self.assertLess(cheap, flagship) class TestCalculateSessionCostEdgeCases(unittest.TestCase):