Skip to content

fix: report usage limits, network and access-policy failures; string-only CCA enums - #65

Merged
yaacovcorcos merged 4 commits into
mainfrom
claude/failure-kinds-20261116
Nov 16, 2026
Merged

yaacovcorcos merged 4 commits into
mainfrom
claude/failure-kinds-20261116

Conversation

@yaacovcorcos

@yaacovcorcos yaacovcorcos commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

What

Failure classification fixes from real failures (Nov 10–16):

  • SA-USAGE-RESET (extended):
    • A usage-limited failure without a status reports httpStatus: 429, so the host shows its quota-blocked state. A status the provider sent is never replaced.
    • "Your usage window resets at " is now read as a reset time. Upstream reads only "reset at". This also changes the agent's own retry wait, which uses the stated time instead of a 30-minute guess.
  • SA-FAILURE-KIND (extended): a new network kind for failures that never reached the provider: DNS, refused connection, offline, or a certificate that failed to verify. A response from the provider is never labelled network.
  • SA-ACCESS-POLICY (new): OpenRouter's "…is only available on agentic harnesses" 403 is now account-policy, not auth-failed. The key is valid.
  • SA-CCA-STRING-ENUMS (new): Cloud Code Assist parameters get string-only enums, as Gemini's already do. Google rejected a tool whose schema had enum values [-1, 1].

Why

These failures reached Scient Desktop as generic errors or were misdiagnosed: a bad key reported when the key was valid, a network failure reported as a provider outage, and a tool schema that Google refused.

Testing

  • Passing: targeted utils/ai/coding-agent suites (59 + 265 + 97), bun test scripts/scient (79), identity and wording checks, divergence-check --footprint, type checks for ai, coding-agent and utils, oxfmt, oxlint.
  • Regression tests: each new test fails with its hook reverted.
  • Not run: live OpenRouter and Google endpoints (tested against a local server and the converter's output), and the full upstream test buckets. CI runs those.
  • Review: Codex found no high-confidence problems.

Upstream candidates: the resets at pattern, the access-policy pattern, and the CCA string-only enums.

Release: none planned. Desktop already ranks network (ScientFactory/scient-desktop#965).

  • bun check passes (CI)
  • Tested locally

🤖 Generated with Claude Code

yaacovcorcos and others added 4 commits November 16, 2026 18:48
A long-wait 429 ("Provider requested 119172000ms wait, exceeds
retry.maxDelayMs (300000ms)") ends the turn at once. Its prompt_result
carried httpStatus only when the transport attached the response's
status to the failed message, and Scient Desktop shows its quota-blocked
state, with the reset time, only for a 429 (or cost_cap). A usage limit
the classifier recognized but that came without a status therefore read
as a generic failure. Such a failure now reports httpStatus 429; a
status the provider sent is never replaced.

The stated reset of "429 Subscription quota exhausted. Your usage
window resets at 2026-11-16T12:15:34Z." was not read either: the retry
hint parser knows "reset at <time>", not "resets at <time>". Scient adds
that wording to the parser's reset-time patterns, so recovery waits for
the stated time and the host gets resetAt from it.

Verified on main first: through the real Anthropic and OpenAI responses
transports against a local server, both observed 429s already carried
httpStatus 429, kinds and resetAt; the status was missing only where the
message had no errorStatus.

Scient-Divergence: SA-USAGE-RESET

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"Couldn't reach chatgpt.com (getaddrinfo ENOTFOUND)..." and "Couldn't
connect to chatgpt.com (connection refused)..." reached the host only as
`transient`, the same kind a provider's own outage gets: the classifier
has no network flag. Oh My Pi's network diagnosis (classifyNetworkCause,
which wrote those sentences through networkFailureMessage) already tells
DNS, refused, offline and certificate failures apart. error.kinds now
adds `network` for a failure with no HTTP status that the diagnosis, or
one of its own sentences, names. The refused sentence keeps no error
code, so the sentences are matched too; a test renders each of upstream's
four sentences so a wording change upstream fails it. A response the
provider sent is never `network`.

Hosts that rank kinds need to learn the label; Scient Desktop's
priority list ignores labels it does not know.

Scient-Divergence: SA-FAILURE-KIND

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
OpenRouter refuses some free models to clients it does not list: "403
thinkingmachines/inkling-2-small:free is only available on agentic
harnesses. Try plugging it into a coding agent or productivity app
listed on https://openrouter.ai/apps". The key is valid, but no pattern
matched the text, so classifyText fell through to its bare-403 rule and
the turn was reported only as auth-failed: the person was told their
sign-in failed.

The classifier now marks this refusal AIError.Flag.AccountPolicy, as it
marks Anthropic's organization denials. The cause-chain classifier still
adds AuthFailed for any 403 response, as it does for those denials, so
the transport path reports both; a host that ranks kinds (Scient
Desktop puts account-policy before auth-failed) reports the policy.
Another 403 stays a failed sign-in. Recovery now treats it like the
other account policies: it tries the model's fallback chain, then
surfaces the error.

A candidate to propose to Oh My Pi.

Scient-Divergence: SA-ACCESS-POLICY

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
google-antigravity refused a session's requests with "Cloud Code Assist
API error (400): Invalid value at 'request.tools[0].function_declarations
[45].parameters.properties[2].value.items.properties[6].value.enum[0]'
(TYPE_STRING), -1": a tool's schema had the numeric enum [-1, 1]. Claude
models on Cloud Code Assist take their tool schemas in the legacy
`parameters` field, a Google Schema whose enum values are strings only.
normalizeSchemaForGoogle (the Gemini parametersJsonSchema path) already
drops a non-string enum with stringEnumsOnly and keeps the node's type;
normalizeSchemaForCCA did not set it. It does now. A boolean or mixed
enum is dropped the same way; a string enum is kept. Arguments are still
checked against the tool's own schema.

A candidate to propose to Oh My Pi.

Scient-Divergence: SA-CCA-STRING-ENUMS

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yaacovcorcos
yaacovcorcos merged commit 2c0d957 into main Nov 16, 2026
21 checks passed
@yaacovcorcos
yaacovcorcos deleted the claude/failure-kinds-20261116 branch November 16, 2026 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant