Skip to content

fix(preview): keep browser calls within their budget and name html_preview refusals - #936

Merged
yaacovcorcos merged 6 commits into
mainfrom
claude/browser-tool-failures-20261115
Nov 15, 2026
Merged

yaacovcorcos merged 6 commits into
mainfrom
claude/browser-tool-failures-20261115

Conversation

@yaacovcorcos

@yaacovcorcos yaacovcorcos commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

What users get

  • preview_open no longer waits up to 15 s on a page that never finishes loading. After an https open (and after preview_navigate), Scient's Accounts check waits at most 3 s for the page's load event. If the page is still loading, it skips the check, logs why, and returns the tab that opened. Before, a page with one stalled image, long-polling request or slow third-party script added the broker's whole 15 s to every open.
  • A browser action keeps to the time the agent gave it. The account check that runs before every action on an existing tab (click, type, wait_for, evaluate, snapshot, …) now spends that action's own budget, measured on a monotonic clock, and hands the rest to the action: the same whole number of milliseconds goes to the broker and into the action's own timeoutMs, which the browser host reads (navigate, resize, click, hover, select, drag, upload, type, wait_for), also when the agent left it to the 15 s default. If the check used the whole budget, nothing is sent and the call fails with PreviewAccountCheckTimeoutError, so no action runs on the page after the agent was told it timed out. Before, the check had its own default budget (15 s, or 660 s on the preferred host), so a slow or stuck check could make a 5 s preview_wait_for take 7 s, 15 s or more.
  • html_preview says which rule a call broke. Its failure result now carries the closed code (invalid_request, capability_denied, orchestration_error) beside the message, and the tool descriptions state the source rules up front: pass exactly one of html or path; only full-access threads may pass a path; the file must start with <!doctype html> or <html>.
  • Analytics name the real failure (no new data leaves the computer; only closed codes):
    • html_preview failures were all reported as orchestrator_mcp_failure. They now report their code, plus a failureClass for the specific refusal: html_source_invalid, html_path_denied, html_render_file, html_render_viewport_height, preview_browser_installing, and so on. html_render gets the same classes.
    • timeout is split: page_not_ready when the browser host answered that the page never became ready (owner external), and host_overrun when the host never answered and the broker gave up (owner scient).
    • preview_evaluate execution failures stay unclassified (owner unknown). The host gives a script that threw on the page and a failed CDP call the same PreviewAutomationExecutionError, so analytics cannot yet tell the agent's code from a product fault; telling them apart needs a closed discriminator carried across the host/broker boundary (T3 mechanics, not in this PR).

Root cause

  • The Accounts check after an open called accountWaitForLoad with no budget of its own. It used the broker's 15 s default with no grace, so any page whose load was late held the open for up to 15 s, after the tab had already opened. Its failures were already swallowed, so the cost was the time, not an error.
  • The pre-action account check called status with no timeoutMs, so it used the broker default for status (15 s, 660 s on the preferred host) on top of the action's budget. Shortening only the broker's timeout is not enough: the hosts run most actions on the input's own timeoutMs (or their 15 s default), so the input has to carry the same remainder.
  • registerImageTool's failure path kept only _tag: OrchestratorMcpFailure, skipped the tool's failure note, and the html handler folded every HtmlRender tag into two codes. Analytics read the wrapper tag.
  • noteToolFailure did not tell a host-answered timeout (remoteTag set) from a broker give-up.

The fix

Scient-owned code, plus small SCIENT-FORK seams in T3 files:

  • scient/accounts/agent/previewProfileFallback.ts: a 3 s load wait (PROFILE_CHECK_LOAD_WAIT_MS); every skip is logged with the operation and the error tag. Profile-grant enforcement in preview_open (authorizePreviewProfile, recordProfileUse, filterPreviewProfiles) is unchanged and still fails the call.
  • mcp/toolkits/preview/handlers.ts (existing Accounts seam): one budget (timeoutMs, else the broker default) starts on Effect's monotonic clock before the check. The pre-action status gets what remains of it; after authorization, a remainder below 1 ms fails with PreviewAccountCheckTimeoutError (Scient-owned, declared in the preview tools' failure union) without dispatch, and otherwise the action gets the remainder in both the envelope and, for operations whose input declares timeoutMs, the input. The broker exports that default as PREVIEW_AUTOMATION_DEFAULT_TIMEOUT_MS (seam).
  • telemetry/ScientMcpAnalytics.ts: browser timeout classes and their owners; noteToolFailureClass; the failure note's code now takes precedence over a wrapper's tag.
  • mcp/toolkits/html/handlers.ts and tools.ts (seams): note each refusal's class before it is folded; describe the source rules.
  • mcp/McpHttpServer.ts imageToolFailure (seam): add the declared failure's closed code to the structured error and write the failure note.

Not in this PR, by decision: T3's browser mechanics (deadlines across queue, select, evaluate, wait, snapshot pause/cleanup). We adopt upstream pingdotgg/t3code#32866 at alignment and remove our SessionControl seam then; the remaining gaps go to an upstream issue.

Tests

New, each failing on main for the reason given (negative controls run):

  • previewProfileFallback.test.ts "a page whose load never ends opens at once, left unchecked": on main the fallback returns after 15 000 ms of the test clock instead of ≤ 3 000 ms.
  • handlers.scient.test.ts "the account check before an action spends the action's budget, not its own": the broker mock now runs on the input's budget like the real hosts. A 5 s preview_wait_for behind a 2 s check is sent with 3 000 ms in both the envelope and the input and ends at 5 000 ms (main: 7 000 ms); with no timeoutMs it is sent with 13 000 ms in both; a check that takes the whole 5 s sends nothing and fails with PreviewAccountCheckTimeoutError; a stuck check ends at 5 000 ms (main: 15 000 ms). The previous head fails it (input 5 000 ≠ 3 000, dispatch after expiry).
  • McpToolAnalytics.scient.test.ts "an HTML refusal reports its own code and reason…": through the production layerHtmlToolkit registration, html_preview reports invalid_request/html_source_invalid, capability_denied/html_path_denied, invalid_request/html_render_file to the agent and analytics, and html_render html_render_file (main: orchestrator_mcp_failure, no class).
  • ScientMcpAnalytics.test.ts "tells a page that never got ready from a host that never answered, and leaves evaluate failures unowned": a script exception (exceptionDetails) and a CDP rejection, both run through the real ServerBrowserPage.evaluate and toOperationError, report execution with owner unknown and no class. The previous head fails it (agent_script_error, owner code).

Ran (macOS, Node 26.13.1, pnpm 12.14.0, live-app environment unset):

  • The four focused files above, plus the existing preview handler, broker, McpHttpServer, html render and tool catalog suites: pass.
  • pnpm run typecheck (root): pass. vp fmt --check, vp lint --report-unused-disable-directives, knip:check (commit hook): pass.
  • node scripts/scient-seam-check.mjs --base origin/main --head HEAD: all manifests pass. node scripts/verify-upstream-provenance.mjs: pass.
  • After the Codex review fixes (third commit): the focused files, every suite under src/mcp, src/telemetry, src/preview, src/scient/accounts and src/provider (3 395 passed, 2 failed: the PortScanner process-group and ClaudeSkills cases below, unrelated), root typecheck, fmt, lint, seam check and provenance: pass. A full-suite rerun was stopped at 30 minutes on a loaded machine; the failures it reached were only the unrelated ones below.
  • Full apps/server suite (first two commits): 16 992 passed, 12 failed in 7 files. One was mine (AgentContextBudget: the tool-list size grew) and is fixed in the second commit. The other 11 also fail on this machine with main's versions of the changed files, and none touches this change: ProviderSwitch prior-images handoffs (6), PortScanner process groups, ClaudeSkills (case-insensitive folder clash), ProjectSetupScriptRunner (expects bash's exit code 2; the shell here is zsh), the OMP native event budget, and the real LaTeX engine.
  • Live check in a candidate dev app (record kept with the review notes): a Claude Haiku thread called preview_open on a local HTTPS page whose image never answers. The Accounts check gave up after 3.0 s and returned the tab (it used to wait up to 15 s). The whole open still took 24.5 s, because T3's own open waits for load for up to 15 s after tab setup; that is T3 browser mechanics, outside this PR, and goes into the upstream issue draft. Three html_preview refusals reached the local analytics outbox as html_source_invalid, html_render_file and html_render_viewport_height (2–3 ms each).

Agent context

The advertised tool list grows by 146 bytes (mcpToolListJsonBytes 91 330 → 91 476, agentContextBudget.scient.json): the html/path parameter descriptions now say "Pass this or path, not both" and "Full-access threads only; others pass html". These are the two rules behind most fast html_preview refusals. The growth needs the owner's agent-context-increase sign-off before merge; the label is theirs to add.

macOS / Windows / Linux

  • No platform branches in the changed code: the 3 s Accounts wait, the shared pre-action budget, the html_preview codes and the analytics classes behave the same on all three.
  • macOS: unit tests and the live check above (Electron desktop host, Claude Haiku).
  • Windows, Linux: not run. The server browser (headless Chromium on Linux servers) uses the same broker path, so the 3 s wait and the budget sharing apply there too. html_preview's path checks are unchanged; their Windows behaviour (drive letters, UNC, the nlink === 1 check on synced folders) was not checked.
  • Not checked live on any platform: the html_path_denied refusal (the live thread was full-access), the pre-action budget, and the timeout classes; unit tests cover them. Only Claude was used live: Claude Code gives the model a tool result's text, not its structured code, so agents on Claude see the specific message rather than the code.

Model and harness: Claude Opus 5.5 in Claude Code (Scient), implementation sub-agent.

🤖 Generated with Claude Code

yaacovcorcos and others added 2 commits November 15, 2026 01:29
…eview refusals

- preview_open/navigate: the Accounts profile check waits at most 3 s for the
  page to load, then skips the check, logs why and returns the opened tab.
- The account check before an action spends the action's own budget; the
  action gets what remains instead of a second full budget.
- html_preview's failure result carries its closed code; html_preview and
  html_render note the specific refusal (source, path access, HtmlRender
  error) for analytics, and the tool descriptions state the source rules.
- Analytics split timeouts into page_not_ready (host answered, external) and
  host_overrun (host never answered, scient), and attribute a thrown
  preview_evaluate script to the agent's code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ol-list size

The two source rules add 146 bytes to the advertised tool list (91330 -> 91476).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yaacovcorcos yaacovcorcos added the agent-context-increase Accept a larger agent context (Agent Context Budget check) label Nov 15, 2026
@github-actions github-actions Bot added the vouch:trusted Author is trusted by repo permissions or the VOUCHED list. label Nov 15, 2026
@yaacovcorcos yaacovcorcos removed the agent-context-increase Accept a larger agent context (Agent Context Budget check) label Nov 15, 2026
…owning evaluate failures

After the Accounts check, the remaining budget is measured on a monotonic clock. With nothing left
the call ends with PreviewAccountCheckTimeoutError and nothing is sent; otherwise the same whole
budget goes in the broker envelope and in the input of every operation whose host reads
input.timeoutMs, also when the agent left it to the default.

An evaluate execution failure is no longer classed agent_script_error: the host gives a script
that threw and a failed CDP call the same tag, so it stays unowned until a closed discriminator
crosses the host boundary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 5.5 KiB 5.5 KiB 0 B (0.0%) 6.8 KiB ✅
Codex Thread snapshot wire 4.2 KiB 4.2 KiB 0 B (0.0%) 4.9 KiB ✅
Codex Live turn WebSocket wire 1.3 KiB 1.3 KiB 0 B (0.0%) 2.0 KiB ✅
Codex Live turn WebSocket decoded 22.3 KiB 22.3 KiB 0 B (0.0%) 24.4 KiB ✅
Codex Live turn messages 1 1 0 (0.0%) 8 ✅
Claude Total thread wire 5.5 KiB 5.5 KiB 0 B (0.0%) 6.8 KiB ✅
Claude Thread snapshot wire 4.2 KiB 4.2 KiB 0 B (0.0%) 4.9 KiB ✅
Claude Live turn WebSocket wire 1.3 KiB 1.3 KiB 0 B (0.0%) 2.0 KiB ✅
Claude Live turn WebSocket decoded 22.7 KiB 22.7 KiB 0 B (0.0%) 24.4 KiB ✅
Claude Live turn messages 1 1 0 (0.0%) 8 ✅

Other scenarios

All 64 scenario measurements match main.

All 64 scenario measurements
Provider Scenario Metric Main baseline This PR Impact PR ceiling
Codex short turn resume WebSocket decoded 12.0 KiB 12.0 KiB 0 B (0.0%) 12.7 KiB ✅
Codex short turn resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex long turn running, cold open HTTP decoded 281.7 KiB 281.7 KiB 0 B (0.0%) 302.7 KiB ✅
Codex long turn running, cold open rows 75 75 0 (0.0%) 75 ✅
Codex long turn running, history page HTTP decoded 92.4 KiB 92.4 KiB 0 B (0.0%) 107.4 KiB ✅
Codex long turn running, history page rows 75 75 0 (0.0%) 75 ✅
Codex long turn finished, cold open HTTP decoded 381.5 KiB 381.5 KiB 0 B (0.0%) 410.2 KiB ✅
Codex long turn finished, cold open rows 151 151 0 (0.0%) 151 ✅
Codex long turn finished, socket cold open WebSocket decoded 381.6 KiB 381.6 KiB 0 B (0.0%) 410.2 KiB ✅
Codex long turn finished, socket cold open WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex long turn finished, socket cold open rows 151 151 0 (0.0%) 151 ✅
Codex long turn finished, work page HTTP decoded 186.2 KiB 186.2 KiB 0 B (0.0%) 200.2 KiB ✅
Codex long turn finished, work page rows 150 150 0 (0.0%) 150 ✅
Codex idle thread resume WebSocket decoded 65 B 65 B 0 B (0.0%) 200 B ✅
Codex idle thread resume WebSocket messages 1 1 0 (0.0%) 1 ✅
Codex fleet shell snapshot HTTP decoded 45.1 KiB 45.1 KiB 0 B (0.0%) 53.7 KiB ✅
Codex fleet shell snapshot rows 27 27 0 (0.0%) 27 ✅
Codex fleet shell subscribe WebSocket decoded 65 B 65 B 0 B (0.0%) 200 B ✅
Codex fleet shell subscribe WebSocket messages 1 1 0 (0.0%) 1 ✅
Codex fleet shell resume check HTTP decoded 18 B 18 B 0 B (0.0%) 100 B ✅
Codex fleet shell resume check rows 0 0 0 0 ✅
Codex stale shell reload HTTP decoded 45.1 KiB 45.1 KiB 0 B (0.0%) 53.7 KiB ✅
Codex stale shell reload rows 27 27 0 (0.0%) 27 ✅
Codex stale shell resume WebSocket decoded 140 B 140 B 0 B (0.0%) 400 B ✅
Codex stale shell resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex stale todo list cold open HTTP decoded 72.4 KiB 72.4 KiB 0 B (0.0%) 83.0 KiB ✅
Codex stale todo list cold open rows 75 75 0 (0.0%) 75 ✅
Codex stale todo list cold open nodes 2 2 0 (0.0%) 4 ✅
Codex stale todo list cold open subagents 2 2 0 (0.0%) 4 ✅
Codex stale todo list cold open plans 1 1 0 (0.0%) 2 ✅
Codex stale todo list cold open providerTurns 1 1 0 (0.0%) 2 ✅
Codex stale todo list cold open checkpoints 1 1 0 (0.0%) 2 ✅
Claude short turn resume WebSocket decoded 12.2 KiB 12.2 KiB 0 B (0.0%) 12.7 KiB ✅
Claude short turn resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude long turn running, cold open HTTP decoded 283.3 KiB 283.3 KiB 0 B (0.0%) 302.7 KiB ✅
Claude long turn running, cold open rows 75 75 0 (0.0%) 75 ✅
Claude long turn running, history page HTTP decoded 93.4 KiB 93.4 KiB 0 B (0.0%) 107.4 KiB ✅
Claude long turn running, history page rows 75 75 0 (0.0%) 75 ✅
Claude long turn finished, cold open HTTP decoded 384.1 KiB 384.1 KiB 0 B (0.0%) 410.2 KiB ✅
Claude long turn finished, cold open rows 151 151 0 (0.0%) 151 ✅
Claude long turn finished, socket cold open WebSocket decoded 384.3 KiB 384.3 KiB 0 B (0.0%) 410.2 KiB ✅
Claude long turn finished, socket cold open WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude long turn finished, socket cold open rows 151 151 0 (0.0%) 151 ✅
Claude long turn finished, work page HTTP decoded 188.1 KiB 188.1 KiB 0 B (0.0%) 200.2 KiB ✅
Claude long turn finished, work page rows 150 150 0 (0.0%) 150 ✅
Claude idle thread resume WebSocket decoded 66 B 66 B 0 B (0.0%) 200 B ✅
Claude idle thread resume WebSocket messages 1 1 0 (0.0%) 1 ✅
Claude fleet shell snapshot HTTP decoded 45.6 KiB 45.6 KiB 0 B (0.0%) 53.7 KiB ✅
Claude fleet shell snapshot rows 27 27 0 (0.0%) 27 ✅
Claude fleet shell subscribe WebSocket decoded 66 B 66 B 0 B (0.0%) 200 B ✅
Claude fleet shell subscribe WebSocket messages 1 1 0 (0.0%) 1 ✅
Claude fleet shell resume check HTTP decoded 18 B 18 B 0 B (0.0%) 100 B ✅
Claude fleet shell resume check rows 0 0 0 0 ✅
Claude stale shell reload HTTP decoded 45.6 KiB 45.6 KiB 0 B (0.0%) 53.7 KiB ✅
Claude stale shell reload rows 27 27 0 (0.0%) 27 ✅
Claude stale shell resume WebSocket decoded 140 B 140 B 0 B (0.0%) 400 B ✅
Claude stale shell resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude stale todo list cold open HTTP decoded 73.4 KiB 73.4 KiB 0 B (0.0%) 83.0 KiB ✅
Claude stale todo list cold open rows 75 75 0 (0.0%) 75 ✅
Claude stale todo list cold open nodes 2 2 0 (0.0%) 4 ✅
Claude stale todo list cold open subagents 2 2 0 (0.0%) 4 ✅
Claude stale todo list cold open plans 1 1 0 (0.0%) 2 ✅
Claude stale todo list cold open providerTurns 1 1 0 (0.0%) 2 ✅
Claude stale todo list cold open checkpoints 1 1 0 (0.0%) 2 ✅

Baseline: d666ac0 · PR result: 4641d45 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.
Bounded compact HTTP snapshot with live cursor.

  • Codex decoded thread snapshot: 88.2 KiB
  • Claude decoded thread snapshot: 88.4 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

yaacovcorcos and others added 2 commits November 15, 2026 04:00
The remaining budget is checked before the Accounts status check as well as
before the action, so nothing reaches the browser once the agent's time is
spent. Found in re-check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ailures-20261115

# Conflicts:
#	apps/server/src/provider/agentContextBudget.scient.json
#	apps/server/src/telemetry/ScientMcpAnalytics.test.ts
#	apps/server/src/telemetry/ScientMcpAnalytics.ts
…ailures-20261115

# Conflicts:
#	apps/server/src/provider/agentContextBudget.scient.json
#	apps/server/src/telemetry/ScientMcpAnalytics.test.ts
@yaacovcorcos
yaacovcorcos added this pull request to the merge queue Nov 15, 2026
Merged via the queue into main with commit 220f377 Nov 15, 2026
59 of 60 checks passed
@yaacovcorcos
yaacovcorcos deleted the claude/browser-tool-failures-20261115 branch November 15, 2026 02:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L vouch:trusted Author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant