feat(integrations): add Grok Build CLI harness by antonvishal · Pull Request #2876 · browserbase/stagehand

antonvishal · 2026-09-02T11:03:02Z

What

Adds the grok_build eval harness using the native Grok Build CLI. Each run uses an isolated Grok home and trusted MCP configuration, with Stagehand browser tools and normalized tool, usage, and completion events.

Partial tool updates remain pending until completion. Wrapped MCP calls retain their actual tool names, arguments, results, and browser evidence. Browser guidance starts with main-frame snapshots and checks every task constraint before the final answer.

Testing

evals run b:webvoyager --harness grok_build --tool stagehand_facade -m grok-build/grok-4.6 -l 1 -t 1 -e browserbase runs and passes.
19 targeted tests passed; evals and Grok SDK typechecks passed.

Summary by cubic

Adds a Grok Build integration and eval harness via a new @browserbasehq/stagehand-integrations-grok-build-sdk session adapter that drives the native grok CLI in headless streaming mode over MCP/stdio.

Registers the grok_build harness with the stagehand_facade, playwright_mcp, and chrome_devtools_mcp tool surfaces.
Runs each eval in an isolated Grok home and workspace, copying cached auth only when XAI_API_KEY is unset and writing the MCP mount into the isolated home's trusted config.
Uses the native --rules policy, turn budget, and session-loss handling, and reports usage and cost from the grok CLI stream.
Supports --always-approve for unattended runs, configurable via EVAL_GROK_BUILD_ALWAYS_APPROVE.
Maps tool, usage, cost, and completion events into the shared verifier trajectory, keeping MCP-wrapped tool names, arguments, results, and browser probe evidence.
Distinguishes normal end_turn stops from max-turn stops and process failures so only genuine failures surface as SDK errors.
Hardens the ACP lifecycle: passes --max-screenshot-base64-bytes=60000 to the facade spawned by the launcher and marks BROWSERBASE_PROJECT_ID optional.
Directs the model to verify every task constraint against observed page data before reporting a final answer.
Adds .grok/config.toml, AGENTS.md, docs, and unit tests for the CLI session, runner, and trajectory adapter.

^{Written for commit 58464db. Summary will update on new commits.}

Resolve overview.mdx conflicts to include both Grok Build and fx entries. Keep the screenshot base64 budget from browserbase#2791 alongside the ACP facade launcher changes. Also pass --max-screenshot-base64-bytes=60000 to the facade spawned by the ACP launcher so ACP transports get the same frame-cap protection as fx, mark BROWSERBASE_PROJECT_ID optional in the Grok Build README per browserbase#2777, and update the affected ACP arg assertions.

changeset-bot · 2026-09-02T11:03:06Z

⚠️ No Changeset found

Latest commit: 58464db

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2026-09-02T11:03:12Z

This PR is from an external contributor and must be approved by a stagehand team member with write access before CI can run.
Approving the latest commit mirrors it into an internal PR owned by the approver.
If new commits are pushed later, the internal PR stays open but is marked stale until someone approves the latest external commit and refreshes it.

# Conflicts: # packages/docs/docs.json # packages/docs/v4/integrations/overview.mdx

…2-pr-2876 # Conflicts: # turbo.json

Bring PR browserbase#2876 onto Miguel's facade/eval stack: native `--rules` policy, turn budget, session-loss, keyed observations, and reported usage/cost from the grok CLI stream. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-10-06T09:50:32Z

The latest approval by @filip-michalsky could not refresh the mirrored PR automatically (push-failed). The external PR stays open, and the mirrored PR should be updated manually before work continues.

filip-michalsky · 2026-10-06T10:06:28Z

@antonvishal CI is failing

The short dependency-wait timeout can abort after fetch has already been sent. That late hit was counted in the next phase and failed TypeScript unit under CI load. Co-authored-by: Cursor <cursoragent@cursor.com>

…sertion" This reverts commit 215a91a.

antonvishal · 2026-10-06T10:56:33Z

@filip-michalsky the CI failure is not related to this PR. So created a separated PR to fix CI.

# Conflicts: # packages/evals/package.json # pnpm-lock.yaml

github-actions · 2026-10-06T18:38:31Z

The latest approval by @miguelg719 could not refresh the mirrored PR automatically (push-failed). The external PR stays open, and the mirrored PR should be updated manually before work continues.

miguelg719 · 2026-10-06T18:40:51Z

            packages/integrations/deepagents-sdk/dist/**
            packages/integrations/fx-sdk/dist/**
            packages/integrations/cursor-sdk/dist/**
+            packages/integrations/grok-build-sdk/dist/**


this blocks the mirror pr @filip-michalsky @antonvishal. closing this one, please reopen without touching .github workflows, we will include this in the mirror pr

Created the new PR: #3120

antonvishal added 9 commits August 18, 2026 22:04

feat(integrations): add Grok Build integration and related docs

41f3557

fix(integrations): harden Grok ACP lifecycle

304c403

docs(integrations): list grok-build in the package overview

88adff8

Merge remote-tracking branch 'upstream/main' into acp+grok-build

8b20f74

Merge remote-tracking branch 'upstream/main' into acp+grok-build

ef54737

fix(integrations): address ACP review feedback

6219756

Merge origin/main into ACP and Grok Build integration

7744b73

feat(integrations): add Grok Build harness

46a63e6

antonvishal requested a review from a team as a code owner September 2, 2026 11:03

github-actions Bot added external-contributor Tracks PRs mirrored from external contributor forks. external-contributor:awaiting-approval Waiting for a stagehand team member to approve the latest external commit. labels Sep 2, 2026

antonvishal mentioned this pull request Sep 2, 2026

feat(integrations): add Grok Build integration + Experimental ACP #2772

Closed

antonvishal and others added 8 commits September 4, 2026 12:18

fix(evals): use trusted Grok MCP config scope

5430351

fix(evals): preserve Grok tool results and browser evidence

8908144

fix(evals): recheck Grok task constraints before final answers

1237840

Merge branch 'browserbase:main' into grok-build

ee547cd

Merge remote-tracking branch 'upstream/main' into grok-build

03efa4a

# Conflicts: # packages/docs/docs.json # packages/docs/v4/integrations/overview.mdx

fix(integrations): handle Grok stop reasons and process failures

7ec8129

Merge remote-tracking branch 'upstream/main' into vishal/resolve-oct0…

1528069

…2-pr-2876 # Conflicts: # turbo.json

filip-michalsky reviewed Oct 6, 2026

View reviewed changes

Comment thread packages/integrations/grok-build-sdk/src/session.ts Outdated

filip-michalsky reviewed Oct 6, 2026

View reviewed changes

Comment thread packages/integrations/grok-build/AGENTS.md

filip-michalsky reviewed Oct 6, 2026

View reviewed changes

Comment thread packages/integrations/grok-build/.grok/config.toml

filip-michalsky reviewed Oct 6, 2026

View reviewed changes

Comment thread packages/integrations/grok-build-sdk/src/index.ts

Enhance Grok Build integration with always-approve configuration

b03e756

antonvishal requested a review from filip-michalsky October 6, 2026 08:52

filip-michalsky approved these changes Oct 6, 2026

View reviewed changes

fix CI

de1b771

antonvishal and others added 2 commits October 6, 2026 16:06

fix(release): drain aborted registry probes before the 503 assertion

215a91a

The short dependency-wait timeout can abort after fetch has already been sent. That late hit was counted in the next phase and failed TypeScript unit under CI load. Co-authored-by: Cursor <cursoragent@cursor.com>

Revert "fix(release): drain aborted registry probes before the 503 as…

beead32

…sertion" This reverts commit 215a91a.

antonvishal mentioned this pull request Oct 6, 2026

fix(release): isolate flaky SDK publication probes #3117

Open

antonvishal force-pushed the grok-build branch from 93b9cba to 623416c Compare October 6, 2026 17:35

Merge commit '7abca76d25cbc378ac363c246dd2fa9423492643' into HEAD

58464db

# Conflicts: # packages/evals/package.json # pnpm-lock.yaml

antonvishal force-pushed the grok-build branch from 623416c to 58464db Compare October 6, 2026 18:01

miguelg719 approved these changes Oct 6, 2026

View reviewed changes

miguelg719 reviewed Oct 6, 2026

View reviewed changes

miguelg719 closed this Oct 6, 2026

antonvishal mentioned this pull request Oct 6, 2026

feat(integrations): add Grok Build CLI harness #3120

Open

Navigation Menu

Repository navigation

feat(integrations): add Grok Build CLI harness - #2876

feat(integrations): add Grok Build CLI harness#2876
antonvishal wants to merge 22 commits into
browserbase:mainfrom
antonvishal:grok-build

antonvishal commented Sep 2, 2026 •

edited by cubic-dev-ai Bot

Loading

Uh oh!

changeset-bot Bot commented Sep 2, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 6, 2026

Uh oh!

filip-michalsky commented Oct 6, 2026

Uh oh!

antonvishal commented Oct 6, 2026

Uh oh!

github-actions Bot commented Oct 6, 2026

Uh oh!

miguelg719 Oct 6, 2026

Uh oh!

antonvishal Oct 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

antonvishal commented Sep 2, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Testing

Summary by cubic

Uh oh!

changeset-bot Bot commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

github-actions Bot commented Sep 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 6, 2026

Uh oh!

filip-michalsky commented Oct 6, 2026

Uh oh!

antonvishal commented Oct 6, 2026

Uh oh!

github-actions Bot commented Oct 6, 2026

Uh oh!

miguelg719 Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

antonvishal Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

antonvishal commented Sep 2, 2026 •

edited by cubic-dev-ai Bot

Loading

changeset-bot Bot commented Sep 2, 2026 •

edited

Loading