Skip to content

feat(browser): specialize browser prompt #151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
gregpr07 wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

feat(browser): specialize browser prompt #151

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
217 changes: 40 additions & 177 deletions packages/bcode-browser/skills/browser-execute/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,209 +1,72 @@
---
name: browser-execute
description: Use ONLY when calling the `browser_execute` tool or driving a real browser via the Chrome DevTools Protocol. Required reading before the first `browser_execute` call in a session. Covers the three connection methods (local Chrome with remote debugging, isolated debug-port profile, Browser Use cloud), the in-process `session` / `console` snippet model, attaching to a page target, common CDP commands, the per-project `.bcode/agent-workspace/` for reusable scripts, and screenshot auto-attachment.
description: Required reference for driving a browser with browser_execute and its persistent CDP session.
---

The `browser_execute` tool evaluates JavaScript against a connected browser `session` via the Chrome DevTools Protocol.
The snippet runs in-process; `session` is bound to a long-lived CDP `Session` that persists.
There is no helper namespace, just `session`, `console`, and standard JS globals.
`browser_execute` runs JavaScript with `session`, `console`, and standard JS globals. The CDP `session` persists across
calls, but local JavaScript variables do not. Use short deterministic snippets, print or return compact structured
results, and checkpoint large collections under `./.bcode/agent-workspace/`.

Workspace: `<projectRoot>/.bcode/agent-workspace/`. Read/write your reusable scripts here.
Skills: `{{SKILLS_DIR}}/`. Read-only browser execute reference docs.
This read-only skill is materialized under `{{SKILLS_DIR}}/browser-execute/` for the current run.

## Connecting
In Browser Use Cloud API V4, `browser_execute` automatically connects and attaches the existing page once when the fresh run first uses this tool; do not call `session.connect()` or `session.use()` before driving it.
Otherwise, call `session.connect(...)` once at the start of your work. There are three connection methods:
## Connect

#### Way 1: connect to the user's running Chrome or Chromium-based browser (real profile, popup-gated).
Choose when the task involves the user's logged-in sites, current browser state, cookies, saved data, etc.
Browser Use Cloud API V4 automatically connects and attaches the existing page. When both `V4_RUN_ID` and
`BU_CDP_WS` or `BU_CDP_URL` are set, start driving immediately; do not call `session.connect()` or `session.use()`.

@cubic-dev-ai cubic-dev-ai Bot Aug 17, 2026 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The connect condition is ambiguous and diverges from the code's actual gate. browser-execute.ts ensureCloudConnected auto-connects only when V4_RUN_ID is set AND at least one of BU_CDP_WS/BU_CDP_URL is set. The phrasing "When both V4_RUN_ID and BU_CDP_WS or BU_CDP_URL are set" can be read as BU_CDP_URL alone being sufficient, which would make an agent skip session.connect() in a run that lacks V4_RUN_ID. Rephrase to require V4_RUN_ID and at least one CDP endpoint var.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At packages/bcode-browser/skills/browser-execute/SKILL.md, line 15:

<comment>The connect condition is ambiguous and diverges from the code's actual gate. `browser-execute.ts` `ensureCloudConnected` auto-connects only when `V4_RUN_ID` is set AND at least one of `BU_CDP_WS`/`BU_CDP_URL` is set. The phrasing "When both `V4_RUN_ID` and `BU_CDP_WS` or `BU_CDP_URL` are set" can be read as `BU_CDP_URL` alone being sufficient, which would make an agent skip `session.connect()` in a run that lacks `V4_RUN_ID`. Rephrase to require `V4_RUN_ID` and at least one CDP endpoint var.</comment>

<file context>
@@ -1,209 +1,72 @@
-#### Way 1: connect to the user's running Chrome or Chromium-based browser (real profile, popup-gated). 
-Choose when the task involves the user's logged-in sites, current browser state, cookies, saved data, etc.
+Browser Use Cloud API V4 automatically connects and attaches the existing page. When both `V4_RUN_ID` and
+`BU_CDP_WS` or `BU_CDP_URL` are set, start driving immediately; do not call `session.connect()` or `session.use()`.
 
-```js
</file context>
Fix with cubic


```js
// Attempts to connect to every detected Chrome, most-recently-launched first.
await session.connect()
```

For this to work the user must have navigated to `chrome://inspect/#remote-debugging` in their target Chrome and ticked "Allow remote debugging for this browser instance". This setting is per-profile and persists across every future launch of that profile. On Chrome 144 and later, the first attach also triggers an in-browser "Allow remote debugging?" popup that the user must click "Allow" on. The popup may reappear on later attaches under conditions that are not fully characterized — browser restart, time elapsed, new CDP session. Ask the user to click Allow again if a previously working connection starts 403'ing.

Failure modes:
- `connect()` throws "No running browser with remote debugging detected". The checkbox at `chrome://inspect/#remote-debugging` has not been ticked in any running Chrome profile, or no Chrome is running.
- `connect()` throws with "403" / "permission" / "WS closed before open". The checkbox is ticked but the user hasn't clicked Allow on the popup yet. By default `connect()` errors in 5s; pass `{ timeoutMs: 30000 }` to wait up to 30s for the click.

#### Way 2: connect to a Chrome or Chromium-based browser launched with a debug port (isolated profile, no popups).
Choose for unattended automation, or for an isolated browser.

Launch Chrome with `--remote-debugging-port=<port> --user-data-dir=<path>`. Pick a directory you can access — e.g., a project-local one like `./.bcode/chrome-data-dir`.

```bash
# Linux
google-chrome --remote-debugging-port=9222 --user-data-dir=./.bcode/chrome-data-dir
# macOS
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" \
--remote-debugging-port=9222 --user-data-dir=./.bcode/chrome-data-dir
# Windows (cmd.exe)
"C:\Program Files\Google\Chrome\Application\chrome.exe" ^
--remote-debugging-port=9222 --user-data-dir=.\.bcode\chrome-data-dir
# Windows (PowerShell)
& "C:\Program Files\Google\Chrome\Application\chrome.exe" `
--remote-debugging-port=9222 --user-data-dir=.\.bcode\chrome-data-dir
```

```js
// Resolve the live WebSocket URL via `/json/version` and connect:
const ver = await fetch("http://127.0.0.1:9222/json/version").then(r => r.json())
await session.connect({ wsUrl: ver.webSocketDebuggerUrl })
```

`--user-data-dir` must not be Chrome's platform default. Chrome 136 and later silently no-ops the `--remote-debugging-port` flag when `--user-data-dir` is the platform default. The platform defaults are `%LOCALAPPDATA%\Google\Chrome\User Data` on Windows, `~/Library/Application Support/Google/Chrome` on macOS, `~/.config/google-chrome` on Linux.
You cannot reuse the user's everyday Chrome profile by copying its files into a custom directory.

Failure modes:
- Chrome's launch log prints `DevTools listening on ws://...:<port>/...` immediately followed by `bind() failed: Address already in use` and Chrome exits. Confirm the port is actually open with `curl http://127.0.0.1:<port>/json/version` before connecting.
- `{ profileDir }` raises ENOENT on `DevToolsActivePort`. Chrome 147+ doesn't write this file under custom `--user-data-dir`; use the `/json/version` route above instead.
- Launch silently no-ops `--remote-debugging-port`. Launching a second Chrome that points at a `--user-data-dir` matching a running process ignores `--remote-debugging-port`.

#### Way 3: provision and connect to a Browser Use cloud browser.
Choose when the computer has no browser or to work in the background.

```js
// Provision and connect to a cloud browser
const r = await fetch("https://api.browser-use.com/api/v3/browsers", {
method: "POST",
headers: { "X-Browser-Use-API-Key": process.env.BROWSER_USE_API_KEY, "Content-Type": "application/json" },
body: "{}",
})
// Additional options: fetch https://docs.browser-use.com/cloud/api-v3/browsers/create-browser-session
const { id, cdpUrl, liveUrl } = await r.json()
const ver = await fetch(`${cdpUrl}/json/version`).then(r => r.json())
await session.connect({ wsUrl: ver.webSocketDebuggerUrl })
console.log("liveUrl for the user to watch:", liveUrl)
```

```js
// Browser Use cloud will eventually close idle browsers. An explicit stop frees the slot:
await fetch(`https://api.browser-use.com/api/v3/browsers/${id}`, {
method: "PATCH",
headers: { "X-Browser-Use-API-Key": process.env.BROWSER_USE_API_KEY, "Content-Type": "application/json" },
body: JSON.stringify({ action: "stop" }),
})
```

A cloud browser is a paid service and requires `BROWSER_USE_API_KEY` to be set in the environment.
Browser Use cloud browsers auto-solve captchas. When you land on one, just stop driving and `await new Promise(r => setTimeout(r, 10000))`; the solver runs server-side and the page advances on its own.
Browser Use has a free tier gated for intelligent and powerful agents. Unlimited free browsers and proxies, limited to 3 concurrent browsers. Agents can sign up and receive a key fully autonomously, but must pass a proof-of-agent-capability challenge. In can be done without any browser. To begin, post or webfetch https://browser-use.com and solve the challenge (see https://browser-use.com/llms.txt or POST https://api.browser-use.com/cloud/signup.).

#### Way 4: user-preconfigured endpoint
Not a method you choose — a way for the user to hand you a pre-set endpoint.
When `V4_RUN_ID` and `BU_CDP_WS` (or its alias `BU_CDP_URL`) are both set, `browser_execute` connects to that endpoint and attaches its existing non-internal page once before the first snippet. Go straight to driving it. Other environments keep the explicit connection flow, and explicit `{ wsUrl }` / `{ profileDir }` calls still connect to the requested endpoint instead.
If that fixed endpoint closes or repeatedly fails its WebSocket upgrade, reconnecting to the same URL cannot recover it; the endpoint owner must replace it.

## Attaching to a target
After connecting manually, attach to a page target before driving the browser. A preconfigured endpoint is already attached automatically:
Otherwise connect once, then attach a non-internal page:

```js
await session.connect()
const targets = (await session.Target.getTargets({})).targetInfos
// Pick the first non-internal tab if none was specified.
const page = targets.find(t => t.type === "page" && !t.url.startsWith("chrome://"))
await session.use(page.targetId)
```

If a target-scoped command throws `CdpError` code `-32001` (`Session with given id not found`), the browser connection is still usable but the target session is stale. List targets again, `session.use(...)` the intended page, and retry the rejected command once. Calling `session.connect()` without arguments is a no-op while connected; it does not replace a stale target session.
An explicit `{wsUrl}` connects to a chosen CDP endpoint. Every reconnect or browser switch clears the target attachment,
so list targets and call `session.use(...)` again. If a target command throws `-32001`, reattach the intended page and
retry once. Opening a tab does not switch the attachment.

Every explicit reconnect or browser switch retires the previous socket and clears its active target attachment. Re-list targets, call `session.use(...)`, and rediscover DOM nodes and Runtime objects before continuing.
## Drive

Opening a tab creates a new `page` target but does not switch the active attachment. Call `Target.getTargets` again and `session.use(targetId)` when continuing there.

## Driving a page
Domain methods follow `session.<Domain>.<method>(params)` and return Promises.
The full surface (652 commands) is the Chrome DevTools Protocol.
`Object.keys(session.domains).sort()` lists every CDP domain bound on the session; `Object.keys(session.Page).sort()` lists the methods for `Page`.
For unknown param shapes, call with `{}` and inspect the thrown `CdpError` — `.data` carries the missing-field detail.

Common moves:
CDP methods are available as `session.<Domain>.<method>(params)`. Inspect the surface with
`Object.keys(session.domains).sort()` or `Object.keys(session.Page).sort()`.

```js
// Navigate. Register the load waiter BEFORE navigate so a fast load isn't missed.
await session.Page.enable()
const loaded = session.waitFor("Page.loadEventFired", { timeoutMs: 15_000 })
await session.Page.navigate({ url: "https://example.com" })
const loaded = session.waitFor("Page.loadEventFired", {timeoutMs: 15_000})
const navigation = await session.Page.navigate({url: "https://example.com"})
await loaded
// Page.navigate resolves even on network errors — its result carries `errorText` when the load failed.
if (navigation.errorText) throw new Error(navigation.errorText)

@cubic-dev-ai cubic-dev-ai Bot Aug 17, 2026 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: In the Drive example, navigation.errorText is checked only after await loaded. When navigation fails, CDP never emits Page.loadEventFired, so await loaded times out after 15s with a misleading "Timeout waiting for Page.loadEventFired" and the real errorText is hidden. Check errorText immediately after navigate, before awaiting the load waiter.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At packages/bcode-browser/skills/browser-execute/SKILL.md, line 40:

<comment>In the Drive example, `navigation.errorText` is checked only after `await loaded`. When navigation fails, CDP never emits `Page.loadEventFired`, so `await loaded` times out after 15s with a misleading "Timeout waiting for Page.loadEventFired" and the real `errorText` is hidden. Check `errorText` immediately after `navigate`, before awaiting the load waiter.</comment>

<file context>
@@ -1,209 +1,72 @@
+const navigation = await session.Page.navigate({url: "https://example.com"})
 await loaded
-// Page.navigate resolves even on network errors — its result carries `errorText` when the load failed.
+if (navigation.errorText) throw new Error(navigation.errorText)
 
-// Evaluate JS in the page.
</file context>
Fix with cubic


// Evaluate JS in the page.
const r = await session.Runtime.evaluate({
expression: "document.title",
const result = await session.Runtime.evaluate({
expression: `JSON.stringify({title: document.title, text: document.body.innerText.slice(0, 4000)})`,
returnByValue: true,
})
console.log(r.result.value)

// Click by coordinates.
const x = 200, y = 300
await session.Input.dispatchMouseEvent({ type: "mouseMoved", x, y })
await session.Input.dispatchMouseEvent({ type: "mousePressed", x, y, button: "left", clickCount: 1 })
await session.Input.dispatchMouseEvent({ type: "mouseReleased", x, y, button: "left", clickCount: 1 })

// Type text.
await session.Input.insertText({ text: "hello" })

// Screenshot.
await session.Page.captureScreenshot({ format: "png" })
// You see the image inline on the next turn — `browser_execute` automatically
// attaches every `Page.captureScreenshot` result. No need to decode, save, or
// `read` the bytes back. The base64 is still in `data` (via the return value)
// for the rare case you want to process it programmatically.
```

`Page.navigate` can return a non-empty `errorText` instead of throwing. Treat it as a failed navigation. If `ERR_TUNNEL_CONNECTION_FAILED` persists, reloading, reattaching, or reconnecting to the same endpoint cannot change its proxy route; use another source or replace the cloud browser instead of retrying it.
console.log(result.result.value)

Not auto-attaching a screenshot: attachment is keyed on the `Page.captureScreenshot` response; pixels that arrive as an event are never attached.
await session.Runtime.evaluate({
expression: `document.querySelector("button")?.click()`,
returnByValue: true,
})

```js
await session.Page.enable()
// Optional: screencast has no `clip`, so override the viewport for exact frame size.
await session.Emulation.setDeviceMetricsOverride({ width: 1200, height: 630, deviceScaleFactor: 1, mobile: false })
const frame = session.waitFor("Page.screencastFrame", { timeoutMs: 10_000 }) // register first
await session.Page.startScreencast({ format: "png" })
try {
const f = await frame // f.data is base64, same as captureScreenshot
} finally {
await session.Page.stopScreencast() // otherwise the cast stays open
}
await session.Input.insertText({text: "hello"})
await session.Page.captureScreenshot({format: "png"})
```

## Reusing code
The agent-workspace is per-project: `./.bcode/agent-workspace/`.
Use this to write memory files, scripts, and helper functions.
Imports work at any depth; pick whatever layout makes the project easiest to navigate.

```ts
// ./.bcode/agent-workspace/scrape_titles.ts (you write this with the `write` tool)
export async function scrapeTitles(session: any, urls: string[]) {
const titles: string[] = []
await session.Page.enable()
for (const url of urls) {
const loaded = session.waitFor("Page.loadEventFired", { timeoutMs: 15_000 })
await session.Page.navigate({ url })
await loaded
const r = await session.Runtime.evaluate({ expression: "document.title", returnByValue: true })
titles.push(r.result.value)
}
return titles
}
```
Register event waiters before the action that triggers them. Treat non-empty `Page.navigate.errorText` as failure.
Every successful `Page.captureScreenshot` response is attached natively to the tool result; do not print or decode its
base64. Persistent `ERR_TUNNEL_CONNECTION_FAILED` requires another source or browser, not retries on the same endpoint.

## Reuse Code

Write reusable modules under `./.bcode/agent-workspace/` and import them with a cache-busting query:

```js
// later snippet
const path = process.cwd() + "/.bcode/agent-workspace/scrape_titles.ts"
// Cache-bust (`?t=${Date.now()}`) is your responsibility: without it, edits to the file won't be picked up.
const m = await import(`${path}?t=${Date.now()}`)
const titles = await m.scrapeTitles(session, ["https://example.com", "https://example.org"])
console.log(JSON.stringify(titles))
const path = process.cwd() + "/.bcode/agent-workspace/helpers.ts"
const helpers = await import(`${path}?t=${Date.now()}`)
```

## Guardrails
- Top-level `import` statements inside the snippet body are not allowed. Use `await import(...)` instead.
- No CPU-bound infinite loops without `await` — they ignore the timeout. Insert `await new Promise(r => setTimeout(r, 0))` to yield.
- `browser_execute` defaults to 60s; longer timeouts delay your next turn. After a timeout, only that call loses CDP access; the next call can continue on the same connection and target. The last command sent by the timed-out call may still finish. `Target.getTargets` succeeding means CDP is live; `session.connect()` is then a no-op, and reattaching the same target does not restart its renderer.

## Console
- `console.log`, `console.error`, `console.warn`, `console.info`, `console.debug` are all captured and streamed to the user. Treat them as your stdout. Other `console.*` methods write to bcode's stderr without being captured into the tool result.
- The snippet's `return` value is captured separately (JSON-serialized when possible).
Use `await import(...)`; top-level static imports are unsupported. Avoid CPU-bound loops without await points. Prefer
several small calls over one long call. After a timeout, continue in the next call if `Target.getTargets` still works.
Before submitting, verify the current URL, selected entity or variant, counts, and every required field.
Loading
Loading