From f11d1ad98c3cad5bc8acc9a758d9cff86185a136 Mon Sep 17 00:00:00 2001 From: Matt Carey Date: Sun, 18 Oct 2026 13:33:10 +0200 Subject: [PATCH 1/3] computer: Add a ws:browser host module createBrowserModule() from @cloudflare/computer/modules/browser lets isolate JavaScript drive a persistent Chrome browser over the Chrome DevTools Protocol: send, attachToTarget, spec, getDebugLog and clearDebugLog. The browser belongs to the host. The module takes anything with the BrowserSource shape, a name and a connect({ scope }) that returns a CDP connection and the scope's active tab. Browser from agents/browser has that shape, so neither package depends on the other. Each run gets its own CDP connection, keyed by the call context's executionId. A deferred function saves the scope's active tab, closes the connection, and writes restarts, closed or shared tabs and tabs the page opened to the run's standard error. sessionId "active", the attach handles that work across runs, scopes, refusing Browser.close and the hints for common CDP mistakes follow the browser tool in agents. The module is experimental. --- .changeset/browser-module.md | 5 + docs/17_isolate_javascript.md | 41 + packages/computer/README.md | 2 + packages/computer/package.json | 4 + packages/computer/rolldown.config.ts | 1 + packages/computer/src/modules/browser.test.ts | 296 +++++++ packages/computer/src/modules/browser.ts | 817 ++++++++++++++++++ .../computer/tests/script-runner-worker.ts | 42 + packages/computer/tests/script-runner.test.ts | 30 + 9 files changed, 1238 insertions(+) create mode 100644 .changeset/browser-module.md create mode 100644 packages/computer/src/modules/browser.test.ts create mode 100644 packages/computer/src/modules/browser.ts diff --git a/.changeset/browser-module.md b/.changeset/browser-module.md new file mode 100644 index 000000000..f1f431738 --- /dev/null +++ b/.changeset/browser-module.md @@ -0,0 +1,5 @@ +--- +"@cloudflare/computer": minor +--- + +Adds the experimental `ws:browser` host module (`@cloudflare/computer/modules/browser`), which lets isolate JavaScript drive a persistent browser over the Chrome DevTools Protocol. diff --git a/docs/17_isolate_javascript.md b/docs/17_isolate_javascript.md index 83243021c..c814dec60 100644 --- a/docs/17_isolate_javascript.md +++ b/docs/17_isolate_javascript.md @@ -434,6 +434,47 @@ On a backend with `access: "read"`, only tools marked `readOnly` run. Every othe The module's description lists the names code can import. With a function of tools, the function is called again each time the backend describes itself, so the `exec` tool, built after the Workspace, sees the current list. A list is read as given; any other iterable may be single-use, so it is not read for the description, and neither is a function that throws. +### `ws:browser` + +> [!NOTE] +> **Experimental.** + +```js +import { send, attachToTarget, spec, getDebugLog, clearDebugLog } from "ws:browser"; +``` + +`createBrowserModule({ browser, scope?, maxResultBytes? })` from `@cloudflare/computer/modules/browser` lets JavaScript drive a persistent Chrome browser over the Chrome DevTools Protocol. The browser belongs to the host: the module takes anything with the `BrowserSource` shape, a `name` and a `connect({ scope })` that returns a CDP connection plus the scope's active tab. `Browser` from `agents/browser` has that shape, so an agent passes its own browser: + +```ts +import { Browser, browserRun } from "agents/browser"; +import { createBrowserModule } from "@cloudflare/computer/modules/browser"; + +browser = new Browser({ provider: browserRun(this.env.BROWSER) }); + +new WorkerJavaScriptBackend({ + loader: this.env.LOADER, + modules: { "ws:browser": createBrowserModule({ browser: this.browser }) }, +}); +``` + +```js +import { writeFile } from "node:fs/promises"; +import { Buffer } from "node:buffer"; +import { send } from "ws:browser"; + +export default async function () { + await send({ method: "Page.navigate", params: { url: "https://example.com" }, sessionId: "active" }); + const { data } = await send({ method: "Page.captureScreenshot", sessionId: "active" }); + await writeFile("/workspace/page.png", Buffer.from(data, "base64")); +} +``` + +`send({ method, params?, sessionId?, timeoutMs? })` returns a command's result without the JSON-RPC envelope. `sessionId: "active"` is the tab the scope is working in. The module remembers it between runs, and opens a new blank tab when the scope has none. `attachToTarget({ targetId })` makes another open tab active and returns a `sessionId` handle that keeps working in later runs. A session id from a raw `Target.attachToTarget` only lasts for one run. `Browser.close` and `Browser.crash` are refused, because the host decides when the browser ends. + +Each run gets its own CDP connection, opened on its first call. When the run ends, the module saves the scope's active tab and closes the connection. The browser stays open. If the browser was replaced since the scope last used it, if another scope's tab closed or shares this one, or if the page opened tabs, the module says so on the run's standard error. The module's description, which the `exec` tool shows, covers the calls and the CDP mistakes models commonly make. + +`scope` keeps conversations that share one browser in their own tabs, while cookies and logins stay shared. A `Page.captureScreenshot` result over `maxResultBytes` (1,000,000 by default) fails with a note to capture the viewport or use JPEG. Keep it under the backend's `maxCapabilityBytes`. + ### `ws:container` `createContainerModule()` from `@cloudflare/computer/modules/container` lets JavaScript run shell commands in the Workspace's container backend. With it, JavaScript is the only backend the model sees, and the container is something that JavaScript can call: diff --git a/packages/computer/README.md b/packages/computer/README.md index d5539d43d..dde327016 100644 --- a/packages/computer/README.md +++ b/packages/computer/README.md @@ -33,6 +33,7 @@ removed in a future release; each one names its replacement. | `LegacyContainerBackend` and `withLegacyWorkspaceContainer` | `@cloudflare/computer/backends/container-legacy` | Deprecated: use `ContainerBackend` | | `WorkerShellBackend` and the shell command groups | `@cloudflare/computer/backends/worker-shell`, `@cloudflare/computer/shell/*` | Experimental | | The `ws:container` host module | `@cloudflare/computer/modules/container` | Experimental | +| The `ws:browser` host module | `@cloudflare/computer/modules/browser` | Experimental | | `workspace.push()`, `workspace.pull()`, and the sync protocol between the durable object and `computerd` | `@cloudflare/computer` | Experimental | | The `computerd` daemon and its container image | `ghcr.io/cloudflare/computer-computerd-linux-x64` | Experimental | | The RPC wire protocol between the durable object and `computerd` (`@cloudflare/computer-rpc`) | Internal | Experimental | @@ -514,6 +515,7 @@ on a computerd instance. | `@cloudflare/computer/modules/git` | `createGitModule()` for `ws:git`: confined Git from isolate JavaScript. | | `@cloudflare/computer/modules/artifacts` | `createArtifactsModule()` for `ws:artifacts`: Artifacts and git tokens from isolate JavaScript. | | `@cloudflare/computer/modules/assets` | `createAssetsModule()` for `ws:assets`: publish a Workspace file from isolate JavaScript. | +| `@cloudflare/computer/modules/browser` | `createBrowserModule()` for `ws:browser`: drive a persistent browser over the Chrome DevTools Protocol from isolate JavaScript. | | `@cloudflare/computer/modules/tools` | `createToolBindings()` for `ws:tools`: an agent's own tools, callable from isolate JavaScript. | | `@cloudflare/computer/tools` | AI SDK tools for agents: `read`, `ls`, `find`, `grep`, `write`, `edit`, `delete`, and optional `exec` and `publish`. | | `@cloudflare/computer/tools/core` | `defineExec()`: the `exec` tool's description, schema, and executor, with no agent library. | diff --git a/packages/computer/package.json b/packages/computer/package.json index 43e4b509f..9903c057e 100644 --- a/packages/computer/package.json +++ b/packages/computer/package.json @@ -51,6 +51,10 @@ "types": "./dist/modules/assets.d.ts", "import": "./dist/modules/assets.js" }, + "./modules/browser": { + "types": "./dist/modules/browser.d.ts", + "import": "./dist/modules/browser.js" + }, "./tools": { "types": "./dist/tools/index.d.ts", "import": "./dist/tools/index.js" diff --git a/packages/computer/rolldown.config.ts b/packages/computer/rolldown.config.ts index 5c8818b4e..2b31e65e0 100644 --- a/packages/computer/rolldown.config.ts +++ b/packages/computer/rolldown.config.ts @@ -40,6 +40,7 @@ export default defineConfig({ "modules/artifacts": "src/modules/artifacts.ts", "modules/tools": "src/modules/tools.ts", "modules/assets": "src/modules/assets.ts", + "modules/browser": "src/modules/browser.ts", "backends/container-legacy/index": "src/backends/container-legacy/index.ts", "backends/container/index": "src/backends/container/index.ts", "backends/worker-javascript/index": "src/backends/worker-javascript/index.ts", diff --git a/packages/computer/src/modules/browser.test.ts b/packages/computer/src/modules/browser.test.ts new file mode 100644 index 000000000..ea0690890 --- /dev/null +++ b/packages/computer/src/modules/browser.test.ts @@ -0,0 +1,296 @@ +import { describe, expect, it } from "vitest"; + +import type { + WorkspaceModuleCallContext, + WorkspaceModuleFunctions, + WorkspaceModuleHost, + WorkspaceRuntimeValue, +} from "../runtime/types.js"; +import { + createBrowserModule, + type BrowserCdp, + type BrowserConnection, + type BrowserModuleOptions, + type BrowserSource, +} from "./browser.js"; + +// SAFETY: The browser module never touches the host's Git, Artifacts, or runtime. +const host = {} as WorkspaceModuleHost; + +class ProtocolError extends Error { + constructor( + readonly code: number, + message: string, + ) { + super(message); + } +} + +interface Tab { + targetId: string; + type: "page"; + url: string; + openerId?: string; +} + +// One browser: tabs shared by every connection, an active tab per scope, +// and Chrome's errors for an unknown session or a page command without one. +class FakeBrowser implements BrowserSource { + readonly name = "work"; + tabs: Tab[] = [{ targetId: "target-1", type: "page", url: "about:blank" }]; + active = new Map(); + connections: { disconnected: boolean }[] = []; + restartNext = false; + screenshotBytes = 10; + #nextTarget = 1; + #nextSession = 0; + #sessions = new Map(); + + addTab(url: string, openerId?: string): string { + const targetId = `target-${++this.#nextTarget}`; + this.tabs.push({ targetId, type: "page", url, ...(openerId ? { openerId } : {}) }); + return targetId; + } + + async connect(options: { scope?: string } = {}): Promise { + const scope = options.scope ?? "shared"; + const restarted = this.restartNext; + if (restarted) { + this.restartNext = false; + this.tabs = []; + this.active.clear(); + } + const state = { disconnected: false }; + this.connections.push(state); + const sessions = new Set(); + const cdp: BrowserCdp = { + send: async (method, params = {}, sendOptions = {}) => { + if (state.disconnected) throw new Error("socket closed"); + return this.#handle(method, params, sendOptions.sessionId); + }, + attachToTarget: async (targetId) => { + const sessionId = `cdp-${++this.#nextSession}`; + this.#sessions.set(sessionId, targetId); + sessions.add(sessionId); + return sessionId; + }, + getDebugLog: () => [{ at: "now", type: "send" }], + clearDebugLog: () => {}, + disconnect: () => { + state.disconnected = true; + for (const sessionId of sessions) this.#sessions.delete(sessionId); + }, + }; + return { + restarted, + activeTargetId: this.active.get(scope), + cdp, + setActiveTarget: async (targetId) => { + this.active.set(scope, targetId); + return true; + }, + targetsInOtherScopes: async () => + new Set( + [...this.active] + .filter(([other, target]) => other !== scope && target !== undefined) + .map(([, target]) => target as string), + ), + spec: async () => ({ + domains: [{ name: "Page", events: [{ event: "Page.loadEventFired" }] }], + }), + }; + } + + #handle(method: string, params: Record, sessionId: string | undefined) { + switch (method) { + case "Target.getTargets": + return { targetInfos: this.tabs }; + case "Target.createTarget": + return { targetId: this.addTab(String(params.url ?? "")) }; + case "Target.closeTarget": + this.tabs = this.tabs.filter((tab) => tab.targetId !== params.targetId); + return { success: true }; + case "Target.attachToTarget": { + const id = `cdp-${++this.#nextSession}`; + this.#sessions.set(id, String(params.targetId)); + return { sessionId: id }; + } + default: + if (sessionId && !this.#sessions.has(sessionId)) { + throw new ProtocolError(-32001, "Session with given id not found."); + } + if (!sessionId) throw new ProtocolError(-32601, `'${method}' wasn't found`); + if (method === "Page.captureScreenshot") return { data: "x".repeat(this.screenshotBytes) }; + if (method === "Runtime.evaluate") { + const target = this.#sessions.get(sessionId); + if (params.expression === "openPopup()") this.addTab("https://popup.example/", target); + return { result: { value: target } }; + } + return {}; + } + } +} + +function setup(options: Partial = {}) { + const browser = new FakeBrowser(); + const module = createBrowserModule({ browser, ...options }); + const functions: WorkspaceModuleFunctions = module(host); + let runs = 0; + + // Run `calls` as one execution, as the JavaScript backend does: every + // call has the run's id, and deferred functions run last first at the end. + async function run(calls: Array<[name: string, args?: WorkspaceRuntimeValue]>) { + const logs: string[] = []; + const deferred: Array<() => unknown> = []; + const context: WorkspaceModuleCallContext = { + executionId: `run-${++runs}`, + signal: new AbortController().signal, + deadline: Date.now() + 60_000, + access: "read-write", + resolvePath: async (path) => path, + log: (text) => logs.push(text), + defer: (fn) => deferred.push(fn), + }; + const results: unknown[] = []; + let error: unknown; + try { + for (const [name, args] of calls) { + results.push(await functions[name](args === undefined ? [] : [args], context)); + } + } catch (caught) { + error = caught; + } + while (deferred.length > 0) await deferred.pop()?.(); + return { results, error, logs }; + } + + return { browser, module, functions, run }; +} + +const evaluate = (expression: string): WorkspaceRuntimeValue => ({ + method: "Runtime.evaluate", + params: { expression }, + sessionId: "active", +}); + +describe("ws:browser", () => { + it("exports the browser calls and describes them for a model", () => { + const { functions, module } = setup(); + expect(Object.keys(functions).sort()).toEqual([ + "attachToTarget", + "clearDebugLog", + "getDebugLog", + "send", + "spec", + ]); + expect(module.description).toContain("send({ method"); + expect(module.description).toContain("standard error"); + expect(module.description).not.toContain("cdp."); + }); + + it("opens a tab for the scope and keeps it between runs", async () => { + const t = setup(); + const first = await t.run([["send", evaluate("1")]]); + expect(first.error).toBeUndefined(); + // Never the browser's initial blank tab, which may be another scope's. + expect(first.results[0]).toEqual({ result: { value: "target-2" } }); + expect(t.browser.active.get("shared")).toBe("target-2"); + expect(t.browser.connections.every((connection) => connection.disconnected)).toBe(true); + + const second = await t.run([["send", evaluate("1")]]); + expect(second.results[0]).toEqual({ result: { value: "target-2" } }); + }); + + it("keeps scopes in their own tabs", async () => { + const shared = setup(); + const other = createBrowserModule({ browser: shared.browser, scope: "other" })(host); + await shared.run([["send", evaluate("1")]]); + const deferred: Array<() => unknown> = []; + const result = await other.send([evaluate("1")], { + executionId: "other-run", + signal: new AbortController().signal, + deadline: Date.now() + 60_000, + access: "read-write", + resolvePath: async (path) => path, + log: () => {}, + defer: (fn) => deferred.push(fn), + }); + while (deferred.length > 0) await deferred.pop()?.(); + expect(result).toEqual({ result: { value: "target-3" } }); + expect(shared.browser.active).toEqual( + new Map([ + ["shared", "target-2"], + ["other", "target-3"], + ]), + ); + }); + + it("returns an attach handle that works in later runs", async () => { + const t = setup(); + const first = await t.run([["attachToTarget", { targetId: "target-1" }]]); + expect(first.results[0]).toEqual({ sessionId: "target:target-1" }); + const second = await t.run([ + [ + "send", + { method: "Runtime.evaluate", params: { expression: "1" }, sessionId: "target:target-1" }, + ], + ]); + expect(second.results[0]).toEqual({ result: { value: "target-1" } }); + }); + + it("writes tabs the page opened to standard error", async () => { + const t = setup(); + const { logs } = await t.run([["send", evaluate("openPopup()")]]); + expect(logs).toEqual([ + 'New tabs opened by the page: [{"targetId":"target-3","url":"https://popup.example/"}]', + ]); + }); + + it("says so on standard error when the browser was replaced", async () => { + const t = setup(); + t.browser.restartNext = true; + const { logs, error } = await t.run([["send", { method: "Target.getTargets" }]]); + expect(error).toBeUndefined(); + expect(logs).toEqual([expect.stringContaining("The browser was restarted before this run")]); + }); + + it("explains a page-scoped command sent without a session", async () => { + const t = setup(); + const { error } = await t.run([["send", { method: "Page.navigate" }]]); + expect(String(error)).toContain('pass sessionId: "active"'); + }); + + it("explains a CDP session id from an earlier run", async () => { + const t = setup(); + const { error } = await t.run([ + ["send", { method: "Runtime.evaluate", params: {}, sessionId: "cdp-99" }], + ]); + expect(String(error)).toContain("CDP session ids don't carry over between runs"); + }); + + it("refuses an oversized screenshot with how to take a smaller one", async () => { + const t = setup({ maxResultBytes: 50 }); + t.browser.screenshotBytes = 100; + const { error } = await t.run([ + ["send", { method: "Page.captureScreenshot", sessionId: "active" }], + ]); + expect(String(error)).toContain("Capture just the viewport"); + }); + + it("refuses bad arguments and commands that would end the browser", async () => { + const t = setup(); + const missing = await t.run([["send", { params: {} }]]); + expect(String(missing.error)).toContain("send needs `method`"); + const notObject = await t.run([["send", "Page.navigate"]]); + expect(String(notObject.error)).toContain("takes one options object"); + const close = await t.run([["send", { method: "Browser.close" }]]); + expect(String(close.error)).toContain("Browser.close is not available"); + }); + + it("leaves the browser alone for a run that never called it", async () => { + const t = setup(); + const { logs } = await t.run([]); + expect(logs).toEqual([]); + expect(t.browser.connections).toEqual([]); + }); +}); diff --git a/packages/computer/src/modules/browser.ts b/packages/computer/src/modules/browser.ts new file mode 100644 index 000000000..f8861d3a3 --- /dev/null +++ b/packages/computer/src/modules/browser.ts @@ -0,0 +1,817 @@ +// `ws:browser`: a persistent Chrome browser for isolate JavaScript, over +// the Chrome DevTools Protocol. +// +// import { send, attachToTarget } from "ws:browser"; +// +// The browser belongs to the host. This module only needs something that +// can connect to it (see `BrowserSource`), such as `Browser` from +// `agents/browser`, so it doesn't depend on any one browser provider. + +import type { + WorkspaceModuleCallContext, + WorkspaceModuleFactory, + WorkspaceModuleFunctions, + WorkspaceRuntimeValue, +} from "../runtime/types.js"; + +/** + * A CDP connection to a browser, as `BrowserSource.connect` returns it. + * `Browser` from `agents/browser` provides one. + */ +export interface BrowserCdp { + /** Send a CDP command and resolve with its result, without the JSON-RPC envelope. */ + send( + method: string, + params?: Record, + options?: { sessionId?: string; timeoutMs?: number }, + ): Promise; + /** Attach to a target and resolve with the CDP session id, valid on this connection only. */ + attachToTarget(targetId: string, options?: { timeoutMs?: number }): Promise; + /** Recent protocol traffic, newest last. */ + getDebugLog(limit?: number): unknown[]; + /** Forget the protocol traffic recorded so far. */ + clearDebugLog(): void; + /** Close the connection. The browser stays open. */ + disconnect(): void; +} + +/** The CDP description a browser serves, as far as this module reads it. */ +export interface BrowserCdpSpec { + domains: Array<{ name: string; events: Array<{ event: string }> }>; +} + +/** One connection to the browser for one scope. */ +export interface BrowserConnection { + /** + * `true` when this scope worked in an earlier browser that has since been + * replaced, so its tabs and logins are gone. + */ + restarted: boolean; + /** The tab this scope last worked in, if it recorded one. */ + activeTargetId?: string; + /** The CDP connection. Closing it doesn't close the browser. */ + cdp: BrowserCdp; + /** Record the tab this scope is working in, or clear it with `undefined`. */ + setActiveTarget(targetId: string | undefined): Promise; + /** The tabs other scopes are working in. */ + targetsInOtherScopes(): Promise>; + /** The Chrome DevTools Protocol description the browser serves. */ + spec(): Promise; +} + +/** + * The browser the module drives. `Browser` from `agents/browser` has this + * shape. `connect` reattaches to the browser, or replaces a lost one, and + * opens a CDP connection for a scope. + */ +export interface BrowserSource { + /** The browser's name, for log messages. */ + readonly name: string; + /** Open a CDP connection for `scope`. */ + connect(options?: { scope?: string }): Promise; +} + +/** Options for {@link createBrowserModule}. */ +export interface BrowserModuleOptions { + /** The browser to drive. */ + readonly browser: BrowserSource; + /** + * Who is driving, when several conversations share one browser: runs with + * the same scope share an active tab, and each scope keeps its own. + * Cookies and logins are shared by every scope. Defaults to the + * browser's own default. + */ + readonly scope?: string; + /** + * The largest `Page.captureScreenshot` result `send` returns, measured as + * JSON. Defaults to 1,000,000 bytes. Keep it under the backend's + * `maxCapabilityBytes`. + */ + readonly maxResultBytes?: number; +} + +/** A tab a page opened on its own during an execution. */ +export interface BrowserNewTab { + targetId: string; + url?: string; + title?: string; +} + +/** + * What happened to the browser during one execution, beyond the code's own + * result. + */ +export interface BrowserExecutionReport { + /** The browser was replaced before this run; earlier page state is gone. */ + restarted: boolean; + /** Tabs the page opened itself (popups, `target=_blank`). */ + newTabs: BrowserNewTab[]; + /** + * The tab this scope was working in was closed by someone else (another + * scope, or a person in Live View), so `"active"` will be a new blank tab. + */ + tabClosed: boolean; + /** Another scope is working in this scope's active tab too. */ + tabShared: boolean; +} + +/** `sessionId` value that addresses the tab the agent is working in. */ +export const ACTIVE_PAGE_SESSION = "active"; + +/** + * `attachToTarget` returns `{ sessionId: "target:" }`, a stable + * handle rather than a connection-scoped CDP session id, so it keeps + * working in later runs. `send` resolves it on the live socket. + */ +const ATTACH_HANDLE_PREFIX = "target:"; + +/** Chrome's JSON-RPC code for a CDP session id it doesn't know. */ +const CDP_SESSION_NOT_FOUND = -32001; + +/** `BrowserDriver#liveParams`: nothing to detach. */ +const DETACHED = Symbol("detached"); + +/** Reports awaiting their tool call; bounded in case a caller never reads. */ +const MAX_PENDING_REPORTS = 100; + +/** + * Commands that would end the browser. The host owns its lifetime, so the + * model can't close it out from under the agent (and lose its logins). + */ +const BROWSER_LIFETIME_COMMANDS = new Set([ + "Browser.close", + "Browser.crash", + "Browser.crashGpuProcess", +]); + +/** A CDP target id: for this module, a tab. */ +type TargetId = string & { readonly __brand: "TargetId" }; + +/** A CDP session id from `Target.attachToTarget`, valid on one socket only. */ +type LiveSessionId = string & { readonly __brand: "LiveSessionId" }; + +/** The stable `sessionId` handle `attachToTarget` gives the model. */ +type AttachHandle = `${typeof ATTACH_HANDLE_PREFIX}${string}`; + +interface TargetInfo { + targetId: TargetId; + type: string; + url?: string; + title?: string; + /** The tab that opened this one (popups, `target=_blank` links). */ + openerId?: TargetId; +} + +/** State for one run. Dropped when the run ends. */ +interface ExecutionState { + executionId: string; + connected: BrowserConnection; + /** Live CDP session per target, valid on this socket only. */ + attached: Map; + /** + * Tabs this run attached to, through `attachToTarget` or a raw + * `Target.attachToTarget`. Their popups are this run's new tabs. + */ + workedIn: Set; + /** Page targets open when the run connected. */ + initialPages: Set; + /** The tab `"active"` resolves to, decided on first use. */ + activeTargetId?: TargetId; + /** Whether {@link activeTargetId} was decided (vs. never needed). */ + activeResolved: boolean; + /** Tabs this run closed with `Target.closeTarget`. */ + closed: Set; +} + +const RESTARTED_NOTICE = + "The browser was restarted before this run (it expired or was closed): earlier tabs, logins, and page state are gone. Your code ran in a fresh browser, so navigate again before relying on page state."; + +const TAB_CLOSED_NOTICE = + 'The tab you were working in was closed (by another conversation using this browser, or a person). `sessionId: "active"` now opens a blank tab, so navigate again before relying on page state.'; + +const TAB_SHARED_NOTICE = + "Another conversation using this browser is also working in your tab, so its page may change between your runs. Open a tab of your own with Target.createTarget if you need one."; + +/** + * Lines telling the model what happened to its browser during a run: it + * was restarted, its tab was closed by someone else, another scope is + * working in its tab too, or the page opened tabs. + */ +function reportLines(report: BrowserExecutionReport | undefined): string[] { + if (!report) return []; + const lines = [ + report.restarted && RESTARTED_NOTICE, + report.tabClosed && !report.restarted && TAB_CLOSED_NOTICE, + report.tabShared && TAB_SHARED_NOTICE, + ].filter((notice) => typeof notice === "string"); + if (report.newTabs.length > 0) { + lines.push(`New tabs opened by the page: ${JSON.stringify(report.newTabs)}`); + } + return lines; +} + +function browserRules(): string[] { + return [ + "The browser persists between runs: tabs, cookies, and logins are still there next time. You never start or close it.", + "If the browser was replaced (idle timeout, crash), standard error says so and earlier tabs and logins are gone. Before an action that matters (submitting a form, a purchase), check you're on the page you expect.", + 'Page-scoped commands (Page.*, Runtime.*, DOM.*, Input.*, Network.*, Emulation.*) need sessionId: "active", your tab, which is remembered between runs. Browser.* and Target.* commands take no sessionId.', + "Other open tabs (Target.getTargets) may belong to other conversations using this browser: don't close, navigate, or attach to a tab you didn't open unless the user asks.", + "Target.createTarget opens a tab and makes it active. To switch to another open tab, call attachToTarget({ targetId }); the sessionId it returns keeps working in later runs, while one from a raw Target.attachToTarget only works in the current run.", + "Tabs a page opens itself (popups, target=_blank links) don't become active; standard error lists them.", + "Runtime.evaluate: pass returnByValue: true, or objects come back as a remote reference with no value, and awaitPromise: true for async expressions. The value is at result.value. A thrown error doesn't reject: check exceptionDetails.", + "Page.navigate returns before the page loads and doesn't throw when it fails: check errorText, then poll document.readyState with Runtime.evaluate until it is 'complete'.", + "Pick out what you need inside Runtime.evaluate and return only that. A single send result over 1 MB (a full-page screenshot, say) fails the call.", + ]; +} + +const DESCRIPTION = [ + "A persistent Chrome browser over the Chrome DevTools Protocol (CDP).", + "- `send({ method, params?, sessionId?, timeoutMs? })` sends a CDP command and returns its result without the JSON-RPC envelope: `Page.navigate` returns `{ frameId, errorText? }`, `Runtime.evaluate` returns `{ result: { value }, exceptionDetails? }`.", + "- `attachToTarget({ targetId })` makes an open tab active and returns `{ sessionId }`.", + '- `spec()` returns the whole protocol (about 650 KB) as `{ domains: [{ name, commands, events, types }] }`. Filter it in code, for example `spec.domains.find((d) => d.name === "Page")`.', + "- `getDebugLog({ limit? })` returns recent protocol traffic, for when a command fails or times out. `clearDebugLog()` clears it.", + '- To look at the page, save a screenshot and open it with the `read` tool: `const { data } = await send({ method: "Page.captureScreenshot", sessionId: "active" }); await writeFile("/workspace/page.png", Buffer.from(data, "base64"));`, with `writeFile` from `node:fs/promises` and `Buffer` from `node:buffer`.', + ...browserRules().map((rule) => `- ${rule}`), +].join("\n"); + +/** + * Build the `ws:browser` host module over a persistent browser. + * + * It exports `send`, `attachToTarget`, `spec`, `getDebugLog` and + * `clearDebugLog`. Each run gets its own CDP connection, opened on its + * first call. When the run ends, the module saves the scope's active tab, + * closes the connection, and writes what happened to the browser (a + * restart, tabs the page opened) to standard error. The browser stays open. + * + * @param options - The browser, the scope to drive it in, and the size limit on screenshots. + * @returns The module to pass as `modules["ws:browser"]`. + */ +export function createBrowserModule(options: BrowserModuleOptions): WorkspaceModuleFactory { + const driver = new BrowserDriver(options); + const running = new Set(); + + const begin = (context: WorkspaceModuleCallContext) => { + const { executionId } = context; + if (running.has(executionId)) return; + running.add(executionId); + context.defer(async () => { + running.delete(executionId); + try { + await driver.end(executionId); + } finally { + for (const line of reportLines(driver.takeReport(executionId))) context.log(line); + } + }); + }; + + const functions: Record = {}; + for (const name of BROWSER_FUNCTION_NAMES) { + functions[name] = async ([value], context) => { + begin(context); + return driver.call(context.executionId, name, value); + }; + } + const create = (): WorkspaceModuleFunctions => functions; + return Object.assign(create, { description: DESCRIPTION }); +} + +/** + * Drives one persistent browser over CDP, with one connection per run. + * + * The model never manages sessions: every run reattaches to the browser, + * or gets a replacement reported as restarted, and `sessionId: "active"` + * addresses the tab the scope last worked in. The active tab is stored + * per scope, so it survives between runs for as long as the tab does, and + * one scope's runs never land in another scope's tab unless the model + * attaches to it. + */ +class BrowserDriver { + readonly #browser: BrowserSource; + readonly #scope: string | undefined; + readonly #maxResultBytes: number; + #states = new Map(); + #connecting = new Map>(); + #reports = new Map(); + + constructor(options: BrowserModuleOptions) { + this.#browser = options.browser; + this.#scope = options.scope; + this.#maxResultBytes = options.maxResultBytes ?? 1_000_000; + } + + /** Answer one call from `executionId`'s code. */ + async call( + executionId: string, + name: BrowserFunctionName, + value: WorkspaceRuntimeValue | undefined, + ): Promise { + const input = parseArguments(name, value); + switch (name) { + case "send": + return this.#send(executionId, input); + case "attachToTarget": + return this.#attachToTarget(executionId, input); + case "spec": + return (await this.#state(executionId)).connected.spec(); + case "getDebugLog": { + const state = await this.#state(executionId); + return state.connected.cdp.getDebugLog(optionalNumber(input, "limit", name)); + } + case "clearDebugLog": { + const state = await this.#state(executionId); + state.connected.cdp.clearDebugLog(); + return null; + } + } + } + + /** + * The run is over: note tabs the page opened, save the scope's active + * tab, note whether another scope shares it, and drop the socket. The + * browser stays alive. + */ + async end(executionId: string): Promise { + const state = this.#states.get(executionId); + if (!state) return; + this.#states.delete(executionId); + try { + const open = await this.#recordNewTabs(executionId, state); + const active = await this.#saveActiveTarget(state, open); + if (active && state.activeResolved) { + await this.#recordSharedTab(state, active); + } + } finally { + state.connected.cdp.disconnect(); + } + } + + /** + * Take (and forget) what happened to the browser during an execution. + * `undefined` when the execution never touched the browser. + */ + takeReport(executionId: string): BrowserExecutionReport | undefined { + const report = this.#reports.get(executionId); + this.#reports.delete(executionId); + return report; + } + + async #send(executionId: string, args: Arguments): Promise { + const method = requiredString(args, "method", "send"); + const params = optionalObject(args, "params", "send"); + const sessionId = optionalString(args, "sessionId", "send"); + const timeoutMs = optionalNumber(args, "timeoutMs", "send"); + if (BROWSER_LIFETIME_COMMANDS.has(method)) { + throw new Error( + `${method} is not available: this browser is managed for you ` + + `and persists between runs. To close a tab, send ` + + `Target.closeTarget with its targetId.`, + ); + } + const state = await this.#state(executionId); + const live = await this.#resolveSessionId(state, sessionId); + const liveParams = this.#liveParams(state, method, params); + if (liveParams === DETACHED) return {}; + let result: unknown; + try { + result = await state.connected.cdp.send(method, liveParams, { + sessionId: live, + timeoutMs, + }); + } catch (error) { + throw await this.#teach(state, error, method, sessionId); + } + this.#observe(state, method, liveParams, result); + if (method === "Page.captureScreenshot") { + this.#assertFits(result); + } + return result; + } + + async #attachToTarget(executionId: string, args: Arguments): Promise<{ sessionId: string }> { + // SAFETY: Any string the caller passes is a target id to Chrome, which rejects ones it doesn't know. + const targetId = requiredString(args, "targetId", "attachToTarget") as TargetId; + const timeoutMs = optionalNumber(args, "timeoutMs", "attachToTarget"); + const state = await this.#state(executionId); + await this.#attach(state, targetId, timeoutMs); + this.#setActive(state, targetId); + return { sessionId: attachHandle(targetId) }; + } + + /** + * The JavaScript backend caps each host call's response, so a result over + * the limit fails with a generic error. A full-page screenshot is the + * usual culprit, so say how to take a smaller one instead. + */ + #assertFits(result: unknown): void { + // Measure the result as JSON, not just the data, so a screenshot just + // under the limit can't slip past with its JSON framing. + const size = JSON.stringify(result ?? null).length; + if (size <= this.#maxResultBytes) return; + const megabytes = (size / 1_000_000).toFixed(1); + const limit = (this.#maxResultBytes / 1_000_000).toFixed(0); + throw new Error( + `The screenshot result is ${megabytes} MB, over the ${limit} MB limit on a ` + + `send result. Capture just the viewport (no captureBeyondViewport ` + + `or clip larger than the window), or pass format: "jpeg" with ` + + `quality: 60.`, + ); + } + + /** + * Connect once per run. Concurrent first calls (model code that uses + * Promise.all despite the instructions) share one connect. + */ + #state(executionId: string): Promise { + const existing = this.#states.get(executionId); + if (existing) return Promise.resolve(existing); + const inFlight = this.#connecting.get(executionId); + if (inFlight) return inFlight; + const promise = this.#connect(executionId).finally(() => { + this.#connecting.delete(executionId); + }); + this.#connecting.set(executionId, promise); + return promise; + } + + async #connect(executionId: string): Promise { + const connected = await this.#browser.connect( + this.#scope === undefined ? undefined : { scope: this.#scope }, + ); + let initialPages: TargetInfo[]; + try { + initialPages = await this.#pages(connected.cdp); + } catch (error) { + connected.cdp.disconnect(); + throw error; + } + const state: ExecutionState = { + executionId, + connected, + attached: new Map(), + workedIn: new Set(), + initialPages: new Set(initialPages.map((page) => page.targetId)), + activeResolved: false, + closed: new Set(), + }; + this.#states.set(executionId, state); + this.#report(executionId, connected.restarted); + return state; + } + + #report(executionId: string, restarted: boolean): BrowserExecutionReport { + let report = this.#reports.get(executionId); + if (!report) { + report = { restarted, newTabs: [], tabClosed: false, tabShared: false }; + this.#reports.set(executionId, report); + if (this.#reports.size > MAX_PENDING_REPORTS) { + const oldest = this.#reports.keys().next().value; + if (oldest !== undefined) this.#reports.delete(oldest); + } + } + // A replacement means earlier page state is gone. + report.restarted ||= restarted; + return report; + } + + /** + * Add tabs the page opened this run to its report, and drop + * earlier ones that have since closed: `newTabs` lists page-opened tabs + * still open when the run ends. Returns the open + * tabs, or `undefined` when the browser couldn't be asked. + */ + async #recordNewTabs( + executionId: string, + state: ExecutionState, + ): Promise | undefined> { + let pages: TargetInfo[]; + try { + pages = await this.#pages(state.connected.cdp); + } catch (error) { + this.#warn("list the open tabs", error); + return undefined; + } + const report = this.#report(executionId, state.connected.restarted); + const opened = new Set(report.newTabs.map((tab) => tab.targetId)); + // A new tab is this run's when its opener is a tab this run + // worked in, or another of its new tabs (a popup's popup). Chrome sets + // openerId only on tabs a page opened, never on Target.createTarget + // tabs, and a popup another scope's page opened meanwhile has that + // scope's tab as its opener. + let grew = true; + while (grew) { + grew = false; + for (const page of pages) { + const opener = page.openerId; + if (!opener || opened.has(page.targetId) || state.initialPages.has(page.targetId)) { + continue; + } + if (state.workedIn.has(opener) || opened.has(opener)) { + opened.add(page.targetId); + grew = true; + } + } + } + report.newTabs = pages + .filter((page) => opened.has(page.targetId)) + .map(({ targetId, url, title }) => ({ targetId, url, title })); + return new Set(pages.map((page) => page.targetId)); + } + + /** + * Save the tab this run settled on, forgetting one that has since closed, + * and return it. Without the list of open tabs, save the choice + * unchecked: the next execution checks the stored tab is still open + * before using it. + */ + async #saveActiveTarget( + state: ExecutionState, + open: Set | undefined, + ): Promise { + const stored = state.connected.activeTargetId as TargetId | undefined; + // A run that never used "active" still sees the stored tab is gone; + // say so now, since the next run won't know there was one. + if (!state.activeResolved && stored && open && !open.has(stored) && !state.closed.has(stored)) { + this.#report(state.executionId, state.connected.restarted).tabClosed = true; + } + const candidate = state.activeResolved ? state.activeTargetId : stored; + const active = candidate && (!open || open.has(candidate)) ? candidate : undefined; + if (active === stored) return active; + try { + await state.connected.setActiveTarget(active); + } catch (error) { + this.#warn("save the active tab", error); + } + return active; + } + + /** Report when another scope is working in this run's tab too. */ + async #recordSharedTab(state: ExecutionState, active: TargetId): Promise { + try { + const others = await state.connected.targetsInOtherScopes(); + if (others.has(active)) { + this.#report(state.executionId, state.connected.restarted).tabShared = true; + } + } catch (error) { + this.#warn("check whether the active tab is shared", error); + } + } + + #warn(action: string, error: unknown): void { + console.warn(`[ws:browser] Failed to ${action} for browser "${this.#browser.name}"`, error); + } + + async #pages(cdp: BrowserCdp): Promise { + const result = (await cdp.send("Target.getTargets")) as { + targetInfos?: TargetInfo[]; + }; + return (result?.targetInfos ?? []).filter((info) => info.type === "page"); + } + + async #resolveSessionId( + state: ExecutionState, + sessionId: string | undefined, + ): Promise { + if (sessionId === ACTIVE_PAGE_SESSION) { + return this.#attach(state, await this.#activeTarget(state)); + } + if (isAttachHandle(sessionId)) { + return this.#attach(state, targetOfHandle(sessionId)); + } + // Omitted, or a raw CDP session id the model got from send(). + return sessionId as LiveSessionId | undefined; + } + + /** + * Decide which tab `"active"` means, once per run: the scope's stored tab + * if it is still open, else a new blank tab. Never a tab the scope didn't + * open, even a blank one: it may be another scope's, or a popup. + */ + async #activeTarget(state: ExecutionState): Promise { + if (state.activeTargetId) return state.activeTargetId; + const { connected } = state; + const pages = await this.#pages(connected.cdp); + const stored = connected.activeTargetId as TargetId | undefined; + let targetId = pages.find((page) => page.targetId === stored)?.targetId; + if (!targetId && stored && !state.closed.has(stored)) { + this.#report(state.executionId, connected.restarted).tabClosed = true; + } + if (!targetId) { + const created = (await connected.cdp.send("Target.createTarget", { + url: "about:blank", + })) as { targetId: TargetId }; + targetId = created.targetId; + } + this.#setActive(state, targetId); + return targetId; + } + + #setActive(state: ExecutionState, targetId: TargetId | undefined): void { + state.activeTargetId = targetId; + state.activeResolved = true; + } + + async #attach( + state: ExecutionState, + targetId: TargetId, + timeoutMs?: number, + ): Promise { + const existing = state.attached.get(targetId); + if (existing) return existing; + const live = (await state.connected.cdp.attachToTarget(targetId, { + timeoutMs, + })) as LiveSessionId; + state.attached.set(targetId, live); + state.workedIn.add(targetId); + return live; + } + + /** Track tab changes the model makes through raw Target commands. */ + #observe( + state: ExecutionState, + method: string, + params: Record | undefined, + result: unknown, + ): void { + const targetId = + typeof params?.targetId === "string" ? (params.targetId as TargetId) : undefined; + if (method === "Target.createTarget") { + const created = (result as { targetId?: unknown } | undefined)?.targetId; + if (typeof created === "string") { + this.#setActive(state, created as TargetId); + } + } else if (method === "Target.attachToTarget" && targetId) { + this.#setActive(state, targetId); + state.workedIn.add(targetId); + } else if (method === "Target.closeTarget" && targetId) { + state.attached.delete(targetId); + state.closed.add(targetId); + // The next "active" use picks a tab afresh. + if (state.activeTargetId === targetId) state.activeTargetId = undefined; + } else if (method === "Target.detachFromTarget") { + // Forget a session the model detached, so the next use reattaches. + for (const [target, live] of state.attached) { + if (target === targetId || live === params?.sessionId) { + state.attached.delete(target); + } + } + } + } + + /** + * Swap a handle in `Target.detachFromTarget`'s `sessionId` param for the + * live CDP session id. Returns {@link DETACHED} when the handle's tab isn't + * attached on this socket, so there is nothing to detach. + */ + #liveParams( + state: ExecutionState, + method: string, + params: Record | undefined, + ): Record | undefined | typeof DETACHED { + const handle = params?.sessionId; + if (method !== "Target.detachFromTarget" || typeof handle !== "string") { + return params; + } + if (!isAttachHandle(handle)) return params; + const live = state.attached.get(targetOfHandle(handle)); + return live ? { ...params, sessionId: live } : DETACHED; + } + + /** + * Turn the common protocol mistakes into instructions: sending an + * event as a command, and a page-scoped command without a session. + */ + async #teach( + state: ExecutionState, + error: unknown, + method: string, + sessionId: string | undefined, + ): Promise { + const code = protocolErrorCode(error); + if (code === undefined) return error; + const message = error instanceof Error ? error.message : String(error); + if ( + code === CDP_SESSION_NOT_FOUND && + sessionId && + sessionId !== ACTIVE_PAGE_SESSION && + !isAttachHandle(sessionId) + ) { + return new Error( + `${message}. CDP session ids don't carry over between runs. ` + + `Use sessionId: "active" for the current tab, or the handle from ` + + `attachToTarget({ targetId }) for another tab.`, + ); + } + if (code !== CDP_METHOD_NOT_FOUND) return error; + if (await this.#isEvent(state, method)) { + return new Error( + `${message}. '${method}' is a CDP *event*, not a command, so it ` + + `cannot be sent. To wait for page state, poll instead (e.g. ` + + `Runtime.evaluate of document.readyState until "complete").`, + ); + } + if (!sessionId) { + return new Error( + `${message}. '${method}' is page-scoped: pass sessionId: ` + + `"active" to run it in the current tab: ` + + `send({ method: "${method}", params, sessionId: "active" }).`, + ); + } + return error; + } + + async #isEvent(state: ExecutionState, method: string): Promise { + try { + const spec = await state.connected.spec(); + const domain = method.split(".")[0]; + return spec.domains.some( + (d) => d.name === domain && d.events.some((e) => e.event === method), + ); + } catch { + return false; + } + } +} + +function attachHandle(targetId: TargetId): AttachHandle { + return `${ATTACH_HANDLE_PREFIX}${targetId}`; +} + +function isAttachHandle(sessionId: string | undefined): sessionId is AttachHandle { + return sessionId?.startsWith(ATTACH_HANDLE_PREFIX) ?? false; +} + +function targetOfHandle(handle: AttachHandle): TargetId { + return handle.slice(ATTACH_HANDLE_PREFIX.length) as TargetId; +} + +/** Chrome's JSON-RPC code for a method it doesn't know. */ +const CDP_METHOD_NOT_FOUND = -32601; + +/** The JSON-RPC code of a CDP protocol error, or `undefined` for any other error. */ +function protocolErrorCode(error: unknown): number | undefined { + if (!(error instanceof Error)) return undefined; + const code: unknown = Reflect.get(error, "code"); + return typeof code === "number" ? code : undefined; +} + +type BrowserFunctionName = "send" | "attachToTarget" | "spec" | "getDebugLog" | "clearDebugLog"; + +const BROWSER_FUNCTION_NAMES: readonly BrowserFunctionName[] = [ + "send", + "attachToTarget", + "spec", + "getDebugLog", + "clearDebugLog", +]; + +type Arguments = Readonly>; + +/** A call's single options object. An omitted argument counts as `{}`. */ +function parseArguments( + name: BrowserFunctionName, + value: WorkspaceRuntimeValue | undefined, +): Arguments { + if (value === undefined || value === null) return {}; + if (typeof value !== "object" || Array.isArray(value)) { + throw new Error(`ws:browser ${name} takes one options object.`); + } + return value; +} + +function requiredString(args: Arguments, key: string, name: string): string { + const value = args[key]; + if (typeof value !== "string" || value === "") { + throw new Error(`ws:browser ${name} needs \`${key}\`, a string.`); + } + return value; +} + +function optionalString(args: Arguments, key: string, name: string): string | undefined { + const value = args[key]; + if (value === undefined || value === null) return undefined; + if (typeof value !== "string") + throw new Error(`ws:browser ${name}: \`${key}\` must be a string.`); + return value; +} + +function optionalNumber(args: Arguments, key: string, name: string): number | undefined { + const value = args[key]; + if (value === undefined || value === null) return undefined; + if (typeof value !== "number" || !Number.isFinite(value) || value <= 0) { + throw new Error(`ws:browser ${name}: \`${key}\` must be a positive number.`); + } + return value; +} + +function optionalObject( + args: Arguments, + key: string, + name: string, +): Record | undefined { + const value = args[key]; + if (value === undefined || value === null) return undefined; + if (typeof value !== "object" || Array.isArray(value)) { + throw new Error(`ws:browser ${name}: \`${key}\` must be an object.`); + } + return value; +} diff --git a/packages/computer/tests/script-runner-worker.ts b/packages/computer/tests/script-runner-worker.ts index e475485e7..4d71660cd 100644 --- a/packages/computer/tests/script-runner-worker.ts +++ b/packages/computer/tests/script-runner-worker.ts @@ -9,6 +9,7 @@ import { createGitClient } from "../src/git/index.js"; import type { WorkspaceRuntimeValue, WorkspaceStub } from "../src/index.js"; import { Workspace } from "../src/index.js"; import { createArtifactsModule } from "../src/modules/artifacts.js"; +import { createBrowserModule, type BrowserSource } from "../src/modules/browser.js"; import { createContainerModule } from "../src/modules/container.js"; import { createGitModule } from "../src/modules/git.js"; @@ -57,6 +58,39 @@ function fakeContainerBackend(): WorkspaceBackend { }; } +// A browser with one tab that answers Runtime.evaluate with the tab it ran +// in, and comes back replaced on the second connection, so a run sees a +// restart on standard error. +let browserConnections = 0; +const fakeBrowser: BrowserSource = { + name: "fake", + async connect() { + browserConnections += 1; + const restarted = browserConnections === 2; + return { + restarted, + activeTargetId: "tab-1", + cdp: { + async send(method) { + if (method === "Target.getTargets") { + return { targetInfos: [{ targetId: "tab-1", type: "page", url: "about:blank" }] }; + } + return { result: { value: method } }; + }, + async attachToTarget(targetId) { + return `session-${targetId}`; + }, + getDebugLog: () => [], + clearDebugLog: () => {}, + disconnect: () => {}, + }, + setActiveTarget: async () => true, + targetsInOtherScopes: async () => new Set(), + spec: async () => ({ domains: [] }), + }; + }, +}; + // Executions whose deferred functions have finished, so a later run can // check that they ran before the earlier run's result was published. const endedRuns = new Set(); @@ -65,6 +99,7 @@ const trackedRuns = new Set(); // The Worker Loader resolves imports differently under the legacy and // the new module registry, so the same backend runs on both. const NEW_REGISTRY_BACKEND = "worker-javascript-new-registry"; +const BROWSER_BACKEND = "worker-javascript-browser"; function javascriptBackend( env: Env, @@ -172,6 +207,13 @@ export class HostDO extends DurableObject { }, }), fakeContainerBackend(), + // Its own backend, since the restart notice is longer than the + // 64 bytes of standard error the others allow. + new WorkerJavaScriptBackend({ + id: BROWSER_BACKEND, + loader: env.LOADER, + modules: { "ws:browser": createBrowserModule({ browser: fakeBrowser }) }, + }), ], }); } diff --git a/packages/computer/tests/script-runner.test.ts b/packages/computer/tests/script-runner.test.ts index 4f718b0b8..f4123e28b 100644 --- a/packages/computer/tests/script-runner.test.ts +++ b/packages/computer/tests/script-runner.test.ts @@ -467,6 +467,36 @@ describe("WorkspaceRuntime", () => { expect(result.stderr).toContain("deferred function failed: boom"); }); + it("drives a browser from ws:browser and reports a restart on standard error", async () => { + const source = ` + import { send } from "ws:browser"; + export default () => send({ method: "Runtime.evaluate", sessionId: "active" }); + `; + const first = await runtime({ + source, + cwd: "/workspace", + backend: "worker-javascript-browser", + }); + const firstText = await first.text(); + expect(first.status, firstText).toBe(200); + expect(JSON.parse(firstText).result).toMatchObject({ + status: "completed", + stderr: "", + value: { result: { value: "Runtime.evaluate" } }, + }); + + const second = await runtime({ + source, + cwd: "/workspace", + backend: "worker-javascript-browser", + }); + const secondText = await second.text(); + expect(second.status, secondText).toBe(200); + const { result } = JSON.parse(secondText); + expect(result.status).toBe("completed"); + expect(result.stderr).toContain("The browser was restarted"); + }); + it("fails to link an import the host module does not export", async () => { const response = await runtime({ source: ` From 7ca1b8a67a62437705571f4d92fcca0a5c3c997b Mon Sep 17 00:00:00 2001 From: Matt Carey Date: Mon, 19 Oct 2026 14:47:26 +0100 Subject: [PATCH 2/3] computer: Run ws:browser on the browser's own session ws:browser kept its own copy of the CDP logic agents/browser already has: the active tab, attach handles, error hints and the end-of-run report. It now takes anything with a name and session(), such as Browser from agents/browser, starts one session per run, and on the run's end closes it and writes the report to standard error. The package still doesn't depend on agents. --- .changeset/browser-module.md | 2 +- docs/17_isolate_javascript.md | 8 +- packages/computer/src/modules/browser.test.ts | 330 +++----- packages/computer/src/modules/browser.ts | 800 +++--------------- .../computer/tests/script-runner-worker.ts | 37 +- 5 files changed, 247 insertions(+), 930 deletions(-) diff --git a/.changeset/browser-module.md b/.changeset/browser-module.md index f1f431738..2a87ced7a 100644 --- a/.changeset/browser-module.md +++ b/.changeset/browser-module.md @@ -2,4 +2,4 @@ "@cloudflare/computer": minor --- -Adds the experimental `ws:browser` host module (`@cloudflare/computer/modules/browser`), which lets isolate JavaScript drive a persistent browser over the Chrome DevTools Protocol. +Adds the experimental `ws:browser` host module (`@cloudflare/computer/modules/browser`), which lets isolate JavaScript drive a persistent browser over the Chrome DevTools Protocol. It takes anything with a `name` and a `session()`, such as `Browser` from `agents/browser`, and opens one session per run. diff --git a/docs/17_isolate_javascript.md b/docs/17_isolate_javascript.md index c814dec60..5991a512f 100644 --- a/docs/17_isolate_javascript.md +++ b/docs/17_isolate_javascript.md @@ -443,7 +443,7 @@ The module's description lists the names code can import. With a function of too import { send, attachToTarget, spec, getDebugLog, clearDebugLog } from "ws:browser"; ``` -`createBrowserModule({ browser, scope?, maxResultBytes? })` from `@cloudflare/computer/modules/browser` lets JavaScript drive a persistent Chrome browser over the Chrome DevTools Protocol. The browser belongs to the host: the module takes anything with the `BrowserSource` shape, a `name` and a `connect({ scope })` that returns a CDP connection plus the scope's active tab. `Browser` from `agents/browser` has that shape, so an agent passes its own browser: +`createBrowserModule({ browser, scope?, maxResultBytes? })` from `@cloudflare/computer/modules/browser` lets JavaScript drive a persistent Chrome browser over the Chrome DevTools Protocol. The browser belongs to the host: the module takes anything with the `BrowserSource` shape, a `name` and a `session({ scope, maxResultBytes })` that starts a session for one run. `Browser` from `agents/browser` has that shape, so an agent passes its own browser, and neither package depends on the other: ```ts import { Browser, browserRun } from "agents/browser"; @@ -469,11 +469,11 @@ export default async function () { } ``` -`send({ method, params?, sessionId?, timeoutMs? })` returns a command's result without the JSON-RPC envelope. `sessionId: "active"` is the tab the scope is working in. The module remembers it between runs, and opens a new blank tab when the scope has none. `attachToTarget({ targetId })` makes another open tab active and returns a `sessionId` handle that keeps working in later runs. A session id from a raw `Target.attachToTarget` only lasts for one run. `Browser.close` and `Browser.crash` are refused, because the host decides when the browser ends. +`send({ method, params?, sessionId?, timeoutMs? })` returns a command's result without the JSON-RPC envelope. `sessionId: "active"` is the tab the scope is working in. The session remembers it between runs, and opens a new blank tab when the scope has none. `attachToTarget({ targetId })` makes another open tab active and returns a `sessionId` handle that keeps working in later runs. A session id from a raw `Target.attachToTarget` only lasts for one run. `Browser.close` and `Browser.crash` are refused, because the host decides when the browser ends. -Each run gets its own CDP connection, opened on its first call. When the run ends, the module saves the scope's active tab and closes the connection. The browser stays open. If the browser was replaced since the scope last used it, if another scope's tab closed or shares this one, or if the page opened tabs, the module says so on the run's standard error. The module's description, which the `exec` tool shows, covers the calls and the CDP mistakes models commonly make. +A run's first call starts a session, and every call in the run goes to it. When the run ends, the module closes the session, which saves the scope's active tab and closes its connection. The browser stays open. If the session reports that the browser was replaced since the scope last used it, that another scope closed or shares this one's tab, or that the page opened tabs, the module says so on the run's standard error. The CDP rules above (the active tab, attach handles, refused commands, error hints) belong to the session, so `agents/browser`'s direct `createBrowserCDPTool` behaves the same way. The module's description, which the `exec` tool shows, covers the calls and the CDP mistakes models commonly make. -`scope` keeps conversations that share one browser in their own tabs, while cookies and logins stay shared. A `Page.captureScreenshot` result over `maxResultBytes` (1,000,000 by default) fails with a note to capture the viewport or use JPEG. Keep it under the backend's `maxCapabilityBytes`. +`scope` keeps conversations that share one browser in their own tabs, while cookies and logins stay shared. A `Page.captureScreenshot` result over `maxResultBytes` (1,000,000 by default in `agents/browser`) fails with a note to capture the viewport or use JPEG. Keep it under the backend's `maxCapabilityBytes`. ### `ws:container` diff --git a/packages/computer/src/modules/browser.test.ts b/packages/computer/src/modules/browser.test.ts index ea0690890..c59426cf6 100644 --- a/packages/computer/src/modules/browser.test.ts +++ b/packages/computer/src/modules/browser.test.ts @@ -2,160 +2,96 @@ import { describe, expect, it } from "vitest"; import type { WorkspaceModuleCallContext, - WorkspaceModuleFunctions, WorkspaceModuleHost, WorkspaceRuntimeValue, } from "../runtime/types.js"; import { createBrowserModule, - type BrowserCdp, - type BrowserConnection, type BrowserModuleOptions, + type BrowserSession, + type BrowserSessionOptions, + type BrowserSessionReport, type BrowserSource, } from "./browser.js"; // SAFETY: The browser module never touches the host's Git, Artifacts, or runtime. const host = {} as WorkspaceModuleHost; -class ProtocolError extends Error { - constructor( - readonly code: number, - message: string, - ) { - super(message); - } -} - -interface Tab { - targetId: string; - type: "page"; - url: string; - openerId?: string; +interface FakeSession extends BrowserSession { + readonly options: BrowserSessionOptions | undefined; + readonly calls: Array<[string, unknown]>; + closed: number; } -// One browser: tabs shared by every connection, an active tab per scope, -// and Chrome's errors for an unknown session or a page command without one. +// Sessions that record their calls. The CDP behaviour behind a session is +// `agents/browser`'s, tested there; this module only routes calls to the +// run's session and closes it. class FakeBrowser implements BrowserSource { readonly name = "work"; - tabs: Tab[] = [{ targetId: "target-1", type: "page", url: "about:blank" }]; - active = new Map(); - connections: { disconnected: boolean }[] = []; - restartNext = false; - screenshotBytes = 10; - #nextTarget = 1; - #nextSession = 0; - #sessions = new Map(); - - addTab(url: string, openerId?: string): string { - const targetId = `target-${++this.#nextTarget}`; - this.tabs.push({ targetId, type: "page", url, ...(openerId ? { openerId } : {}) }); - return targetId; - } - - async connect(options: { scope?: string } = {}): Promise { - const scope = options.scope ?? "shared"; - const restarted = this.restartNext; - if (restarted) { - this.restartNext = false; - this.tabs = []; - this.active.clear(); - } - const state = { disconnected: false }; - this.connections.push(state); - const sessions = new Set(); - const cdp: BrowserCdp = { - send: async (method, params = {}, sendOptions = {}) => { - if (state.disconnected) throw new Error("socket closed"); - return this.#handle(method, params, sendOptions.sessionId); - }, - attachToTarget: async (targetId) => { - const sessionId = `cdp-${++this.#nextSession}`; - this.#sessions.set(sessionId, targetId); - sessions.add(sessionId); - return sessionId; - }, - getDebugLog: () => [{ at: "now", type: "send" }], - clearDebugLog: () => {}, - disconnect: () => { - state.disconnected = true; - for (const sessionId of sessions) this.#sessions.delete(sessionId); - }, + readonly sessions: FakeSession[] = []; + report: BrowserSessionReport | undefined = { + restarted: false, + newTabs: [], + tabClosed: false, + tabShared: false, + }; + + session(options?: BrowserSessionOptions): BrowserSession { + const calls: Array<[string, unknown]> = []; + const record = (name: string) => async (input?: unknown) => { + calls.push([name, input]); + if (name === "send" && (input as { method?: string })?.method === "Browser.close") { + throw new Error("Browser.close is not available"); + } + return { called: name, input: input ?? null }; }; - return { - restarted, - activeTargetId: this.active.get(scope), - cdp, - setActiveTarget: async (targetId) => { - this.active.set(scope, targetId); - return true; + const session: FakeSession = { + options, + calls, + closed: 0, + send: record("send"), + attachToTarget: record("attachToTarget"), + spec: record("spec"), + getDebugLog: record("getDebugLog"), + clearDebugLog: record("clearDebugLog"), + close: async () => { + session.closed += 1; + return this.report; }, - targetsInOtherScopes: async () => - new Set( - [...this.active] - .filter(([other, target]) => other !== scope && target !== undefined) - .map(([, target]) => target as string), - ), - spec: async () => ({ - domains: [{ name: "Page", events: [{ event: "Page.loadEventFired" }] }], - }), }; - } - - #handle(method: string, params: Record, sessionId: string | undefined) { - switch (method) { - case "Target.getTargets": - return { targetInfos: this.tabs }; - case "Target.createTarget": - return { targetId: this.addTab(String(params.url ?? "")) }; - case "Target.closeTarget": - this.tabs = this.tabs.filter((tab) => tab.targetId !== params.targetId); - return { success: true }; - case "Target.attachToTarget": { - const id = `cdp-${++this.#nextSession}`; - this.#sessions.set(id, String(params.targetId)); - return { sessionId: id }; - } - default: - if (sessionId && !this.#sessions.has(sessionId)) { - throw new ProtocolError(-32001, "Session with given id not found."); - } - if (!sessionId) throw new ProtocolError(-32601, `'${method}' wasn't found`); - if (method === "Page.captureScreenshot") return { data: "x".repeat(this.screenshotBytes) }; - if (method === "Runtime.evaluate") { - const target = this.#sessions.get(sessionId); - if (params.expression === "openPopup()") this.addTab("https://popup.example/", target); - return { result: { value: target } }; - } - return {}; - } + this.sessions.push(session); + return session; } } function setup(options: Partial = {}) { const browser = new FakeBrowser(); const module = createBrowserModule({ browser, ...options }); - const functions: WorkspaceModuleFunctions = module(host); + const functions = module(host); let runs = 0; - // Run `calls` as one execution, as the JavaScript backend does: every - // call has the run's id, and deferred functions run last first at the end. - async function run(calls: Array<[name: string, args?: WorkspaceRuntimeValue]>) { - const logs: string[] = []; - const deferred: Array<() => unknown> = []; - const context: WorkspaceModuleCallContext = { - executionId: `run-${++runs}`, + function context(executionId: string, logs: string[], deferred: Array<() => unknown>) { + return { + executionId, signal: new AbortController().signal, deadline: Date.now() + 60_000, access: "read-write", - resolvePath: async (path) => path, - log: (text) => logs.push(text), - defer: (fn) => deferred.push(fn), - }; + resolvePath: async (path: string) => path, + log: (text: string) => logs.push(text), + defer: (fn: () => unknown) => deferred.push(fn), + } satisfies WorkspaceModuleCallContext; + } + + /** Make `calls` as one run, then run what it deferred, as the backend does. */ + async function run(calls: Array<[string, ...WorkspaceRuntimeValue[]]>) { + const logs: string[] = []; + const deferred: Array<() => unknown> = []; + const ctx = context(`run-${++runs}`, logs, deferred); const results: unknown[] = []; let error: unknown; try { - for (const [name, args] of calls) { - results.push(await functions[name](args === undefined ? [] : [args], context)); + for (const [name, ...args] of calls) { + results.push(await functions[name](args, ctx)); } } catch (caught) { error = caught; @@ -164,15 +100,9 @@ function setup(options: Partial = {}) { return { results, error, logs }; } - return { browser, module, functions, run }; + return { browser, module, functions, run, context }; } -const evaluate = (expression: string): WorkspaceRuntimeValue => ({ - method: "Runtime.evaluate", - params: { expression }, - sessionId: "active", -}); - describe("ws:browser", () => { it("exports the browser calls and describes them for a model", () => { const { functions, module } = setup(); @@ -188,109 +118,99 @@ describe("ws:browser", () => { expect(module.description).not.toContain("cdp."); }); - it("opens a tab for the scope and keeps it between runs", async () => { + it("starts one session per run and closes it when the run ends", async () => { const t = setup(); - const first = await t.run([["send", evaluate("1")]]); + const first = await t.run([ + ["send", { method: "Page.navigate", sessionId: "active" }], + ["attachToTarget", { targetId: "target-1" }], + ["spec"], + ["getDebugLog", { limit: 5 }], + ["clearDebugLog"], + ]); expect(first.error).toBeUndefined(); - // Never the browser's initial blank tab, which may be another scope's. - expect(first.results[0]).toEqual({ result: { value: "target-2" } }); - expect(t.browser.active.get("shared")).toBe("target-2"); - expect(t.browser.connections.every((connection) => connection.disconnected)).toBe(true); + expect(first.results[0]).toEqual({ + called: "send", + input: { method: "Page.navigate", sessionId: "active" }, + }); + expect(t.browser.sessions).toHaveLength(1); + expect(t.browser.sessions[0].calls.map(([name]) => name)).toEqual([ + "send", + "attachToTarget", + "spec", + "getDebugLog", + "clearDebugLog", + ]); + expect(t.browser.sessions[0].closed).toBe(1); - const second = await t.run([["send", evaluate("1")]]); - expect(second.results[0]).toEqual({ result: { value: "target-2" } }); + await t.run([["send", { method: "Target.getTargets" }]]); + expect(t.browser.sessions).toHaveLength(2); + expect(t.browser.sessions[1].closed).toBe(1); }); - it("keeps scopes in their own tabs", async () => { - const shared = setup(); - const other = createBrowserModule({ browser: shared.browser, scope: "other" })(host); - await shared.run([["send", evaluate("1")]]); + it("shares one session between a run's concurrent calls", async () => { + const t = setup(); + const logs: string[] = []; const deferred: Array<() => unknown> = []; - const result = await other.send([evaluate("1")], { - executionId: "other-run", - signal: new AbortController().signal, - deadline: Date.now() + 60_000, - access: "read-write", - resolvePath: async (path) => path, - log: () => {}, - defer: (fn) => deferred.push(fn), - }); - while (deferred.length > 0) await deferred.pop()?.(); - expect(result).toEqual({ result: { value: "target-3" } }); - expect(shared.browser.active).toEqual( - new Map([ - ["shared", "target-2"], - ["other", "target-3"], - ]), - ); + const ctx = t.context("parallel", logs, deferred); + await Promise.all([ + t.functions.send([{ method: "Target.getTargets" }], ctx), + t.functions.send([{ method: "Browser.getVersion" }], ctx), + ]); + expect(t.browser.sessions).toHaveLength(1); + expect(deferred).toHaveLength(1); }); - it("returns an attach handle that works in later runs", async () => { - const t = setup(); - const first = await t.run([["attachToTarget", { targetId: "target-1" }]]); - expect(first.results[0]).toEqual({ sessionId: "target:target-1" }); - const second = await t.run([ - [ - "send", - { method: "Runtime.evaluate", params: { expression: "1" }, sessionId: "target:target-1" }, - ], - ]); - expect(second.results[0]).toEqual({ result: { value: "target-1" } }); + it("passes the scope and screenshot limit to the session", async () => { + const t = setup({ scope: "thread-1", maxResultBytes: 500 }); + await t.run([["spec"]]); + expect(t.browser.sessions[0].options).toEqual({ scope: "thread-1", maxResultBytes: 500 }); + + const plain = setup(); + await plain.run([["spec"]]); + expect(plain.browser.sessions[0].options).toEqual({}); }); - it("writes tabs the page opened to standard error", async () => { + it("writes what happened to the browser to standard error", async () => { const t = setup(); - const { logs } = await t.run([["send", evaluate("openPopup()")]]); + t.browser.report = { + restarted: true, + newTabs: [{ targetId: "target-3", url: "https://popup.example/" }], + tabClosed: true, + tabShared: true, + }; + const { logs } = await t.run([["send", { method: "Target.getTargets" }]]); expect(logs).toEqual([ + expect.stringContaining("The browser was restarted before this run"), + expect.stringContaining( + "Another conversation using this browser is also working in your tab", + ), 'New tabs opened by the page: [{"targetId":"target-3","url":"https://popup.example/"}]', ]); - }); - - it("says so on standard error when the browser was replaced", async () => { - const t = setup(); - t.browser.restartNext = true; - const { logs, error } = await t.run([["send", { method: "Target.getTargets" }]]); - expect(error).toBeUndefined(); - expect(logs).toEqual([expect.stringContaining("The browser was restarted before this run")]); - }); - - it("explains a page-scoped command sent without a session", async () => { - const t = setup(); - const { error } = await t.run([["send", { method: "Page.navigate" }]]); - expect(String(error)).toContain('pass sessionId: "active"'); - }); - it("explains a CDP session id from an earlier run", async () => { - const t = setup(); - const { error } = await t.run([ - ["send", { method: "Runtime.evaluate", params: {}, sessionId: "cdp-99" }], + t.browser.report = { restarted: false, newTabs: [], tabClosed: true, tabShared: false }; + const closed = await t.run([["send", { method: "Target.getTargets" }]]); + expect(closed.logs).toEqual([ + expect.stringContaining("The tab you were working in was closed"), ]); - expect(String(error)).toContain("CDP session ids don't carry over between runs"); }); - it("refuses an oversized screenshot with how to take a smaller one", async () => { - const t = setup({ maxResultBytes: 50 }); - t.browser.screenshotBytes = 100; - const { error } = await t.run([ - ["send", { method: "Page.captureScreenshot", sessionId: "active" }], - ]); - expect(String(error)).toContain("Capture just the viewport"); + it("passes the session's errors to the code and still closes it", async () => { + const t = setup(); + const { error } = await t.run([["send", { method: "Browser.close" }]]); + expect(String(error)).toContain("Browser.close is not available"); + expect(t.browser.sessions[0].closed).toBe(1); }); - it("refuses bad arguments and commands that would end the browser", async () => { + it("refuses more than one argument", async () => { const t = setup(); - const missing = await t.run([["send", { params: {} }]]); - expect(String(missing.error)).toContain("send needs `method`"); - const notObject = await t.run([["send", "Page.navigate"]]); - expect(String(notObject.error)).toContain("takes one options object"); - const close = await t.run([["send", { method: "Browser.close" }]]); - expect(String(close.error)).toContain("Browser.close is not available"); + const { error } = await t.run([["send", { method: "Page.navigate" }, "extra"]]); + expect(String(error)).toContain("takes one options object"); }); it("leaves the browser alone for a run that never called it", async () => { const t = setup(); const { logs } = await t.run([]); expect(logs).toEqual([]); - expect(t.browser.connections).toEqual([]); + expect(t.browser.sessions).toEqual([]); }); }); diff --git a/packages/computer/src/modules/browser.ts b/packages/computer/src/modules/browser.ts index f8861d3a3..1ad7f0525 100644 --- a/packages/computer/src/modules/browser.ts +++ b/packages/computer/src/modules/browser.ts @@ -3,94 +3,20 @@ // // import { send, attachToTarget } from "ws:browser"; // -// The browser belongs to the host. This module only needs something that -// can connect to it (see `BrowserSource`), such as `Browser` from -// `agents/browser`, so it doesn't depend on any one browser provider. +// The browser belongs to the host. This module opens a session on it for +// each run and closes it when the run ends. It needs only something with a +// `name` and `session()` (see `BrowserSource`), such as `Browser` from +// `agents/browser`, so it doesn't depend on any one browser provider. The +// session does the CDP work: the "active" tab, attach handles that last +// between runs, error hints, and the report of what happened to the browser. import type { WorkspaceModuleCallContext, WorkspaceModuleFactory, WorkspaceModuleFunctions, - WorkspaceRuntimeValue, } from "../runtime/types.js"; -/** - * A CDP connection to a browser, as `BrowserSource.connect` returns it. - * `Browser` from `agents/browser` provides one. - */ -export interface BrowserCdp { - /** Send a CDP command and resolve with its result, without the JSON-RPC envelope. */ - send( - method: string, - params?: Record, - options?: { sessionId?: string; timeoutMs?: number }, - ): Promise; - /** Attach to a target and resolve with the CDP session id, valid on this connection only. */ - attachToTarget(targetId: string, options?: { timeoutMs?: number }): Promise; - /** Recent protocol traffic, newest last. */ - getDebugLog(limit?: number): unknown[]; - /** Forget the protocol traffic recorded so far. */ - clearDebugLog(): void; - /** Close the connection. The browser stays open. */ - disconnect(): void; -} - -/** The CDP description a browser serves, as far as this module reads it. */ -export interface BrowserCdpSpec { - domains: Array<{ name: string; events: Array<{ event: string }> }>; -} - -/** One connection to the browser for one scope. */ -export interface BrowserConnection { - /** - * `true` when this scope worked in an earlier browser that has since been - * replaced, so its tabs and logins are gone. - */ - restarted: boolean; - /** The tab this scope last worked in, if it recorded one. */ - activeTargetId?: string; - /** The CDP connection. Closing it doesn't close the browser. */ - cdp: BrowserCdp; - /** Record the tab this scope is working in, or clear it with `undefined`. */ - setActiveTarget(targetId: string | undefined): Promise; - /** The tabs other scopes are working in. */ - targetsInOtherScopes(): Promise>; - /** The Chrome DevTools Protocol description the browser serves. */ - spec(): Promise; -} - -/** - * The browser the module drives. `Browser` from `agents/browser` has this - * shape. `connect` reattaches to the browser, or replaces a lost one, and - * opens a CDP connection for a scope. - */ -export interface BrowserSource { - /** The browser's name, for log messages. */ - readonly name: string; - /** Open a CDP connection for `scope`. */ - connect(options?: { scope?: string }): Promise; -} - -/** Options for {@link createBrowserModule}. */ -export interface BrowserModuleOptions { - /** The browser to drive. */ - readonly browser: BrowserSource; - /** - * Who is driving, when several conversations share one browser: runs with - * the same scope share an active tab, and each scope keeps its own. - * Cookies and logins are shared by every scope. Defaults to the - * browser's own default. - */ - readonly scope?: string; - /** - * The largest `Page.captureScreenshot` result `send` returns, measured as - * JSON. Defaults to 1,000,000 bytes. Keep it under the backend's - * `maxCapabilityBytes`. - */ - readonly maxResultBytes?: number; -} - -/** A tab a page opened on its own during an execution. */ +/** A tab a page opened on its own during a run. */ export interface BrowserNewTab { targetId: string; url?: string; @@ -98,10 +24,10 @@ export interface BrowserNewTab { } /** - * What happened to the browser during one execution, beyond the code's own - * result. + * What happened to the browser during one run, beyond its calls' results. + * `BrowserSession.close` resolves with it. */ -export interface BrowserExecutionReport { +export interface BrowserSessionReport { /** The browser was replaced before this run; earlier page state is gone. */ restarted: boolean; /** Tabs the page opened itself (popups, `target=_blank`). */ @@ -115,72 +41,64 @@ export interface BrowserExecutionReport { tabShared: boolean; } -/** `sessionId` value that addresses the tab the agent is working in. */ -export const ACTIVE_PAGE_SESSION = "active"; - /** - * `attachToTarget` returns `{ sessionId: "target:" }`, a stable - * handle rather than a connection-scoped CDP session id, so it keeps - * working in later runs. `send` resolves it on the live socket. + * One run's session on the browser. `browser.session()` from + * `agents/browser` returns one. Each method takes the single options + * object the isolate passed, which comes from untrusted code, so the + * session must validate it. */ -const ATTACH_HANDLE_PREFIX = "target:"; - -/** Chrome's JSON-RPC code for a CDP session id it doesn't know. */ -const CDP_SESSION_NOT_FOUND = -32001; - -/** `BrowserDriver#liveParams`: nothing to detach. */ -const DETACHED = Symbol("detached"); +export interface BrowserSession { + /** Send a CDP command and resolve with its result. */ + send(input: unknown): Promise; + /** Make an open tab active and resolve with `{ sessionId }`. */ + attachToTarget(input: unknown): Promise; + /** The Chrome DevTools Protocol description the browser serves. */ + spec(): Promise; + /** Recent protocol traffic, newest last. */ + getDebugLog(input?: unknown): Promise; + /** Forget the protocol traffic recorded so far. */ + clearDebugLog(): Promise; + /** + * End the session and resolve with its report, or `undefined` when it + * never connected. The browser stays open. + */ + close(): Promise; +} -/** Reports awaiting their tool call; bounded in case a caller never reads. */ -const MAX_PENDING_REPORTS = 100; +/** Options the module passes to {@link BrowserSource.session}. */ +export interface BrowserSessionOptions { + scope?: string; + maxResultBytes?: number; +} /** - * Commands that would end the browser. The host owns its lifetime, so the - * model can't close it out from under the agent (and lose its logins). + * The browser the module drives. `Browser` from `agents/browser` has this + * shape. */ -const BROWSER_LIFETIME_COMMANDS = new Set([ - "Browser.close", - "Browser.crash", - "Browser.crashGpuProcess", -]); - -/** A CDP target id: for this module, a tab. */ -type TargetId = string & { readonly __brand: "TargetId" }; - -/** A CDP session id from `Target.attachToTarget`, valid on one socket only. */ -type LiveSessionId = string & { readonly __brand: "LiveSessionId" }; - -/** The stable `sessionId` handle `attachToTarget` gives the model. */ -type AttachHandle = `${typeof ATTACH_HANDLE_PREFIX}${string}`; - -interface TargetInfo { - targetId: TargetId; - type: string; - url?: string; - title?: string; - /** The tab that opened this one (popups, `target=_blank` links). */ - openerId?: TargetId; +export interface BrowserSource { + /** The browser's name. */ + readonly name: string; + /** Start a session for one run. */ + session(options?: BrowserSessionOptions): BrowserSession; } -/** State for one run. Dropped when the run ends. */ -interface ExecutionState { - executionId: string; - connected: BrowserConnection; - /** Live CDP session per target, valid on this socket only. */ - attached: Map; +/** Options for {@link createBrowserModule}. */ +export interface BrowserModuleOptions { + /** The browser to drive. */ + readonly browser: BrowserSource; + /** + * Who is driving, when several conversations share one browser: runs with + * the same scope share an active tab, and each scope keeps its own. + * Cookies and logins are shared by every scope. Defaults to the + * browser's own default. + */ + readonly scope?: string; /** - * Tabs this run attached to, through `attachToTarget` or a raw - * `Target.attachToTarget`. Their popups are this run's new tabs. + * The largest `Page.captureScreenshot` result `send` returns, measured as + * JSON. Defaults to the session's own limit (1,000,000 bytes for + * `agents/browser`). Keep it under the backend's `maxCapabilityBytes`. */ - workedIn: Set; - /** Page targets open when the run connected. */ - initialPages: Set; - /** The tab `"active"` resolves to, decided on first use. */ - activeTargetId?: TargetId; - /** Whether {@link activeTargetId} was decided (vs. never needed). */ - activeResolved: boolean; - /** Tabs this run closed with `Target.closeTarget`. */ - closed: Set; + readonly maxResultBytes?: number; } const RESTARTED_NOTICE = @@ -197,7 +115,7 @@ const TAB_SHARED_NOTICE = * was restarted, its tab was closed by someone else, another scope is * working in its tab too, or the page opened tabs. */ -function reportLines(report: BrowserExecutionReport | undefined): string[] { +function reportLines(report: BrowserSessionReport | undefined): string[] { if (!report) return []; const lines = [ report.restarted && RESTARTED_NOTICE, @@ -234,584 +152,72 @@ const DESCRIPTION = [ ...browserRules().map((rule) => `- ${rule}`), ].join("\n"); +type BrowserFunctionName = "send" | "attachToTarget" | "spec" | "getDebugLog" | "clearDebugLog"; + +const BROWSER_FUNCTION_NAMES: readonly BrowserFunctionName[] = [ + "send", + "attachToTarget", + "spec", + "getDebugLog", + "clearDebugLog", +]; + /** * Build the `ws:browser` host module over a persistent browser. * * It exports `send`, `attachToTarget`, `spec`, `getDebugLog` and - * `clearDebugLog`. Each run gets its own CDP connection, opened on its - * first call. When the run ends, the module saves the scope's active tab, - * closes the connection, and writes what happened to the browser (a - * restart, tabs the page opened) to standard error. The browser stays open. + * `clearDebugLog`. A run's first call starts a session on the browser. + * When the run ends, the module closes the session, which saves the + * scope's active tab, and writes what happened to the browser (a restart, + * tabs the page opened) to standard error. The browser stays open. * * @param options - The browser, the scope to drive it in, and the size limit on screenshots. * @returns The module to pass as `modules["ws:browser"]`. */ export function createBrowserModule(options: BrowserModuleOptions): WorkspaceModuleFactory { - const driver = new BrowserDriver(options); - const running = new Set(); + const sessionOptions: BrowserSessionOptions = { + ...(options.scope === undefined ? {} : { scope: options.scope }), + ...(options.maxResultBytes === undefined ? {} : { maxResultBytes: options.maxResultBytes }), + }; + const sessions = new Map(); - const begin = (context: WorkspaceModuleCallContext) => { + const sessionFor = (context: WorkspaceModuleCallContext): BrowserSession => { const { executionId } = context; - if (running.has(executionId)) return; - running.add(executionId); + const existing = sessions.get(executionId); + if (existing) return existing; + const session = options.browser.session(sessionOptions); + sessions.set(executionId, session); context.defer(async () => { - running.delete(executionId); - try { - await driver.end(executionId); - } finally { - for (const line of reportLines(driver.takeReport(executionId))) context.log(line); - } + sessions.delete(executionId); + for (const line of reportLines(await session.close())) context.log(line); }); + return session; }; - const functions: Record = {}; - for (const name of BROWSER_FUNCTION_NAMES) { - functions[name] = async ([value], context) => { - begin(context); - return driver.call(context.executionId, name, value); - }; - } - const create = (): WorkspaceModuleFunctions => functions; - return Object.assign(create, { description: DESCRIPTION }); -} - -/** - * Drives one persistent browser over CDP, with one connection per run. - * - * The model never manages sessions: every run reattaches to the browser, - * or gets a replacement reported as restarted, and `sessionId: "active"` - * addresses the tab the scope last worked in. The active tab is stored - * per scope, so it survives between runs for as long as the tab does, and - * one scope's runs never land in another scope's tab unless the model - * attaches to it. - */ -class BrowserDriver { - readonly #browser: BrowserSource; - readonly #scope: string | undefined; - readonly #maxResultBytes: number; - #states = new Map(); - #connecting = new Map>(); - #reports = new Map(); - - constructor(options: BrowserModuleOptions) { - this.#browser = options.browser; - this.#scope = options.scope; - this.#maxResultBytes = options.maxResultBytes ?? 1_000_000; - } - - /** Answer one call from `executionId`'s code. */ - async call( - executionId: string, - name: BrowserFunctionName, - value: WorkspaceRuntimeValue | undefined, - ): Promise { - const input = parseArguments(name, value); + const call = (session: BrowserSession, name: BrowserFunctionName, input: unknown) => { switch (name) { case "send": - return this.#send(executionId, input); + return session.send(input); case "attachToTarget": - return this.#attachToTarget(executionId, input); + return session.attachToTarget(input); case "spec": - return (await this.#state(executionId)).connected.spec(); - case "getDebugLog": { - const state = await this.#state(executionId); - return state.connected.cdp.getDebugLog(optionalNumber(input, "limit", name)); - } - case "clearDebugLog": { - const state = await this.#state(executionId); - state.connected.cdp.clearDebugLog(); - return null; - } - } - } - - /** - * The run is over: note tabs the page opened, save the scope's active - * tab, note whether another scope shares it, and drop the socket. The - * browser stays alive. - */ - async end(executionId: string): Promise { - const state = this.#states.get(executionId); - if (!state) return; - this.#states.delete(executionId); - try { - const open = await this.#recordNewTabs(executionId, state); - const active = await this.#saveActiveTarget(state, open); - if (active && state.activeResolved) { - await this.#recordSharedTab(state, active); - } - } finally { - state.connected.cdp.disconnect(); - } - } - - /** - * Take (and forget) what happened to the browser during an execution. - * `undefined` when the execution never touched the browser. - */ - takeReport(executionId: string): BrowserExecutionReport | undefined { - const report = this.#reports.get(executionId); - this.#reports.delete(executionId); - return report; - } - - async #send(executionId: string, args: Arguments): Promise { - const method = requiredString(args, "method", "send"); - const params = optionalObject(args, "params", "send"); - const sessionId = optionalString(args, "sessionId", "send"); - const timeoutMs = optionalNumber(args, "timeoutMs", "send"); - if (BROWSER_LIFETIME_COMMANDS.has(method)) { - throw new Error( - `${method} is not available: this browser is managed for you ` + - `and persists between runs. To close a tab, send ` + - `Target.closeTarget with its targetId.`, - ); - } - const state = await this.#state(executionId); - const live = await this.#resolveSessionId(state, sessionId); - const liveParams = this.#liveParams(state, method, params); - if (liveParams === DETACHED) return {}; - let result: unknown; - try { - result = await state.connected.cdp.send(method, liveParams, { - sessionId: live, - timeoutMs, - }); - } catch (error) { - throw await this.#teach(state, error, method, sessionId); + return session.spec(); + case "getDebugLog": + return session.getDebugLog(input); + case "clearDebugLog": + return session.clearDebugLog(); } - this.#observe(state, method, liveParams, result); - if (method === "Page.captureScreenshot") { - this.#assertFits(result); - } - return result; - } - - async #attachToTarget(executionId: string, args: Arguments): Promise<{ sessionId: string }> { - // SAFETY: Any string the caller passes is a target id to Chrome, which rejects ones it doesn't know. - const targetId = requiredString(args, "targetId", "attachToTarget") as TargetId; - const timeoutMs = optionalNumber(args, "timeoutMs", "attachToTarget"); - const state = await this.#state(executionId); - await this.#attach(state, targetId, timeoutMs); - this.#setActive(state, targetId); - return { sessionId: attachHandle(targetId) }; - } - - /** - * The JavaScript backend caps each host call's response, so a result over - * the limit fails with a generic error. A full-page screenshot is the - * usual culprit, so say how to take a smaller one instead. - */ - #assertFits(result: unknown): void { - // Measure the result as JSON, not just the data, so a screenshot just - // under the limit can't slip past with its JSON framing. - const size = JSON.stringify(result ?? null).length; - if (size <= this.#maxResultBytes) return; - const megabytes = (size / 1_000_000).toFixed(1); - const limit = (this.#maxResultBytes / 1_000_000).toFixed(0); - throw new Error( - `The screenshot result is ${megabytes} MB, over the ${limit} MB limit on a ` + - `send result. Capture just the viewport (no captureBeyondViewport ` + - `or clip larger than the window), or pass format: "jpeg" with ` + - `quality: 60.`, - ); - } - - /** - * Connect once per run. Concurrent first calls (model code that uses - * Promise.all despite the instructions) share one connect. - */ - #state(executionId: string): Promise { - const existing = this.#states.get(executionId); - if (existing) return Promise.resolve(existing); - const inFlight = this.#connecting.get(executionId); - if (inFlight) return inFlight; - const promise = this.#connect(executionId).finally(() => { - this.#connecting.delete(executionId); - }); - this.#connecting.set(executionId, promise); - return promise; - } - - async #connect(executionId: string): Promise { - const connected = await this.#browser.connect( - this.#scope === undefined ? undefined : { scope: this.#scope }, - ); - let initialPages: TargetInfo[]; - try { - initialPages = await this.#pages(connected.cdp); - } catch (error) { - connected.cdp.disconnect(); - throw error; - } - const state: ExecutionState = { - executionId, - connected, - attached: new Map(), - workedIn: new Set(), - initialPages: new Set(initialPages.map((page) => page.targetId)), - activeResolved: false, - closed: new Set(), - }; - this.#states.set(executionId, state); - this.#report(executionId, connected.restarted); - return state; - } - - #report(executionId: string, restarted: boolean): BrowserExecutionReport { - let report = this.#reports.get(executionId); - if (!report) { - report = { restarted, newTabs: [], tabClosed: false, tabShared: false }; - this.#reports.set(executionId, report); - if (this.#reports.size > MAX_PENDING_REPORTS) { - const oldest = this.#reports.keys().next().value; - if (oldest !== undefined) this.#reports.delete(oldest); - } - } - // A replacement means earlier page state is gone. - report.restarted ||= restarted; - return report; - } - - /** - * Add tabs the page opened this run to its report, and drop - * earlier ones that have since closed: `newTabs` lists page-opened tabs - * still open when the run ends. Returns the open - * tabs, or `undefined` when the browser couldn't be asked. - */ - async #recordNewTabs( - executionId: string, - state: ExecutionState, - ): Promise | undefined> { - let pages: TargetInfo[]; - try { - pages = await this.#pages(state.connected.cdp); - } catch (error) { - this.#warn("list the open tabs", error); - return undefined; - } - const report = this.#report(executionId, state.connected.restarted); - const opened = new Set(report.newTabs.map((tab) => tab.targetId)); - // A new tab is this run's when its opener is a tab this run - // worked in, or another of its new tabs (a popup's popup). Chrome sets - // openerId only on tabs a page opened, never on Target.createTarget - // tabs, and a popup another scope's page opened meanwhile has that - // scope's tab as its opener. - let grew = true; - while (grew) { - grew = false; - for (const page of pages) { - const opener = page.openerId; - if (!opener || opened.has(page.targetId) || state.initialPages.has(page.targetId)) { - continue; - } - if (state.workedIn.has(opener) || opened.has(opener)) { - opened.add(page.targetId); - grew = true; - } - } - } - report.newTabs = pages - .filter((page) => opened.has(page.targetId)) - .map(({ targetId, url, title }) => ({ targetId, url, title })); - return new Set(pages.map((page) => page.targetId)); - } - - /** - * Save the tab this run settled on, forgetting one that has since closed, - * and return it. Without the list of open tabs, save the choice - * unchecked: the next execution checks the stored tab is still open - * before using it. - */ - async #saveActiveTarget( - state: ExecutionState, - open: Set | undefined, - ): Promise { - const stored = state.connected.activeTargetId as TargetId | undefined; - // A run that never used "active" still sees the stored tab is gone; - // say so now, since the next run won't know there was one. - if (!state.activeResolved && stored && open && !open.has(stored) && !state.closed.has(stored)) { - this.#report(state.executionId, state.connected.restarted).tabClosed = true; - } - const candidate = state.activeResolved ? state.activeTargetId : stored; - const active = candidate && (!open || open.has(candidate)) ? candidate : undefined; - if (active === stored) return active; - try { - await state.connected.setActiveTarget(active); - } catch (error) { - this.#warn("save the active tab", error); - } - return active; - } + }; - /** Report when another scope is working in this run's tab too. */ - async #recordSharedTab(state: ExecutionState, active: TargetId): Promise { - try { - const others = await state.connected.targetsInOtherScopes(); - if (others.has(active)) { - this.#report(state.executionId, state.connected.restarted).tabShared = true; + const functions: Record = {}; + for (const name of BROWSER_FUNCTION_NAMES) { + functions[name] = async (args, context) => { + if (args.length > 1) { + throw new Error(`ws:browser ${name} takes one options object.`); } - } catch (error) { - this.#warn("check whether the active tab is shared", error); - } - } - - #warn(action: string, error: unknown): void { - console.warn(`[ws:browser] Failed to ${action} for browser "${this.#browser.name}"`, error); - } - - async #pages(cdp: BrowserCdp): Promise { - const result = (await cdp.send("Target.getTargets")) as { - targetInfos?: TargetInfo[]; + return call(sessionFor(context), name, args[0] ?? undefined); }; - return (result?.targetInfos ?? []).filter((info) => info.type === "page"); - } - - async #resolveSessionId( - state: ExecutionState, - sessionId: string | undefined, - ): Promise { - if (sessionId === ACTIVE_PAGE_SESSION) { - return this.#attach(state, await this.#activeTarget(state)); - } - if (isAttachHandle(sessionId)) { - return this.#attach(state, targetOfHandle(sessionId)); - } - // Omitted, or a raw CDP session id the model got from send(). - return sessionId as LiveSessionId | undefined; - } - - /** - * Decide which tab `"active"` means, once per run: the scope's stored tab - * if it is still open, else a new blank tab. Never a tab the scope didn't - * open, even a blank one: it may be another scope's, or a popup. - */ - async #activeTarget(state: ExecutionState): Promise { - if (state.activeTargetId) return state.activeTargetId; - const { connected } = state; - const pages = await this.#pages(connected.cdp); - const stored = connected.activeTargetId as TargetId | undefined; - let targetId = pages.find((page) => page.targetId === stored)?.targetId; - if (!targetId && stored && !state.closed.has(stored)) { - this.#report(state.executionId, connected.restarted).tabClosed = true; - } - if (!targetId) { - const created = (await connected.cdp.send("Target.createTarget", { - url: "about:blank", - })) as { targetId: TargetId }; - targetId = created.targetId; - } - this.#setActive(state, targetId); - return targetId; - } - - #setActive(state: ExecutionState, targetId: TargetId | undefined): void { - state.activeTargetId = targetId; - state.activeResolved = true; - } - - async #attach( - state: ExecutionState, - targetId: TargetId, - timeoutMs?: number, - ): Promise { - const existing = state.attached.get(targetId); - if (existing) return existing; - const live = (await state.connected.cdp.attachToTarget(targetId, { - timeoutMs, - })) as LiveSessionId; - state.attached.set(targetId, live); - state.workedIn.add(targetId); - return live; } - - /** Track tab changes the model makes through raw Target commands. */ - #observe( - state: ExecutionState, - method: string, - params: Record | undefined, - result: unknown, - ): void { - const targetId = - typeof params?.targetId === "string" ? (params.targetId as TargetId) : undefined; - if (method === "Target.createTarget") { - const created = (result as { targetId?: unknown } | undefined)?.targetId; - if (typeof created === "string") { - this.#setActive(state, created as TargetId); - } - } else if (method === "Target.attachToTarget" && targetId) { - this.#setActive(state, targetId); - state.workedIn.add(targetId); - } else if (method === "Target.closeTarget" && targetId) { - state.attached.delete(targetId); - state.closed.add(targetId); - // The next "active" use picks a tab afresh. - if (state.activeTargetId === targetId) state.activeTargetId = undefined; - } else if (method === "Target.detachFromTarget") { - // Forget a session the model detached, so the next use reattaches. - for (const [target, live] of state.attached) { - if (target === targetId || live === params?.sessionId) { - state.attached.delete(target); - } - } - } - } - - /** - * Swap a handle in `Target.detachFromTarget`'s `sessionId` param for the - * live CDP session id. Returns {@link DETACHED} when the handle's tab isn't - * attached on this socket, so there is nothing to detach. - */ - #liveParams( - state: ExecutionState, - method: string, - params: Record | undefined, - ): Record | undefined | typeof DETACHED { - const handle = params?.sessionId; - if (method !== "Target.detachFromTarget" || typeof handle !== "string") { - return params; - } - if (!isAttachHandle(handle)) return params; - const live = state.attached.get(targetOfHandle(handle)); - return live ? { ...params, sessionId: live } : DETACHED; - } - - /** - * Turn the common protocol mistakes into instructions: sending an - * event as a command, and a page-scoped command without a session. - */ - async #teach( - state: ExecutionState, - error: unknown, - method: string, - sessionId: string | undefined, - ): Promise { - const code = protocolErrorCode(error); - if (code === undefined) return error; - const message = error instanceof Error ? error.message : String(error); - if ( - code === CDP_SESSION_NOT_FOUND && - sessionId && - sessionId !== ACTIVE_PAGE_SESSION && - !isAttachHandle(sessionId) - ) { - return new Error( - `${message}. CDP session ids don't carry over between runs. ` + - `Use sessionId: "active" for the current tab, or the handle from ` + - `attachToTarget({ targetId }) for another tab.`, - ); - } - if (code !== CDP_METHOD_NOT_FOUND) return error; - if (await this.#isEvent(state, method)) { - return new Error( - `${message}. '${method}' is a CDP *event*, not a command, so it ` + - `cannot be sent. To wait for page state, poll instead (e.g. ` + - `Runtime.evaluate of document.readyState until "complete").`, - ); - } - if (!sessionId) { - return new Error( - `${message}. '${method}' is page-scoped: pass sessionId: ` + - `"active" to run it in the current tab: ` + - `send({ method: "${method}", params, sessionId: "active" }).`, - ); - } - return error; - } - - async #isEvent(state: ExecutionState, method: string): Promise { - try { - const spec = await state.connected.spec(); - const domain = method.split(".")[0]; - return spec.domains.some( - (d) => d.name === domain && d.events.some((e) => e.event === method), - ); - } catch { - return false; - } - } -} - -function attachHandle(targetId: TargetId): AttachHandle { - return `${ATTACH_HANDLE_PREFIX}${targetId}`; -} - -function isAttachHandle(sessionId: string | undefined): sessionId is AttachHandle { - return sessionId?.startsWith(ATTACH_HANDLE_PREFIX) ?? false; -} - -function targetOfHandle(handle: AttachHandle): TargetId { - return handle.slice(ATTACH_HANDLE_PREFIX.length) as TargetId; -} - -/** Chrome's JSON-RPC code for a method it doesn't know. */ -const CDP_METHOD_NOT_FOUND = -32601; - -/** The JSON-RPC code of a CDP protocol error, or `undefined` for any other error. */ -function protocolErrorCode(error: unknown): number | undefined { - if (!(error instanceof Error)) return undefined; - const code: unknown = Reflect.get(error, "code"); - return typeof code === "number" ? code : undefined; -} - -type BrowserFunctionName = "send" | "attachToTarget" | "spec" | "getDebugLog" | "clearDebugLog"; - -const BROWSER_FUNCTION_NAMES: readonly BrowserFunctionName[] = [ - "send", - "attachToTarget", - "spec", - "getDebugLog", - "clearDebugLog", -]; - -type Arguments = Readonly>; - -/** A call's single options object. An omitted argument counts as `{}`. */ -function parseArguments( - name: BrowserFunctionName, - value: WorkspaceRuntimeValue | undefined, -): Arguments { - if (value === undefined || value === null) return {}; - if (typeof value !== "object" || Array.isArray(value)) { - throw new Error(`ws:browser ${name} takes one options object.`); - } - return value; -} - -function requiredString(args: Arguments, key: string, name: string): string { - const value = args[key]; - if (typeof value !== "string" || value === "") { - throw new Error(`ws:browser ${name} needs \`${key}\`, a string.`); - } - return value; -} - -function optionalString(args: Arguments, key: string, name: string): string | undefined { - const value = args[key]; - if (value === undefined || value === null) return undefined; - if (typeof value !== "string") - throw new Error(`ws:browser ${name}: \`${key}\` must be a string.`); - return value; -} - -function optionalNumber(args: Arguments, key: string, name: string): number | undefined { - const value = args[key]; - if (value === undefined || value === null) return undefined; - if (typeof value !== "number" || !Number.isFinite(value) || value <= 0) { - throw new Error(`ws:browser ${name}: \`${key}\` must be a positive number.`); - } - return value; -} - -function optionalObject( - args: Arguments, - key: string, - name: string, -): Record | undefined { - const value = args[key]; - if (value === undefined || value === null) return undefined; - if (typeof value !== "object" || Array.isArray(value)) { - throw new Error(`ws:browser ${name}: \`${key}\` must be an object.`); - } - return value; + const create = (): WorkspaceModuleFunctions => functions; + return Object.assign(create, { description: DESCRIPTION }); } diff --git a/packages/computer/tests/script-runner-worker.ts b/packages/computer/tests/script-runner-worker.ts index 4d71660cd..2570b15fa 100644 --- a/packages/computer/tests/script-runner-worker.ts +++ b/packages/computer/tests/script-runner-worker.ts @@ -58,35 +58,26 @@ function fakeContainerBackend(): WorkspaceBackend { }; } -// A browser with one tab that answers Runtime.evaluate with the tab it ran -// in, and comes back replaced on the second connection, so a run sees a +// A browser whose sessions answer send with the method they were sent, +// and whose second session reports a replaced browser, so a run sees a // restart on standard error. -let browserConnections = 0; +let browserSessions = 0; const fakeBrowser: BrowserSource = { name: "fake", - async connect() { - browserConnections += 1; - const restarted = browserConnections === 2; + session() { + browserSessions += 1; + const restarted = browserSessions === 2; return { - restarted, - activeTargetId: "tab-1", - cdp: { - async send(method) { - if (method === "Target.getTargets") { - return { targetInfos: [{ targetId: "tab-1", type: "page", url: "about:blank" }] }; - } - return { result: { value: method } }; - }, - async attachToTarget(targetId) { - return `session-${targetId}`; - }, - getDebugLog: () => [], - clearDebugLog: () => {}, - disconnect: () => {}, + async send(input) { + return { result: { value: (input as { method: string }).method } }; + }, + async attachToTarget(input) { + return { sessionId: `target:${(input as { targetId: string }).targetId}` }; }, - setActiveTarget: async () => true, - targetsInOtherScopes: async () => new Set(), spec: async () => ({ domains: [] }), + getDebugLog: async () => [], + clearDebugLog: async () => null, + close: async () => ({ restarted, newTabs: [], tabClosed: false, tabShared: false }), }; }, }; From e1e67d3cb617782d52d93b8973badad512d47b56 Mon Sep 17 00:00:00 2001 From: Matt Carey Date: Mon, 19 Oct 2026 15:07:50 +0100 Subject: [PATCH 3/3] docs: Name agents' direct browser tool createBrowserTool --- docs/17_isolate_javascript.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/17_isolate_javascript.md b/docs/17_isolate_javascript.md index 5991a512f..e28fe658f 100644 --- a/docs/17_isolate_javascript.md +++ b/docs/17_isolate_javascript.md @@ -471,7 +471,7 @@ export default async function () { `send({ method, params?, sessionId?, timeoutMs? })` returns a command's result without the JSON-RPC envelope. `sessionId: "active"` is the tab the scope is working in. The session remembers it between runs, and opens a new blank tab when the scope has none. `attachToTarget({ targetId })` makes another open tab active and returns a `sessionId` handle that keeps working in later runs. A session id from a raw `Target.attachToTarget` only lasts for one run. `Browser.close` and `Browser.crash` are refused, because the host decides when the browser ends. -A run's first call starts a session, and every call in the run goes to it. When the run ends, the module closes the session, which saves the scope's active tab and closes its connection. The browser stays open. If the session reports that the browser was replaced since the scope last used it, that another scope closed or shares this one's tab, or that the page opened tabs, the module says so on the run's standard error. The CDP rules above (the active tab, attach handles, refused commands, error hints) belong to the session, so `agents/browser`'s direct `createBrowserCDPTool` behaves the same way. The module's description, which the `exec` tool shows, covers the calls and the CDP mistakes models commonly make. +A run's first call starts a session, and every call in the run goes to it. When the run ends, the module closes the session, which saves the scope's active tab and closes its connection. The browser stays open. If the session reports that the browser was replaced since the scope last used it, that another scope closed or shares this one's tab, or that the page opened tabs, the module says so on the run's standard error. The CDP rules above (the active tab, attach handles, refused commands, error hints) belong to the session, so `agents/browser`'s direct `createBrowserTool` behaves the same way. The module's description, which the `exec` tool shows, covers the calls and the CDP mistakes models commonly make. `scope` keeps conversations that share one browser in their own tabs, while cookies and logins stay shared. A `Page.captureScreenshot` result over `maxResultBytes` (1,000,000 by default in `agents/browser`) fails with a note to capture the viewport or use JPEG. Keep it under the backend's `maxCapabilityBytes`.