diff --git a/.cursor/rules/project.mdc b/.cursor/rules/project.mdc index 923df13..14a093c 100644 --- a/.cursor/rules/project.mdc +++ b/.cursor/rules/project.mdc @@ -30,6 +30,10 @@ Agent ─stdio─ thin MCP client (index.ts) ─IPC─► daemon (daemon.ts, own - `autoReSnapshot`: on ref failure (virtualized feed), snapshot the tab and embed `freshRefs` in the response so the agent retries in one step. Do NOT auto-retry the click (non-idempotent). `browser_scroll` returns `refsMayBeStale: true` as a hint. - `isNew`: each snapshot tracks `role|name` fingerprints per tab; elements appearing since the last snapshot are tagged `isNew: true` so the agent can focus on what changed (big token saver after an overlay/dropdown opens). State lives in `lastSnapshotFingerprints` Map, cleared on tab close. +## Native accessibility tree (`browser_snapshot source:"native"`) + +`handlers/ax-snapshot.js` + pure `lib/ax-native.js`. CDP `Accessibility.getFullAXTree` per frame (main + same-process child frames, grafted under their `Iframe` node) → same tree shape as the DOM snapshot. Refs are bound into the SAME page registry (`__bcDom.registry`) by structural path: main world `DOM.resolveNode` + `Runtime.callFunctionOn(pathOfTarget)` → isolated world walks the path (`-1` = enter shadow root, `-2` = enter iframe document, else element-child index), verifies the tag, registers, stores the smart-selector fallback. No DOM mutation. Gotchas: batch `callFunctionOn` PER FRAME (Chrome rejects mixed JS contexts); AX ids repeat across frames (namespaced on merge); user-agent shadow nodes (e.g. a date input's sub-fields) are unreachable: no ref, and omitted from the compact tree (`unreachableNodes`); scoping resolves in the page, then finds the node in a `DOM.getDocument({pierce:true})` dump with the same path grammar. CDP failure → DOM snapshot + `nativeUnavailable`. Default `source:"dom"` never touches the debugger. + ## Concurrency (extension/lib/tab-concurrency.js — pure, unit-tested) - `TabMutexMap`: same-tab serializes, cross-tab parallelizes. diff --git a/README.md b/README.md index d94f647..b1e4913 100644 --- a/README.md +++ b/README.md @@ -45,6 +45,7 @@ It already has your browser open right there. It just can't see it. - **Batches.** `browser_batch` runs a list of tool calls in one round-trip and stops at the first failure — a click → type → Tab → wait → read sequence is one call instead of five. - **Console-style JavaScript.** `browser_evaluate` accepts code as you'd type it in DevTools: top-level `await`, several statements, the last expression's value is returned, DOM nodes come back as readable descriptions — and page CSP doesn't block it. - **Refs that stay right.** Snapshot/find refs resolve through one shared page runtime: the registered element first, then the first *visible* selector match (across open and closed shadow roots and same-origin iframes), then a verified fallback that only re-binds when role, tag and name identify one element — an ambiguous match is reported as gone instead of clicked. Hidden duplicates are skipped. +- **Native Chrome accessibility tree.** `browser_snapshot { source: "native" }` reads the tree Chrome itself computes (CDP `Accessibility.getFullAXTree`): exact roles, accessible names (`aria-labelledby`, `