Repository navigation
Add agent workflow observations, MCP consent and isolated Chrome automation - #18
Merged
Merged
Conversation
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Suggest exposed menu routes after button clicks, keep screenshot labels tied to verified AX refs, and retain polling and final signature checks when observing UI changes. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
… diffs Keep action approval and retained screen baselines scoped to the connection, bound modern consent continuations to one call, and preserve existing text/image responses and cancellation behavior. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Use a private headless profile, current web policy, bounded CDP queues and guarded DOM targets. Verify typed values, reject unseen or changed UIDs, handle dialogs without blocking their answers, and close the session on cancellation or pause. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Describe supported routes, limits, connection scope and headless regression tests. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Wait for retired browsers to exit before final profile removal, bound retiring sessions, and prevent an abandoned command from closing the next session. Cover cancellation, timeout recovery and complete profile cleanup. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Poll with a stop flag so teardown is bounded even when shutdown does not wake accept. Rechecked both live Chrome regressions after the fixture-only change. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Avoid transient bridged strings during asynchronous remote registration and per-action observer churn. Include process birth identity when reusing an observer. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Avoid restarting quiet time when a later poll notices a notification, and sample at the quiet deadline rather than overshooting it. Cover delayed observation without relaxing signature verification. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Agents can now inspect installed scripting terminology, request connection-scoped snapshot changes, and label app screenshots with existing accessibility refs. Action results can suggest exposed menu routes, while notification-assisted settling preserves signature checks and avoids adding a full polling interval after the quiet deadline.
The same branch adds structured MCP results, opt-in progress, optional per-connection human consent, and an explicit isolated Chrome backend for the existing web tools. Chrome uses a disposable automation profile with guarded DOM actions, policy enforcement, bounded transport queues, cancellation recovery and cleanup. Existing text/image responses and Safari defaults remain compatible. Diagnostic probes are excluded from real-use reports, and multiline details stay on one report row.
Validation
4d6fb02; after Xcode component setup, the two blocked Xcode tasks passed atd86f9e7. All 31 tasks have passing results across these runs, rather than one fresh final-tip run.Scope and limits
Chrome requires an installed Google Chrome and supports plain article-text extraction and standard input/textarea editing; rich contenteditable editors, modified key chords and cross-frame actions remain unsupported. Tool definitions and server instructions use a 26,624-byte budget to accommodate the new schemas and options. This PR does not bump the version or publish a release.
GPT-6.1 Sol in cooperation with Trevor.