Repository navigation
fix(perception): keep system dialogs clickable, only filter true status/nav bars - #181
Open
RanjithRagavan wants to merge 1 commit into
Open
RanjithRagavan wants to merge 1 commit into
RanjithRagavan wants to merge 1 commit into
Conversation
…us/nav bars com.android.systemui was treated as a high-confidence fixed-bar identity with no height bound, so any full-width SystemUI window anchored to the top or bottom screen edge — including full-screen alert dialog windows (USB debugging authorization, crash/ANR alerts) and tall bottom-anchored dialog panels — was classified as a fixed bar. The bogus bar then made _clamp_by_fixed_bars clamp or silently discard neighbouring elements and shift tap targets. Similarly, _detect_status_bar_height accepted the first systemui node with top == 0 without a height check; a full-screen SystemUI dialog window yielded a 2400px 'status bar', pushing every OCR coordinate off screen in _map_coordinates_back and silently dropping all OCR results for the frame. - Cap fixed-bar height in all cases (<=15% of screen height at the bottom, <=12% at the top), even for high-confidence system packages. - Never classify com.android.permissioncontroller nodes as bars; the package only owns dialogs. - Only accept thin top-anchored strips as the status bar in _detect_status_bar_height, and keep scanning for the real one. Fixes google#180
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi team, when you get a chance, could someone take a look at this PR?
It's a small, contained fix (~120 lines, 4 files):
Root cause (reproduced on main): full-screen SystemUI/permissioncontroller dialogs were classified as fixed bars (no height cap), so their buttons were clamped out of the clickable set — and status-bar detection returned 2400px, shifting all OCR coordinates off-screen for the frame.
Tested: 6 new unit tests the 4 defect-demonstrating ones fail on main, pass with this fix (verified via stash round-trip). Full suite: 2499 passed, 6 skipped; ruff clean.
Value: unblocks every first-run flow nearly any real-world task hits a runtime permission dialog, and today the agent stalls there until the step budget exhausts.
Happy to add an emulator repro script or adjust the approach if you'd prefer a different direction. Thanks!