Repository navigation
Add Maestro UI tests - #383
Open
stephanlindauer wants to merge 7 commits into
Open
stephanlindauer wants to merge 7 commits into
stephanlindauer wants to merge 7 commits into
Conversation
Run 13 flows against the release APK on an emulator for every push to master and every pull request. They use the production server, so each run publishes a test rider in the open Atlantic for a few minutes. Posting to the public chat is not part of CI and only runs by hand. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The command line tools preinstalled on the runner don't have pixel_8 yet. pixel_6 has the same screen resolution. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Maestro's device server died on the emulator with two cores. Use the same number of cores as the local runs where the flows pass, and keep Maestro's debug output and logcat as artifacts to make failures diagnosable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the 37.0 x86_64 image Android restarts itself shortly after the map is reopened, which kills Maestro's device server. Also keep more logcat and the tombstones, so a crash like this can be traced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
System UI is slow with software rendering. Its "isn't responding" dialog blocked every flow, although the app itself was fine. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
System apps stopped responding because everything renders in software. Use a smaller screen, and make hiding error dialogs take effect by triggering a configuration change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds UI tests that run without any manual work: Maestro flows that drive the release APK on an emulator, on every push to
masterand every PR. Testing the release APK also covers R8.The 13 flows cover first launch, navigation, chat input, settings, about, rotation, dark mode, GPX import, camera, the notification, observer mode, offline mode and publishing a position.
.maestro/README.mdexplains how to run them and the rules for new flows.These tests use the production server
manual/chat_sendposts a real message, so it only runs when started by hand:./scripts/run-ui-tests.sh --public-chat.Maestro grants every permission by default, which would make the app start publishing positions. All flows therefore set
permissionsexplicitly.Because the job depends on the production server and the map tile server, an outage there turns it red without a code problem. I suggest not making it a required check, at least for now.
CI setup
GitHub's runners have no GPU, so the emulator renders everything in software. That shaped the configuration:
Nexus 4, 768×1280). At 1080×2400, System UI and the launcher stopped responding, and their dialogs blocked every flow.A run takes about 16 minutes, of which the flows take about 12. Maestro's screenshots and logs, logcat and tombstones are kept as artifacts for 14 days.
Testing
known-bugs/back_navigationfails as intended. It documents the Back button bug described in Update dependencies, Gradle and CI actions #382 and moves into the regular run together with a fix.run-ui-tests.shpasses a failing test run on as its exit code (checked with stubs).manual/chat_send: It passed its one run: the message reached the server and appeared in the app.This branch builds
master's dependencies, so the CI setup hasn't run with #382 yet. The two PRs don't touch the same files, and the first push tomasterwith both will run that combination.Not covered: zoom, activity recreation with "Don't keep activities", languages and external links. The dark mode and GPX flows check that the app keeps working, not how it looks.
🤖 Generated with Claude Code