Repository navigation
fix(replay): fresh-account routines replay on a new account - #200
Merged
Merged
Conversation
Two things broke every new-user retest (bugpatrol#197):
- One-time values. The explorer recorded literal usernames
(`bugpatrolnew77`, `bug-patrol-77`) into the onboarding routines. Dev keeps
a username taken after its account is deleted, so on the next fresh account
Save Profile stays disabled, the replay never leaves the Profile step, and
every later step falls back to stored points. Vars now carries a public
`{{RUN_ID}}`, new on each run, and the explorer is told to put it in any
value the app accepts only once. Its trail already keeps placeholders, so a
routine records `bugpatrolnew{{RUN_ID}}` and each replay types a new name.
- The connect hand-off. A connect URL like `/test/e2e-login?token=...` signs
in ~1s after load and then redirects itself. A routine that starts with
`open /` reloaded the page before that, and the app came up signed out
(2 of 2 immediate opens; 0 of 1 after the redirect). connect() now waits,
up to 15s, for a connect URL with a query to move on.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Nebula AI Agent <277912164+nebula-ai-agent[bot]@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
Bugpatrol review: dashboardNo problem introduced · tested Tested
Not tested
Updated on each push. Tests only what the diff can affect. Does not block the merge. |
Bugpatrol review: fixture-appNo problem introduced · tested Tested
Not tested
Updated on each push. Tests only what the diff can affect. Does not block the merge. |
Bugpatrol review: cliNo problem introduced · tested
TestedThe explorer stopped before it finished (timeout), so it tested a part of the change only. Updated on each push. Tests only what the diff can affect. Does not block the merge. |
compactSteps cuts a detour when the app comes back to an earlier place,
and the place was the screen key plus the element names. Field values were
not part of it, so on the onboarding Team step "type not-an-email, Send
invite (error), type {{TEAMMATE_EMAIL}}" looked like a return to the same,
unchanged form, and the real invite was cut: 19 Nebula Mobile routines
replayed only the invalid address and stalled on Team (#197).
The location key now includes each element's value.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Nebula AI Agent <277912164+nebula-ai-agent[bot]@users.noreply.github.com>
4 tasks done
Contributor
|
Merged as a216d5b. |
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #197.
What broke
Every new-user retest on 11-16 came back
unclear, because the onboarding routines it depends on could not replay on a fresh account. There were four causes. I reproduced each one on dev with step-level replay logging.bugpatrolnew77,bug-patrol-77, and others). Dev's/user/username/checkstill reports these as taken after their accounts are deleted. So Save Profile stays disabled and the replay never leaves the Profile step.screen-onboarding-agents-runstarts withopen /. The replay ran it straight after connect, while/test/e2e-login?token=...was still seeding the session (it redirects itself about 1s after load). The app came up on "Welcome to Nebula". In a Playwright probe, 2 of 2 immediate opens landed signed out; an open after the redirect stayed signed in.networkidlefires before the redirect, so it doesn't help.resolveTargetfound no named match yet and fell straight to the stored point (step 2 type … degraded=true). The value went nowhere, and every later step typed into the Profile page through the bareinputselector or clicked stored points.compactStepsremoves a "detour" when the app returns to an earlier place. That place was the screen key plus element names, with no field values. So "typenot-an-email, Send invite (error), type{{TEAMMATE_EMAIL}}" looked like a return to the same unchanged form, and the step that typed the real address was dropped. 19 mobile routines replayed only the invalid address and stalled on Team.Change
{{RUN_ID}}(vars.ts,prompts.ts, the explorer'stypetool): every run gets a publicRUN_IDof 8 lowercase letters and digits, and a setup capture of the same name overrides it. The explorer is told to put it in any value the app accepts only once. The trail stores the unresolved text, so routines keep the template.web.ts): when the connect URL has a query,connect()waits up to 15s for the page to leave it.dom.ts,web.ts): a replayed target with a test id, or a role and a name, gets up to 5s to appear before the text, selector or point stands in. Explorer actions use refs and are unchanged.explorer.tslocationKey): each element's value is part of the place, so typing that changes a form is never taken for a detour.configuration.mddocuments{{RUN_ID}}and the hand-off wait.Verification
web-origin.test.ts(hand-off): withawaitHandOffcommented out, it received/test/login?token=abc.dom.test.ts(named-target wait): fails with the wait loop disabled.agents-flow.test.ts(form retyped after an error): with the old key, the routine lostfriend@example.comand its tap.vars.test.ts(RUN_ID): unknown variable without the change.pnpm typecheck,pnpm checkandpnpm knipare clean.pnpm test: everything passes except the 2launchElectrontests, which need a full Chromium build this machine doesn't have. CI has it.screen-onboarding-agents-livefailed at step 3 (Save Profile)./onboarding/team. The replay then stalled on Team, which exposed cause 4.iss_a850d7b4195bcame back fixed andiss_9c35fb29d760came back fixed.Rollout notes
~/.bughunters/routines-backup-20261116-bp197/:bugpatrol…usernames were changed tobugpatrolnew{{RUN_ID}};{{TEAMMATE_EMAIL}}back.bugpatrolnew{{RUN_ID}}. Merge them only after a release that includes this PR.🤖 Generated with Claude Code