Repository navigation
feat: test: lighter suites: one pass per route, targeted local runs, machine-wide lock, native VRT - #547
Merged
Conversation
mvilacad
force-pushed
the
feat/546-test-lighter-suites-one-pass-per
branch
2 times, most recently
from
October 25, 2026 04:37
99d3ec7 to
6c66fb1
Compare
mvilacad
marked this pull request as ready for review
October 25, 2026 04:37
mvilacad
force-pushed
the
feat/546-test-lighter-suites-one-pass-per
branch
from
October 25, 2026 06:55
6c66fb1 to
13b5015
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #546
Not ready to land:
test:fulldoes not meet its 240 s budget (see the table). Everything else below is pushed andvalidatepasses on every push.What changed
scripts/test-run.ts,BUDGETS):changed60 s,validate90 s, land 180 s,full240 s. Over budget fails the run and prints the 10 slowest tests. On an oversubscribed machine (load above the core count) the verdict is printed asNOT ENFORCED, because it would only measure other processes.bun run test=scripts/test-changed.ts. Component tests are picked by a static import graph (scripts/test-graph.ts), because Vitest's--changedselected 198 of 198 files for a one-line Avatar edit in browser mode. Routes byscripts/route-impact.ts, which fails safe (shared path or unknown file selects every route and says why). Shared files select every component test.routes.spec.ts(smoke + layout audit),routes.axe.spec.ts(ownaxeproject, contrast on in dark and light, forced colors on every route,<pre>excluded),dist-html.spec.ts(no browser).makeAxerunsaxe.runin the page: 4 to 7 times faster than@axe-core/playwright's builder with identical findings (checked on 8 page/scheme pairs with injected defects, frames included).vrt-docker.shremoved, site shots are viewport-only for landing/templates/theme. Docker is gone. CI is untouched and stays disabled.settled()jumps running animations to their end state, the pointer park only runs after a pointer event, Carousel's 4 real waits run on a fake clock, Shiki palette contrast is a component test.release.tsrunstest:fullbefore versioning;issue-land.shrunstest-changed --land(validate runs on the push).postinstallnow installs WebKit (theiphoneproject needs it andtest:fullincludes it).Found and fixed on the way
reuseExistingServerwas on, so the suite compared that site with itself. Now: one port per checkout, no reuse, and a global setup that fails the run if the served<title>is not CodeAtlas. Both proven with a deliberate collision. Baselines regenerated and checked twice against the real site.validateand failed only there; two manifest tests had no timeout under load.Measured (wall time; machine shared, load noted; never a quiet window for the full runs)
test(one-component edit, fresh dist)validatetest:fullStill open
test:fullis over budget by construction: components 109 + screenshots ~105 + routes 168 + unit 40 is ~7 minutes. Needs a decision on the 240 s budget or on what moves out of it.settled()effect on the full component run have no clean measurement yet.isolate: falsewas measured: 775 of 2095 tests fail, 642 s. Not an option.