Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ Unit tests run on `bun test` (repo root, `bunfig.toml`); component and visual te

**Budgets are enforced.** `scripts/test-run.ts --budget=<name>` starts the clock once it holds the lock, prints the 10 slowest tests when the run ends (runners append to `CA_TEST_TIMINGS`: `tests/timing-reporter.ts` for Playwright, `packages/ui/tests/timing-reporter.ts` for Vitest) and **fails** a run over its budget. A budget is wall time, so on an oversubscribed machine (load average above the core count) the verdict is printed as `OVER BUDGET, NOT ENFORCED` and does not fail the run: it would only measure the other processes. Rerun when the load is below the core count.

**The Release workflow's Verify is unbudgeted, on purpose (#643).** It runs `bun scripts/validate.ts` (every check, no wall time) instead of `bun run validate`: budgets are measured on the owner's machine, and the 4-core GitHub runner took 110.6 s against the 90 s budget for 4.6.0 while every check passed. The budgets stay enforced locally (`validate` on push, the land gate, and the four release commands `bun run release` runs before it tags), so a slow change is still caught before a tag exists. If CI is re-enabled, give each CI job its own measured budget in its own commit instead of reusing these.

**One suite at a time.** A run takes a machine-wide lock (`$TMPDIR/ca-test-lock`, shared by every worktree and checkout; `CA_TEST_LOCK_DIR` overrides it) and a second run waits, printing who holds it, instead of starting more browsers. A holder that died is taken over, and a nested `test-run` (the commands above call each other) reuses its parent's lock.

**The package build skips itself when nothing it reads changed.** `bun run build` in `packages/ui` is `scripts/build-package.ts`: it hashes the bytes of every input (`INPUTS` in `packages/ui/scripts/build-inputs.ts`: the package sources and scripts, the site's demos, docs, data and changelog parser, `bun.lock`, the versions of bun and svelte-package; `scripts/content-hash.ts`) as they are on disk, uncommitted edits included, and stops with `build: dist is current` when the hash is the one `dist/.ca-input-hash` holds (0.1 s instead of 12 to 15 s; the land gate builds, then `validate` on the push builds again). The stamp is written only after the last step succeeds. `tests/build-cache.test.ts` fails when a file the build scripts import is not under an input. A release never builds from the skip: `release.js` and `test:full` set `CA_BUILD_COLD=1`; set it yourself to force a build.
Expand Down
2 changes: 2 additions & 0 deletions site/src/content/changelog/4.6.0.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,8 @@ The suites froze a 12-core machine: 2433 Playwright tests loaded all 182 docs ro

Every command has a time budget that is enforced: a run over its budget fails and prints its 10 slowest tests. On an oversubscribed machine the verdict is printed as not enforced, because it would only measure the other processes.

One gate runs without a budget: the GitHub Release workflow's verify step runs every check of `validate` but not its 90 s clock, because a 4-core hosted runner took 110.6 s for this release while every check passed (issue 643). The four release commands above run with their budgets on the maintainer's machine before any tag is created.

<Fig id="gate-before-after" />

<Bench title="One-component edit, bun run test" unit="s" rows={[{ label: "Total", value: 28, good: true }, { label: "Routes", value: 16.6 }, { label: "Screenshots", value: 6.7 }, { label: "Components", value: 4.6 }, { label: "Unit", value: 0.4 }]} conditions="A one-component edit with a fresh dist, wall time on the maintainer's 12-core machine at load 15 (shared with other work), from the pull request that introduced the system (547). The budget for this command is 60 s." />
Expand Down