Repository navigation
test(web): track every page's Lighthouse results over time and flag regressions - #355
Merged
Merged
Conversation
adapter-node 6 takes every request to be https unless PROTOCOL_HEADER names a header that says otherwise, so SvelteKit's CSRF check refused the sign-in form posted from http://127.0.0.1. dev-network.ts --production now sets PROTOCOL_HEADER=x-forwarded-proto. The OAuth client also refused the network's http PDS outside dev. It now allows http when ALLOW_PRIVATE_NETWORK is set, which is already what lets the production build reach a loopback PDS.
lighthouse/pages.json lists the pages to measure, each signed out or signed in as a seeded account, and lighthouserc.cjs takes its pull request pages from it. `pnpm test:lighthouse:full` starts the network once, signs in with Playwright, and runs lhci collect on every page on a phone and a desktop against the existing budgets. scripts/lighthouse-history.ts then holds each page's medians against the last five recorded runs and fails on a regression past a metric's tolerance or on a bad result (a failed run, a redirect, an error status, a failed budget). The e2e sign-in helper moves to tests/sign-in.ts so both use it.
Web scheduled runs scripts/lighthouse.ts in one job per form factor, then compares their summaries with main's history in a job that installs nothing. The history is the lighthouse-history artifact, which each run on main downloads, adds to and uploads again. A regression or bad result fails the run, which opens the scheduled-failure issue.
…n main The history job took the newest lighthouse-history artifact from any run whose head branch was named main, which a pull request from a fork with a branch of that name also has, and unpacked it over apps/web, where it could replace the script the job runs next. It now only looks at this workflow's scheduled and hand-started runs in this repository, unpacks each into an empty directory, and copies out the history file alone.
# Conflicts: # .github/workflows/web-ci.yml # .github/workflows/web-scheduled.yml
github-actions
Bot
added
documentation
bnb
approved these changes
Nov 1, 2026
Coverage Report for apps/web
File Coverage
SizeCompared with main. KiB are 1,024 bytes.
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Coverage Report for apps/localtoolbox
File CoverageNo changed files found.SizeCompared with main. KiB are 1,024 bytes.
|
Contributor
Author
|
Closing and reopening so CI runs against current main (with #353's CodeQL fix). Branch unchanged. |
Rust coverageFrom run 33. Both are held to the thresholds in .cargo/config.toml.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Pull requests measured 3 pages against fixed budgets, with no history, so slow drift and signed-in pages went unseen. Added:
lighthouse/pages.json(41 pages, signed out or asneon.test/ada.test, with a schema),scripts/lighthouse.ts(one network start, Playwright sign-in,lhci collectper page on phone and desktop, 3 runs each, existing budgets),scripts/lighthouse-history.tswithscripts/lib/lighthouse.ts(per-page medians in a JSON Lines history; fails on a regression past each metric's tolerance against the last 5 runs, or a bad result: failed run, redirect, error status, failed budget; notes slow creep, unstable runs and machine or Lighthouse changes),pnpm test:lighthouse:full, and weekly jobs inweb-scheduled.ymlthat keep main's history as thelighthouse-historyartifact and open an issue throughscheduled-issue.ymlwhen they fail.lighthouserc.cjsnow takes its pages from the list (the ones markedpr).Changed: the production build on the local network couldn't be signed in. adapter-node 6 treats requests as https, so the CSRF check refused the form;
dev-network.ts --productionnow setsPROTOCOL_HEADER. And the OAuth client refused http PDSes outside dev; it now allows them whenALLOW_PRIVATE_NETWORKis set. The e2e sign-in helper moved totests/sign-in.tsso both use it.The history job reads history only from this workflow's own scheduled or hand-started runs on main, unpacked into an empty directory with just the history file copied out, so a fork's pull request can't plant one. The Lighthouse jobs share the e2e tests'
changesgate from #352; a skipped week only uploads the history again so it doesn't expire.Signed-in pages aren't held to an SEO budget. The first weekly runs will report the existing bad results (8 public pages without a meta description, and layout shift on 5 pages) until they're fixed separately.
How it was tested
Unit tests for the comparison and summaries, checked by breaking the logic on purpose; an OAuth test that fails without the
allowHttpchange. A full local run: all 41 pages on both form factors, 3 runs each. Chromium e2e: one failure that also fails on main. Coverage, lint, type check, knip, actionlint, zizmor andtools/policy.shpass. The weekly jobs and the artifact hand-off only run on GitHub.