Repository navigation
test: modernize visual regression with pinned Playwright - #2552
Merged
Merged
Conversation
…ywright-visual-regression # Conflicts: # yarn.lock
- Updated visual tests for Menu, RadioButtonGroup, RadioButton, SegmentedButton, and SidebarNavigation components. - Refactored test cases to use async functions for better organization and clarity. - Improved accessibility checks and assertions for various states and interactions. - Added keyboard navigation tests for SidebarNavigation and SegmentedButton components. - Enhanced screenshot capturing for different states to ensure visual consistency.
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
State captures remain contaminated, nested shadow components can still be measured prematurely, and the reported coverage totals conflict.
Review effort: Balanced
Findings: 3
Open (4)
What changed in this PR
Migrates visual regression testing from Jest/Puppeteer to pinned Playwright, adds deterministic readiness checks, and fixes DataGrid auto-width timing.
Changes:
- Introduces Playwright fixtures, containerized runners, reports, and snapshot policy checks.
- Migrates component visual and interaction suites to explicit state captures.
- Waits for component/font readiness before DataGrid width measurement.
| File | Description |
|---|---|
.github/workflows/build-pr.yml |
Splits Storybook build and visual comparison jobs. |
docs/2026-10-07-visual-test-modernization.md |
Documents research and migration decisions. |
packages/components/package.json |
Declares Jest dependencies directly. |
packages/components/src/components/data-grid/data-grid.spec.ts |
Tests delayed auto-width measurement. |
packages/components/src/components/data-grid/data-grid.tsx |
Adds asynchronous measurement readiness. |
packages/visual-tests/.gitignore |
Ignores Playwright outputs. |
packages/visual-tests/README.md |
Documents the new browser-test workflow. |
packages/visual-tests/inject-fail-images.js |
Removes legacy report customization. |
packages/visual-tests/jest-puppeteer.config.js |
Removes Puppeteer configuration. |
packages/visual-tests/jest.config.js |
Removes Jest visual-test configuration. |
packages/visual-tests/outdated-snapshot-reporter.js |
Removes legacy snapshot cleanup. |
packages/visual-tests/package.json |
Replaces visual-test dependencies and scripts. |
packages/visual-tests/playwright.config.js |
Configures Playwright projects and reporting. |
packages/visual-tests/scripts/check-snapshot-policy.js |
Verifies read-only snapshot comparisons. |
packages/visual-tests/scripts/container-entry.js |
Runs Playwright inside the pinned container. |
packages/visual-tests/scripts/prepare.js |
Builds and stages Storybook. |
packages/visual-tests/scripts/run-container.js |
Starts the canonical Docker environment. |
packages/visual-tests/setup.js |
Removes the legacy server setup. |
packages/visual-tests/teardown.js |
Removes legacy teardown/report copying. |
packages/visual-tests/test-environment-setup.js |
Removes global Puppeteer helpers. |
packages/visual-tests/src/accordion.visual.spec.js |
Migrates accordion states. |
packages/visual-tests/src/brand-header.visual.spec.js |
Migrates skipped brand-header coverage. |
packages/visual-tests/src/breadcrumb.visual.spec.js |
Migrates breadcrumb states. |
packages/visual-tests/src/button.visual.spec.js |
Migrates button variants and interactions. |
packages/visual-tests/src/callout.visual.spec.js |
Migrates skipped callout coverage. |
packages/visual-tests/src/card.visual.spec.js |
Migrates card states. |
packages/visual-tests/src/checkbox-group.visual.spec.js |
Migrates checkbox-group states. |
packages/visual-tests/src/checkbox.visual.spec.js |
Migrates checkbox variants and states. |
packages/visual-tests/src/chip.visual.spec.js |
Migrates chip variants. |
packages/visual-tests/src/components.interaction.spec.js |
Adds native interaction checks. |
packages/visual-tests/src/datagrid.visual.spec.js |
Waits for DataGrid measurement completion. |
packages/visual-tests/src/date-picker.visual.spec.js |
Migrates date-picker states. |
packages/visual-tests/src/divider.visual.spec.js |
Migrates divider variants. |
packages/visual-tests/src/dropdown-select.visual.spec.js |
Adds dropdown-select behavior coverage. |
packages/visual-tests/src/dropdown.visual.spec.js |
Migrates deprecated dropdown states. |
packages/visual-tests/src/footer.visual.spec.js |
Migrates footer variants. |
packages/visual-tests/src/icon.visual.spec.js |
Migrates icon variants. |
packages/visual-tests/src/link.visual.spec.js |
Migrates link states. |
packages/visual-tests/src/list.visual.spec.js |
Migrates list variants. |
packages/visual-tests/src/logo.visual.spec.js |
Migrates logo states. |
packages/visual-tests/src/menu-flyout.visual.spec.js |
Adds flyout interaction coverage. |
packages/visual-tests/src/notification-badge.visual.spec.js |
Migrates badge variants. |
packages/visual-tests/src/notification.visual.spec.js |
Migrates notification variants. |
packages/visual-tests/src/pagination.visual.spec.js |
Migrates pagination states. |
packages/visual-tests/src/progressbar.visual.spec.js |
Migrates progress-bar variants. |
packages/visual-tests/src/radio-button-group.visual.spec.js |
Restores radio-group coverage. |
packages/visual-tests/src/radio-button.visual.spec.js |
Restores radio-button coverage. |
packages/visual-tests/src/rating-stars.visual.spec.js |
Migrates rating-star states. |
packages/visual-tests/src/segmented-button.visual.spec.js |
Restores segmented-button interactions. |
packages/visual-tests/src/sidebar-navigation.visual.spec.js |
Restores sidebar behavior coverage. |
packages/visual-tests/src/slider.visual.spec.js |
Migrates slider states. |
packages/visual-tests/src/switch.visual.spec.js |
Migrates switch states. |
packages/visual-tests/src/tab-navigation.visual.spec.js |
Migrates tab-navigation states. |
packages/visual-tests/src/table.visual.spec.js |
Migrates table variants and hover. |
packages/visual-tests/src/tag.visual.spec.js |
Migrates tag variants. |
packages/visual-tests/src/test-fixtures.js |
Adds shared readiness and screenshot fixtures. |
packages/visual-tests/src/textarea.visual.spec.js |
Migrates textarea states. |
packages/visual-tests/src/textfield.visual.spec.js |
Migrates text-field states. |
packages/visual-tests/src/toggle-group.visual.spec.js |
Migrates skipped toggle-group coverage. |
packages/visual-tests/src/tooltip.visual.spec.js |
Migrates tooltip hover coverage. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+743
to
+754
| await Promise.all( | ||
| Array.from( | ||
| this.hostElement.shadowRoot.querySelectorAll( | ||
| `.${name}__auto-width-check td *` | ||
| ), | ||
| ( | ||
| element: HTMLElement & { | ||
| componentOnReady?: () => Promise<HTMLElement>; | ||
| } | ||
| ) => element.componentOnReady?.() | ||
| ) | ||
| ); |
Comment on lines
+22
to
+25
| await page.mouse.move(60, 40); | ||
| await page.mouse.down(); | ||
| await story.screenshot('active.png'); | ||
| await page.keyboard.press('Tab'); |
Comment on lines
+24
to
+26
| await page.mouse.down(); | ||
| await story.screenshot('active.png'); | ||
| await page.keyboard.press('Tab'); |
Comment on lines
+130
to
+132
| All 38 legacy files are migrated: 478 active visual cases plus four native | ||
| interaction cases across both themes. Button interaction states now run in both | ||
| themes instead of inheriting the previous suite's theme. |
This was referenced Oct 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Removes deprecated visual-only Docker/Puppeteer/Jest dependencies. Core declares its existing Jest dependencies directly. DataGrid now waits for nested components and fonts before automatic width measurement, with a regression spec.
Evidence
Before: Chromium 108 and implicit baseline updates; reproduced DataGrid measuring an unready cell as 12px instead of 120px.
After, local:
Implementation checks are green at c7678a3: build-pr and CodeQL. Remote visual report: 400 passed, 128 skipped, zero unexpected/flaky results, 4.1 minutes. Final published head 01b8670 is also green: all eight build-pr jobs and CodeQL. Its downloaded report confirms 400 passed, 128 skipped, zero unexpected/flaky results, 474 expected PNG attachments, 4.4 minutes. Artifact names use Actions PR merge SHA 3d0f3d2; its parent is the exact final head, and the Storybook producer checkout was verified. Final browser evidence. CI also fixes scoped Git checkout trust and explicitly installs the existing core browser after dependency-cache restoration; image assertions are unchanged. Visual artifacts include HTML, JSON, JUnit, failure images and traces. Nine previously skipped suites remain visible coverage debt.
Merge Danger
Door: Two-way. Blast radius: Visual tests plus DataGrid initial auto-width timing.
Chromium 108 → 153 requires baseline replacement and named state paths. Passing comparisons do not establish design approval. Review baseline changes and the HTML report before merge. This PR stays draft and unmerged.
Setup and baseline review: packages/visual-tests/README.md. Research: docs/2026-10-07-visual-test-modernization.md.
Playwright CLI assessment is complete in the research document, after green checks. Keep the Playwright Test CLI already used by local/Actions gates. The separate session CLI is an optional inspection/debugging companion, not a replacement for PNG comparison, fixtures, or reports. The pinned 1.63.0 package exposes playwright cli --help; standalone attachment compatibility is not runtime-tested. No second runner migration or performance claim is included.