Skip to content

test: characterization suite for every feature and SVG source - #44

Merged
herewithme merged 5 commits into
developfrom
test/unit-coverage
Sep 10, 2026
Merged

herewithme merged 5 commits into
developfrom
test/unit-coverage

Conversation

@herewithme

@herewithme herewithme commented Sep 10, 2026 •

Copy link
Copy Markdown
Member

Why

The plugin had no tests. This adds them before the performance work in #42 lands, so that PR can be rebased on top and shown not to change behaviour — rather than asserted not to.

Written against develop as it stands today: 176 tests, 427 assertions, all green on the unmodified plugin.

These are characterization tests

They assert observable behaviour — how many icons come back, with what names, labels, types and contents, and what markup reaches a page — and say nothing about when or how often files are read.

That is the whole design point. A change that alters only the mechanism (lazy loading, different cache keys, a new source type) leaves the suite green. A change that alters what a caller can see does not. That is what makes it a regression net for #42 rather than a cast of the current implementation.

Coverage

file subject
Icon/CollectionItemsFactoryTest.php every SVG source: folder, sprite, single file
Icon/CollectionTest.php the container: lookup, dotted-name keys, count, search
Icon/CollectionItemTest.php a single icon, and what survives the object cache
BlockRendererTest.php front-end markup, in full
RegistrationTest.php the public API a theme calls, plus editor preload paths
Rest/CollectionsControllerTest.php collection discovery and its permissions
Rest/IconsControllerTest.php listing, pagination, search, single item, permissions
Helpers/CacheTest.php the object-cache wrapper every source goes through
Helpers/KsesTest.php the KSES allowances that keep saved block markup valid

Worth calling out specifically:

  • Every SVG source, including icon_map labels, sprite version cache-busting, the public URL a sprite icon resolves to, empty files, unreadable paths, non-SVG files being ignored, and the blockparty_icons_svg_parse_tags filter.
  • All colour formats the renderer accepts (hex, short hex, var:preset|color|x, var(--x), var(--x, #fff)) and the ones it must reject (named colours, rgb(), javascript:, expression(), a quote-break injection attempt).
  • Permissions at every role on both REST endpoints — anonymous 401, subscriber 403, editor 200.
  • That <script> inside an SVG is still stripped by KSES despite the plugin's allowances.
  • That an empty cached result is a hit, not a miss — the distinction the factories depend on.

Infrastructure

Integration tests against a real WordPress, not mocks: the plugin leans on WP_Query, the object cache, WP_HTML_Tag_Processor, WP_REST_Controller and a set of filters, and mocking that surface would test the mocks.

Two ways to run, both verified:

# through wp-env, which supplies WordPress, the test library and a database
WP_ENV_PORT=8890 WP_ENV_TESTS_PORT=8891 npx wp-env start
npm run test:php

# anywhere else, against the Composer copy of WordPress — the path CI takes
cp tests/wp-tests-config-sample.php wp-tests-config.php   # then edit credentials
composer test

CI runs the suite on PHP 8.1, 8.2, 8.3 and 8.4 (.github/workflows/test-php.yml).

Findings

Three things surfaced while writing this. None is caused by these tests; all are recorded in tests/README.md.

  1. Collection's Iterator never yields anything. It walks an integer $position while add() keys items by name, so valid() asks for $items[0], which never exists — foreach over a populated collection yields nothing. Nothing in the plugin iterates a collection (every caller uses all()), so it is latent. test_known_defect_iteration_yields_nothing_even_when_populated records it and will fail when someone fixes it; update that test then rather than deleting it.

  2. BlockRenderer::render() cannot be called outside a block render pass on PHP 8.4. It calls get_block_wrapper_attributes(), which reads WP_Block_Supports::$block_to_render; that is null outside a render pass and PHP 8.4 warns. The tests go through render_block() instead, which is both closer to reality and warning-free. Not a plugin bug — real rendering always has that context — but worth knowing if anyone calls the renderer directly.

  3. The development-mode cache bypass is not coverable from a plugin suite. WordPress resolves wp_is_development_mode() from a constant behind a static cache and only lets its own core suite vary it. The suite therefore runs with caching active, which is the production path anyway.

For the reviewer

  • composer.json gained phpunit/phpunit ^9.6, yoast/phpunit-polyfills and wp-phpunit/wp-phpunit as dev dependencies. Resolving them needed -W, which moved some sebastian/* versions; Psalm still reports the same 221 errors as before and phpcs is clean, so the toolchain is unaffected.
  • PHPUnit 9.6 rather than 10, because the WordPress core test suite still requires 9.x.
  • wp-tests-config.php is git-ignored (it holds credentials and machine paths); a sample lives at tests/wp-tests-config-sample.php.
  • tests/ is excluded from releases already, via the existing /tests entry in .distignore.
  • Note for rebasing perf(icons): load SVG payloads on demand, and add a benchmark to prove it #42: that branch excludes ./tests/ from phpcs.xml.dist for its perf harness, which would also stop linting these test files. Narrow it to ./tests/perf/ on rebase — the PHPUnit suite is phpcs-clean and should stay that way.

Next

Once this is in, #42 rebases onto it. Any behaviour the optimisation changes will show up as a failing test.

🤖 Generated with Claude Code


Note

Low Risk
Adds tests and CI only; production plugin code is unchanged, so runtime risk is minimal aside from new dev dependencies and workflow secrets for private Composer repos.

Overview
Introduces a PHPUnit integration test suite (~176 tests) that runs against a real WordPress install via wp-env or Composer + wp-tests-config.php, documented in tests/README.md.

The tests are characterization-style: they lock in observable behaviour for icon loading (folder, sprite, single file), collections, registration APIs, block front-end rendering (including colour validation and escaping), both REST controllers, object-cache helpers, and KSES allowances. Shared fixtures and isolation live in tests/phpunit/TestCase.php; bootstrap enforces compiled build/block.json before rendering tests run.

Tooling and CI: dev dependencies add PHPUnit 9.6, wp-phpunit, and polyfills; composer test and npm run test:php (via tests/bin/phpunit.sh in the wp-env CLI container) run the suite. A new .github/workflows/test-php.yml matrix runs on PHP 8.1–8.4 (build assets, composer install, wp-env start, then PHPUnit). Local PHPUnit config paths are git-ignored; wp-tests-config.php is excluded from dist builds.

Reviewed by Cursor Bugbot for commit fc07015. Bugbot is set up for automated code reviews on this repo. Configure here.

The plugin had no tests. This adds 176 of them, describing what the plugin does
today so that a later refactor of how icons load and cache can be shown not to
change it.

Covered:

- every SVG source — folder, sprite, single file — including icon_map labels,
  sprite version cache-busting, the public URL a sprite icon resolves to, empty
  and unreadable inputs, and the blockparty_icons_svg_parse_tags filter
- the collection container: lookup by name, the dotted-name key, replacement,
  count, search across both name and label
- the registration API a theme calls, duplicate rejection, unknown types, and
  the editor's REST preload paths
- front-end markup: raw and sprite icons, size, border radius, link and
  aria-label, all accepted and rejected colour formats, and URL/label escaping
- both REST controllers: payload shape, pagination headers, search, single item,
  404s, and the permission checks at every role
- the object-cache wrapper, including that an empty result is a hit and not a miss
- the KSES allowances that keep saved block markup valid, and that <script>
  inside an SVG is still stripped

They are characterization tests: they assert observable behaviour — how many
icons, with what names, labels, types and contents, and what markup reaches a
page — and say nothing about when or how often files are read. A change that
alters the mechanism leaves them green; a change that alters what a caller sees
does not.

Infrastructure: integration tests against a real WordPress rather than mocks,
since the plugin leans on WP_Query, the object cache, WP_HTML_Tag_Processor,
WP_REST_Controller and a set of filters. The bootstrap runs both through wp-env
(npm run test:php) and against the Composer copy of WordPress (composer test),
which is the path CI takes on PHP 8.1 through 8.4.

Two findings recorded along the way, both in tests/README.md: Collection's
Iterator never yields anything because it walks integer positions over
name-keyed items, and the development-mode cache bypass cannot be exercised from
a plugin suite.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread tests/wp-tests-config-sample.php
Comment thread .github/workflows/test-php.yml Outdated
All four PHPUnit jobs failed on their first run: 23 failures, every one a
rendering test receiving an empty string, plus the assertion that the block type
is registered.

The cause was not the tests. The plugin calls register_block_type() on its build/
directory, build/ is git-ignored, and the workflow never compiled it — so the
block never registered and render_block() had nothing to return. Confirmed
locally by moving build/ aside: the same 23 failures over the same 408
assertions as CI.

The workflow now builds the assets once in its own job and hands them to the
matrix through an artifact, rather than running npm four times over, and checks
build/block.json exists before handing over to PHPUnit.

The bootstrap gained the same check with a pointed message, because a fresh
clone lands in exactly this state and two dozen unexplained failures are a poor
welcome.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 6bbf18d. Configure here.

Comment thread tests/bin/phpunit.sh
herewithme and others added 3 commits September 10, 2026 16:39
The workflow was standing up its own MariaDB service and generating a
wp-tests-config.php, duplicating what wp-env already provides — and it is where
the first CI failure came from. wp-env supplies WordPress, the WordPress PHPUnit
library and the database, so the workflow now runs `npm run test:php`, the same
command as locally. A green run on a laptop and a green run on the pull request
now mean the same thing.

The PHP matrix is kept: wp-env takes the version from WP_ENV_PHP_VERSION and
pulls the matching upstream wordpress:php8.x image. Documented in
tests/README.md, since it is also how you reproduce a single matrix job locally.

The hand-rolled path stays supported for anyone without Docker — the bootstrap
still falls back to the Composer copy of WordPress and tests/wp-tests-config-sample.php
is still there — it just is not what CI uses any more.

On failure the job prints the wp-env test-environment logs, which is the first
thing anyone would ask for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
86 of 176 tests failed in CI on every PHP version, all with the same message:
the fixture directory under wp-content could not be created.

The runner script was doing `docker exec -u 33`. wp-env builds its containers
around the *host* user — wp-content is owned by that UID and Apache runs as it —
so 33 is only right by accident. On macOS it works because Docker Desktop's file
sharing ignores ownership; on a Linux runner, where the host UID is 1001 and
ownership is real, www-data cannot write there. The same mismatch was behind the
PHPUnit result-cache permission warning.

Resolving the UID at run time matches what wp-env actually configures, in both
places.

Verified locally on PHP 8.1, 8.2, 8.3 and 8.4 through wp-env: 176 tests, 427
assertions, green on each.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review flagged the runner as never setting WP_TESTS_DIR, and concluded the suite
must be falling back to the Composer copy of WordPress and a root
wp-tests-config.php. It is not: wp-env sets the variable on its containers and
docker exec inherits it, so the bootstrap takes its managed-environment branch
and uses wp-env's WordPress, test library and database.

The reading was wrong but the confusion was fair — the script leaned on a
container-level variable it never mentioned. Both ends of that coupling now say
so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@herewithme
herewithme merged commit 2885074 into develop Sep 10, 2026
7 checks passed
@herewithme
herewithme deleted the test/unit-coverage branch September 24, 2026 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant