Skip to content

Make every port notice a fixture nothing decodes - #49

Merged
ergofobe merged 2 commits into
mainfrom
issue-48-conformance-guard
Sep 16, 2026
Merged

ergofobe merged 2 commits into
mainfrom
issue-48-conformance-guard

Conversation

@ergofobe

@ergofobe ergofobe commented Sep 16, 2026

Copy link
Copy Markdown
Owner

Closes #48.

Since #39 the TypeScript suite has asserted that every fixture in models.json
has a schema to decode it. The other four ports hand-maintained their lists with
nothing holding them up against the file, so a thirty-seventh fixture would have
been silently unchecked in Rust, Python, Swift and Kotlin — the same mechanism
that let TypeScript drift to 26 of 36 without anyone noticing.

What changed

  • conformance/models.json: assetPage now carries one real asset. With items
    empty the fixture proved the envelope decodes and never decoded an Asset through
    it, so a port whose page held the wrong element type passed everywhere. All five
    already type the items as assets, so nothing failed — the fixture now holds them to it.
  • Each port asserts coverage and fails naming the fixture nothing decodes.
    Python's map was already a dict, so it only needed the assertion; Rust, Swift and
    Kotlin turn their run of check calls into a table keyed by fixture name, which the
    guard reads. $comment and version describe the file rather than a model, and are
    skipped in all five the way TypeScript already skipped them.

Proved to bite

A guard that cannot fail is worse than none, so each was run against a fixture
(assetSidecar) added to models.json with no registered type, then the fixture removed:

Port Failure
Rust fixtures in the contract with no type to decode them: ["assetSidecar"]
Python AssertionError: fixtures in the contract with no model to decode them: ['assetSidecar']
Swift FAIL every fixture in the contract has a type to decode it (1)fixtures in the contract with no type to decode them: expected [], got ["assetSidecar"]
Kotlin AssertionFailedError: fixtures in the contract with no type to decode them ==> expected: <[]> but was: <[assetSidecar]>
TypeScript already guarded by #39; fails the same way on the same canary

Additive?

Yes — additive, and in substance a test-only change. Nothing on the wire moves: no
production code is touched, no endpoint, request field or response field is added or
retyped. The assetPage payload gains an item, which changes what every port must
decode in its own test run — but it asks for nothing a server has to send, and an
AssetPage carrying assets is what the contract always said. The one way this could
break a consumer is if a port had modelled the page's items as something other than
Asset; all five were checked and none had, and that disagreement is exactly what the
item exists to catch from now on.

One limit, carried over

items.0.duration: null inherits the limitation #47 wrote down: every port's walker
reads an absent key as null, so a null assert cannot tell a dropped field from a carried
one. The other nine items.0.* asserts are non-null and do catch a dropped field, so the
item is exercised either way — but the null one is a weaker check than it looks.

Verified

  • typescript: bun install && bun run verify — 201 pass, 0 fail
  • rust: cargo test && cargo clippy --all-targets && cargo fmt --check — 18 conformance tests pass, clippy clean, fmt clean
  • python: uv run pytest && uv run ruff check && uv run ruff format --check . — 55 passed, clean
  • swift: swift run ImogenSDKConformance — 258 checks passed
  • kotlin: ./gradlew build --rerun-tasks — BUILD SUCCESSFUL

Found, not fixed

  • The Kotlin test task does not declare conformance/ as an input (it reads the files
    through user.dir at run time), so a contract-only change leaves ./gradlew build
    UP-TO-DATE and the suite does not re-run locally. CI starts clean, so it only bites a
    developer on loki; --rerun-tasks works around it. Nit-to-significant, not filed.

🤖 Generated with Claude Code

ergofobe and others added 2 commits September 15, 2026 22:38
With `items` empty the fixture proved the page envelope decodes and never
decoded an Asset through it, so a port whose page carried the wrong element
type would have passed in all five languages.

All five already model the items as assets, so nothing failed — the fixture
now holds them to it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TypeScript has asserted since #39 that every fixture in `models.json` has a
schema to decode it. Rust, Python, Swift and Kotlin hand-maintained their
lists with nothing holding them up against the file, so a thirty-seventh
fixture would have been skipped in silence in four languages — the mechanism
that let TypeScript drift to 26 of 36 without anyone noticing.

Each port now asserts coverage and names what is missing. Python's map was
already a dict, so it only needed the assertion; Rust, Swift and Kotlin turn
their run of calls into a table keyed by fixture name, which the guard reads.
Each was proved to bite by adding a fixture with no registered type and
watching that port's suite fail naming it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Only the TypeScript conformance suite checks that every fixture is registered

1 participant