[stephrobert]

Summary

Five hand-written statements about the size of the falsify specs were stale, and nothing recomputed any of them: CONTRIBUTING.md and its French twin said 128 specs naming 186 files, tools/testplan/specs.go had been corrected by hand to 251 and was wrong one merge later, falsify.yml said "twenty-three times", and mise.toml, the selftest docstring and CONTRIBUTING disagreed on whether the selftest holds three mistakes or four.

This does not correct the numbers: it gives them a control. specFigureProblems (tools/testplan/figures.go) recounts the specs and refuses a published figure that is not the live count, in the four places that state it as a fact about today. It runs in go test ./..., so in prepush.

It has already caught a real drift. Rebasing this branch onto #952, which added a spec, made the check refuse this very pull request's own figures without anyone provoking it:

CONTRIBUTING.md: publishes 254 specs, the specs hold 255
tools/testplan/specs.go: publishes 1701 mutations, the specs hold 1709

Without it, this pull request would have been born with wrong numbers. It also caught its author once while being written: adding its own falsify spec moved the counts and it named the figures just written by hand.

Present-tense counts tied to a timing measured on a given day are dated rather than recounted (falsify.yml, nightly_reporting_test.go, mise.toml, falsify.py): recounting the number alone would leave the timing beside it stale. The selftest count is four everywhere, all four entries in b39e4af (#187).

What makes it red

Three outcomes, never two. Red when a published figure differs from the live count; when the sentence it reads matches zero or several times (a rewritten sentence must not pass silently); when the specs directory is unreadable, empty, or holds a spec with no mutation; when a quantity name is unknown (the default refuses); and when no site is declared. Each has its own test asserting its own reason, built from a world where the three quantities differ (2 specs, 4 mutations, 3 files), so no case can pass for another's reason.

Type of change

• A route added or changed
• Bug fix
• Refactor
• Documentation
• Chore / tooling
• Security / supply chain

Checklist

Always

• mise run check passes, through mise run prepush, green in 110.61 s on the rebase onto ecd639a
• No new external Go dependency
• internal/core untouched
• Nothing provider-specific
• mise run testplan was run. Played: falsify -- published-spec-figures.json (18/18 biting, restoration green, replayed after the rebase), falsify:lint, docs:check, lint-shell, and the six specs it names (a-plan-that-cannot-rot, a-run-ends-where-it-could-start, a-silent-night-is-refused, evidence-leg-mode, the-night-fetches-what-it-extracts, unkillable-dhcp-orphan), all exit 0, 82 mutations biting, none not biting. Not played locally: conformance:leg -- probe, named because mise.toml moved; the mise.toml diff is comments only (checked), and CI plays the probe leg on this pull request.

When a model wrote a substantive part of this

• Assisted-by: trailer names the tool and the model version
• mise run conformance: not run, no route, response or runtime path changed. CI plays the client legs.
• Field names: N/A

When a route is added or changed

N/A.

When behaviour a client can observe changes

N/A: contributor documentation and an internal check only.

When .github/workflows/ is touched

• falsify.yml changed in comments only (checked: no non-comment line in the diff). No action, permission, job name or required check changed. actionlint green locally.

When a machine runtime is involved

N/A.

Reviewed before the commit

falsifier on the uncommitted tree. It found three gaps, all closed in this pull request and re-measured: three present-tense stale counts outside the check's list (now dated); two tests that passed with their guard removed (one asserted only a message prefix shared with another guard, one guard had no test); three mutations that bit but were undeclared (the spec went from 13 to 18).

Not covered, each filed

• a present-tense count written after this search is noticed by nothing, the list is checked against a second hand-written list: Nothing notices a new present-tense count of the falsify specs written after #937's search #951
• the "four" in CONTRIBUTING is not checked against len(HISTORY): The "four historical mistakes" count is written by hand in both CONTRIBUTING pages and agrees with len(HISTORY) by nobody's measurement #949
• a spec path written ./x is counted twice and never claimed by testplan; latent, no such path today: A falsify spec naming ./x.go instead of x.go is never claimed by testplan, and falsify:lint accepts both #950

Related issues

Closes #937.

🤖 Generated with Claude Code