Skip to content

fixtures: a corpus with ground truth, and the defect it found - #9

Merged
inquerium merged 2 commits into
VedSoni-dev:masterfrom
inquerium:fixture-corpus
Aug 5, 2026
Merged

inquerium merged 2 commits into
VedSoni-dev:masterfrom
inquerium:fixture-corpus

Conversation

@inquerium

Copy link
Copy Markdown
Collaborator

Adds test/fixtures/, which docs/OPENCLAW.md already requires and which did not exist. Eight synthetic learners, each built to exhibit one property of the mastery model, each carrying an expect block stating what is true of it by construction.

Independent of #7. Touches nothing under world/.

Why

Every claim about the model was checkable only by reading the code. Nothing that examines the record, the scheduler, or a generated activity could be measured against anything.

test/fixtures.test.ts holds the corpus to its own claims. A persona that quietly stops exhibiting the property it is named for does not fail loudly. It makes everything measured against it report success against nothing.

The personas

id what it is by construction
cold-start Eight years old, empty record. Placement opens.
cold-start-young Five years old, empty record. Placement is withheld on purpose.
stuck-blending Nine attempts, two correct. Sorts to the top of the queue.
lapsed-summer Three crammed skills and one well-spaced one, same child, same summer off.
crammer Twenty right answers in one sitting.
vowel-confuser Reads every short e as short i, with short i mastered. Deliberately unlabelled.
accommodated Three active accommodations.
above-band Six years old doing grade-2 place value.

Two properties are now arithmetic rather than assumption. Eight correct answers across widening gaps buy a half-life near 65 days, so a well-spaced skill survives a summer and only lapses around 150. The same eight crammed into six consecutive days lapse in 60. lapsed-summer carries both in one child, so the spacing effect is visible as a difference rather than asserted.

vowel-confuser carries a real misconception in the response column with no misconception row. A detector scored against a record that already names the answer has found nothing.

The defect it found

nextTargets tested opportunities >= 6 && p < 0.5 before it tested lapsed. Six opportunities is an ordinary number of tries to learn something, so a child who mastered a skill in spring and took the summer off came back reported as stuck: the tutor was told to change modality or drop to a prerequisite for a child who needed a two-minute review of something she used to know.

peak_p_known is the only thing that separates "lost it" from "never had it". statusOf already reads it. The stuck branch now consults it.

stuck-blending still reports stuck, which is the case the branch is for.

Checks

npm test passes, 193 tests. CI cannot run: the upstream account is billing-locked and every workflow is refused before it starts.

🤖 Generated with Claude Code

Every claim this repository makes about the mastery model was checkable only
by reading the code. test/fixtures/ makes them checkable by running it: eight
synthetic learners, each built to exhibit one property, each carrying an
expect block stating what is true of it by construction.

They are also the substrate the pipeline needs. docs/OPENCLAW.md already says
all pipeline work uses synthetic fixtures in test/fixtures/, and pointed at a
directory that did not exist. Nothing that examines the record, the scheduler,
or a generated activity could be measured against anything until it did.

test/fixtures.test.ts holds the corpus to its own claims. That file is the
point of the exercise. A persona that quietly stops exhibiting the property it
is named for does not fail loudly, it makes everything measured against it
report success against nothing.

Two properties worth naming, because both were assumptions until they were
arithmetic. Eight correct answers across widening gaps buy a half-life of
about sixty-five days, so a well-spaced skill survives a summer away and only
lapses around a hundred and fifty. The same eight answers crammed into six
consecutive days lapse in sixty. lapsed-summer carries both, in one child, so
the spacing effect is visible as a difference rather than asserted as a claim.

vowel-confuser is deliberately unlabelled. It carries a real misconception in
the response column, bed read as bid and pen as pin, with short i mastered so
that not knowing short i is ruled out, and no misconception row at all. A
detector scored against a record that already names the answer has found
nothing.

The corpus found something on the first run. nextTargets tested
opportunities >= 6 && p < 0.5 before it tested lapsed, and six opportunities
is an ordinary number of tries to learn something, so a child who mastered a
skill in spring and took the summer off came back reported as stuck. The
tutor was told to change modality or drop to a prerequisite for a child who
needed a two minute review of something she used to know. peak_p_known is the
only thing that can separate lost it from never had it, statusOf already
reads it, and the stuck branch now consults it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The assertion read `validateInterface.length === 1`, meaning to fail the day
the validator learned who an activity was for. It would not have. Giving a
function a second parameter with a default leaves Function.length at 1, so the
check would have kept passing after the capability landed and would have
quietly asserted nothing.

That is precisely the silent rot this corpus exists to prevent, in the corpus.
A persona that stops exhibiting its property fails loudly; a pin that stops
pinning does not.

So the gap is now pinned by behavior: a page setting 9px text is accepted
today, and nothing reports which accommodations went unchecked. Both statements
become false the moment the constraint is enforced, and the test says what to
assert instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@inquerium
inquerium merged commit 18cbb76 into VedSoni-dev:master Aug 5, 2026
1 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant