Skip to content

Let the card check tell a site that grew from a wrong card - #212

Merged
realmarcin merged 8 commits into
mainfrom
fix/card-check-semantics
Sep 25, 2026
Merged

realmarcin merged 8 commits into
mainfrom
fix/card-check-semantics

Conversation

@realmarcin

@realmarcin realmarcin commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Closes #148, closes #113, closes #115, closes #176, closes #114, closes #217, closes #218, closes #219, closes #220, closes #221, closes #222, closes #231, closes #232, closes #233, closes #234, closes #235, closes #239, closes #240, closes #241, closes #242, closes #243, closes #247, closes #248, closes #249, closes #250, closes #257, closes #258, closes #259, closes #260, closes #261, closes #262, closes #263, closes #264.

What changes

scripts/fleet/check_cards.py sorts each card into one verdict, judged against _fleet/data/site_audit.json, which the refresh writes:

Verdict When Exit
ok site equals card pass
grew site ahead of card, at most 14 days after the audit's pinned_at_utc (GRACE_DAYS), and at most 50% ahead (MAX_LEAD) warning
STALE site ahead past either limit, or no audit fail
WRONG card differs from figure_at_pin, the figure the audit read from the source's committed copy at the pin fail (#217, #231)
SHRANK site behind card fail
GONE source answered a 4xx other than a throttle fail (#176)
CHANGED source arrived with no readable figure, or CultureMech's generated block is missing fail (#176)
MARKUP a card without exactly one readable headline tile, a figure outside every card, or two cards for one Mech fail (#218, #261, #262)
UNCARDED a card with no SOURCES entry, or the reverse fail
AUDIT site_audit.json missing or malformed (including null, {} or no repositories), a figure_at_pin missing or not a whole number, or pinned_at_utc missing, unreadable or in the future (a time with no offset is taken as UTC) fail (#232, #242, #250)
unread DNS, timeout, dropped connection, 5xx, or a 408/425/429 throttle warning (#219, #220)
UNCHECKED more than half the sources unread fail (#115)

Evidence

Judgement calls

  • The grace period and lead limit (14 days, 50%). While any Mech grows, the nightly asks for a full refresh a fortnight after the pins, or sooner for a Mech as fast as CellStructureMech (under a week). Both are constants at the top of the script.
  • ProteinTraitsMech has no figure_at_pin. Its data file is built in CI, so there is no committed copy at the pin. Its card is checked only against the live site.
  • figure_at_pin in the current audit was read from the committed copies at the Refresh the X-Mech page from the Mechs as they are now, and add the skill that does it #120 pins in the refresh snapshot. Each copy's sha256 matched the audit's recorded *_sha256_at_pin, and each figure equals its card.

🤖 Generated with Claude Code

@realmarcin
realmarcin force-pushed the fix/card-check-semantics branch from 1710250 to b008369 Compare September 25, 2026 20:48
@realmarcin
realmarcin force-pushed the fix/card-check-semantics branch from b008369 to c15af8a Compare September 25, 2026 21:05
realmarcin and others added 7 commits September 25, 2026 14:44
The nightly check failed on every card a fast Mech had outgrown since the
refresh's pins, and passed on a source that returned 404, lost its figure, or
could not be reached at all. It now warns on a site up to 10% ahead of its
card and fails past that, on a site behind its card, on a 4xx, on a source with
no readable figure, on an unpaired card or SOURCES entry, and when more than
half the sources are unread (#148, #113, #115, #176). CultureMech's figure is
read only from the README block its generator writes (#176).

The cards are now parsed once, card by card, in card_markup.py, which the
assembler, the check and the tests share, so a card without a figure is
missing rather than reading its neighbour's (#114).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A 10% allowance went red a day after every refresh, because CellStructureMech
adds about 9% a day, and it let a card mistyped inside 10% pass (#217). Growth
now warns for 14 days after the pins in site_audit.json while the site is at
most half as large again as the card, and a card that differs from a source
still byte-identical to its pin fails as WRONG.

Throttles (408, 425, 429) and a connection dropped mid-body are unread, not GONE
or a traceback (#219, #220). A card without exactly one headline figure, or a
figure outside every card, is a MARKUP failure here and an error in the
assembler (#218). The update skill derives audit figures through read_source(),
which applies REGIONS (#221), and the tests cover the parse-exception path, the
more-than-half boundary and a half-marked region (#222).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
WRONG compared the live source's hash with its pinned copy, so it stopped
working the moment a source changed at all, and five of ten already had (#231).
site_audit.json now records figure_at_pin, read from each source's committed
copy at the pin with check_cards.figure(), the nightly's own parser; a test
requires it to equal the card, so a typo fails on the PR, and the nightly
reports WRONG from it however far the site has moved.

A pin time with no offset is UTC and an unreadable one is an AUDIT failure, not
a traceback (#232). The docstring states the lead limit the code applies
(#233), the tests cover the assembler's stray-figure refusal, more 4xx codes and
main()'s exit for WRONG, STALE and UNCHECKED (#234), and the README names every
failing verdict's remedy (#235).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…remedy

A mistyped card fails the provenance tests first, and the card check used to be
skipped with them, so its WRONG line and every other card's verdict never
printed; the step now runs unless the job was cancelled (#239). WRONG's remedy
is to correct the card or the audit, not a full refresh (#240). The README's
remedy list no longer swallows the next paragraph (#241). A pin time in the
future is an AUDIT failure, and the provenance test bounds it by the newest
pinned commit and the check (#242). The tests pin the 14-day grace, a missing
pin time, AUDIT on its own, "no offset means UTC" under a non-UTC zone, and the
400/499/500 edges (#243).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ve a bad audit

CultureMech's CHANGED detail said its missing generated block was "found"
(#247). WRONG's remedy names every copy of a card figure and says to rerun the
assembler, and a new test holds each MECHS records: to its card (#248). A
missing or malformed site_audit.json is an AUDIT row instead of a traceback, so
the rest of the report prints (#250). The tests assert the remedy text, AUDIT on
its own, a pin a minute ahead and an overstated WRONG card (#249).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…he figure

An audit of null, {} or one without repositories got no AUDIT row: grown sites
read STALE and a mistyped card could pass (#257). WRONG's remedy now says to grep
the tree for the old figure, as the update skill's step 6 does, since it also
appears in the MECHS extra: prose and cross-references (#258). The tests cover
those audit shapes and the remedy's content (#259).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…at_pin

A second tile with a stray space escaped the one-figure rule, and "<b>,</b>"
parsed and then crashed int(""); the parser now counts every headline tile and
reads a figure only when it starts with a digit (#261). A second card for one
Mech is MARKUP, not a card the nightly silently skips (#262). A figure_at_pin
that is a string, float or missing is an AUDIT row, and the provenance test
requires an int (#260). The review skill names both growth limits (#263).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
"32,06" was read as 3206 and passed every gate while the page showed it
malformed; a figure is now plain digits or digits grouped in threes (#264).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@realmarcin
realmarcin merged commit d035915 into main Sep 25, 2026
1 check passed
@realmarcin
realmarcin deleted the fix/card-check-semantics branch September 25, 2026 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment