From the round-2 review of #212 (follows #217).
Where: scripts/fleet/check_cards.py classify(), and the audit builder.
The gap: WRONG compared the live source's sha256 with the audit's *_sha256_at_pin. Any byte change to the source turns WRONG off, even one that leaves the figure alone. After that, a card up to 50% low reads as "grew" for 14 days.
- Already exposed: NaturalProductMech and MediaIngredientMech changed during the refresh itself, so WRONG could never fire for them. CellStructureMech and TraitMech have moved since, and ProteinTraitsMech has no pin hash. That is five of the ten sources.
- Reproduced: with each card lowered by 30% (for example NaturalProductMech typed 2,115 for 3,115), the check stays green until the grace period ends.
No gate catches it either:
- the audit builder takes
card_records from the template, and its growth guard accepts figure > card;
RefreshProvenanceTests checks only card_records == card;
- no test exercises the json/
data_sha256_at_pin branch, or checks the committed audit's hashes against SOURCES.
Fix: record figure_at_pin in site_audit.json, read by the check's own parser from each source's committed copy at the pin.
- A unit test requires it to equal the card, so a typo fails at PR time.
- The nightly reports WRONG whenever the card differs from it, however the site has moved since.
From the round-2 review of #212 (follows #217).
Where:
scripts/fleet/check_cards.pyclassify(), and the audit builder.The gap: WRONG compared the live source's sha256 with the audit's
*_sha256_at_pin. Any byte change to the source turns WRONG off, even one that leaves the figure alone. After that, a card up to 50% low reads as "grew" for 14 days.No gate catches it either:
card_recordsfrom the template, and its growth guard accepts figure > card;RefreshProvenanceTestschecks onlycard_records == card;data_sha256_at_pinbranch, or checks the committed audit's hashes againstSOURCES.Fix: record
figure_at_pininsite_audit.json, read by the check's own parser from each source's committed copy at the pin.