Skip to content

check_cards: WRONG fires only while a source is byte-identical to its pin, so a mistyped card on a moved source passes #231

Description

@realmarcin

From the round-2 review of #212 (follows #217).

Where: scripts/fleet/check_cards.py classify(), and the audit builder.

The gap: WRONG compared the live source's sha256 with the audit's *_sha256_at_pin. Any byte change to the source turns WRONG off, even one that leaves the figure alone. After that, a card up to 50% low reads as "grew" for 14 days.

  • Already exposed: NaturalProductMech and MediaIngredientMech changed during the refresh itself, so WRONG could never fire for them. CellStructureMech and TraitMech have moved since, and ProteinTraitsMech has no pin hash. That is five of the ten sources.
  • Reproduced: with each card lowered by 30% (for example NaturalProductMech typed 2,115 for 3,115), the check stays green until the grace period ends.

No gate catches it either:

  • the audit builder takes card_records from the template, and its growth guard accepts figure > card;
  • RefreshProvenanceTests checks only card_records == card;
  • no test exercises the json/data_sha256_at_pin branch, or checks the committed audit's hashes against SOURCES.

Fix: record figure_at_pin in site_audit.json, read by the check's own parser from each source's committed copy at the pin.

  • A unit test requires it to equal the card, so a typo fails at PR time.
  • The nightly reports WRONG whenever the card differs from it, however the site has moved since.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions