From 33b6edb0f504530ecdbbd4d39c47e1dff6202814 Mon Sep 17 00:00:00 2001 From: "marcin p. joachimiak" <4625870+realmarcin@users.noreply.github.com> Date: Fri, 25 Sep 2026 15:01:12 -0700 Subject: [PATCH 1/5] Commit the site-audit builder, with the audited notes as its only hand input _fleet/data/site_audit.json was written by a script that existed only in a session scratchpad, so each refresh had to rewrite it from the skill's prose and nothing checked that the rewrite derived each field the same way (#238). scripts/fleet/build_site_audit.py now derives every mechanical field from the snapshot, mech_stats.json and check_cards.figure(), appends the sentence saying whether each live copy still matches its pin, and refuses a card that differs from its figure at the pin or exceeds its site. The audited notes and scope move to _fleet/audit_notes.json, the builder's only hand-written input. Run in memory against the #120 snapshot and the live sites, it reproduces the committed audit exactly, key order and notes included, except CellStructureMech, whose site has grown from 577 to 584 since the audit was written. Co-Authored-By: Claude Opus 5.5 (1M context) --- .claude/skills/update-xmech-page/SKILL.md | 49 +++--- _fleet/README.md | 10 +- _fleet/audit_notes.json | 16 ++ scripts/fleet/build_site_audit.py | 204 ++++++++++++++++++++++ tests/test_fleet_page.py | 89 +++++++++- 5 files changed, 340 insertions(+), 28 deletions(-) create mode 100644 _fleet/audit_notes.json create mode 100644 scripts/fleet/build_site_audit.py diff --git a/.claude/skills/update-xmech-page/SKILL.md b/.claude/skills/update-xmech-page/SKILL.md index 339f511..bc58a65 100644 --- a/.claude/skills/update-xmech-page/SKILL.md +++ b/.claude/skills/update-xmech-page/SKILL.md @@ -227,26 +227,29 @@ either being stale alone. ### 7. Provenance -Rewrite `_fleet/data/site_audit.json` for the run: per repository the pinned -`sha` from `$SNAP/revisions.json` (not from `mech_stats.json`, or the audit-pin -test compares a value with itself, #125) and its commit date, the URL each card figure is read from, the figure, the -sha256 of the fetched HTML and of any data file, merged PRs, and short notes on -how the site figure relates to the repo count. Set `checked_at_utc`, -`local_date`, `pinned_at_utc` (ISO, with its offset) and `scope`. Record -`figure_at_pin` for every source with a committed copy: read the copy at the pin -with `check_cards.figure()`, the nightly's own parser with `REGIONS` applied, -never from the template. The provenance tests require it to equal each card, so -a mistyped card fails on the PR, and the nightly reports WRONG from it (#231). -ProteinTraitsMech's data file is built in CI and has none. Derive the other -mechanical fields rather than typing them: the live figure through -`check_cards.read_source()`, merged PRs from -`mech_stats.json`, SHAs and commit dates from the pins, and assert that the -pins equal the stats' `source_revision` before writing. Hash the served page as -committed at the pin too (`git show :pages/index.html`, or `docs/`), record -it beside the live hash, and let the builder say whether the two match. Never -type "byte-identical" into a note: sites publish during the run, and a -hand-written claim of identity went stale for four Mechs in the first run (#155). The provenance tests require its SHAs to equal the -stats' `source_revision`. +Update the audited notes in `_fleet/audit_notes.json`, one per Mech plus CLAW +and the `scope`: how each site's figure relates to its records at the pin, as +re-checked in step 5. Then write the audit: + +```bash +python3 scripts/fleet/build_site_audit.py --snapshot "$SNAP" +``` + +The builder derives every mechanical field (#238): the pinned `sha` and commit +date from `$SNAP/revisions.json` (not from `mech_stats.json`, or the audit-pin +test compares a value with itself, #125), merged PRs from `mech_stats.json` +after checking it was counted at the pins, the live figure and `figure_at_pin` +through `check_cards.figure()` on the served file and on its committed copy at +the pin, and the sha256 of both. It appends one sentence to each note saying +whether the live copy still matches the pin, so never type "byte-identical" into +a note: sites publish during the run, and a hand-written claim went stale for +four Mechs in the first run (#155). It refuses to write when a card differs from +its figure at the pin (a typo, #231) or when a site states fewer than its card, +and it records a site that grew as `site_figure_at_check`. A source with no +committed copy must be listed in `check_cards.NO_PIN_COPY` (ProteinTraitsMech, +built in CI). The provenance tests then require the audit's SHAs to equal the +stats' `source_revision`, each `figure_at_pin` to equal its card, and each note +to start with its text in `_fleet/audit_notes.json`. `CLAUDE.md` at the repository root records when the page was last refreshed. Update its "Last refreshed" sentence in the same PR, so the guidance and the @@ -269,9 +272,9 @@ live site, so it is the only gate that sees that. When a site has moved past its pin, do not re-pin that one Mech: the census and overlaps are computed across Mechs, so a single re-pin is a partial rerun, and a fast Mech moves again before the rerun finishes. Keep the page a consistent -snapshot at the pins, record the live figure as `site_figure_at_check` in that -Mech's `site_audit.json` entry, and say in the PR which cards the check reports -as grown. That stays a warning for `GRACE_DAYS` (14) after `pinned_at_utc` while +snapshot at the pins (the builder records the live figure as +`site_figure_at_check` in that Mech's `site_audit.json` entry), and say in the PR +which cards the check reports as grown. That stays a warning for `GRACE_DAYS` (14) after `pinned_at_utc` while the site is at most half as large again as the card; past either limit the check reports STALE and fails, and the page is due a full refresh. A card that differs from its audit's `figure_at_pin` fails as WRONG however far the site has diff --git a/_fleet/README.md b/_fleet/README.md index 5a70da4..0815466 100644 --- a/_fleet/README.md +++ b/_fleet/README.md @@ -9,7 +9,7 @@ hand-curated at the top of the script. - `data/manifest.json` — membership and all capability declarations from a pinned commit of CLAW's canonical manifest, plus the canonical artifact count. -- Other `data/` files — derived numbers: `prefix_census.json`, `subsets_summary.json`, `fleet_data.json`, `mech_stats.json`; `site_audit.json` is the hand-written provenance record. +- Other `data/` files — derived numbers: `prefix_census.json`, `subsets_summary.json`, `fleet_data.json`, `mech_stats.json`; `site_audit.json` is the provenance record, written by `scripts/fleet/build_site_audit.py` from the snapshot, the live sites and the audited notes in `_fleet/audit_notes.json`. ## Membership and capability updates @@ -195,8 +195,12 @@ procedure. `data/site_audit.json` records, per repository, the pinned revision, the URL each card figure is read from (a Pages URL, except CultureMech's committed README on -`main`), the figure, response hashes and merged -pull-request totals. The three dedicated pages link their descriptions and +`main`), the figure, the figure its source stated at the pin (`figure_at_pin`), +response hashes and merged pull-request totals. `scripts/fleet/build_site_audit.py +--snapshot "$SNAP"` writes it: every mechanical field is derived, and only the +notes on how each site's figure relates to its records, kept in +`_fleet/audit_notes.json`, are written by hand. It refuses a card that differs +from its figure at the pin or exceeds its site (#238). The three dedicated pages link their descriptions and commands to those same revisions. Follow client-side meta refreshes from site roots to `pages/` or `app/`. Read diff --git a/_fleet/audit_notes.json b/_fleet/audit_notes.json new file mode 100644 index 0000000..612ebe2 --- /dev/null +++ b/_fleet/audit_notes.json @@ -0,0 +1,16 @@ +{ + "scope": "Public main revisions pinned once for the whole refresh, and the deployed GitHub Pages responses, except CultureMech, whose figure is read from its committed README on main because no page it reliably serves states the count; the vocabulary census, subsets and card stats were regenerated at the same pins.", + "notes": { + "AntibioticMech": "2,939 data/antibiotics records at the pin (2,669 ontology-grounded, 270 minted; 80 REVIEWED, 30 PROPOSED, 2,829 SEEDED), equal to the stat tile.", + "CellStructureMech": "542 data/structures records at the pin, equal to the stat tile (13 categories, 475 GO-grounded, 196 membrane organelles). All 542 are PROPOSED, so none is reviewed.", + "CommunityMech": "The landing tile counts 422 kb/communities records, equal to the pin. data/isolates adds four more, so the census and mech_stats.json count 426. The landing's '16 categories' tile is hard-coded upstream; the schema enum, the records and the browser facet all have 15. No GitHub release has been published.", + "CultureMech": "The committed README's corpus snapshot states 15,878 normalized and 6,288 merged records, equal to data/merge_yaml/merged at the pin; no page CultureMech reliably serves states the canonical count. The app/ landing tile still reads 10,657, a legacy figure that matches no data layer. The pages/ media index is deployed by CultureMech's generate-pages workflow and replaced by the branch-based Pages build on other pushes to main, so it was live on 2026-09-24 and gone on 2026-09-25 (#175).", + "HabitatMech": "3,206 data/habitats records at the pin (686 REVIEWED, 2,520 SEEDED), equal to the stat tile. The 263 commits since the previous audit add only record-review reports.", + "MediaIngredientMech": "The landing tiles are computed from data/ingredients.json: 2,953 ingredients, 2,611 MAPPED (88%), 261 UNMAPPED, 80 REJECTED, 1 AMBIGUOUS. browser.html reads directory counts instead (2,680 under mapped/).", + "NaturalProductMech": "The landing prose states 3,115 natural product structures, equal to data/natural_products at the pin; all are SEEDED.", + "ProteinTraitsMech": "429,293 records and 34 source labels, computed at runtime from data/facets.json and equal to data/traits at the pin. The static HTML still carries a legacy fallback of 408,978 / 29.", + "TaxonMech": "625,960 data/taxa records at the pin, equal to the stat tile and including 14 retained DEPRECATED records; none is REVIEWED.", + "TraitMech": "763 data/traits records at the pin (427 REVIEWED, 248 PROPOSED, 50 DEPRECATED, 38 SEEDED; 519 with causal graphs), equal to the landing page, which shows 62.5% embedded (477 of 763).", + "culturebotai-claw": "10 fleet members and 16 governed artifacts. Projected membership and capabilities are unchanged from the previous snapshot; provenance advanced to the pinned main revision. Every Mech vendors claw at 44db08d, and fleet.yaml, README.md and vendored_artifacts.json are identical between 44db08d and this pin." + } +} diff --git a/scripts/fleet/build_site_audit.py b/scripts/fleet/build_site_audit.py new file mode 100644 index 0000000..57242ec --- /dev/null +++ b/scripts/fleet/build_site_audit.py @@ -0,0 +1,204 @@ +"""Write _fleet/data/site_audit.json for a refresh (update-xmech-page, step 7). + +Run from the site root after the pipeline, with the snapshot the refresh built: + + python3 scripts/fleet/build_site_audit.py --snapshot "$SNAP" + +Every mechanical field is derived here rather than typed: the pins and commit +dates from $SNAP/revisions.json, merged PRs from mech_stats.json, each source's +live figure and figure_at_pin through check_cards.figure() (the nightly's own +parser, REGIONS applied), and the sha256 of the served file and of its committed +copy at the pin. What only a person can check, the notes on how each site's +figure relates to its records and the audit's scope, comes from +_fleet/audit_notes.json. The builder adds one sentence per source saying whether +the live copy still matches the pin, so no note ever claims that by hand (#155). + +It refuses to write when a card differs from the figure its source stated at the +pin, or when a site states fewer than its card: either is a wrong card, not a +site that grew (#231). Until #238 this script lived only in a session +scratchpad, and each refresh rewrote it from the skill's prose. +""" +from __future__ import annotations + +import argparse +import datetime +import hashlib +import json +import subprocess +import sys +import urllib.request +from pathlib import Path + +import check_cards +from card_markup import card_figures + +REPO = Path(__file__).resolve().parents[2] +AUDIT = REPO / "_fleet/data/site_audit.json" +NOTES = REPO / "_fleet/audit_notes.json" +STATS = REPO / "_fleet/data/mech_stats.json" +CLAW = "culturebotai-claw" + + +def fetch_bytes(url: str) -> bytes: + request = urllib.request.Request(url, headers={"User-Agent": "culturebotai-site-audit"}) + return urllib.request.urlopen(request, timeout=60).read() + + +def committed_candidates(path: str) -> list[str]: + """Where a SOURCES path's file sits in its repository, most likely first. + + Pages sites here publish from the repository root, pages/ or docs/, and + CultureMech's source is a file read straight from the repository. + """ + if path.startswith("https://raw.githubusercontent.com/"): + return [path.split("/main/", 1)[1]] + rel = path.split("/", 1)[1] if "/" in path else "" + if rel == "" or rel.endswith("/"): + rel += "index.html" + return [rel, "docs/" + rel] + + +def committed_copy(show, path: str) -> tuple[str | None, bytes | None]: + """The first candidate that show(candidate) finds, as (path, bytes).""" + for candidate in committed_candidates(path): + body = show(candidate) + if body is not None: + return candidate, body + return None, None + + +def git_show(clone: Path, sha: str): + """show(candidate) for one Mech's snapshot clone at its pin.""" + def show(candidate: str) -> bytes | None: + done = subprocess.run(["git", "-C", str(clone), "show", f"{sha}:{candidate}"], capture_output=True) + return done.stdout if done.returncode == 0 else None + return show + + +def match_sentence(key: str, live: bytes, path: str | None, pinned: bytes | None) -> str: + """Whether the live copy still matches the pin, in the audit's words.""" + if pinned is None: + return " The served file is built in CI rather than committed, so there is no copy at the pin to compare." + name = path.rsplit("/", 1)[-1] + if pinned == live: + return f" The live {name} is byte-identical to {path} at the pin." + return (f" The live {name} had moved past the pin when this audit was written; " + f"{key} is the live copy and {key.replace('sha256', 'sha256_at_pin')} is {path} at the pin.") + + +def read_figure(mech: str, kind: str, body: bytes, selector: str, what: str) -> int: + value = check_cards.figure(mech, kind, body.decode("utf-8", "replace"), selector) + if not isinstance(value, int) or isinstance(value, bool): + raise SystemExit(f"{mech}: no figure could be read from {what}") + return value + + +def build_entry(mech: str, source: tuple[str, str, str], pin: dict, stats: dict, card: int, + note: str, fetch, show) -> dict: + """One repository's audit entry. fetch(url) and show(path) supply the bytes.""" + kind, path, selector = source + url = check_cards.source_url(path) + if stats["source_revision"] != pin["sha"]: + raise SystemExit(f"{mech}: mech_stats.json was not counted at the pin") + body = fetch(url) + live = read_figure(mech, kind, body, selector, url) + pinned_path, pinned = committed_copy(show, path) + entry = { + "repo": stats["repo"], + "sha": pin["sha"], + "commit_date": pin["commit_date"], + "readme_url": f"https://github.com/CultureBotAI/{stats['repo']}/blob/{pin['sha']}/README.md", + "card_records": card, + } + if pinned is not None: + at_pin = read_figure(mech, kind, pinned, selector, f"{pinned_path} at the pin") + if at_pin != card: + raise SystemExit(f"{mech}: the card says {card:,} but {pinned_path} stated {at_pin:,} at the pin") + entry["figure_at_pin"] = at_pin + elif mech not in check_cards.NO_PIN_COPY: + raise SystemExit(f"{mech}: no committed copy of {path} at the pin; if it is built in CI, " + "add it to check_cards.NO_PIN_COPY") + if live < card: + raise SystemExit(f"{mech}: the site says {live:,}, below the card's {card:,}; that is not growth") + entry["merged_prs"] = stats["merged_prs"] + if kind == "json": + # The page's markup carries a placeholder; hash both it and the data. + page = url.rsplit("/data/", 1)[0] + "/" + entry["site"] = page + entry["site_html_sha256"] = hashlib.sha256(fetch(page)).hexdigest() + entry["data_url"] = url + entry["data_sha256"] = hashlib.sha256(body).hexdigest() + key = "data_sha256" + else: + entry["site"] = url + entry["site_html_sha256"] = hashlib.sha256(body).hexdigest() + key = "site_html_sha256" + if pinned is not None: + entry[key.replace("sha256", "sha256_at_pin")] = hashlib.sha256(pinned).hexdigest() + entry["notes"] = note + match_sentence(key, body, pinned_path, pinned) + if live > card: + # The corpus grew between the pin and this fetch. The page stays a + # snapshot at the pins; the audit says what the site showed instead. + entry["site_figure_at_check"] = live + entry["notes"] += (f" The site moved past the pin during the refresh and showed {live:,} when this" + f" audit was written; the card keeps the pinned figure, {card:,}.") + return entry + + +def utc(iso: str) -> str: + moment = datetime.datetime.fromisoformat(iso.replace("Z", "+00:00")) + return moment.astimezone(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") + + +def build(pins: dict, notes: dict, template: str, stats: dict, fetch, shower, claw_date: str, + now: datetime.datetime) -> dict: + """The whole audit. shower(mech, sha) returns that Mech's show(path).""" + cards = card_figures(template) + by_mech = {m["mech"]: m for m in stats["mechs"]} + missing = sorted(set(check_cards.SOURCES) - set(notes["notes"])) + if missing or CLAW not in notes["notes"]: + raise SystemExit(f"_fleet/audit_notes.json has no notes for: {', '.join(missing + [CLAW])}") + repositories = [] + for mech, source in sorted(check_cards.SOURCES.items()): + pin = dict(pins["mechs"][mech], commit_date=utc(pins["mechs"][mech]["commit_date"])) + repositories.append(build_entry(mech, source, pin, by_mech[mech], cards[mech], + notes["notes"][mech], fetch, shower(mech, pin["sha"]))) + claw = pins["claw"] + repositories.append({ + "repo": CLAW, + "sha": claw, + "commit_date": claw_date, + "readme_url": f"https://github.com/CultureBotAI/{CLAW}/blob/{claw}/README.md", + "notes": notes["notes"][CLAW], + }) + repositories.sort(key=lambda r: r["repo"].lower()) + return { + "checked_at_utc": now.astimezone(datetime.timezone.utc).isoformat(timespec="seconds"), + # The local date of that same moment, not a typed one (#178). + "local_date": now.astimezone().date().isoformat(), + "pinned_at_utc": pins["pinned_at_utc"], + "scope": notes["scope"], + "repositories": repositories, + } + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + parser.add_argument("--snapshot", required=True, help="the refresh's $SNAP") + parser.add_argument("--notes", default=str(NOTES)) + args = parser.parse_args() + snap = Path(args.snapshot) + pins = json.loads((snap / "revisions.json").read_text()) + claw_date = subprocess.check_output( + ["gh", "api", f"repos/CultureBotAI/{CLAW}/commits/{pins['claw']}", "--jq", ".commit.committer.date"], + text=True).strip() + audit = build(pins, json.loads(Path(args.notes).read_text()), (REPO / "_fleet/mechs_template.md").read_text(), + json.loads(STATS.read_text()), fetch_bytes, lambda mech, sha: git_show(snap / "mechs" / mech, sha), + claw_date, datetime.datetime.now(datetime.timezone.utc)) + AUDIT.write_text(json.dumps(audit, indent=1, ensure_ascii=False) + "\n") + print(f"Wrote {AUDIT.relative_to(REPO)} ({len(audit['repositories'])} repositories)") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/test_fleet_page.py b/tests/test_fleet_page.py index d64b102..9c25a7b 100644 --- a/tests/test_fleet_page.py +++ b/tests/test_fleet_page.py @@ -792,6 +792,91 @@ def test_main_exits_by_the_same_rule(self): self.assertEqual(self.check_cards.main(), 1) +class SiteAuditBuilderTests(unittest.TestCase): + """The committed audit builder derives every mechanical field (#238).""" + + PAGE = b'
3,206habitat records
\n' + + def setUp(self): + import build_site_audit + self.b = build_site_audit + self.stats = {"repo": "HabitatMech", "source_revision": "a" * 40, "merged_prs": 7} + self.pin = {"sha": "a" * 40, "commit_date": "2026-09-25T01:40:32Z"} + self.source = ("html", "HabitatMech/pages/index.html", "habitat records") + + def entry(self, live=PAGE, pinned=PAGE, card=3206, mech="HabitatMech", source=None): + return self.b.build_entry(mech, source or self.source, self.pin, self.stats, card, "Audited.", + lambda url: live, lambda path: pinned if path == "pages/index.html" else None) + + def test_committed_copies_are_looked_for_where_pages_sites_publish(self): + candidates = self.b.committed_candidates + self.assertEqual(candidates("HabitatMech/pages/index.html"), ["pages/index.html", "docs/pages/index.html"]) + self.assertEqual(candidates("CommunityMech/"), ["index.html", "docs/index.html"]) + self.assertEqual(candidates("MediaIngredientMech/data/ingredients.json"), + ["data/ingredients.json", "docs/data/ingredients.json"]) + self.assertEqual(candidates("https://raw.githubusercontent.com/CultureBotAI/CultureMech/main/README.md"), + ["README.md"]) + + def test_an_unchanged_source_records_its_figure_at_the_pin(self): + entry = self.entry() + self.assertEqual(entry["figure_at_pin"], 3206) + self.assertEqual(entry["site_html_sha256"], entry["site_html_sha256_at_pin"]) + self.assertTrue(entry["notes"].endswith("The live index.html is byte-identical to pages/index.html at the pin.")) + self.assertNotIn("site_figure_at_check", entry) + + def test_a_site_that_grew_keeps_the_pinned_card_and_says_so(self): + entry = self.entry(live=self.PAGE.replace(b"3,206", b"3,300")) + self.assertEqual((entry["figure_at_pin"], entry["site_figure_at_check"]), (3206, 3300)) + self.assertIn("had moved past the pin", entry["notes"]) + self.assertIn("showed 3,300", entry["notes"]) + + def test_a_card_that_differs_from_its_pin_or_exceeds_its_site_is_refused(self): + # #231: the old builder read the card from the template and passed a typo. + with self.assertRaisesRegex(SystemExit, "stated 3,206 at the pin"): + self.entry(card=3026, live=self.PAGE.replace(b"3,206", b"3,300")) + with self.assertRaisesRegex(SystemExit, "not growth"): + self.entry(live=self.PAGE.replace(b"3,206", b"3,100")) + with self.assertRaisesRegex(SystemExit, "no figure"): + self.entry(live=b"
nothing
") + + def test_a_source_with_no_committed_copy_must_be_declared(self): + with self.assertRaisesRegex(SystemExit, "NO_PIN_COPY"): + self.entry(pinned=None) + from unittest import mock + with mock.patch.object(self.b.check_cards, "NO_PIN_COPY", ("HabitatMech",)): + entry = self.entry(pinned=None) + self.assertNotIn("figure_at_pin", entry) + self.assertIn("built in CI", entry["notes"]) + + def test_a_count_from_another_revision_is_refused(self): + self.stats["source_revision"] = "b" * 40 + with self.assertRaisesRegex(SystemExit, "not counted at the pin"): + self.entry() + + def test_a_data_source_hashes_both_the_page_and_the_data(self): + body = b'{"ingredients": [1, 2, 3]}' + source = ("json", "MediaIngredientMech/data/ingredients.json", "ingredients") + entry = self.b.build_entry("MediaIngredientMech", source, self.pin, self.stats, 3, "Audited.", + lambda url: body if url.endswith(".json") else b"", + lambda path: body if path == "docs/data/ingredients.json" else None) + self.assertEqual(entry["site"], "https://culturebotai.github.io/MediaIngredientMech/") + self.assertEqual(entry["data_url"], "https://culturebotai.github.io/MediaIngredientMech/data/ingredients.json") + self.assertEqual(entry["data_sha256"], entry["data_sha256_at_pin"]) + self.assertEqual(entry["figure_at_pin"], 3) + + def test_the_committed_audit_notes_come_from_the_notes_file(self): + # The builder appends one sentence per source; everything before it is + # the audited text in _fleet/audit_notes.json, which must stay in step. + notes = json.loads((ROOT / "_fleet/audit_notes.json").read_text()) + audit = json.loads((ROOT / "_fleet/data/site_audit.json").read_text()) + self.assertEqual(audit["scope"], notes["scope"]) + by_repo = {r["repo"].lower(): r for r in audit["repositories"]} + self.assertEqual(set(by_repo), {k.lower() for k in notes["notes"]}) + for name, text in notes["notes"].items(): + with self.subTest(name=name): + self.assertTrue(by_repo[name.lower()]["notes"].startswith(text)) + + class RefreshProvenanceTests(unittest.TestCase): """The derived numbers must all come from one set of checkouts (#85). @@ -844,8 +929,8 @@ def test_every_graph_panel_states_its_card_figure(self): self.assertEqual(panels, card_figures((ROOT / "_fleet/mechs_template.md").read_text())) def test_every_card_equals_the_figure_its_source_stated_at_the_pin(self): - # #231: the audit builder read card_records from the template, so the - # audit could only repeat a mistyped card. figure_at_pin is read from the + # #231: the old scratchpad audit builder read card_records from the + # template, so the audit could only repeat a mistyped card. figure_at_pin is read from the # source's committed copy at the pin with check_cards.figure(), and the # nightly's WRONG verdict depends on it being there. import check_cards From 0e00ce20844e3b1c664359d43e3250b2a0494e6a Mon Sep 17 00:00:00 2001 From: "marcin p. joachimiak" <4625870+realmarcin@users.noreply.github.com> Date: Fri, 25 Sep 2026 15:12:28 -0700 Subject: [PATCH 2/5] Test build() itself, name only missing notes, and refuse a bad pin time build() had no test, so its notes refusal, UTC commit dates, CLAW entry and order could regress unnoticed; an offline test now covers them (#266). The refusal names only the notes actually missing and the file read (#267). The WRONG remedy says to re-derive figure_at_pin with the builder, never edit it (#268). Step 1 gives the revisions.json layout and pin-time format the builder reads, and the builder refuses a pin time the nightly would report as AUDIT (#269). Co-Authored-By: Claude Opus 5.5 (1M context) --- .claude/skills/update-xmech-page/SKILL.md | 14 +++++- _fleet/README.md | 6 ++- scripts/fleet/build_site_audit.py | 17 +++++-- scripts/fleet/check_cards.py | 5 +- tests/test_fleet_page.py | 56 +++++++++++++++++++++++ 5 files changed, 87 insertions(+), 11 deletions(-) diff --git a/.claude/skills/update-xmech-page/SKILL.md b/.claude/skills/update-xmech-page/SKILL.md index bc58a65..53ab433 100644 --- a/.claude/skills/update-xmech-page/SKILL.md +++ b/.claude/skills/update-xmech-page/SKILL.md @@ -100,8 +100,18 @@ Then, for each Mech in the refreshed `_fleet/data/manifest.json`: `git -C "$SRC/" fetch -q origin`, take `origin/main`, and cross-check it against `gh api repos/CultureBotAI//commits/main --jq .sha`. The repository name is not always the Mech name (`proteintraitsmech`). Write the -pins and CLAW's to `$SNAP/revisions.json` with the pin time. Corpora move within -minutes, so pin once and do not re-pin mid-run. +pins and CLAW's to `$SNAP/revisions.json` with the pin time, in the layout +`build_site_audit.py` reads in step 7 (#269): + +```json +{"pinned_at_utc": "2026-09-25T02:06:03+00:00", + "mechs": {"": {"repo": "", "sha": "<40-hex sha>", "commit_date": ""}}, + "claw": "<40-hex sha>"} +``` + +`pinned_at_utc` is an ISO time with its offset; the nightly card check reads it +from the audit and fails on one it cannot read. Corpora move within minutes, so +pin once and do not re-pin mid-run. ### 2. Snapshot at the pins diff --git a/_fleet/README.md b/_fleet/README.md index 0815466..3557ad9 100644 --- a/_fleet/README.md +++ b/_fleet/README.md @@ -63,8 +63,10 @@ remedy for each failing verdict (#235): and every other occurrence of its figure, found by grepping the tree for it as step 6 of the update skill does: the MECHS `records:` and `extra:` text in `fleet_fragment.html`, cross-references, `card_records` in `site_audit.json` - and the pages that repeat it. Or correct `figure_at_pin` if that is what was - wrong. Then rerun `assemble_page.py`; no re-pin (#248, #258). The unit tests catch this on the + and the pages that repeat it. Then rerun `assemble_page.py`; no re-pin (#248, + #258). If `figure_at_pin` itself is wrong, re-derive it with + `build_site_audit.py` against a snapshot at the audit's pins (update skill + step 11); it is derived, never typed (#268). The unit tests catch this on the PR and in the nightly, which still runs the card check after a failed test step so its report prints (#239, #240). - GONE or CHANGED: a `SOURCES` entry needs repointing. diff --git a/scripts/fleet/build_site_audit.py b/scripts/fleet/build_site_audit.py index 57242ec..700a576 100644 --- a/scripts/fleet/build_site_audit.py +++ b/scripts/fleet/build_site_audit.py @@ -151,13 +151,20 @@ def utc(iso: str) -> str: def build(pins: dict, notes: dict, template: str, stats: dict, fetch, shower, claw_date: str, - now: datetime.datetime) -> dict: + now: datetime.datetime, notes_path: str = "_fleet/audit_notes.json") -> dict: """The whole audit. shower(mech, sha) returns that Mech's show(path).""" cards = card_figures(template) by_mech = {m["mech"]: m for m in stats["mechs"]} - missing = sorted(set(check_cards.SOURCES) - set(notes["notes"])) - if missing or CLAW not in notes["notes"]: - raise SystemExit(f"_fleet/audit_notes.json has no notes for: {', '.join(missing + [CLAW])}") + missing = sorted((set(check_cards.SOURCES) | {CLAW}) - set(notes["notes"])) + if missing: + raise SystemExit(f"{notes_path} has no notes for: {', '.join(missing)}") # #267 + # The nightly reads this pin time; refuse one it would report as AUDIT (#269). + try: + pinned = check_cards.pin_time(pins) + except ValueError as error: + raise SystemExit(f"revisions.json: {error}") + if pinned > now: + raise SystemExit(f"revisions.json: pinned_at_utc {pins['pinned_at_utc']} is in the future") repositories = [] for mech, source in sorted(check_cards.SOURCES.items()): pin = dict(pins["mechs"][mech], commit_date=utc(pins["mechs"][mech]["commit_date"])) @@ -194,7 +201,7 @@ def main() -> int: text=True).strip() audit = build(pins, json.loads(Path(args.notes).read_text()), (REPO / "_fleet/mechs_template.md").read_text(), json.loads(STATS.read_text()), fetch_bytes, lambda mech, sha: git_show(snap / "mechs" / mech, sha), - claw_date, datetime.datetime.now(datetime.timezone.utc)) + claw_date, datetime.datetime.now(datetime.timezone.utc), args.notes) AUDIT.write_text(json.dumps(audit, indent=1, ensure_ascii=False) + "\n") print(f"Wrote {AUDIT.relative_to(REPO)} ({len(audit['repositories'])} repositories)") return 0 diff --git a/scripts/fleet/check_cards.py b/scripts/fleet/check_cards.py index b4439f3..d455984 100644 --- a/scripts/fleet/check_cards.py +++ b/scripts/fleet/check_cards.py @@ -366,8 +366,9 @@ def main() -> int: "assemble_page.py. WRONG: correct the card and every other occurrence of its " "figure, found by grepping the tree for it as update-xmech-page step 6 does (the MECHS " "records: and extra: text in _fleet/fleet_fragment.html, cross-references, card_records " - "in site_audit.json, the pages that repeat it), or the audit's figure_at_pin if that is " - "what was mistyped, then rerun assemble_page.py; no re-pin. GONE or CHANGED: repoint that Mech's SOURCES entry. MARKUP or " + "in site_audit.json, the pages that repeat it), then rerun assemble_page.py; no re-pin. " + "If figure_at_pin itself is wrong, re-derive it with build_site_audit.py against a " + "snapshot at the audit's pins (update-xmech-page step 11); never edit it by hand. GONE or CHANGED: repoint that Mech's SOURCES entry. MARKUP or " "UNCARDED: fix the card or its SOURCES entry. AUDIT: fix site_audit.json. " "UNCHECKED: the run could not reach most sites; rerun before changing anything.") return 1 diff --git a/tests/test_fleet_page.py b/tests/test_fleet_page.py index 9c25a7b..d703e41 100644 --- a/tests/test_fleet_page.py +++ b/tests/test_fleet_page.py @@ -864,6 +864,62 @@ def test_a_data_source_hashes_both_the_page_and_the_data(self): self.assertEqual(entry["data_sha256"], entry["data_sha256_at_pin"]) self.assertEqual(entry["figure_at_pin"], 3) + def whole_build(self, notes=None, pinned_at="2026-09-25T02:06:03+00:00", now=None): + """build() over two fake Mechs, one of them with no committed copy.""" + from unittest import mock + sources = {"HabitatMech": ("html", "HabitatMech/pages/index.html", "habitat records"), + "ProteinTraitsMech": ("json", "proteintraitsmech/data/facets.json", "total")} + pins = {"pinned_at_utc": pinned_at, "claw": "c" * 40, "mechs": { + "HabitatMech": {"repo": "HabitatMech", "sha": "a" * 40, "commit_date": "2026-09-25T03:40:32+02:00"}, + "ProteinTraitsMech": {"repo": "proteintraitsmech", "sha": "b" * 40, "commit_date": "2026-09-25T01:36:06Z"}}} + stats = {"mechs": [ + {"mech": "HabitatMech", "repo": "HabitatMech", "source_revision": "a" * 40, "merged_prs": 7}, + {"mech": "ProteinTraitsMech", "repo": "proteintraitsmech", "source_revision": "b" * 40, "merged_prs": 9}]} + template = ('
3,206
' + '
5
') + notes = notes or {"scope": "Scope.", "notes": {"HabitatMech": "H.", "ProteinTraitsMech": "P.", + "culturebotai-claw": "C."}} + def fetch(url): + return b'{"total": 5}' if url.endswith(".json") else self.PAGE + def shower(mech, sha): + return lambda path: self.PAGE if (mech, path) == ("HabitatMech", "pages/index.html") else None + now = now or datetime.datetime(2026, 9, 25, 19, 0, tzinfo=datetime.timezone.utc) + with mock.patch.dict(self.b.check_cards.SOURCES, sources, clear=True): + return self.b.build(pins, notes, template, stats, fetch, shower, "2026-09-22T07:18:11Z", now, + "notes.json") + + def test_the_whole_audit_is_derived_in_order(self): + # #266: build() itself, not just build_entry(). + audit = self.whole_build() + self.assertEqual(list(audit), ["checked_at_utc", "local_date", "pinned_at_utc", "scope", "repositories"]) + self.assertEqual(audit["checked_at_utc"], "2026-09-25T19:00:00+00:00") + self.assertEqual((audit["pinned_at_utc"], audit["scope"]), ("2026-09-25T02:06:03+00:00", "Scope.")) + repos = audit["repositories"] + self.assertEqual([r["repo"] for r in repos], ["culturebotai-claw", "HabitatMech", "proteintraitsmech"]) + claw, habitat, proteins = repos + self.assertEqual(claw, {"repo": "culturebotai-claw", "sha": "c" * 40, "commit_date": "2026-09-22T07:18:11Z", + "readme_url": f"https://github.com/CultureBotAI/culturebotai-claw/blob/{'c' * 40}/README.md", + "notes": "C."}) + self.assertEqual(habitat["commit_date"], "2026-09-25T01:40:32Z") # +02:00 read as UTC + self.assertEqual(habitat["figure_at_pin"], 3206) + self.assertNotIn("figure_at_pin", proteins) + self.assertIn("built in CI", proteins["notes"]) + + def test_missing_notes_or_a_bad_pin_time_are_refused(self): + # #267: name only what is missing, and the file that was read. + notes = {"scope": "S.", "notes": {"ProteinTraitsMech": "P.", "culturebotai-claw": "C."}} + with self.assertRaises(SystemExit) as caught: + self.whole_build(notes=notes) + self.assertEqual(str(caught.exception), "notes.json has no notes for: HabitatMech") + notes = {"scope": "S.", "notes": {"HabitatMech": "H.", "ProteinTraitsMech": "P."}} + with self.assertRaisesRegex(SystemExit, "no notes for: culturebotai-claw$"): + self.whole_build(notes=notes) + # #269: a pin time the nightly would report as AUDIT is refused here. + with self.assertRaisesRegex(SystemExit, "revisions.json"): + self.whole_build(pinned_at="25 September 2026") + with self.assertRaisesRegex(SystemExit, "in the future"): + self.whole_build(pinned_at="2026-09-26T02:06:03+00:00") + def test_the_committed_audit_notes_come_from_the_notes_file(self): # The builder appends one sentence per source; everything before it is # the audited text in _fleet/audit_notes.json, which must stay in step. From 537aa059d8caa6f9659f1fccd55eaaad8cecff2d Mon Sep 17 00:00:00 2001 From: "marcin p. joachimiak" <4625870+realmarcin@users.noreply.github.com> Date: Fri, 25 Sep 2026 15:24:53 -0700 Subject: [PATCH 3/5] Send AUDIT to the builder, refuse offset-less commit dates, test every derived field AUDIT's remedy said to fix site_audit.json by hand, though figure_at_pin and the pin time are derived; it now says to regenerate the audit with the builder (#273). A commit date with no offset is refused instead of being read as local time, and step 1 says to use the committer date with its offset (#274). The tests now check local_date under a non-UTC zone, each hash against its own copy, a data source's page hash, readme_url and merged_prs (#275). Co-Authored-By: Claude Opus 5.5 (1M context) --- .claude/skills/update-xmech-page/SKILL.md | 3 +- _fleet/README.md | 7 +++-- scripts/fleet/build_site_audit.py | 4 +++ scripts/fleet/check_cards.py | 4 ++- tests/test_fleet_page.py | 36 ++++++++++++++++++++++- 5 files changed, 49 insertions(+), 5 deletions(-) diff --git a/.claude/skills/update-xmech-page/SKILL.md b/.claude/skills/update-xmech-page/SKILL.md index 53ab433..f1f16ea 100644 --- a/.claude/skills/update-xmech-page/SKILL.md +++ b/.claude/skills/update-xmech-page/SKILL.md @@ -105,7 +105,8 @@ pins and CLAW's to `$SNAP/revisions.json` with the pin time, in the layout ```json {"pinned_at_utc": "2026-09-25T02:06:03+00:00", - "mechs": {"": {"repo": "", "sha": "<40-hex sha>", "commit_date": ""}}, + "mechs": {"": {"repo": "", "sha": "<40-hex sha>", + "commit_date": ">"}}, "claw": "<40-hex sha>"} ``` diff --git a/_fleet/README.md b/_fleet/README.md index 3557ad9..8cb1954 100644 --- a/_fleet/README.md +++ b/_fleet/README.md @@ -59,7 +59,7 @@ remedy for each failing verdict (#235): - STALE or SHRANK: the card figures in `mechs_template.md` and the `MECHS` block in `fleet_fragment.html` need a full refresh, since the page is a snapshot. -- WRONG: a card, or the audit's `figure_at_pin`, was mistyped. Correct the card +- WRONG: a card differs from the figure its source stated at the pin. Correct the card and every other occurrence of its figure, found by grepping the tree for it as step 6 of the update skill does: the MECHS `records:` and `extra:` text in `fleet_fragment.html`, cross-references, `card_records` in `site_audit.json` @@ -70,7 +70,10 @@ remedy for each failing verdict (#235): PR and in the nightly, which still runs the card check after a failed test step so its report prints (#239, #240). - GONE or CHANGED: a `SOURCES` entry needs repointing. -- MARKUP, UNCARDED or AUDIT: fix the card markup, `SOURCES` or `site_audit.json`. +- MARKUP or UNCARDED: fix the card markup or `SOURCES`. +- AUDIT: regenerate `site_audit.json` with `build_site_audit.py` against a + snapshot at its pins (update skill steps 7 and 11); never edit it by hand, since + `figure_at_pin` and the pin time are derived (#273). - UNCHECKED: most sites could not be reached; rerun before changing anything. A new card needs a `SOURCES` entry; a test enforces that. The cards diff --git a/scripts/fleet/build_site_audit.py b/scripts/fleet/build_site_audit.py index 700a576..8cb6d4a 100644 --- a/scripts/fleet/build_site_audit.py +++ b/scripts/fleet/build_site_audit.py @@ -146,7 +146,11 @@ def build_entry(mech: str, source: tuple[str, str, str], pin: dict, stats: dict, def utc(iso: str) -> str: + """A commit date with its offset, as UTC. One with no offset is refused: read + as local time it would shift by the machine's zone without a word (#274).""" moment = datetime.datetime.fromisoformat(iso.replace("Z", "+00:00")) + if moment.tzinfo is None: + raise SystemExit(f"revisions.json: commit_date {iso} has no offset") return moment.astimezone(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") diff --git a/scripts/fleet/check_cards.py b/scripts/fleet/check_cards.py index d455984..58d281a 100644 --- a/scripts/fleet/check_cards.py +++ b/scripts/fleet/check_cards.py @@ -369,7 +369,9 @@ def main() -> int: "in site_audit.json, the pages that repeat it), then rerun assemble_page.py; no re-pin. " "If figure_at_pin itself is wrong, re-derive it with build_site_audit.py against a " "snapshot at the audit's pins (update-xmech-page step 11); never edit it by hand. GONE or CHANGED: repoint that Mech's SOURCES entry. MARKUP or " - "UNCARDED: fix the card or its SOURCES entry. AUDIT: fix site_audit.json. " + "UNCARDED: fix the card or its SOURCES entry. AUDIT: regenerate site_audit.json with " + "build_site_audit.py against a snapshot at its pins (update-xmech-page steps 7 and " + "11), never by hand: its figure_at_pin and pin time are derived. " "UNCHECKED: the run could not reach most sites; rerun before changing anything.") return 1 if any(status in ("grew", "unread") for status, _, _ in rows): diff --git a/tests/test_fleet_page.py b/tests/test_fleet_page.py index d703e41..a986d89 100644 --- a/tests/test_fleet_page.py +++ b/tests/test_fleet_page.py @@ -825,8 +825,15 @@ def test_an_unchanged_source_records_its_figure_at_the_pin(self): self.assertNotIn("site_figure_at_check", entry) def test_a_site_that_grew_keeps_the_pinned_card_and_says_so(self): - entry = self.entry(live=self.PAGE.replace(b"3,206", b"3,300")) + import hashlib + live = self.PAGE.replace(b"3,206", b"3,300") + entry = self.entry(live=live) self.assertEqual((entry["figure_at_pin"], entry["site_figure_at_check"]), (3206, 3300)) + # #275: each hash is of its own copy, and the other fields come from the pin. + self.assertEqual(entry["site_html_sha256"], hashlib.sha256(live).hexdigest()) + self.assertEqual(entry["site_html_sha256_at_pin"], hashlib.sha256(self.PAGE).hexdigest()) + self.assertEqual(entry["readme_url"], f"https://github.com/CultureBotAI/HabitatMech/blob/{'a' * 40}/README.md") + self.assertEqual(entry["merged_prs"], 7) self.assertIn("had moved past the pin", entry["notes"]) self.assertIn("showed 3,300", entry["notes"]) @@ -860,6 +867,9 @@ def test_a_data_source_hashes_both_the_page_and_the_data(self): lambda url: body if url.endswith(".json") else b"", lambda path: body if path == "docs/data/ingredients.json" else None) self.assertEqual(entry["site"], "https://culturebotai.github.io/MediaIngredientMech/") + import hashlib # #275: the page is hashed as the page, the data as the data + self.assertEqual(entry["site_html_sha256"], hashlib.sha256(b"").hexdigest()) + self.assertEqual(entry["data_sha256"], hashlib.sha256(body).hexdigest()) self.assertEqual(entry["data_url"], "https://culturebotai.github.io/MediaIngredientMech/data/ingredients.json") self.assertEqual(entry["data_sha256"], entry["data_sha256_at_pin"]) self.assertEqual(entry["figure_at_pin"], 3) @@ -905,6 +915,30 @@ def test_the_whole_audit_is_derived_in_order(self): self.assertNotIn("figure_at_pin", proteins) self.assertIn("built in CI", proteins["notes"]) + def test_local_date_is_the_machine_date_of_the_check(self): + # #275, #178: 03:00 UTC on the 26th is still the 25th in Los Angeles. + import time + saved = os.environ.get("TZ") + os.environ["TZ"] = "America/Los_Angeles" + time.tzset() + try: + audit = self.whole_build(now=datetime.datetime(2026, 9, 26, 3, 0, 5, 123456, + tzinfo=datetime.timezone.utc)) + finally: + if saved is None: + os.environ.pop("TZ", None) + else: + os.environ["TZ"] = saved + time.tzset() + self.assertEqual(audit["local_date"], "2026-09-25") + self.assertEqual(audit["checked_at_utc"], "2026-09-26T03:00:05+00:00") + + def test_a_commit_date_without_an_offset_is_refused(self): + # #274: read as local time it would shift by the machine's zone. + self.assertEqual(self.b.utc("2026-09-25T03:40:32+02:00"), "2026-09-25T01:40:32Z") + with self.assertRaisesRegex(SystemExit, "no offset"): + self.b.utc("2026-09-25T03:40:32") + def test_missing_notes_or_a_bad_pin_time_are_refused(self): # #267: name only what is missing, and the file that was read. notes = {"scope": "S.", "notes": {"ProteinTraitsMech": "P.", "culturebotai-claw": "C."}} From 16dbf6e2368cdf754393561c271da2aeca041a66 Mon Sep 17 00:00:00 2001 From: "marcin p. joachimiak" <4625870+realmarcin@users.noreply.github.com> Date: Fri, 25 Sep 2026 15:38:29 -0700 Subject: [PATCH 4/5] Split the AUDIT remedy by cause, so a bad pin time does not loop Rebuilding from the failing audit reproduces a bad pin time, and a missing audit or a Mech with no entry has no pins to rebuild from. The remedy now says where each comes from: rebuild for figure_at_pin, the last builder-written audit or the PR body for the pins, a full refresh for a new Mech (#276). Co-Authored-By: Claude Opus 5.5 (1M context) --- _fleet/README.md | 10 +++++++--- scripts/fleet/check_cards.py | 9 ++++++--- 2 files changed, 13 insertions(+), 6 deletions(-) diff --git a/_fleet/README.md b/_fleet/README.md index 8cb1954..fa111cf 100644 --- a/_fleet/README.md +++ b/_fleet/README.md @@ -71,9 +71,13 @@ remedy for each failing verdict (#235): step so its report prints (#239, #240). - GONE or CHANGED: a `SOURCES` entry needs repointing. - MARKUP or UNCARDED: fix the card markup or `SOURCES`. -- AUDIT: regenerate `site_audit.json` with `build_site_audit.py` against a - snapshot at its pins (update skill steps 7 and 11); never edit it by hand, since - `figure_at_pin` and the pin time are derived (#273). +- AUDIT: never edit `site_audit.json` by hand (#273, #276). A missing or + non-integer `figure_at_pin`: regenerate it with `build_site_audit.py` against a + snapshot at its pins (update skill steps 7 and 11). A bad pin time or a missing + audit: the builder copies the pin time from `revisions.json`, so take the pins + and pin time from the last audit the builder wrote (`git log -p + _fleet/data/site_audit.json`) or the refresh PR body, then regenerate. A Mech + with no entry, such as a new member, needs a full refresh with new pins. - UNCHECKED: most sites could not be reached; rerun before changing anything. A new card needs a `SOURCES` entry; a test enforces that. The cards diff --git a/scripts/fleet/check_cards.py b/scripts/fleet/check_cards.py index 58d281a..ba6cdd6 100644 --- a/scripts/fleet/check_cards.py +++ b/scripts/fleet/check_cards.py @@ -369,9 +369,12 @@ def main() -> int: "in site_audit.json, the pages that repeat it), then rerun assemble_page.py; no re-pin. " "If figure_at_pin itself is wrong, re-derive it with build_site_audit.py against a " "snapshot at the audit's pins (update-xmech-page step 11); never edit it by hand. GONE or CHANGED: repoint that Mech's SOURCES entry. MARKUP or " - "UNCARDED: fix the card or its SOURCES entry. AUDIT: regenerate site_audit.json with " - "build_site_audit.py against a snapshot at its pins (update-xmech-page steps 7 and " - "11), never by hand: its figure_at_pin and pin time are derived. " + "UNCARDED: fix the card or its SOURCES entry. AUDIT: never edit site_audit.json by hand. " + "A missing or non-integer figure_at_pin: regenerate it with build_site_audit.py " + "against a snapshot at its pins (update-xmech-page steps 7 and 11). A bad pin time " + "or a missing audit: take the pins and pin time from the last audit the builder " + "wrote (git log -p _fleet/data/site_audit.json) or the refresh PR body, then " + "regenerate. A Mech with no entry: a full refresh with new pins. " "UNCHECKED: the run could not reach most sites; rerun before changing anything.") return 1 if any(status in ("grew", "unread") for status, _, _ in rows): From 0f95e58a6486b3bf1a57ed8f736790125970611b Mon Sep 17 00:00:00 2001 From: "marcin p. joachimiak" <4625870+realmarcin@users.noreply.github.com> Date: Fri, 25 Sep 2026 15:48:55 -0700 Subject: [PATCH 5/5] Make the remedies' recovery steps work as written git log needs -- before a path that may be deleted, and the refresh PR body has no pin time, so the AUDIT remedy now names only the last builder-written audit. A malformed audit is covered, and card_records is named as the one audit field a WRONG fix edits (#277). Co-Authored-By: Claude Opus 5.5 (1M context) --- _fleet/README.md | 16 +++++++++------- scripts/fleet/check_cards.py | 15 ++++++++------- 2 files changed, 17 insertions(+), 14 deletions(-) diff --git a/_fleet/README.md b/_fleet/README.md index fa111cf..54936de 100644 --- a/_fleet/README.md +++ b/_fleet/README.md @@ -62,8 +62,9 @@ remedy for each failing verdict (#235): - WRONG: a card differs from the figure its source stated at the pin. Correct the card and every other occurrence of its figure, found by grepping the tree for it as step 6 of the update skill does: the MECHS `records:` and `extra:` text in - `fleet_fragment.html`, cross-references, `card_records` in `site_audit.json` - and the pages that repeat it. Then rerun `assemble_page.py`; no re-pin (#248, + `fleet_fragment.html`, cross-references and the pages that repeat it, and + `card_records` in `site_audit.json`, the one audit field a WRONG fix edits, to + match the corrected card (a test requires the two to agree; #277). Then rerun `assemble_page.py`; no re-pin (#248, #258). If `figure_at_pin` itself is wrong, re-derive it with `build_site_audit.py` against a snapshot at the audit's pins (update skill step 11); it is derived, never typed (#268). The unit tests catch this on the @@ -71,12 +72,13 @@ remedy for each failing verdict (#235): step so its report prints (#239, #240). - GONE or CHANGED: a `SOURCES` entry needs repointing. - MARKUP or UNCARDED: fix the card markup or `SOURCES`. -- AUDIT: never edit `site_audit.json` by hand (#273, #276). A missing or +- AUDIT: never edit `site_audit.json` by hand, apart from `card_records` under + WRONG (#273, #276, #277). A missing or non-integer `figure_at_pin`: regenerate it with `build_site_audit.py` against a - snapshot at its pins (update skill steps 7 and 11). A bad pin time or a missing - audit: the builder copies the pin time from `revisions.json`, so take the pins - and pin time from the last audit the builder wrote (`git log -p - _fleet/data/site_audit.json`) or the refresh PR body, then regenerate. A Mech + snapshot at its pins (update skill steps 7 and 11). A bad pin time, or a missing + or malformed audit: the builder copies the pin time from `revisions.json`, so + take the pins and pin time from the last audit the builder wrote (`git log -p -- + _fleet/data/site_audit.json`), then regenerate. A Mech with no entry, such as a new member, needs a full refresh with new pins. - UNCHECKED: most sites could not be reached; rerun before changing anything. diff --git a/scripts/fleet/check_cards.py b/scripts/fleet/check_cards.py index ba6cdd6..2dd6b41 100644 --- a/scripts/fleet/check_cards.py +++ b/scripts/fleet/check_cards.py @@ -365,16 +365,17 @@ def main() -> int: "and the MECHS block in _fleet/fleet_fragment.html (update-xmech-page), then rerun " "assemble_page.py. WRONG: correct the card and every other occurrence of its " "figure, found by grepping the tree for it as update-xmech-page step 6 does (the MECHS " - "records: and extra: text in _fleet/fleet_fragment.html, cross-references, card_records " - "in site_audit.json, the pages that repeat it), then rerun assemble_page.py; no re-pin. " + "records: and extra: text in _fleet/fleet_fragment.html, cross-references, the pages " + "that repeat it, and card_records in site_audit.json, the one audit field a WRONG fix " + "edits, to match the corrected card), then rerun assemble_page.py; no re-pin. " "If figure_at_pin itself is wrong, re-derive it with build_site_audit.py against a " "snapshot at the audit's pins (update-xmech-page step 11); never edit it by hand. GONE or CHANGED: repoint that Mech's SOURCES entry. MARKUP or " - "UNCARDED: fix the card or its SOURCES entry. AUDIT: never edit site_audit.json by hand. " + "UNCARDED: fix the card or its SOURCES entry. AUDIT: never edit site_audit.json by hand, apart from card_records under WRONG. " "A missing or non-integer figure_at_pin: regenerate it with build_site_audit.py " - "against a snapshot at its pins (update-xmech-page steps 7 and 11). A bad pin time " - "or a missing audit: take the pins and pin time from the last audit the builder " - "wrote (git log -p _fleet/data/site_audit.json) or the refresh PR body, then " - "regenerate. A Mech with no entry: a full refresh with new pins. " + "against a snapshot at its pins (update-xmech-page steps 7 and 11). A bad pin time, " + "or a missing or malformed audit: take the pins and pin time from the last audit the " + "builder wrote (git log -p -- _fleet/data/site_audit.json), then regenerate. A Mech " + "with no entry: a full refresh with new pins. " "UNCHECKED: the run could not reach most sites; rerun before changing anything.") return 1 if any(status in ("grew", "unread") for status, _, _ in rows):