Refresh the X-Mech page from the Mechs as they are now, and add the skill that does it - #120
Merged
Merged
Conversation
… now There was no procedure for this, only the pipeline reference in _fleet/README.md, and the last refreshes show what that costs: a census taken from one checkout and card stats from another, a census that read a shared checkout's uncommitted files, and figures on five other pages left behind the cards. The skill pins every Mech and CLAW once, builds a sparse shared snapshot at those commits instead of pulling the shared checkouts, runs the pipeline canary-first with the side effects checked after each stage, re-checks every hand-curated claim against the live sites and the pinned repos, propagates the figures to every page that repeats them, rewrites the provenance record, and stops at a reviewed PR. It keeps the methodology fixed; census scope changes (#84, #87) stay out. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…able The census and the card stats are written by different scripts, and nothing said which checkout either had read. The last refresh put 364 CommunityMech records in the census and 396 in the stats, and its ProteinTraitsMech census counted 770,276 UniProt references where the commit it claimed holds 657,598, because it read a shared checkout's uncommitted files. roots.revision() now names the commit, marked +dirty for a checkout with local changes; prefix_census.py writes it per Mech under _revisions and mech_stats.py writes it as source_revision. The latter field was already in the committed file, added by hand, so rerunning the script would have dropped it. build_subsets.py wrote two orders that came from iterating a set of strings, which changes with each process's hash seed: tied prefixes in each edge's "by" breakdown (#107) and, not in that issue, the order of the per-cell summary, which reaches subsets_summary.json and the page's embedded data. Both now sort. Two runs over the same snapshot under different seeds are byte-identical. Closes #107. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
What the 2026-09-24 run hit that the procedure did not yet say: scanning with resource warnings as errors floods the log because the census never closes its files (#118); git grep -E has no \b; three cross-references pointed the wrong way, so direction needs checking as well as existence; HUB text renders after a prefix; and two sites published new records mid-run. For the last, the skill now says to keep the page a consistent snapshot at the pins and record the live figure in the audit, rather than re-pinning one Mech, which would be a partial rerun. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every number was re-derived from one set of revisions: each Mech's GitHub main and CLAW's, pinned once and read from sparse clones of the shared checkouts, never their working trees. Several of those lagged their remotes by up to 121 commits, and ProteinTraitsMech's carried 45,049 uncommitted files. Derived layer. The census, subsets, fleet data and card stats were regenerated at the pins, and the manifest snapshot advanced to CLAW's pin (semantically unchanged). The census now names its revisions, and RefreshProvenanceTests pins the census, the stats and the site audit to the same commits and record counts. On the old data that test fails for exactly the known partial rerun, 364 vs 396 CommunityMech records; each of its four invariants was mutation-tested. The #105 citation filter takes effect: 379 citation matches leave the edge weights and all 36 edges survive. The previous census's ProteinTraitsMech counts had been read from a dirty checkout (770,276 UniProt references where the revision the last audit pinned holds 657,598), so the lower figures are a correction. Hand-curated layer. Twelve read-only auditors checked 533 claims against the live sites and the pinned repositories; an independent skeptic re-derived each proposed change, and a second round verified what the first round had missed. Beyond the moved figures (CommunityMech 422 and 15 categories, TraitMech 763, CellStructureMech 542, MediaIngredientMech 2,953 / 2,611 mapped, CultureMech 6,288, ProteinTraitsMech 429,293): - three cross-references pointed away from the consumer and now point at it; three new ones are implemented upstream (TraitMech into ProteinTraitsMech, TraitMech and HabitatMech into TaxonMech), and the list now matches XREFS; - the kg-microbe ties claimed exports that do not exist: only MediaIngredientMech feeds the graph, CultureMech and CommunityMech have exporters but no release, and TraitMech receives embeddings rather than exporting SSSOM; - "ENVO-grounded", "a superset of TraitMech's node vocabulary", "one node vocabulary", "content-hashed CURIE" and "evidence with PMIDs or DOIs" were fleet-wide generalizations the sources contradict, and now say what holds; - CultureMech's pages/ index is live again, so its Browse link and card cite it. The figures are propagated to every page that repeats them, and the three dedicated pages' permalinks move to the pins (all 33 anchors checked). CellStructureMech (544) and TraitMech (764) published after the pins; their cards keep the pinned figures and site_audit.json records what the sites showed. Closes #85, #113. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Sep 25, 2026
Closed
…EADME exception everywhere From the Codex review of the #175 fix. The text parser collapsed no whitespace, so a formatting-only wrap between "merged" and "records" in CultureMech's README returned None; whitespace now collapses before matching, with a test for a wrap inside the label (#207). _fleet/README.md and the review skill's general descriptions of the card sources now name CultureMech's README exception (#208). Closes #207. Closes #208. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Verification review of e115229: the fleet README's census paragraph, the check_cards.py docstring and assemble_page.py's fleet_records docstring still said every card follows a published site; the review skill now says the README figure is on the card and in MECHS (#211). Closes #211. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
realmarcin
added a commit
that referenced
this pull request
Sep 25, 2026
The guidance agents follow here lived in an untracked CLAUDE.md one directory above the git root, so its edits were never reviewed, never reached a clone, and during #120 changed three times outside any diff (#173). It is now CLAUDE.md at the repository root, excluded from the Jekyll build. Once this merges, the outer file becomes a pointer to it. Corrected on the way in, each against the repository or the Mechs' records: - the outer file's "Repository Root Structure" put the git root one level too high, with a .github/profile the repository does not have; - MediaIngredientMech is described by what its records map to, mostly ChEBI, not "ChEBI/PubChem/METPO", and the naming rule now says to describe a Mech's groundings by what its records use (#150); - IMPLEMENTATION_SUMMARY.md is already in exclude, so it is not deployed; - culturemech.md is 15,878 normalized recipes and 6,288 canonical media, not "10,000+"; the fleet assets are about 10.5 MB, not 3.5; the census covers ten. It also names the CI workflow, the two repository skills, the git workflow, and the fact that Jekyll is not installed in most working copies. Closes #173. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Sep 25, 2026
realmarcin
added a commit
that referenced
this pull request
Sep 25, 2026
…d snapshot build_subsets.py takes about six minutes over the ten-Mech snapshot, less than the census's eight, in every logged run at the #120 pins (#230). Step 11 also says a later PR can rebuild the snapshot from the pins in site_audit.json. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
realmarcin
added a commit
that referenced
this pull request
Sep 25, 2026
* Remove a stray zero-byte file named ] from the repository root Tracked since 917fcde (2026-03-26), almost certainly a shell slip, and served by GitHub Pages at /]. Nothing references it. Closes #187. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Track the repository's agent guidance in the repository The guidance agents follow here lived in an untracked CLAUDE.md one directory above the git root, so its edits were never reviewed, never reached a clone, and during #120 changed three times outside any diff (#173). It is now CLAUDE.md at the repository root, excluded from the Jekyll build. Once this merges, the outer file becomes a pointer to it. Corrected on the way in, each against the repository or the Mechs' records: - the outer file's "Repository Root Structure" put the git root one level too high, with a .github/profile the repository does not have; - MediaIngredientMech is described by what its records map to, mostly ChEBI, not "ChEBI/PubChem/METPO", and the naming rule now says to describe a Mech's groundings by what its records use (#150); - IMPLEMENTATION_SUMMARY.md is already in exclude, so it is not deployed; - culturemech.md is 15,878 normalized recipes and 6,288 canonical media, not "10,000+"; the fleet assets are about 10.5 MB, not 3.5; the census covers ten. It also names the CI workflow, the two repository skills, the git workflow, and the fact that Jekyll is not installed in most working copies. Closes #173. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Point the skills at the tracked guidance, and correct what review found in it Round-1 review of #192. Both repository skills sent agents to the untracked ../CLAUDE.md; review-open-issues now names the tracked file, and update-xmech- page's step 7 updates its Last refreshed sentence inside the PR (#196). The MediaIngredientMech line takes #186's final wording (#197). The layout list drops the deleted _includes/header.html and adds the real overrides (#198). The Mech colour tokens are placed in _fleet/fleet_fragment.html, not custom.css (#199). The capability table and badges are credited to claw's manifest snapshot and refresh_manifest.py rather than the census pipeline (#200). The organization profile README is pointed to in CultureBotAI/.github (#201). Closes #196. Closes #197. Closes #198. Closes #199. Closes #200. Closes #201. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Say what the profile README and the stat strip actually hold The organization profile does not list the suite; it names only CultureMech, with a stale figure (#213). Of the stat strip's four tiles only the vocabulary and merged-PR tiles are pipeline output: the records tile sums the hand-curated cards and the Mech count follows the manifest (#214). The MediaIngredientMech line follows #186's final wording (#215). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Give the skill's layer table the stat strip's real sources, and keep the CultureMech exception The update skill filed the whole stat strip as pipeline output; it now has a manifest row, and the records tile sits with the hand-curated cards it sums (#225). CLAUDE.md said the card check reads every live site; CultureMech's figure comes from its committed README (#226). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
realmarcin
added a commit
that referenced
this pull request
Sep 25, 2026
* Link NaturalProductMech records to its own pages, not GitHub NaturalProductMech commits one page per record, pages/<class>/<slug>.html mirroring data/natural_products/<class>/<slug>.yaml, and its branch build serves them, so its census record links now open those pages (#149). The subsets were rebuilt at the #120 pins from the same snapshot: only the fifteen NaturalProductMech cell and edge indexes changed, and only in their link base and slug suffix; all 2,576 distinct records they link have a committed page. CultureMech stays on GitHub while its pages/ deployment comes and goes (#223). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Say MediaIngredientMech has no per-record pages, not unreliable ones The comment grouped it with CultureMech's flickering pages/media/; it publishes none (#229). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
realmarcin
added a commit
that referenced
this pull request
Sep 25, 2026
…d snapshot build_subsets.py takes about six minutes over the ten-Mech snapshot, less than the census's eight, in every logged run at the #120 pins (#230). Step 11 also says a later PR can rebuild the snapshot from the pins in site_audit.json. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Sep 25, 2026
Closed
realmarcin
added a commit
that referenced
this pull request
Sep 25, 2026
…e merge (#216) * Tell the update skill to remove its snapshot after the merge A refresh builds sparse shared clones of every Mech in the scratchpad, about 6 GB, and nothing removed them (#174). Step 11 removes the snapshot once the PR has merged, guarded against an unset variable, and step 2 gives its real size. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Give the snapshot its own directory, and remove only that The skill named the session scratchpad itself as $SNAP, so step 11's rm -rf could take the scratchpad and every worktree in it (#227). Step 1 now creates a dedicated directory and refuses to reuse one, the PR records its absolute path because the merge may come in another session (#228), and step 11 removes the path from the PR only if it holds revisions.json, mechs/ and claw/. Step 4's timings follow _fleet/README.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Give the pipeline's measured timings, and say how to rebuild a removed snapshot build_subsets.py takes about six minutes over the ten-Mech snapshot, less than the census's eight, in every logged run at the #120 pins (#230). Step 11 also says a later PR can rebuild the snapshot from the pins in site_audit.json. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Stop step 11 on anything but snapshot data, and spell out a rebuild at the same pins Step 11 said step 2 could rebuild a removed snapshot, but step 2 makes only the Mech clones; a rerun at the same pins also needs CLAW at the audited sha and a revisions.json, and the skill now says how (#236). The removal also stops if the directory holds anything but the clones, revisions.json and logs, and the preamble says to keep working files outside it (#237). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Keep before/after copies out of the snapshot, and give revisions.json's shape Step 11 listed before/after copies of the derived data among what it removes while its guard stops on them; they are working files and belong outside $SNAP (#245). The same-pins rebuild recipe now shows the revisions.json shape step 7 reads and says to set SRC before step 2 (#246). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Name snapshot logs <stage>.log, the only logs step 11 removes The preamble allowed any log in $SNAP while step 11's guard accepts only *.log; the preamble and step 4 now say to use that suffix (#251). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Compare step 11's entries literally, and let a clean snapshot pass under set -e The listing matched names as regular expressions, so revisionsXjson passed as revisions.json, and it took a directory named *.log for a log (#252). It also exited 1 when nothing was extra, which stops a set -e shell before the removal (#253). A case loop compares literally, requires a *.log to be a regular file and exits 0 on a clean snapshot; tested under bash 3.2, zsh and sh with set -e. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
realmarcin
added a commit
that referenced
this pull request
Sep 25, 2026
…ured A scan of every census record at the #120 pins found more spellings of counted registries: SwissProt/swissprot/Swissprot/UNIPROT, TAXON, PDBe/pdbe, interpro, KEGG_PATHWAY/kegg.module/kegg.glycan, MeSH and PubChem_Compound (#255). The census and, for heatmap columns, build_subsets.py now fold them; the comment and README say the list covers the spellings measured at the pins, not every spelling there could be. The test comment no longer counts norm's entries (#256). Rerun at the same pins: ProteinTraitsMech UniProt +17, PDB +10, NCBITaxon +5, KEGG +5, InterPro +2 and AntibioticMech KEGG +1; four chords gain one term each; the vocabulary tile stays at 48. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
realmarcin
added a commit
that referenced
this pull request
Sep 25, 2026
…ens allowed The spelling scan skipped hyphenated prefixes, so CAS-RN and Swiss went unmeasured (#270), and the census folded entity-qualified namespaces for some registries but not others (#271). The rule is now written down and applied evenly: every namespace named for or qualified by a counted registry folds into it, as gold.ecosystem and pubchem.compound always did. Added: gtdb.genome, pdb.ligand, pdb-ccd, RHEA-COMP, CAS-RN, uniprot.location, uniprot.ptm, UniProtKB-KW and Swiss. build_subsets.py keeps Rhea compounds and PDB ligand codes in their own term space, so an overlap never pairs a compound with a reaction or a ligand with an entry, while they still count toward their column. Rerun at the #120 pins: TaxonMech GTDB +135,164, ProteinTraitsMech PDB +14,863, RHEA +11,169 and UniProt +617, CellStructureMech UniProt 337 to 993, AntibioticMech PDB 6 to 339, MediaIngredientMech CAS 727 to 1,463; the vocabulary tile stays at 48. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
realmarcin
added a commit
that referenced
this pull request
Sep 25, 2026
…d input (#265) * Commit the site-audit builder, with the audited notes as its only hand input _fleet/data/site_audit.json was written by a script that existed only in a session scratchpad, so each refresh had to rewrite it from the skill's prose and nothing checked that the rewrite derived each field the same way (#238). scripts/fleet/build_site_audit.py now derives every mechanical field from the snapshot, mech_stats.json and check_cards.figure(), appends the sentence saying whether each live copy still matches its pin, and refuses a card that differs from its figure at the pin or exceeds its site. The audited notes and scope move to _fleet/audit_notes.json, the builder's only hand-written input. Run in memory against the #120 snapshot and the live sites, it reproduces the committed audit exactly, key order and notes included, except CellStructureMech, whose site has grown from 577 to 584 since the audit was written. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Test build() itself, name only missing notes, and refuse a bad pin time build() had no test, so its notes refusal, UTC commit dates, CLAW entry and order could regress unnoticed; an offline test now covers them (#266). The refusal names only the notes actually missing and the file read (#267). The WRONG remedy says to re-derive figure_at_pin with the builder, never edit it (#268). Step 1 gives the revisions.json layout and pin-time format the builder reads, and the builder refuses a pin time the nightly would report as AUDIT (#269). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Send AUDIT to the builder, refuse offset-less commit dates, test every derived field AUDIT's remedy said to fix site_audit.json by hand, though figure_at_pin and the pin time are derived; it now says to regenerate the audit with the builder (#273). A commit date with no offset is refused instead of being read as local time, and step 1 says to use the committer date with its offset (#274). The tests now check local_date under a non-UTC zone, each hash against its own copy, a data source's page hash, readme_url and merged_prs (#275). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Split the AUDIT remedy by cause, so a bad pin time does not loop Rebuilding from the failing audit reproduces a bad pin time, and a missing audit or a Mech with no entry has no pins to rebuild from. The remedy now says where each comes from: rebuild for figure_at_pin, the last builder-written audit or the PR body for the pins, a full refresh for a new Mech (#276). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Make the remedies' recovery steps work as written git log needs -- before a path that may be deleted, and the refresh PR body has no pin time, so the AUDIT remedy now names only the last builder-written audit. A malformed audit is covered, and card_records is named as the one audit field a WRONG fix edits (#277). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
realmarcin
added a commit
that referenced
this pull request
Sep 25, 2026
The census counted GOLD, BacDive, IMG, DSMZ, KEGG, PubChem, MediaDive, KOMODO, ChEMBL, EC, ChEBI, PDB and PMID under one spelling each and dropped the others. TaxonMech writes lowercase bioregistry prefixes, so 1,492,983 GOLD, 556,160 BacDive, 68,303 IMG and 31,553 DSMZ identifiers were never counted; so were kegg.compound/kegg.drug/KEGG_REACTION, pubchem/pubchem.aid, mediadive.*, komodo.medium, chembl, ec, ChEBI, RCSB_PDB and PubMed/PUBMED elsewhere (#84). build_subsets.py folds the spellings of heatmap columns the same way, so AntibioticMech now has a KEGG cell and a kegg.compound term is the same term as a KEGG one. Rerun at the #120 pins from the same snapshot: revisions and record counts are unchanged; only folded prefixes moved. The vocabulary tile goes from 46 to 48 (ChEMBL and KOMODO were only ever written the uncounted way). Which new namespaces to count stays open on #84. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
realmarcin
added a commit
that referenced
this pull request
Sep 25, 2026
…ured A scan of every census record at the #120 pins found more spellings of counted registries: SwissProt/swissprot/Swissprot/UNIPROT, TAXON, PDBe/pdbe, interpro, KEGG_PATHWAY/kegg.module/kegg.glycan, MeSH and PubChem_Compound (#255). The census and, for heatmap columns, build_subsets.py now fold them; the comment and README say the list covers the spellings measured at the pins, not every spelling there could be. The test comment no longer counts norm's entries (#256). Rerun at the same pins: ProteinTraitsMech UniProt +17, PDB +10, NCBITaxon +5, KEGG +5, InterPro +2 and AntibioticMech KEGG +1; four chords gain one term each; the vocabulary tile stays at 48. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
realmarcin
added a commit
that referenced
this pull request
Sep 25, 2026
…ens allowed The spelling scan skipped hyphenated prefixes, so CAS-RN and Swiss went unmeasured (#270), and the census folded entity-qualified namespaces for some registries but not others (#271). The rule is now written down and applied evenly: every namespace named for or qualified by a counted registry folds into it, as gold.ecosystem and pubchem.compound always did. Added: gtdb.genome, pdb.ligand, pdb-ccd, RHEA-COMP, CAS-RN, uniprot.location, uniprot.ptm, UniProtKB-KW and Swiss. build_subsets.py keeps Rhea compounds and PDB ligand codes in their own term space, so an overlap never pairs a compound with a reaction or a ligand with an entry, while they still count toward their column. Rerun at the #120 pins: TaxonMech GTDB +135,164, ProteinTraitsMech PDB +14,863, RHEA +11,169 and UniProt +617, CellStructureMech UniProt 337 to 993, AntibioticMech PDB 6 to 339, MediaIngredientMech CAS 727 to 1,463; the vocabulary tile stays at 48. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
realmarcin
added a commit
that referenced
this pull request
Sep 26, 2026
* Count every spelling of a registry the census already counts The census counted GOLD, BacDive, IMG, DSMZ, KEGG, PubChem, MediaDive, KOMODO, ChEMBL, EC, ChEBI, PDB and PMID under one spelling each and dropped the others. TaxonMech writes lowercase bioregistry prefixes, so 1,492,983 GOLD, 556,160 BacDive, 68,303 IMG and 31,553 DSMZ identifiers were never counted; so were kegg.compound/kegg.drug/KEGG_REACTION, pubchem/pubchem.aid, mediadive.*, komodo.medium, chembl, ec, ChEBI, RCSB_PDB and PubMed/PUBMED elsewhere (#84). build_subsets.py folds the spellings of heatmap columns the same way, so AntibioticMech now has a KEGG cell and a kegg.compound term is the same term as a KEGG one. Rerun at the #120 pins from the same snapshot: revisions and record counts are unchanged; only folded prefixes moved. The vocabulary tile goes from 46 to 48 (ChEMBL and KOMODO were only ever written the uncounted way). Which new namespaces to count stays open on #84. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Fold the rest of the measured spellings, and claim only what was measured A scan of every census record at the #120 pins found more spellings of counted registries: SwissProt/swissprot/Swissprot/UNIPROT, TAXON, PDBe/pdbe, interpro, KEGG_PATHWAY/kegg.module/kegg.glycan, MeSH and PubChem_Compound (#255). The census and, for heatmap columns, build_subsets.py now fold them; the comment and README say the list covers the spellings measured at the pins, not every spelling there could be. The test comment no longer counts norm's entries (#256). Rerun at the same pins: ProteinTraitsMech UniProt +17, PDB +10, NCBITaxon +5, KEGG +5, InterPro +2 and AntibioticMech KEGG +1; four chords gain one term each; the vocabulary tile stays at 48. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Fold every namespace named for a counted registry, measured with hyphens allowed The spelling scan skipped hyphenated prefixes, so CAS-RN and Swiss went unmeasured (#270), and the census folded entity-qualified namespaces for some registries but not others (#271). The rule is now written down and applied evenly: every namespace named for or qualified by a counted registry folds into it, as gold.ecosystem and pubchem.compound always did. Added: gtdb.genome, pdb.ligand, pdb-ccd, RHEA-COMP, CAS-RN, uniprot.location, uniprot.ptm, UniProtKB-KW and Swiss. build_subsets.py keeps Rhea compounds and PDB ligand codes in their own term space, so an overlap never pairs a compound with a reaction or a ligand with an entry, while they still count toward their column. Rerun at the #120 pins: TaxonMech GTDB +135,164, ProteinTraitsMech PDB +14,863, RHEA +11,169 and UniProt +617, CellStructureMech UniProt 337 to 993, AntibioticMech PDB 6 to 339, MediaIngredientMech CAS 727 to 1,463; the vocabulary tile stays at 48. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Fold TC into TCDB, scope the rule, and let the prefix test read escaped hyphens TC: is TCDB's own prefix; ProteinTraitsMech cites 194 transporters that way beside 4,705 TCDB: ones (#278). The rule now names its exception, reference and curator collections such as GO_REF, and no longer says the lowercase spellings come from TaxonMech alone; TERM_SPACE gives its real reason for PDB ligand codes (#279). PrefixListTests unescapes every regex escape, not only dots, and checks that every norm key is a spelling the census matches (#280). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Say what #84 says about GO_REF, PO_REF and GOC GO_REF and PO_REF are literature-like, GO_REF a CITATION candidate in #84; GOC is curator attribution, which #84 proposes never to count (#282). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Let the edge panel's column filter find terms kept in their own term space The panel pre-fills its filter with the column ("PDB:"), which PDB-CCD ligand codes and RHEA-COMP compounds never contain, so a refiltered edge could show "PDB 140" and list none. Terms now also match the column they count toward, through a TERM_COLUMN map a test holds equal to build_subsets.COLUMN_OF (#284). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Fold MSH into MESH MSH is MeSH's synonym in GO's db-xrefs registry; ten ProteinTraitsMech records cite MeSH that way, so its MESH count goes from 4 to 14 (#285). Census only: MESH has no heatmap column. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #85
Closes #87
Closes #107
Closes #121
Closes #122
Closes #123
Closes #124
Closes #125
Closes #126
Closes #127
Closes #128
Closes #129
Closes #130
Closes #133
Closes #134
Closes #135
Closes #136
Closes #138
Closes #139
Closes #140
Closes #141
Closes #142
Closes #143
Closes #145
Closes #155
Closes #156
Closes #158
Closes #159
Closes #160
Closes #162
Closes #163
Closes #164
Closes #165
Closes #166
Closes #167
Closes #168
Closes #169
Closes #170
Closes #171
Closes #175
Closes #177
Closes #178
Closes #179
Closes #180
Closes #181
Closes #182
Closes #183
Closes #184
Closes #185
Closes #188
Closes #189
Closes #193
Closes #202
Closes #204
Closes #207
Closes #208
Closes #211
The adversarial review below filed #121 to #154. This PR fixes #121 to #143 except #131, #132, #137 and #144, which were about the PR itself or are upstream. Those three are resolved by this description, #147 and the evidence comment. #113 stays open: the nightly card check is still red for two Mechs that moved past their pins.
There was no procedure for refreshing
/mechs/, so this adds one,.claude/skills/update-xmech-page/, and this PR is its first run.One set of revisions for everything
Every Mech and CLAW were pinned once, at GitHub
main, and every number was derived from those commits. The corpora were read from sparse shared clones, never from the shared checkouts' working trees, which lagged by up to 121 commits (HabitatMech) and in one case carried 45,049 uncommitted files (ProteinTraitsMech).66d68c242cc23b8505a56faaf033b16e309dfce1c936662aa5f05fca972fbd7a21c5aaCLAW is pinned at
2637172; the manifest is semantically unchanged. CommunityMech's census adds four isolate records its site does not list.The census and the stats could not disagree unnoticed before; now they cannot
The committed data had the census at 364 CommunityMech records and the stats at 396, from different checkouts.
roots.revision()now names the commit each count was read at, andRefreshProvenanceTestsrequires the census, the stats andsite_audit.jsonto agree on revisions and record counts. On the old data it fails for exactly that inconsistency. Each invariant was mutation-tested: a census revision from another checkout, a different corpus, a dirty checkout, a mismatched audit pin, and a census with no revisions each fail the intended test.The previous census's ProteinTraitsMech counts turned out to come from the dirty checkout: 770,276 UniProt references where the revision the last audit pinned holds 657,598. The lower figures here are a correction.
Canary and determinism
ls-tree; the unquoted form miscounts the non-ASCII filenames), and every clone reports a clean revision.ResourceWarningan error and the census never closes its files (build_subsets.py and build_data.py still write with unclosed handles, and no script writes atomically #118). I killed it; the committed census was untouched. The relaunch was verified before the rest ran.build_subsets.pyruns under different hash seeds are byte-identical. The fix also covers a second leak the issue missed: the per-cell summary was keyed in set-iteration order.Hand-curated claims
Twelve read-only auditors, one per Mech plus fleet-wide and cross-reference scopes, checked 533 claims against the live sites and the pinned repos. An independent skeptic re-derived every proposed change, and a second round verified the 60 things the first round missed. Of 174 first-round changes, none was refuted and 9 were corrected. Beyond the moved figures:
XREFS(14 pairs).pages/index, its recipe browser's data and its dashboard are deployed by an Actions workflow, while its Pages source is set to branch builds. Each branch build replaces that deployment, so they alternate between live and 404, and all three were gone on September 25 (CultureMech's pages/ index is gone again, and #120 points the card and Browse link at it #175). The Browse link therefore stays on the site root. The upstream fix is recorded on File the upstream defects from #147 in their sibling repositories #172.The figures are propagated to every page that repeats them. The three dedicated pages' permalinks move to the pins, with all 33 anchors checked.
Known drift at merge time
CellStructureMech and TraitMech published new records after the pins; at the last audit the sites showed 577 and 768, recorded in
site_audit.json. Their cards keep the pinned figures andsite_audit.jsonrecords what the sites showed.check_cards.pywill report both as drifted, and re-pinning one Mech would be a partial rerun.Review
A separate read-only pass had six reviewers, one per dimension, and an independent verifier for each finding. It produced 30 findings: 28 were confirmed or partly confirmed and 1 was refuted (#146, closed). The data-integrity reviewer re-derived every derived file, count and record link and confirmed them. Its one finding, audit notes claiming byte-identity with the pin, is #155, fixed in "Hash each audited page at the pin". The fixes are in "Address the adversarial review of the refresh":
revision()now compares the counted records with git's tracked files, and no longer relies ongit status.A rerun at the same pins changed only CommunityMech's record lists. The per-edit evidence is in the PR comments.
Complete arrows, and TaxonMech measured (follow-up)
The graph drew only two arrows at TaxonMech and none leaving it. A second sweep read ingest scripts, vendored snapshots, source catalogues, CI workflows and site generators in all ten repositories at the pins. An independent verifier checked each of 32 candidate pairs. The graph now draws 30 arrows, one per ordered pair:
Taxa tie TaxonMech to the fleet through shared NCBI Taxonomy identifiers, which the page draws as vocabulary chords. So TaxonMech now joins the census (#87), with eight overlaps:
A second review of that commit found 16 issues, #156 to #171; 14 held up and are fixed here, and 2 are closed with the rule that excludes them. The biggest fix was geometry. Arcs bowed away from the chord midpoint, so opposite nodes collapsed into one line through the hub. They now run outside the ring, or around the hub when they cannot. Measured in a browser:
TaxonMech also became the first label authority for taxa, which halves the unlabelled overlap terms. The record-subset assets grow to about 10.5 MB. The largest overlap files load only on click.
Not in this PR
../CLAUDE.mdis untracked and above the repository. I updated its "last computed" note and HabitatMech description, so that change is not in this diff.Checks: 48 tests pass; a Codex review ran after the fifth internal round on the #175 fix, and its two findings are fixed;
assemble_page.py --checkandrefresh_manifest.py --checkare clean; the page script passesnode --check.🤖 Generated with Claude Code