Skip to content

Refresh the fleet from its published sites, and say ten everywhere - #89

Closed
realmarcin wants to merge 3 commits into
fix/mech-palette-and-records-labelfrom
feat/refresh-fleet-and-ten-mech-consistency
Closed

realmarcin wants to merge 3 commits into
fix/mech-palette-and-records-labelfrom
feat/refresh-fleet-and-ten-mech-consistency

Conversation

@realmarcin

Copy link
Copy Markdown
Contributor

Stacked on #83base is fix/mech-palette-and-records-label, because both branches regenerate mechs.md. Review #83 first, or read this diff alone.

Cards had drifted from the Mechs they describe

Mech Card was Card now Source
TraitMech 630 696 live site
CellStructureMech 360 421 live site (also 306 → 367 GO-grounded)
CommunityMech 374 379 live site
ProteinTraitsMech 408,978 429,291 data/facets.json
CultureMech 10,657 6,286 see below

Every changed figure was read off the live site, then re-derived independently — against data/facets.json, the local checkout, and a second fetch — before it went in. The records tile sums the cards by construction, so it moved to 1,075,244.

Two sites lie in their own HTML

ProteinTraitsMech ships 408,978 records and 29 sources as static markup and overwrites both at runtime from data/facets.json, which serves 429,291 and 34. MediaIngredientMech ships literal ellipsis placeholders filled the same way. A scraper and a visitor see different numbers. _fleet/README.md now says to read the rendered figure, and to fetch the pages/ or app/ URL directly — those landing redirects are client-side meta refreshes that curl -L does not follow.

CultureMech's card no longer takes its Mech's headline tile

That tile reads 10,657 "recipes". It is hand-typed, was written once on 2026-06-30, and has never matched any data layer — at that commit the corpus was already 15,878 normalized / 6,103 merged. Nothing regenerates it.

The card now states 6,286 canonical media, which is what CultureMech's own pages/index.html publishes and what the fleet census counts — so the card half and the census half of the page finally agree. The card note says why this one card is sourced differently. This reverses the choice made in e626d77; the evidence that it was never right is new.

A census defect, fixed

Six timestamped backups under data/ingredients/mapped/backups/ were counted as MediaIngredientMech records, and their prefixes double-counted in the heatmap: the census read 2,957 where the Mech says 2,951 (2,679 mapped + 272 unmapped). Fixed by an EXCLUDE_DIRS map in roots.py — a ** glob cannot exclude a subtree, and enumerating mapped/ and unmapped/ explicitly would silently miss a new record directory, which is the failure record_paths() exists to prevent. Full pipeline rerun after. (#88)

Ten, everywhere

The fleet has had ten members since TaxonMech joined, but the site still said nine and resources.md listed eight while giving sections to five.

  • Every enumeration names all ten and links to /mechs/
  • resources.md gains sections for CellStructureMech, HabitatMech, AntibioticMech, NaturalProductMech and TaxonMech
  • index.md now carries a number for all ten and links the claw orchestrator
  • communitymech.md claimed 304 communities and pointed both its browser and embedding links at the landing page

Filed, not fixed here

Checks

assemble_page.py --check, refresh_manifest.py --check (10 members), and all 16 tests pass. Jekyll is not installed locally, so rendering is unverified beyond the assembler's own placeholder and Liquid guards.

Note: CultureBotAI/CLAUDE.md sits above the repository root and is untracked, so corrections made to it there are not in this diff.

🤖 Generated with Claude Code

Four cards had drifted away from the Mechs they describe: TraitMech 630 → 696,
CellStructureMech 360 → 421 (and 306 → 367 GO-grounded), CommunityMech 374 →
379, ProteinTraitsMech 408,978 → 429,291. Each number was read off the live
site and then re-derived independently before it went in.

Two of those sites lie in their own HTML, so the figure a scraper sees is not
the figure a visitor sees. ProteinTraitsMech ships 408,978 records and 29
sources as static markup and overwrites both at runtime from data/facets.json,
which serves 429,291 and 34. MediaIngredientMech ships ellipsis placeholders
filled the same way. _fleet/README.md now says to read the rendered number.

CultureMech's card no longer takes its Mech's headline tile. That tile reads
10,657 "recipes" and has never matched any data layer: when it was hand-written
on 2026-06-30 the corpus was already 15,878 normalized and 6,103 merged, and
nothing regenerates it. The card states 6,286 canonical media instead — what
CultureMech's own pages/index.html publishes, and what the fleet census counts,
so the two halves of the page now agree. The card note says why this one card
is sourced differently.

Six timestamped backups under data/ingredients/mapped/backups/ were being
counted as MediaIngredientMech records, and their prefixes double-counted in
the heatmap, so the census read 2,957 where the Mech says 2,951 (#88). A `**`
glob cannot exclude a subtree, so roots.py grew an EXCLUDE_DIRS map rather than
enumerating mapped/ and unmapped/ explicitly — enumerating would silently miss
a new record directory, which is the failure record_paths() exists to prevent.

The fleet has had ten members since TaxonMech joined, but the site still said
nine, and resources.md listed eight and gave sections to five. Every
enumeration now names all ten and links to /mechs/; resources.md gains sections
for CellStructureMech, HabitatMech, AntibioticMech, NaturalProductMech and
TaxonMech. communitymech.md still claimed 304 communities and pointed both its
browser and embedding links at the landing page.

Census reread on 2026-09-19; card totals rechecked against the live sites on
2026-09-20. Known gaps filed rather than fixed here: the prefix regex omits
TOGO, CultureMech's second namespace at 2,833 occurrences (#84); the
CommunityMech and TraitMech checkouts lag their remotes (#85); upstream site
defects (#86); TaxonMech still has no census row (#87).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An adversarial pass over the branch raised 27 findings; 7 survived
independent verification. All seven are addressed here.

The normalized-recipe count was off by one across the site, and this branch
made that visible by stating the right figure on the card. Six places said
15,877; the corpus holds 15,878, which CultureMech's own merge_stats.json
records as its input_recipes and its README states in a CI-checked block.
Corrected in culturemech.md, resources.md and research.md rather than
reverting the card.

"Has never matched any data layer" was too strong a claim about CultureMech's
10,657 tile, and wrong. It is the March 2026 normalized count, recorded as
input_recipes in the Mech's tracked merge_stats_2026.json, and was already four
months stale when the tile was written. Saying which layer it came from is both
true and more useful than calling it baseless.

The NaturalProductMech card listed NPAtlas among its sources, which contradicts
the resources.md section added two commits ago, the guidance file, and the
Mech's own README — NPAtlas is CC BY-NC and explicitly cannot be seeded into a
CC BY 4.0 corpus. It is a cross-reference target, which is why it stays in
`vocab` and leaves `sources`; NPClassifier and AntibioticMech, both genuinely
adopted, take its place, restoring the nine the README names.

Two of the new resources.md sections stated partial coverage as universal.
AntibioticMech does not record a mode of action for every antimicrobial: 454 of
2,939 have one and 282 a molecular target. NaturalProductMech carries
bioactivities on 176 of 3,115 records, not all, and only 805 of its 3,407
producer claims rest on evidence that addressed the organism.

The TaxonMech bullet linked pages/straininfo.html, which 404s, and its card
claimed browsers for StrainInfo deposits and AllTheBacteria assemblies that the
site does not publish. TaxonMech publishes browse, type-strains and sources;
the bullet now points at type-strains. Every culturebotai.github.io link the
page carries was then fetched, and all resolve.

record_paths() had no test, so the exclusion added for #88 could regress
silently. Four cover it: backups are excluded, a sibling directory whose name
merely starts with "backups" is not, an exclusion that empties a corpus still
raises rather than returning zero, and every excluded Mech is one the census
reads — which catches an entry left behind by a rename.

Filed rather than fixed: enumerations outside the X-Mech pages that are still
short of ten, the site-wide footer among them (#90), and two Quick Navigation
anchors that miss the emoji leading-hyphen id (#91).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@realmarcin

Copy link
Copy Markdown
Contributor Author

Review pass

Reviewed the diff adversarially across five dimensions (numbers, ten-Mech consistency, markup/links, the roots.py change, prose), then had each finding independently refuted before accepting it. 27 raised, 7 survived. All seven are fixed in 438a177.

Confirmed and fixed

  1. The normalized count was off by one, site-wide. Six places said 15,877; the corpus holds 15,878 — CultureMech's merge_stats.json records input_recipes: 15878 → output_recipes: 6286, and its README says so in a CI-checked block. This branch made the discrepancy visible by putting the right figure on the card. Corrected in culturemech.md (×3), resources.md (×2) and research.md, rather than reverting the card.

  2. "Has never matched any data layer" was wrong. 10,657 is the March 2026 normalized count, recorded as input_recipes in the Mech's tracked merge_stats_2026.json. It was four months stale when the tile was written, not baseless. The README now says which layer it came from — true, and more useful.

  3. The NaturalProductMech card listed NPAtlas as a source, contradicting the resources.md section added two commits earlier, CLAUDE.md, and the Mech's own README — NPAtlas is CC BY-NC and explicitly cannot be seeded into a CC BY 4.0 corpus. It stays in vocab (the cross-references are real) and leaves sources; NPClassifier and AntibioticMech take its place, restoring the nine the README names.

  4. Two new resources.md sections stated partial coverage as universal. AntibioticMech: 454 of 2,939 records have a mode of action, 282 a molecular target. NaturalProductMech: bioactivities on 176 of 3,115, and only 805 of 3,407 producer claims rest on evidence that addressed the organism.

  5. The TaxonMech bullet linked a 404 (pages/straininfo.html), and its card claimed StrainInfo and AllTheBacteria browsers the site does not publish. TaxonMech publishes browse, type-strains and sources. I then fetched every culturebotai.github.io link the page carries — all resolve.

  6. record_paths() had no test, so the Backup copies were counted as MediaIngredientMech records #88 exclusion could regress silently. Four added: backups excluded; a sibling named backups_archive not excluded; an exclusion that empties a corpus still raises rather than returning zero; and every excluded Mech is one the census reads, which catches an entry orphaned by a rename.

Refuted

The other 20 were dismissed on verification — most were pre-existing and untouched by this diff, or faults in generated mechs.md that belong to its sources. Three were real but out of scope and are now filed: #90 (the site-wide footer still names five Mechs, plus microgrowagents.md and publications.md) and #91 (two Quick Navigation anchors miss the emoji leading-hyphen id).

Checks

20 tests pass, assemble_page.py --check and refresh_manifest.py --check (10 members) clean. Jekyll is not installed locally, so rendering is still unverified beyond the assembler's placeholder and Liquid guards.

One thing #90 raises that is worth a thought beyond the fix: nothing mechanically catches an enumeration falling behind. A test asserting that any page naming two or more Mechs names all ten would have caught the footer, microgrowagents.md, and the nine-vs-ten drift this PR cleaned up by hand — and would catch the eleventh Mech too.

The sweep stopped at the X-Mech pages. Three enumerations elsewhere were still
short (#90), the most visible by far being the site-wide footer, which named
five Mechs and linked no hub — on every page of the site. It now names all ten
and ends with a link to /mechs/, which also gives the two task-organized tool
lists (kg-microbe.md, research.md) a route to the hub without distorting them:
those group tools by what a reader is trying to do, so "all ten under For
Culture Optimization" would be wrong, and they are left alone deliberately.

The separator before MicroGrowAgents, which marks it as the one entry that is
not a Mech, needed a margin of its own. The anchors around it carry .55rem
leading and .1rem trailing, so an unstyled separator sits 5.3px from the link
on its left and 12.6px from the one on its right, reading as part of the hub
link rather than as a divider. Measured in a headless browser against the real
stylesheet: .45rem evens it to 12.5px on both sides.

microgrowagents.md's Related Tools named five Mechs and now names ten. The
bibliography entry in publications.md enumerates the organization's
repositories, so it names all ten and the orchestrator rather than the same
five, and links the hub.

Both Quick Navigation anchors for emoji headings were dead (#91). The repo
already documents that an emoji heading renders with a leading hyphen in its
id, and the deployed HTML confirms it: `## 🔬 Advanced Research Tools` emits
`-advanced-research-tools`, where the link said `#advanced-research-tools`.

`## 🛠️ Developer Resources` is the awkward one. Its emoji carries a variation
selector, and GitHub Pages keeps that codepoint while dropping the glyph, so
the live id is U+FE0F followed by `-developer-resources` — an anchor beginning
with an invisible character. Dropping the selector from the heading should make
it the ordinary `-developer-resources`, matching what the 🔬 heading already
emits. That one is an inference from the parallel case rather than something
this branch can prove: a locally installed kramdown reports different ids, so
it needs checking against the deployed page after merge.

Seven of the eight anchors now resolve against the live HTML; the eighth is the
one whose heading this commit changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@realmarcin
realmarcin force-pushed the feat/refresh-fleet-and-ten-mech-consistency branch from d77f902 to 826235d Compare September 20, 2026 23:49
@realmarcin
realmarcin deleted the branch fix/mech-palette-and-records-label September 20, 2026 23:50
@realmarcin realmarcin closed this Sep 20, 2026
@realmarcin

Copy link
Copy Markdown
Contributor Author

Closed by GitHub when #83 merged and this PR's base branch fix/mech-palette-and-records-label was deleted. The work continues in #92 — same branch, rebased onto main with the palette commits dropped (they landed as ba88182), tree byte-identical to what was reviewed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant