Skip to content

feat: feat: the index, phase 1: facet schema, 4-digit id ranges, backfill of every demo, index.json and .md twins - #570

Merged
mvilacad merged 7 commits into
mainfrom
feat/569-feat-the-index-phase-1-facet
Oct 25, 2026
Merged

mvilacad merged 7 commits into
mainfrom
feat/569-feat-the-index-phase-1-facet

Conversation

@mvilacad

@mvilacad mvilacad commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Closes #569

What this is

Phase 1 of the gallery index: facet schema in every Variant.meta.json, 4-digit id ranges, backfill of all 377 demos, site/public/api/v1/index.json and one .md twin per id at /gallery/<id>.md. No UI change. catalog.ts is untouched.

Regression tests (fail before, pass after)

New packages/ui/tests/demos-index.test.ts (schema rejections, id ranges, legacy exemptions, backfilled facets vs. source, publishing gate, index output). Fail-before evidence: with one sidecar set to the wrong kind and cost, 5 tests fail (catalog validation, range, legacy, tech/cost, committed output); with industry forced to published at a 6.8% error rate, 4 gate tests fail. Pass-after: bun test tests/demos-index.test.ts tests/demos-catalog.test.ts tests/llms-budget.test.ts, 54 pass, 0 fail. The llms-full cap is unaffected (it does not read the catalog).

Facet audit

Hand-read facets (industry, style, motion, interaction, layout, colorMode, density, aliases, related) were audited before publishing. Inferred facets (kind, subtype, components, tech, cost, added, reduced motion) come from the code and are always published.

What was and was not verified. A person read 203 of 377 entries against their source and sidecar summary: 130 in pass 1 (seed 569: 66 random, stratified by folder, plus all 64 immersive/expressive entries) and 73 in a holdout (seed 1031: random, stratified by folder, none in pass 1, read after the fixes). 174 entries were never re-read. They hold the first pass's value as changed by two rules (below) and are unverified beyond those rules. The hand-read values were written by six agents that each finished in under two minutes.

Gate. A field is published only when the one-sided 95% upper bound of its error rate is at most 5%. The bound is used, not the point estimate: 0 wrong of 73 means "at most ~4% wrong" (exact bound 4.0%, rule of three 4.1%), not "0% wrong". Fields above the limit stay in the sidecar as review: pending and are withheld from index.json, the facet counts and the .md twins.

field pass 1 errors, random 66 holdout errors upper bound (holdout) gate
industry 19 (28.8%) 5 of 73 (6.8%) ~14% pending
style 5 (7.6%) 0 of 73 ~4% published
motion 1 (1.5%) 0 of 73 ~4% published
interaction 2 (3.0%) 0 of 73 ~4% published
layout 1 (1.5%) 0 of 73 ~4% published
colorMode 0 of 66 0 of 73 ~4% published
density 0 of 66 0 of 73 ~4% published
aliases 1 (1.5%) 0 of 73 ~4% published
related 0 of 66 0 of 73 ~4% published

Fixes between pass 1 and the holdout: the sampled errors by hand, then two rules applied to all 377 entries: a non-default style stays only when the demo source shows it (minimal dropped; 35 entries changed), and an industry stays only when the demo's own copy or data names that domain, otherwise general (132 entries changed). Because the rules were written after seeing pass-1 errors, the holdout shows they generalise; it does not show the original values were good (industry was wrong on 28.8% of the random sample).

Why industry is hidden. Two reasons, neither a quality badge: it is unreliable (5 wrong of 73 on the holdout, upper bound ~14%) and it is uninformative. After the rule, 200 of 377 entries are general-only and 177 carry a specific industry (saas 107, fintech 39, devtools 29, media 14, e-commerce 12, agency 5, ai 4, travel 3, education 2, real-estate 2). A filter that returns "general" for more than half the catalog and saas for most of the rest does not narrow anything, and the specific values are the ones the audit found wrong. It stays pending until it is corrected and re-audited on a new seed.

Checks run

@mvilacad
mvilacad force-pushed the feat/569-feat-the-index-phase-1-facet branch from acfaa1f to 25330a8 Compare October 25, 2026 10:01
@mvilacad
mvilacad marked this pull request as ready for review October 25, 2026 10:01
@mvilacad
mvilacad merged commit b14695e into main Oct 25, 2026
@mvilacad
mvilacad deleted the feat/569-feat-the-index-phase-1-facet branch October 25, 2026 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: the index, phase 1: facet schema, 4-digit id ranges, backfill of every demo, index.json and .md twins

1 participant