docs(catalog): a home for a project's gitignored data (HF trial) + template prompt - #5
Conversation
…t in the CLAUDE.md template
Every data project gitignores data/{bronze,silver,gold}, but the kit never said where that
data lives or how to rebuild it. Evaluated HF dataset repos, HF Storage Buckets, DVC and
git-annex + rclone against the current practice ($SILVER on an own server, regenerable
silver). Verdict: trial — no project in the stack uses HF yet.
Not wired as a default: the template gains a "Where it lives" fill prompt listing the
options, so each new data project states where its data lives and never publishes
third-party or personal data.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Reviewer's GuideThis documentation-only change records a research-setup trial for storing gitignored data and adds a rendered CLAUDE.md prompt so each scaffolded data project documents its storage location, rebuild path, and privacy considerations. HF remains an optional, explicitly private-by-default workflow rather than a scaffold default. Flow diagram for documenting gitignored data storageflowchart LR
Layers[Gitignored data layers<br/>bronze, silver, gold] --> Prompt[CLAUDE.md Data Conventions<br/>Where it lives]
Prompt --> Choice{Choose storage and rebuild path}
Choice --> Regenerate[Regenerate from source]
Choice --> Silver[Remote silver via $SILVER]
Choice --> Bucket[Private HF Storage Bucket]
Choice --> Dataset[HF dataset repo<br/>card and license]
Prompt --> Privacy[Never publish third-party<br/>or personal data]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
| <!-- Adjust/remove if this is not a data project. Defaults reflect the medallion + polars/duckdb stack. --> | ||
|
|
||
| - **Layers**: raw → `data/bronze/`, cleaned → `data/silver/`, analytic → `data/gold/`. All gitignored. | ||
| - **Where it lives**: <!-- fill: gitignored data still needs a home — say where each layer lives and how |
There was a problem hiding this comment.
This changes a template shipped by the plugin but doesn't bump .claude-plugin/plugin.json version (still 0.1.0). /project-kit:scaffold runs $CLAUDE_PLUGIN_ROOT/scripts/scaffold.sh, which reads templates from the cached project-kit/0.1.0/ copy.
After merge, projects scaffolded through the plugin still get the old CLAUDE.md with no "Where it lives" prompt, because the cache is keyed on the unchanged version. The PR's claimed effect only shows up when running the script straight from the checkout. PR #8's own contract requires a version bump in the same PR.
Automated review (Claude Code, AI-generated). Validate before acting.
There was a problem hiding this comment.
Fixed in 8dc3f37: plugin bumped to 0.2.0. (Claude Opus 5.5, on behalf of Rody)
| repo (card + license) for data we publish and a **private** HF Storage Bucket for working data; DVC | ||
| or git-annex only if a project needs dataset versions pinned to commits. Never publish third-party or | ||
| personal data (course material, class recordings, PII). Revisit after one project ships. | ||
| - **Wiring**: not a default. `templates/CLAUDE.md.tmpl` gains a "Where it lives" fill prompt in Data |
There was a problem hiding this comment.
Convention break: a trial verdict gets wired into the default template. The catalog format says "Wiring (if adopt)", research-setup says "If adopt, offer wiring", and the SQLite precedent says trial means "per-project only, never in the default template".
The rule in CLAUDE.md is: "templates/ ... are kept faithful to real repos (tesis_redes, exp_tesis, infelix) — reuse, don't invent." The new template bullet recommends HF buckets and dataset repos, which this entry admits "no project in the stack uses ... yet". So every new data project gets an unproven option in its default CLAUDE.md. Keep the prompt to the proven options (regenerate / $SILVER) until HF is adopted.
Automated review (Claude Code, AI-generated). Validate before acting.
There was a problem hiding this comment.
Kept on purpose (Rody's call). 8dc3f37 records it in the entry as a deliberate exception: the prompt asks the question and installs no HF tooling. (Claude Opus 5.5, on behalf of Rody)
| <!-- Adjust/remove if this is not a data project. Defaults reflect the medallion + polars/duckdb stack. --> | ||
|
|
||
| - **Layers**: raw → `data/bronze/`, cleaned → `data/silver/`, analytic → `data/gold/`. All gitignored. | ||
| - **Where it lives**: <!-- fill: gitignored data still needs a home — say where each layer lives and how |
There was a problem hiding this comment.
The new prompt repeats the $SILVER option that the existing Remote silver bullet (L39) already covers. It also points to catalog/setup-options.md in project-kit, a path that doesn't exist in the scaffolded repo.
The rendered CLAUDE.md now describes remote silver twice in one section, so the two can drift, and the bare relative path can't be resolved from the consumer repo. Fold "Where it lives" into the Remote silver bullet, or drop the $SILVER item, and link the catalog by URL.
Automated review (Claude Code, AI-generated). Validate before acting.
There was a problem hiding this comment.
Fixed in 8dc3f37: it now says "remote silver (see below)" and points to the catalog by URL. (Claude Opus 5.5, on behalf of Rody)
The template change must reach the plugin cache, which is keyed on the plugin version. Drop the duplicate $SILVER mention and point to the catalog by URL, since the scaffolded repo has no catalog/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 participant
Summary
Every data project scaffolded by the kit gitignores
data/{bronze,silver,gold}, but nothing says where that data lives or how to rebuild it. This records aresearch-setupevaluation in the catalog and adds a fill prompt to theCLAUDE.mdtemplate, so each new project answers that question. HF is not made a default.Changes
catalog/setup-options.md: new entry, "a home for a project's gitignored data". Candidates:--privateis passed.$SILVERon an own server (tesis_redes) and regenerable silver (infelix).Verdict: trial, since no project in the stack uses HF yet.
templates/CLAUDE.md.tmpl: a "Where it lives" fill prompt in Data Conventions. It lists regenerate-from-source,$SILVER, a private HF bucket, and an HF dataset repo (card + license), and warns never to publish third-party or personal data.Kept as a prompt rather than a default, to respect the kit's rule that templates stay faithful to real repos.
Testing
bash -n scripts/scaffold.shandscaffold.sh … --dry-run: OK. No script changes.--no-github --no-mcp). The renderedCLAUDE.mdshows the new prompt under Data Conventions.Related
Comes from the backup and data-storage split discussed for chrome-helper (R0SEWT/chrome-helper#10). No bead: I didn't touch this repo's beads DB because the main checkout has an uncommitted
.beads/issues.jsonlchange.🤖 Generated with Claude Code
Summary by Sourcery
Document where gitignored project data should live and prompt new projects to record its storage and regeneration approach.
Enhancements:
Documentation:
Tests:
Chores: