From 9966def001171b300238a691785fca1a90b0c481 Mon Sep 17 00:00:00 2001 From: cmj Date: Thu, 17 Sep 2026 06:26:38 +0800 Subject: [PATCH] docs: correct four figures and one claim both guides had outgrown A sweep of every mechanically checkable number in CLAUDE.md and README.md against the repo. Five were wrong, all in the same direction -- written true, then overtaken by work that had no reason to revisit them. - description average 173 -> 204, and README's "kept near 160 characters" with it. #14 and #22 gave every description a negative boundary clause; that is the trade that bought fewer misfires, so the paragraph now names it rather than quoting a figure from before it. - README's "all 19 together cost a session under 800 tokens" -> about 1,000. Same cause: name + description is 4,063 characters. - allowed-tools grants containing a space, 70 -> 79, now stated as a share of the 173 total so the next drift is visible in the sentence itself. - The references/ example was Phase 2.5 in both guides, and README still said the machinery had "gone unused". Neither is true: what moved is Phase 5, into references/iteration.md, because Autonomous mode is the only mode that runs it. Phase 2.5 is named as the counter-example it actually is -- conditional, but eight lines, so a fetch costs more than it saves. Verified and left alone: 19 plugins and skills, 380 queries at 20 per plugin with a 10/10 label split and a 12/8 train/validation split, 13 eval cases at runs: 3, five verdict markers, the 320/500/300/100 thresholds, the model-tier roster, and spec-writer's 5000 words / 800 lines. No plugin changed, so nothing bumps -- neither behaviour nor catalogue copy. Co-Authored-By: Claude Opus 5 (1M context) --- CLAUDE.md | 36 +++++++++++++++++++++--------------- README.md | 26 ++++++++++++++------------ 2 files changed, 35 insertions(+), 27 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 20adf76..a7103fe 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -34,20 +34,23 @@ layers, and `scripts/check-skill-spec` enforces the split. Anything outside both which reject an unknown key outright rather than ignoring it. Three carry rules worth stating: - `description`: what the skill does **and when to use it**, third person. The repo ceiling is - 320 characters and its skills average 173, because the same string is catalogue copy in four - places: `/plugin`, `marketplace.json`, the plugin `README.md` blockquote and the skill's own - frontmatter. All four are compared; the blockquote is line-wrapped, so an edit by exact - string match reaches three of them and leaves the fourth stale. + 320 characters and its skills average 204 — up from 173 once every one of them gained a + negative boundary clause, which is the trade that bought fewer misfires. The same string is + catalogue copy in four places: `/plugin`, `marketplace.json`, the plugin `README.md` + blockquote and the skill's own frontmatter. All four are compared; the blockquote is + line-wrapped, so an edit by exact string match reaches three of them and leaves the fourth + stale. - `allowed-tools`: a **YAML list**, the one place this repo knowingly departs from the - standard's space-separated string. 70 grants contain a space (`Bash(git status:*)`), and the - space-free spelling `Bash(git:*)` widens the grant from one subcommand to all of git. Of the - two remaining spellings, a comma-separated string is split on spaces into patterns that - match nothing — a permission dropped in silence — while a list at worst reaches a consumer - that cannot use it as given. The field is still typed as a space-separated string and still - marked Experimental, so it is the one to leave. Note the claim this rests on has narrowed: - as of September 2026 tools are normalising a list into a string rather than rejecting it - loudly, which is the quiet handling the list was chosen to avoid. `CLAUDE_CODE_EXTENSIONS` - in `scripts/check-skill-spec` records what would make this decision worth revisiting. + standard's space-separated string. 79 of the 173 grants contain a space + (`Bash(git status:*)`), and the space-free spelling `Bash(git:*)` widens the grant from one + subcommand to all of git. Of the two remaining spellings, a comma-separated string is split + on spaces into patterns that match nothing — a permission dropped in silence — while a list + at worst reaches a consumer that cannot use it as given. The field is still typed as a + space-separated string and still marked Experimental, so it is the one to leave. Note the + claim this rests on has narrowed: as of September 2026 tools are normalising a list into a + string rather than rejecting it loudly, which is the quiet handling the list was chosen to + avoid. `CLAUDE_CODE_EXTENSIONS` in `scripts/check-skill-spec` records what would make this + decision worth revisiting. - `metadata`: string keys to string values only — hence `version: "2.0.0"`, quoted. **Claude Code extensions adopted here** — each costs portability, so each records its reason @@ -76,8 +79,11 @@ Whether a procedure moves is a second, orthogonal question: unconditional or con procedure every run needs is read anyway, one fetch later, so moving it costs a round trip and buys nothing. A procedure only some runs need is what `references/` is for — put it there and name the file where the condition is stated, because nothing in `references/` is read unless -the body says to read it. `agent-smith`'s Phase 2.5 is the example: it now runs only when the -plan touches a user-visible surface, so on most dispatches its detail is cost with no reader. +the body says to read it. `agent-smith`'s Phase 5 is the example, and so far the only one: it +runs in Autonomous mode alone, so its scoring table and iteration log sit in +`references/iteration.md` and a Partner-mode dispatch never pays for them. Phase 2.5 is +conditional too and still does not qualify — at eight lines, most of them the condition +itself, the fetch would cost more than the lines it saves. ### Skill Discovery (Three Tiers) diff --git a/README.md b/README.md index 9d50353..6dfed0c 100644 --- a/README.md +++ b/README.md @@ -225,23 +225,25 @@ runs need is what `references/` is for. Put it there, and name the file at the p condition is stated: loading is just-in-time, so nothing in `references/` is read unless the body says to read it. -`agent-smith`'s Phase 2.5 is the example. It now runs only when the plan changes a -user-visible surface, so on every dispatch that does not, its detail is context paid for with -no reader. That is the shape `references/` exists for, and the reason this repo's own -machinery for it — resolved links, a required `## Contents` past 100 lines, no nesting — has -so far gone unused. +`agent-smith`'s Phase 5 is the example. It runs in Autonomous mode only, so its scoring table +and iteration log sit in `references/iteration.md`, and a Partner-mode dispatch never pays for +them. That is the shape `references/` exists for, and this repo's own machinery for it — +resolved links, a required `## Contents` past 100 lines, no nesting — has exactly that one +user so far. Phase 2.5 is conditional too and still does not qualify: at eight lines, most of +them the condition itself, the fetch would cost more than the lines it saves. ### Trigger evals A skill only helps if it activates, and the `description` is the only thing Claude has at the moment it decides. It is also the only part of a skill loaded on every session, whether or -not the skill ever fires, so a description here is kept near 160 characters: long enough to -say what the skill does and when to use it, short enough that all 19 together cost a session -under 800 tokens. Every plugin carries `evals/trigger-queries.json`: 20 labelled prompts, ten -that should activate the skill and ten near-misses that share its vocabulary but need something -else. Split 60/40 into train and validation, so a reworded description is tuned on one half and -judged on the other rather than fitted to the phrasings used to tune it. `scripts/validate` -fails a plugin that carries no such file. +not the skill ever fires, so a description here stays well under the 320-character ceiling: +they average 204 — long enough to say what the skill does, when to use it and what it is +_not_ for, short enough that all 19 together cost a session about 1,000 tokens. Every plugin +carries `evals/trigger-queries.json`: 20 labelled prompts, ten that should activate the skill +and ten near-misses that share its vocabulary but need something else. Split 60/40 into train +and validation, so a reworded description is tuned on one half and judged on the other rather +than fitted to the phrasings used to tune it. `scripts/validate` fails a plugin that carries +no such file. The scrum agents carry one too. They used to be exempt on the grounds that `agent-smith` dispatches them rather than a prompt triggering them, and that each is the next one's nearest