Repository navigation
process(routing, mdl-agent): mdl helper loads MDL rules per statement type at Stage 5 (lazy-load PR 2b) - #572
Merged
Conversation
bin/mdl-rules.sh <kind...> | --for <script.mdl> | --render | --check prints the always-needed core plus the sections bin/lib/mdl-rule-kinds.tsv maps to each statement kind (page, microflow, access, workflow, entity, any), located by file + heading text, and closes with "Rules for kinds K: STOP rows n/n listed". --for reuses exec.sh --subjects. skills/generated/rules-<kind>.md are the rendered, drift-checked plain-read copies; CI runs --check. No routing change: lazy-load PR 2b demotes the mdl preloads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
…loaded (lazy-load PR 2b) Demote learned-microflow-patterns, microflow-preflight and ui-preflight-pages to on-demand; learned-mdl-preflight, design-spacing, learned-detection-gaps, security-is-not-a-later-script and retesting-learned-rules to on-demand for mdl only (mdl?). New baseline row bin/mdl-rules.sh, counted as its core output. mdl stub gains step 4a + report line "STOP rows for kinds K: n/n checked". gate-check build-ready item 2 accepts bin/mdl-rules.sh, so synced projects do not flip to FAIL (test-mdl-rules.sh T7, both layouts). Baseline 72,891 -> 63,877 words; mdl Stage 5 29,388 -> 14,600. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
Conflicts: bin/mdl-rules.sh and bin/lib/mdl-rule-kinds.tsv take master's (reviewed) versions, which 2b never changed; 2b's gate fixture moves to T13 after master's T7-T12; CHANGELOG keeps both lines. test-mdl-rules 43/0, mdl-rules --check clean, baseline 63,877. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
…flight The lazy-load 2b A/B scored L-mf-r3 mfpre=0 although it posted two full checklist lines written 'microflow-preflight — Sales."…" — tier: Guided'. The pattern now takes ':', an em/en dash or ' - ' after the name. Re-scored all 20 page/mf outputs of that run: the only change is L-mf-r3 0 -> 1; no M score moves; the no-pack fixture N-mf-r1 still scores 0. L-mf-r3 is added as a captured fixture with its own assertion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
tests/behaviour/results/2026-10-23-lazy-pr2b.tsv: M (master pack) vs L (2b pack + bin/mdl-rules.sh output), 15 page reps per arm, 5 mf, 8 trap kinds. L holds: page uixref L 14/15 vs M 11/15; every other metric equal or better except crumbs at 3/3 (M 2/15, L 1/15, same median 2/3). CLAUDE.md: the always-on set no longer lists learned-microflow-patterns.md, which 2b routes on demand; the mdl helper runs bin/mdl-rules.sh instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
…les in context packs, Stage 6 - agents/mdl-agent.md: step 4a's --for run and the report line name <toolkit>/bin/mdl-rules.sh (a project's bin/ does not carry it). - project-bin/context-pack.sh: the "Rules" list names `<tk>bin/mdl-rules.sh <kinds>` with the step's kinds (page / microflow / entity); microflow-preflight, security-is-not-a-later-script, design-spacing, learned-detection-gaps and retesting-learned-rules now reach a pack-fed helper only through it. Fixture asserts a page step and a logic step. - skill-routing.tsv: mdl-rules row stages 5,6, so Stage 6 mdl keeps detection-gaps and retesting (mdl Stage 6 6,129 -> 9,476 words, below 10,093). Re-rendered README.md / ROUTING.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
Re-review of #572: the preload figure read as a flat 14.8k saving; give the measured per-script range and what the A/B covered. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Direct lane
Classification: DISCUSS. The maintainer approved this as option C of the lazy-load plan (decision #16 = B: run more page reps before opening).
What changed and why (one paragraph):
At Stage 5 the mdl helper used to preload 14 baseline files (29,388 routed words), whatever the script wrote. Now it gets a smaller core and runs
bin/mdl-rules.sh <kinds>(added in PR 2a, #571) to load the rules for the statement types each script writes. The STOP rows, gotchas and preflight sections still reach it in full, but only for the types the script writes. Three files move to on demand:learned-microflow-patterns.md,microflow-preflight.mdandui-preflight-pages.md. Five files go on demand for the mdl role only (mdl?) and stay as they were for every other role:learned-mdl-preflight.md,design-spacing.md,learned-detection-gaps.md,security-is-not-a-later-script.mdandretesting-learned-rules.md. A new baseline rowmdl-rules(Stages 5 and 6) takes their place. The mdl stub gains step 4a and the report line "STOP rows for kinds K: n/n checked". No skill text changed.Also in this PR:
ui-preflight-pages.md, so a project synced after this change would have flipped to FAIL.bin/gate-check.shnow acceptsbin/mdl-rules.shas well asui-preflight-pages. A project synced before the change still passes on its old table. Fixture:tests/wave2/test-mdl-rules.shT13 covers both layouts.tests/behaviour/score.shmfprenow accepts:, an em/en dash or-aftermicroflow-preflight. Captured fixturetests/behaviour/fixtures/L-mf-r3.md, asserted intests/wave2/test-behaviour-score.sh.learned-microflow-patterns.md.tests/behaviour/results/2026-10-23-lazy-pr2b.tsv.agents/mdl-agent.mdstep 4a and the report line name<toolkit>/bin/mdl-rules.sh --for(a consuming project'sbin/does not carry it).project-bin/context-pack.sh"Rules" list now namesbin/mdl-rules.sh <kinds>under the toolkit root (or<toolkit>/when the root is not found) with the step's kinds (page / logic → microflow / domain → entity), read ALL of it. After this PR,microflow-preflight,security-is-not-a-later-script,design-spacing,learned-detection-gapsandretesting-learned-rulesreach a pack-fed helper only throughmdl-rules.tests/wave2/test-context-pack.shasserts it for a page step and a logic step.mdl-rulesrouting row is stage5,6, so Stage 6 mdl keepsdetection-gapsandretesting: mdl Stage 6 6,129 → 9,476 words (below 10,093).Impact
bin/mdl-rules.shper script (stub step 4a); a pack-fed helper is told to by the context pack. Gate-check item 2 accepts either routing name, so no verdict flips to FAIL.bin/lib/skill-routing.tsv,bin/lib/skill-routing.shand their re-render (README.md, ROUTING.md),agents/mdl-agent.md,bin/gate-check.sh,project-bin/context-pack.sh. Depends on PR 2a (#571, merged). New artifact read:bin/mdl-rules.shoutput, produced by the mandatory mdl stub step 4a.mdl-rules.shcounted, like the runbook slice). Per script the helper then readsmdl-rulesoutput on top, measured by kind: page only ≈21.4k words in total (≈8k saved), microflow+access ≈22.5k (≈6.9k saved), page+microflow ≈28.1k (≈1.3k saved), all kinds ≈29.5k (≈0 saved). So the per-script saving is ≈0–8k words depending on the kinds a script writes, not the full 14.8k. mdl Stage 6: 9,476 words.mdl-rules.shoutput. Verdict: L no worse than M except crumbs at 3/3 (M 2/15, L 1/15, one rep, same median 2/3; reported per RUNBOOK:67-68). Table below.mdl-rules.shruns in under a second per call.A/B (
tests/behaviour/RUNBOOK.md)Setup
claude -pagent per rep, with Read/Write only, started from an empty cwd with no CLAUDE.md above it.mdl-rules.shoutput for the task's statement types.read_all=1).What the A/B tested: the L arm was handed mdl-rules output with kinds pre-selected, so the A/B shows the rules suffice when read; it does not test that the agent runs step 4a on its own — the context-pack line (this PR) and PR 3 cover delivery.
Reproducibility: the trap tasks and the trap-18 scorer are not committed, so trap rows are not reproducible from the repo.
Runs
uixref4/5 against M 5/5.Re-scoring run 1 with the fixed scorers
mfpre: only L-mf-r3 changed, 0→1. Its two checklist lines are writtenmicroflow-preflight — … — tier: Guided.-- FILE n of 2:headers inside the single fenced block the task format requires. The trap scorer now counts per script, meaning a fenced block or one of those headers. It is the session's own scorer and is not committed.Page, 15 reps per arm (median; count of reps at the good value)
The
uixrefmisses look the same in both arms: the Step 5 checks are posted as NOT RUN, with no named "UI cross-reference" block.Finding, reported as the decision rule requires: crumbs reached 3/3 in 2/15 M reps against 1/15 L reps. The median is 2/3 in both arms, so this is a one-rep difference.
Microflow, 5 reps per arm
Planted traps (planted / flagged)
Field evidence: owed. This PR has only the A/B; a field run on a real project comes with PR 3.
test-mdl-rules43/0,test-context-pack59/0,test-baseline-pack22/0,test-routing-tier26/0,test-render-routing5/0,test-runbook-slice31/0,test-bug12-sync62/0,test-bug03-gates47/0;render-routing.sh --check,mdl-rules.sh --check,check-portability.shclean. Earlier: themfpreregex was replayed over the 4 committed fixtures (no change; N-mf-r1 stays 0) and over all 20 run-1 page/mf outputs (only L-mf-r3 changed);score.shon the new fixture printsmfpre=1. Behaviour A/B run (above).CHANGELOG.mdline appended (updated for the context pack, Stage 6 and the per-script saving)Review: MERGE-AFTER-FIX → fixed (two review rounds; remaining findings were text only, fixed in 4fa05df)
Generated by Claude Code