Skip to content

feat(mdl-rules): load MDL rules per statement type instead of preloading (lazy-load PR 2a) - #571

Merged
MendixMau merged 3 commits into
masterfrom
claude/loving-gates-hauc1n-lazy-pr2
Oct 23, 2026
Merged

MendixMau merged 3 commits into
masterfrom
claude/loving-gates-hauc1n-lazy-pr2

Conversation

@MendixMau

Copy link
Copy Markdown
Owner

Direct lane

Classification: DISCUSS. The maintainer approved this as option C of the lazy-loading plan. PR 1 is #570.

Split. The whole of PR 2 changes about 630 hand-written lines plus 2,634 generated lines, so it is split per the brief:

  • 2a (this PR): the lookup script, the kind table, the generated files, the fixture and the CI check. No routing change.
  • 2b (branch claude/loving-gates-hauc1n-lazy-pr2b, stacked on this one, pushed, not opened yet): the routing demotion, the gate-check fix, the mdl stub line and its fixture cases.

The word counts and the A/B below cover both halves, because the A/B arm L is the 2b preload.

What changed and why. Today the mdl helper preloads every kind-specific MDL skill at Stage 5: 29,388 words. A page script needs none of the microflow rules, and a microflow script needs none of the page rules. This PR adds the per-statement-kind lookup.

  • bin/mdl-rules.sh <kind...> | --for <script.mdl> | --render | --check. It prints the always-needed core, then the sections that bin/lib/mdl-rule-kinds.tsv maps to each kind (page, microflow, access, workflow, entity, any). It ends with Rules for kinds K: STOP rows n/n listed. The denominator is counted by a second, independent pass over the STOP table. If the two counts disagree, the run fails.
  • Sections are located by file and heading text, never by line number (the runbook-span.sh precedent). The table covers:
    • the STOP rows, filtered by PR 1's Kind column
    • the kind-specific gotchas (per bullet)
    • learned-microflow-patterns.md
    • microflow-preflight.md
    • ui-preflight-pages.md
    • design-spacing.md §1–3
    • the detection-gaps register rows, by an ERE over the construct cell
    • security-is-not-a-later-script.md
    • retesting-learned-rules.md
  • --for reuses the existing classifier by calling project-bin/exec.sh --subjects. That is the same lexer exec.sh, affected-tests.sh and the change class use. There is no second copy, and exec.sh is not refactored.
  • It never prints empty. A missing heading, a bullet lead that matches nothing, an ERE that matches no row, or a STOP count mismatch each give exit 1, a message on stderr and nothing on stdout.
  • skills/generated/rules-<kind>.md hold the same bytes as bin/mdl-rules.sh <kind>, for Copilot, Cursor and Windsurf, which read files. They have a routing-exempt.tsv line: they are rendered output, not routed one by one.
  • --check runs in CI (new step next to render-routing.sh --check). It checks the denominators: every STOP row has a known Kind, every gotcha bullet and every register row reaches at least one kind, and the generated files match a fresh render. A new gotcha or register row that no kind selects fails CI instead of silently not reaching the mdl helper.
  • tests/behaviour/build-packs.sh --rules "<kinds>" expands the script in a pack to its output for those kinds. That is how the dispatcher will deliver it in PR 3.
  • No skill text changes. Bash 3.2 and POSIX awk. It makes no mxcli or mxbuild calls.
Row Answer
Process 2a: nothing new blocks a project; CI gains bin/mdl-rules.sh --check. 2b: the mdl stub gains workflow step 4a: run bin/mdl-rules.sh (or read rules-<kind>.md) before drafting, and report "STOP rows for kinds K: n/n checked" in PREFLIGHT.
Dependencies It reads project-bin/exec.sh --subjects, and its output is unchanged. Producer: a toolkit checkout; the script needs no project. 2b touches skill-routing.tsv/.sh, gate-check.sh and agents/mdl-agent.md.
Context cost 2a: none (the generated files are exempt and not routed); baseline stays at 72,891. With 2b (render-routing.sh --check, LC_ALL=C): baseline 72,891 → 63,877 (−9,014, the three mdl-only files go on-demand). mdl Stage 5 29,388 → 14,600 words / 14 → 7 files. The 14,600 counts the 3,347-word core of mdl-rules.sh, like the runbook slice. Per task the helper adds its kinds on top: page 10,223, microflow 10,186, access 4,730, entity 5,183, workflow 3,732 (each includes the core). So a page task reads about 21.5k words instead of 29.4k. This is above the plan's 12–13k estimate because the core keeps Step 0, the 3 any STOP rows, the gotchas, the final self-check, the detection-gaps ladder and operating rules, and retesting-learned-rules.md whole. Other roles do not change.
Behaviour A/B below. The decision rule as written is not met: there are three single-rep losses, reported, not averaged away.
Runtime mdl-rules.sh page takes about 0.1 s; --check takes about 1 s.

Gate / sync fix (in 2b)

gate-check.sh build-ready item 2 grepped CLAUDE.local.md for ui-preflight-pages. Once that row is on-demand, a re-synced baseline block no longer names it, so item 2 would flip to FAIL. 2b accepts ui-preflight-pages or bin/mdl-rules.sh, which is the new baseline row.

The sync-project.sh "§2c" check the brief mentions no longer exists. §2d now renders the routing block instead of grepping for it (see the T9 note in test-bug12-sync.sh), so there is nothing to change in sync. The only check is the gate's.

Fixture test-mdl-rules.sh T7 (2b) covers:

  • a freshly synced project, single-tree and two-tree (.mpr under app/): item 2 passes, and the rendered block holds bin/mdl-rules.sh and no ui-preflight-pages
  • an old, unsynced table: still passes
  • a file with neither name: still fails

Fixtures (scoped runs only; full suites not run)

  • test-mdl-rules.sh bin/mdl-rules.sh: 26/26 on 2a, 30/30 on 2b. It covers:
    • each kind (own sections, none of another kind's, n/n, byte-equal to the generated file, core present)
    • --for on the real mixed page+entity repro pipelines/mendix-mpr/samples/.../D2-page-repeater-content.mdl
    • --for on real workflow, access, microflow and scheduled-event scripts
    • an unknown kind (exit 2, empty stdout)
    • a renamed heading and a dead bullet lead (exit 1, empty stdout, named on stderr)
    • --check drift, an unmapped new gotcha, an unmapped register row and an unknown STOP Kind (all on a temp copy)
  • Golden input is the real skills and the repo's real .mdl files; nothing is hand-written.
  • Run on 2b: test-mdl-scan 52/0, test-baseline-pack 22/0, test-render-routing 5/0, test-routing-tier 26/0, test-runbook-slice 31/0 (T5 now counts mdl-rules.sh as its core output), test-bug03-gates 47/0, test-source-sufficiency-gate 7/0, test-source-ledger 55/0, test-bug12-sync 62/0, test-authoring-audit 5/0.
  • render-routing.sh --check is clean. The authoring-audit.py routing advisory is unchanged (top-1 49/55, top-3 55/55). check-portability and check-scripts are clean.

Behaviour A/B (tests/behaviour/RUNBOOK.md)

  • Arms. M = origin/master @76e1610, pack 33,296 words. L = 2b head @785bef2: the mdl Stage-5 preload, with bin/mdl-rules.sh expanded to its output for the task's kinds via the new build-packs.sh --rules mode. L packs are 19.9k–26.7k words.
  • Kinds per task:
    • page task: page, access
    • mf task: microflow, access
    • trap 12: page
    • traps 26, 1b: page, access
    • traps 33, 29: microflow, access
    • trap 16: access
    • trap 18: workflow, microflow, access
  • Setup. Fresh claude -p agents with only Read and Write, model claude-opus-5-5, run from an empty directory with no CLAUDE.md, arms interleaved.
  • Reps. 5 page + 5 mf per arm (3 planned, plus 2 more to separate noise from loss), 1 per trap per arm, and 2 for trap 18.
  • Traps. STOP 4 is planted in the page task and STOP 21 in the mf task (score.sh planted). PR 1's traps cover 12, 26, 1b, 33 and 29. New traps: 16 (access XPath with a path inside [%CurrentUser%]) and 18 (callee microflow and workflow in one script).
Metric M L
read_all 30/30 30/30
page: sections, report, scan, planted (STOP 4), crumbs, txtcrumb, h1, empty, folder equal in every rep equal
page: quoted (median) 100 (62–100) 100 (all 100)
page: uixref 5/5 4/5 — LOSS (L-page-r3 posted Step 5 as NOT RUN lines but no "UI cross-reference" block)
mf: sections, report, scan, planted (STOP 21), cinloop, rinloop, pos, vfb, folder equal equal
mf: quoted (median) 90 100
mf: mfpre 5/5 4/5 — LOSS (L-mf-r3 posted both checklists with tiers but wrote microflow-preflight — where the scorer wants microflow-preflight:)
traps 12, 26, 1b, 33, 29, 16: planted / flagged 1/1 each 1/1 each
trap 18: planted (mechanical) 2/2 0/2 — LOSS by the scorer
tokens (mean page / mf) 323k / 311k 314k / 272k

About trap 18. Both L reps did split into two scripts (scripts: 2, "FILE 1 of 2 / FILE 2 of 2", STOP #18 cited). They wrote both files inside the single ```mdl fence the task format demands, and the ad-hoc scorer counts per fence. Both M reps left the workflow undrafted. Read by hand, both arms obey STOP 18. Read mechanically, L loses.

Verdict. Under the decision rule ("no worse on every metric, count at the good value"), L is not shown no-worse. There are three losses, each a single rep, and two of them are output-format misses where the content was present. This goes to the maintainer to decide.

Outputs and scores are in the builder's session scratch. They can be reproduced with build-packs.sh and the PR 1 trap tasks plus the two new ones.

Field evidence. None. A field run is owed: this was not run on a real project. It is planned with PR 3, the dispatcher that hands mdl-rules.sh output to the helper.

  • No client data anywhere in the diff
  • Size cap: 2a is about 430 hand-written lines plus 2,634 generated lines (skills/generated/, rendered output) — split 2a/2b per the brief
  • Test tier: scoped wave2 fixtures (listed), plus the behaviour A/B
  • Golden input captured (real skills, real .mdl), both layouts (T7 in 2b); Bash 3.2-safe constructs and Git Bash considered (no .exe probing needed, only bash and awk), but not run on macOS or Windows
  • CHANGELOG.md line in this PR (and one in 2b)

Open questions for the maintainer

  • This repo's CLAUDE.md still names learned-microflow-patterns.md in the "always-on set". After 2b it is on-demand. That one-line edit was left to the maintainer.
  • bin/init-project.sh item 5 and project-bin/context-pack.sh still point at the skill files directly. They are lead/dispatcher surfaces and left for PR 3.

Review: pending independent review


Generated by Claude Code

bin/mdl-rules.sh <kind...> | --for <script.mdl> | --render | --check prints the
always-needed core plus the sections bin/lib/mdl-rule-kinds.tsv maps to each
statement kind (page, microflow, access, workflow, entity, any), located by
file + heading text, and closes with "Rules for kinds K: STOP rows n/n listed".
--for reuses exec.sh --subjects. skills/generated/rules-<kind>.md are the
rendered, drift-checked plain-read copies; CI runs --check. No routing change:
lazy-load PR 2b demotes the mdl preloads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
…Es, unselected sections

- F1: `--for` with a failing classifier exited 0 and printed the core alone (kinds_for's exit 1
  ran in a subshell inside a heredoc). It now runs first, `|| exit 1`. Fixture T7.
- F2: a register ERE matching no row hid behind its kind's union. Each ERE is now checked on its
  own, in the lookup and in --check. Fixture T8.
- F3: a new `##` section no kind selects passed --check. Every `##` section of a file the table
  reads in parts must be selected or named by an `exclude` row with a reason (seeded:
  design-spacing.md §4 Review-side, §Related); a stale exclude fails too. Fixture T9.
- --for maps exec.sh's `action` and `mapping` subjects to microflow and demo-user heads to
  access (T10, on fixtures/import-mapping-range-repro/maptest.mdl plus minimal scripts).
- Headings inside code fences are not headings (T11); `page page` is labelled once (T11).

Generated files unchanged; baseline 72,891 words.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
… the sections after it

Re-review finding A on #571: an odd number of fence lines in a file the
table reads silently swallowed every later heading, so a new rule section
after it reached no kind while --check passed. load() and the coverage
pass now report "unclosed code fence". Fixture T12 (fails on e7529c5,
passes now); test-mdl-rules 39/0; --check clean on the real tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
@MendixMau
MendixMau merged commit 87676f8 into master Oct 23, 2026
2 checks passed
MendixMau pushed a commit that referenced this pull request Oct 23, 2026
Conflicts: bin/mdl-rules.sh and bin/lib/mdl-rule-kinds.tsv take master's
(reviewed) versions, which 2b never changed; 2b's gate fixture moves to
T13 after master's T7-T12; CHANGELOG keeps both lines.
test-mdl-rules 43/0, mdl-rules --check clean, baseline 63,877.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants