fix(bug-lookup): also search the project's own bug-logs drafts

CHANGELOG.md

@@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m
 
 ## Unreleased
 
+- fix(bug-lookup): **`bin/bug-lookup.sh` also searches the project's own `bug-logs/**.md` drafts, marked `(project draft: <file>:<line>)`, after the ledger's output.** It searched only the toolkit ledger, so a bug a session had already written down in the project was invisible to the lookup run before the next experiment on the same error; the ledger's output and ranking are unchanged (drafts are ranked in a separate pass), `--full` and `--index` stay ledger-only, and `MXTK_PROJECT_DIR=` turns drafts off. Field: a Mendix 10.24 app with mxcli v0.27.0, 43 files in `bug-logs/`; the query "cannot be spliced" listed 0 project drafts at 393d253 and lists 3 (3 distinct files) now, each pointing at a `## ` line; the project was not written. Pinned by group P of `tests/wave2/test-bug-lookup.sh` (3 of its checks fail on 393d253) on synthetic drafts; the run under /bin/bash 3.2.57 passes, other awks not run. — saingge
 - fix(progress-report, build-plan-status): **HARNESS rows are no longer counted as built rows, and a module's `verify-module.sh` row shows once, as that module's sign-off status on its phase card, instead of as a row.** A HARNESS step is a module's sign-off, not a build, so it is out of every "x of y built" count: the phase cards, the plan rollup and the Stage 5 card in `progress-report.html`, and the lede of `build-plan-status.sh --html` ("N build rows", plus a "Not counted: N HARNESS rows" note). The verify row's evidence is read exactly as #501 reads it (`harness_verify`'s rule, now `verify_evidence` + `module_status` in `progress-report.py`; nothing new measured, no extra script run, no mxcli or mxbuild call) and shown as one line per module: `Verified ✓ <date> (LOOK n/n)`, `Verify owed - LOOK + CONFIRM`, `Verify stale (N builds since)` (or `(a script was marked done- after it)`), `Verify not run`, `built? no verify summary`, `Verify not clean <date> (<why>)` for a summary with a FAULT, FINDING or SKIPPED row, or `Verify clean <date>, LOOK n/n - held: State says <State>` when State is blocked or pending a person (held is not signed off, as #501 never promoted it); a State cell that disagrees is flagged under it, the rollup says how many modules are verified, and the Stage 5 card, KPI and owed text add ", k of n modules signed off". The Phase state badge is read from the build rows only (a phase of HARNESS rows alone says "sign-off only" and its card "Sign-off phase: no build rows"). A module with no verify row shows no status. Other HARNESS rows (a coherence pass, a harness run) stay in the row list, marked "(not counted)". `build-plan.md`, `build-plan.json` and the text output are unchanged, so agents still see every HARNESS step. **Behaviour change to check in your project:** the progress report's built counts drop the HARNESS rows (a plan of 10 rows with 2 HARNESS reads "x of 8"), and the verify rows leave the row tables. Fixture: `test-progress-harness-verify.sh` (H1-H10, N1-N6, both layouts). — MendixMau (decision 2026-10-21)
 - fix(exec): **a script restated from a stale DESCRIBE now WARNs instead of silently reverting an earlier exec** (#261). A script drafted from a DESCRIBE taken before an earlier script of the batch ran put four old messages back, with check, mxbuild and lint green. New `project-bin/describe-baseline.sh stamp` writes `-- baseline: <Doc> sha256=<hash>` (plus `-- baseline-mxcli: <version>`) from a DESCRIBE at draft time; `exec.sh` runs `verify` only for a script that carries the header, so other scripts cost one grep and no mxcli call. It reports DRIFT with the first lines of the diff, STALEHEADER, and RESTAMP after an mxcli upgrade (DESCRIBE text may change, so that is not called drift). A re-run by the script's own last recorded write (the #269 ledger) and a fresh model are not drift. Nothing is refused and the exit codes do not change. Preflight STOP 21 gains one clause (`-- supersedes:`, `stamp`, exec.sh only warns) and its trigger names `create or modify` (+24 words). Not-found DESCRIBE and `--version` output are captured from mxcli v0.26.0. Lands after the #269 PR; with it, supersedes draft #311. **Behaviour change to check in your project:** `exec.sh` output can carry `WARN: stale draft: …` lines, and the BUILD-LOG detail cell `WARN stale baseline: <docs>`; `MXTK_RESTATE_GUARD=off` skips the check. — AlvinnYeo (#261)
 - fix(exec): **re-running an older `mdlsource/` script over a newer owner's live work now WARNs instead of passing silently** (#269). Three field apps had 99/579, 130/357 and 38/159 documents created by more than one script, and a replay of an older one reverted the newer with exit 0 and every gate green. New `project-bin/script-owners.sh` maps each microflow, nanoflow, page and snippet to the scripts that create or restate it, in run order (digit runs compared as numbers, so `2-core/` runs before `11-fixes/`; `done-` ignored), and reads `-- supersedes:` / `-- superseded-by:` headers. `exec.sh` records every kept write in a machine-local last-writer ledger (`.claude/loop/script-owners/`, self-gitignored, compacted per write); `restore-mpr.sh` marks it on a manual restore. A replay WARNs only when the newer owner's write is live: ledger first, else one DESCRIBE per document, at most 5 per exec (the rest print `CONFLICT (not checked)`). A new owner without a supersedes header gets one `NOTE:` line on its first run, kept out of BUILD-LOG. Nothing is refused and the exit codes do not change. With no co-owner, `exec.sh` runs one grep over `mdlsource/` and no mxcli call. `gate-check.sh` prints the co-owned count when it is above 0. Part of the rebuilt draft #311 (with the #261 PR). **Behaviour change to check in your project:** `exec.sh` output can carry `WARN: restatement: …` and `NOTE: new owner of …` lines; the BUILD-LOG detail cell can carry `WARN reverts: <docs>`. Set `MXTK_RESTATE_GUARD=off` to skip the check. — MendixMau (#269)

bin/bug-lookup.sh

@@ -7,7 +7,7 @@
 #   bin/bug-lookup.sh '<pasted error message>'    # ranked: top 3 — header, score, one excerpt line
 #   bin/bug-lookup.sh --full 2 '<same message>'   # the 2nd ranked entry, in full
 #   bin/bug-lookup.sh --exact "widget name"       # literal phrase match, case-insensitive
-#   bin/bug-lookup.sh --index                     # one line per entry: id, title, status flags
+#   bin/bug-lookup.sh --index                     # one line per entry: id, title, status flags (ledger only)
 #
 # WHY. bug-logs/mxcli-bugs.md is far too large to read on every session (`wc -w` it for the
 # current size; a number written here goes stale). Nobody needs all of it; they need the entry
@@ -20,8 +20,18 @@
 # term in the header x1.5, an exact BUG id in the header +100. "No confident match" is printed
 # when the best entry covers < MXTK_BUG_MINCOV (default 0.40) of the query's term weight, or
 # when the query's CE code is in no entry. Measured on tests/wave2/fixtures/bug-lookup/.
-# Exit: 0 entries printed (the "no confident match" banner too: it still lists the nearest 3),
-# 1 nothing matches or no query, 2 no ledger or a bad --full rank.
+# PROJECT DRAFTS. Run from inside a wired project, it also searches that project's own
+# bug-logs/**.md: the drafts a session writes for a bug it hit, one `## ` entry each, which the
+# ledger does not carry until someone merges them. Draft hits come AFTER the ledger's output,
+# under a "Project drafts" line, each marked `(project draft: <file>:<line>)`. The ledger's
+# output and ranking are the same with or without drafts (they are ranked in a separate pass,
+# so they never move the ledger's IDF). A ranked draft is shown only when it covers 40% of the
+# query's term weight; a ledger hit below that is shown under a banner, a draft is dropped.
+# The project is found as every toolkit script finds it (project-bin/_common.sh, mxtk_walk_root
+# from $PWD); MXTK_PROJECT_DIR=<dir> names it, MXTK_PROJECT_DIR= (empty) turns drafts off.
+# --full N and --index are ledger-only.
+# Exit: 0 entries printed (the "no confident match" banner too: it still lists the nearest 3;
+# a draft hit alone counts), 1 nothing matches or no query, 2 no ledger or a bad --full rank.
 # Bash 3.2 + POSIX awk (mawk, gawk, BWK awk); no gawk-isms.
 set -eu
 ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
@@ -48,13 +58,41 @@ done
 Q="$*"
 [ -n "$Q" ] || usage
 
+# --- The project's own drafts: bug-logs/**.md beside the project, never the ledger itself ---
+DRAFTS=()
+if [ -n "${MXTK_PROJECT_DIR+x}" ]; then PD="$MXTK_PROJECT_DIR"
+else PD="$( . "$ROOT/project-bin/_common.sh" >/dev/null 2>&1; mxtk_walk_root "$PWD" 2>/dev/null )" || PD=""; fi
+if [ -n "$PD" ] && [ -d "$PD/bug-logs" ]; then
+  while IFS= read -r _f; do [ "$_f" = "$LOG" ] || DRAFTS+=("$_f"); done \
+    < <(find "$PD/bug-logs" -name '*.md' -type f | sort)
+fi
+# draft_word: the draft entries that contain "$Q" (literal, ASCII case-insensitive), in full up
+# to 6 of them, headers only above. Prints nothing and returns 1 when none match.
+draft_word() {
+  [ "${#DRAFTS[@]}" -gt 0 ] || return 1
+  BUGQ="$Q" LC_ALL=C awk -v PD="$PD/" '
+  function flush(   i) { if (cur && index(tolower(body), q)) { n++; H[n] = hdr; W[n] = where; B[n] = body } cur = 0 }
+  BEGIN { q = tolower(ENVIRON["BUGQ"]) }
+  FNR == 1 { flush() }
+  /^## / { flush(); if ($0 ~ /^## Provenance/) next
+           cur = 1; hdr = $0; body = $0 "\n"; f = FILENAME
+           if (index(f, PD) == 1) f = substr(f, length(PD) + 1)
+           where = "(project draft: " f ":" FNR ")"; next }
+  cur { body = body $0 "\n" }
+  END { flush(); if (!n) exit 1
+        if (n > 6) { print "bug-lookup: project drafts: " n " entries mention \"" ENVIRON["BUGQ"] "\" — headers only:"
+                     for (i = 1; i <= n; i++) print "  " substr(H[i], 4) "  " W[i]; exit 0 }
+        print "bug-lookup: project drafts (" n "):"
+        for (i = 1; i <= n; i++) { print W[i]; printf "%s\n", B[i] } }' "${DRAFTS[@]}"
+}
+
 # --- Ranked path: several words, or --full ------------------------------------------------
 case "$Q" in *[[:space:]]*) MULTI=1 ;; *) MULTI=0 ;; esac
 if [ -n "$FULL" ] || { [ "$MULTI" = 1 ] && [ "$EXACT" = 0 ]; }; then
   TOP="${MXTK_BUG_TOP:-3}"; [ -n "$FULL" ] && [ "$FULL" -gt "$TOP" ] && TOP="$FULL"
   # The query goes in through ENVIRON, not -v: -v expands backslash escapes (a pasted Windows
   # path would change). LC_ALL=C: byte semantics, so every awk tokenises and truncates alike.
-  OUT="$(BUGQ="$Q" LC_ALL=C awk -v TOP="$TOP" -v MINCOV="${MXTK_BUG_MINCOV:-0.40}" -v FULL="${FULL:-0}" '
+  RANK_AWK='
 function toks(s, out,   n, i, a, t, m) {        # s -> out[token]=1 ; returns count
   s = tolower(s); n = 0
   while (index(s, "bug-") && match(s, /(bug-[a-z]+[0-9]+|bug-[0-9]+|bug-draft-[a-z0-9-]+)/)) {
@@ -77,7 +115,7 @@ BEGIN {
   nq = toks(ENVIRON["BUGQ"], QT)
 }
 /^## / { if ($0 ~ /^## Provenance/) { cur = 0; next }
-         N++; cur = N; HDR[N] = substr($0, 4); START[N] = NR
+         N++; cur = N; HDR[N] = substr($0, 4); START[N] = NR; SRC[N] = FILENAME; LNO[N] = FNR
          delete H; toks(HDR[N], H); for (t in H) HT[N SUBSEP t] = 1 }
 cur {
   delete L; toks($0, L)
@@ -98,22 +136,37 @@ END {
   for (k = 1; k <= TOP; k++) { m = 0; mi = 0
     for (e = 1; e <= N; e++) if (!(e in USED) && SC[e] > m) { m = SC[e]; mi = e }
     if (!mi) break; USED[mi] = 1; R[k] = mi }
-  if (!(1 in R)) { print "bug-lookup: no ledger entry shares a term with the query — likely not a known tool quirk."; exit 1 }
+  if (!(1 in R)) { if (!DRAFT) print "bug-lookup: no ledger entry shares a term with the query — likely not a known tool quirk."; exit 1 }
   if (FULL > 0) { if (FULL in R) { print "@" START[R[FULL]]; exit 0 }
                   print "bug-lookup: fewer than " FULL " entries share a term with the query."; exit 1 }
   cov = (TOT > 0) ? SC[R[1]] / (TOT * 1.5) : 0; if (cov > 1) cov = 1
+  if (DRAFT && (UNK != "" || cov < MINCOV)) exit 1
   if (UNK != "") printf "bug-lookup: no ledger entry mentions%s — likely not a known tool quirk. Nearest by wording:\n", UNK
   else if (cov < MINCOV) printf "bug-lookup: no confident match (best covers %.0f%% of query weight) — likely not a known tool quirk. Nearest:\n", cov * 100
   for (k = 1; k <= TOP && (k in R); k++) { e = R[k]
+    if (DRAFT && TOT > 0 && SC[e] / (TOT * 1.5) < MINCOV) break
     x = EX[e SUBSEP BT[e]]; gsub(/^[ \t>*`-]+/, "", x)
     if (length(x) > 160) { x = substr(x, 1, 157); sub(/[\300-\377][\200-\277]*$/, "", x); x = x "..." }
-    printf "%d. [%.1f] %s\n     %s\n", k, SC[e], HDR[e], x }
-  print "(--full N with the same query prints entry N in full)"
-}' "$LOG")" || { printf '%s\n' "$OUT"; exit 1; }
-  case "$OUT" in
-    @*) awk -v S="${OUT#@}" 'NR >= S { if (NR > S && /^## /) exit; print }' "$LOG" ;;
-    *)  printf '%s\n' "$OUT" ;;
-  esac
+    if (DRAFT) { f = SRC[e]; if (index(f, PD) == 1) f = substr(f, length(PD) + 1)
+                 printf "%d. [%.1f] %s  (project draft: %s:%d)\n     %s\n", k, SC[e], HDR[e], f, LNO[e], x }
+    else printf "%d. [%.1f] %s\n     %s\n", k, SC[e], HDR[e], x }
+  if (!DRAFT) print "(--full N with the same query prints entry N in full)"
+}'
+  # rank <full-rank> <draft 0|1> <file>...: the ledger (draft 0) or the project drafts (draft 1)
+  rank() { BUGQ="$Q" LC_ALL=C awk -v TOP="$TOP" -v MINCOV="${MXTK_BUG_MINCOV:-0.40}" -v FULL="$1" -v DRAFT="$2" -v PD="$PD/" "$RANK_AWK" "${@:3}"; }
+  RC=0; OUT="$(rank "${FULL:-0}" 0 "$LOG")" || RC=$?
+  DOUT=""
+  if [ -z "$FULL" ] && [ "${#DRAFTS[@]}" -gt 0 ]; then DOUT="$(rank 0 1 "${DRAFTS[@]}")" || DOUT=""; fi
+  if [ "$RC" -ne 0 ]; then
+    printf '%s\n' "$OUT"
+    [ -n "$DOUT" ] || exit 1
+  else
+    case "$OUT" in
+      @*) awk -v S="${OUT#@}" 'NR >= S { if (NR > S && /^## /) exit; print }' "$LOG" ;;
+      *)  printf '%s\n' "$OUT" ;;
+    esac
+  fi
+  [ -z "$DOUT" ] || printf 'Project drafts (not in the ledger):\n%s\n' "$DOUT"
   exit 0
 fi
 
@@ -134,9 +187,10 @@ for h in "${HDR[@]}"; do
   if sed -n "${a},${b}p" "$LOG" | grep -qiF -- "$Q"; then matches="$matches $h"; fi
 done
 set -- $matches
-if [ "$#" -eq 0 ]; then echo "bug-lookup: nothing in the ledger mentions '$Q' — it may be a modelling mistake, not a tool quirk. Try a shorter keyword or the CE code."; exit 1; fi
+if [ "$#" -eq 0 ]; then echo "bug-lookup: nothing in the ledger mentions '$Q' — it may be a modelling mistake, not a tool quirk. Try a shorter keyword or the CE code."; draft_word && exit 0; exit 1; fi
 if [ "$#" -gt 6 ]; then
   echo "bug-lookup: $# entries mention '$Q' — headers only; narrow the query to read one:"
-  for h in "$@"; do sed -n "${h}p" "$LOG" | sed 's/^## /  /'; done; exit 0
+  for h in "$@"; do sed -n "${h}p" "$LOG" | sed 's/^## /  /'; done; draft_word || true; exit 0
 fi
 for h in "$@"; do read -r a b < <(entry_range "$h"); sed -n "${a},${b}p" "$LOG"; echo; done
+draft_word || true

tests/wave2/fixtures/bug-lookup/project-drafts/bug-logs/2026-01-02-order-flow.md

@@ -0,0 +1,17 @@
+# Draft findings, synthetic (test fixture: invented names and messages, no real project)
+
+## BUG-DRAFT-rewire-refusal: ALTER MICROFLOW refuses to rewire an activity (2026-01-02)
+
+Version: mxcli v0.27.0, Mendix 10.24.
+
+- Expected: `ALTER MICROFLOW Shop.ACT_Order` moves the Approve activity behind the new decision.
+- Actual: `Shop.ACT_Order cannot be rewired: no base activity survives the edit`.
+- Workaround: drop the flow and create it again in one script.
+
+## BUG-DRAFT-gallery-lost-ref: a gallery action leaves no catalog edge (2026-01-02)
+
+Version: mxcli v0.27.0.
+
+- Expected: `show references to Shop.DS_Orders` lists the page.
+- Actual: `no references found` while `describe page Shop.Order_Overview` prints the datasource.
+- Workaround: grep the `describe page` output.

tests/wave2/fixtures/bug-lookup/project-drafts/bug-logs/2026-01-03-misc.md

@@ -0,0 +1,10 @@
+## BUG-DRAFT-frobnicate-limit: the frobnicate command stops at 50 items (2026-01-03)
+
+Version: mxcli v0.27.0.
+
+- Actual: `frobnicate limit reached: 50` with no way to raise it.
+- Workaround: run it once per folder.
+
+## Provenance
+
+The rewire refusal above was seen twice in one week.