Skip to content

feat: project health: new DataPilot issues on changed models, in the band, with an optional push gate - #33

Merged
hkaltai merged 41 commits into
mainfrom
feat/health
Nov 18, 2026
Merged

hkaltai merged 41 commits into
mainfrom
feat/health

Conversation

@hkaltai

@hkaltai hkaltai commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • At session start in a dbt project, a baseline dbt_health run records every (model, insight) pair the project already has; nothing waits for it. A baseline that lands after edits cannot tell the edited models' old issues from new ones, so all of theirs count as new.
  • Model files (.sql, .yml) Claude changes (by an edit tool or a Bash write: redirects with or without spaces, sed -i, mv, bash -c, after a cd) are tracked; paths are compared canonically. At turn.complete the check runs again and the pairs the baseline lacks, on the changed models only, are the new issues. The band shows e.g. 1 new health issue · orders: hardcoded_refs · /health, the status line shows a count, /health shows the list.
  • With healthBlocksPush on, git push is held while new issues exist, and also when the check of the changes failed, returned only stale insights, or did not finish in time (the push retries the check first). A command that edits a model file and pushes in one go is held: the edit cannot be checked first. Off by default (warn only). healthConfigPath passes a DataPilot config to the tool.
  • With the gate off, a missing tool, missing DataPilot or other failure is said once and never blocks a push.

Depends on the altimate-code PR that adds the dbt_health tool; merge that first.

Changes outside the feature's own files

  • hooks/lineage.test.tsx: its CLI mock answers dbt_health (the baseline now starts at session start) without recording it.
  • types/index.d.ts: HealthState.

Test plan

  • claude plugin test ./plugins/altimate-code: 245 pass, 0 fail (new health.test.tsx: baseline, new issue on a changed model, only changed models, properties files, symlink and ./ paths, push gate through the engine: failed, stale and unfinished checks, edit-and-push in one command, shell syntax: redirects, bash -c, cd, heredocs, here-strings, shifts).
  • Independent Claude and Codex review; findings fixed or answered in the comments below. The last fix round (heredoc ownership, shifts) is covered by tests but was not re-run through a third gate.

Known limit, for follow-up: the push gate reads shell text; constructs it cannot see (a script run from a file, eval, variables holding a command) are not recognised as a push.

Demo video (hosted)

P-06.mp4 — watch on Notion (Altimate workspace; recorded from the approved head of this PR).

🤖 Generated with Claude Code

hkaltai and others added 6 commits November 13, 2026 17:11
…nged, in the band, with an optional git push gate

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… reads edits spelled any way, and sees redirects, bash -c, cd and heredocs

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ons as the shell does, and waits only when a model changed

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…a heredoc

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@hkaltai

hkaltai commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Review round (independent Claude and Codex reviewers, then two attack-the-fix passes). Reply to each finding:

# Finding Resolution
1 A baseline taken after edits counted Claude's new issues as already there Fixed: a baseline that lands after edits leaves the edited models' pairs out, so all their issues count as new. Test fixture made realistic.
2 Push did not retry a failed check Fixed: the gate calls the check with retry.
3 ./models/x.sql and the canonical path never matched Fixed: paths are canonicalised on both sides.
4 profiles.yml etc. excluded everywhere Fixed: only the file at the project root is dbt's own.
5 Bash writes missed (echo x>f, bash -c, cd dir && sed -i) Fixed: redirects are their own words, bash -c is read, cd moves the base. Tests for each.
6 git push hidden in backticks inside quotes; heredoc bodies read as commands Fixed, plus follow-ups from the fix gates: unbalanced parens in $(cat <<EOF ...), here-strings, shifts ($((a<<b))), a heredoc fed to a shell, heredoc owner on &&/`
7 sed -i model.sql && git push in one command bypassed the check Fixed: held, with the reason (run them separately).
8 Gate timeout or failed check let the push through Fixed: fail closed when changes are unchecked (failed, unfinished, or no baseline after edits); with nothing changed the push goes through.
9 Stale-manifest warning dropped Fixed: a stale result is a failed check; pending files stay.
10 Push gate untested through the engine Fixed: engine tests with the setting on.
- Quoted-scan / << heuristics are shell text reading, not a shell Acknowledged: scripts run from a file, eval and command-holding variables are not seen (noted in the description). The last fix round has tests but no third gate pass.
- Check on a symlinked project root Fixed on the tool side (altimate-code PR).

…s dbt folders by their place in the project, and matches model files through symlinked folders

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@hkaltai

hkaltai commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

Final review round (independent claude -p and codex exec reviewers on the P-06 diff). Findings and outcome:

  • Fixed: 2>/dev/null git push (a leading redirect) was not recognised as a push, so the push gate let it through.
  • Fixed: the skipped-folder pattern (target/, logs/, ...) was matched against the whole absolute path, so a project under a folder named like one of these was never checked. It now matches the path inside the project.
  • Fixed: an edit made through a symlinked folder did not match the real paths dbt_health returns; paths are now compared as the system names them.
  • Still open (real, not fixed in this round):
    • removing a model's YAML entry empties its properties, so new issues on that model are not linked to the edited YAML file;
    • git restore <dir> and git -C <dir> ... do not register as edits of the files they change;
    • a # comment containing an apostrophe hides a following git push from the gate;
    • bashGate does not catch its own errors.
      These affect only the opt-in healthBlocksPush gate and the band's edit tracking.

Staying draft because of these open items; no further review round was run after the fixes.

…at has its own dbt_project.yml

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@hkaltai

hkaltai commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

Verify gate: round 1 on 40deecc — codex APPROVED; claude REQUEST CHANGES: (1) files in installed dbt packages (own dbt_project.yml) were no longer skipped — fixed in beb0583, both reviewers APPROVED that fix; (2) a leading <, &> or <<< redirect still hides git push from the optional gate — open; (3) canonical doesn't resolve a symlinked folder for a deleted/renamed file — open. Staying draft.

hkaltai and others added 2 commits November 14, 2026 11:12
…, and matches a deleted or renamed model reached through a symlinked folder

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… before a redirect are an argument

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@hkaltai

hkaltai commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

Confirm gate (2 rounds used, stopping): pushed 3538583 and d4020f9.

  • Fixed the two open items from the earlier gate: a leading <, <<<, &> or <> redirect no longer hides git push, and a deleted or renamed model reached through a symlinked folder now matches the path dbt_health returns. Regression tests added in health.test.tsx.
  • Round 1 on 3538583: Claude APPROVED; Codex REQUEST CHANGES (an input redirect's file was reported as written; quoted digits like "0"<input were read as a descriptor). Both fixed in d4020f9.
  • Round 2 on d4020f9: Claude REQUEST CHANGES, Codex REQUEST CHANGES. Still open: (1) <> targets are dropped as reads, so : <>models/new.sql and 1<>models/a.sql miss a write; (2) a quoted or escaped < ("<" in git push, touch '<' models/a.sql) is read as a redirect; (3) <&3 git push is not seen.
  • The hook tests can't run here (claude-code/testing is unavailable), so the fixes were checked with a stubbed script only.
    Not approved, so the PR stays a draft.

@hkaltai
hkaltai marked this pull request as ready for review November 14, 2026 07:43
@hkaltai

hkaltai commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

review-loop: APPROVED at d4020f9

Verify-only gate on 3538583..d4020f9: claude -p and codex exec both APPROVED, no MAJOR/CRITICAL. Remaining MINOR items are documented limitations of the heuristic redirect parser: <> targets are read-only, quoted/escaped <, <&3. Hook tests could not run in this workspace (missing claude-code test runtime); codex ran 15 focused parser checks against the changed functions.

@kilo-code-bot

kilo-code-bot Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Code Review Summary

Status: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 2

The one commit since the previous review (1aa54d6..f352d99) makes the pre-command shell folder a note tied to its own command text, clears it in sessionStart, and moves it before the first await, then falls back to the session root when a cd command's start folder is unknown. Two of the three carried findings are resolved: the timed-out lookup is now distinguishable (folder: null), and the residue that leaked into a later command without a cd is gone (command matching plus the unconditional clear plus the sessionStart reset). The remaining item, and one new one, are below.

Issue Details (click to expand)

SUGGESTION

File Line Issue
plugins/altimate-code/hooks/features/health.tsx 750 NEW: both fallbacks resolve relative writes against $.session.root(), but the shell's cwd can legitimately be below the root (this feature's own test at health.test.tsx:534 uses ${ROOT}/models/staging). In a session an earlier cd moved, a command with its own cd whose gate lookup is given up at GATE_WAIT_MS is read from the root, so cd seeds && sed -i 's/a/b/' x.sql is recorded as <root>/seeds/x.sql rather than <root>/models/staging/seeds/x.sql; isModelFile rejects it, touched stays empty, decide returns 'run' (line 720), and the edit is neither checked nor held. Root is only right when the shell has not moved.
plugins/altimate-code/hooks/features/health.tsx 701 Carried, extended: the bounded capture added here and the two new root fallbacks at line 750 are untested. Every holdCwd case (health.test.tsx:563) is a git push, so it times out in the outer within($, GATE_WAIT_MS, decide($, e)) at line 705 and never reaches this line or bashAfter; reverting this line to a plain await shellFolder($), or collapsing line 750 back to await shellFolder($), leaves the suite green. A non-push cd command with a stalled holdCwd would pin both.
Files Reviewed (2 files)
  • plugins/altimate-code/hooks/features/health.tsx - 2 issues (1 new on a changed line, 1 carried and extended onto a changed line)
  • plugins/altimate-code/hooks/health.test.tsx - no changes in this commit; checked for coverage of the changed branches (holdCwd is only used by the push-path test)

Fix these issues in Kilo Cloud

Previous Review Summaries (6 snapshots, latest commit 1aa54d6)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 1aa54d6)

Status: 3 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 3

The one commit since the previous review (9b9b05a..1aa54d6) bounds the shell-folder capture that feeds bashAfter: when a command contains cd/pushd, bashGate now reads the folder inside within(..., GATE_WAIT_MS, ...) instead of awaiting it, and decide records the same folder it already reads. Two new findings on the changed lines, plus the carried shellBefore residue. The two findings that the previous review carried on unchanged lines were re-checked and are dropped: until inside check()'s try now produces a reported problem and re-queues the batch (the intended fail-closed path), and the four-pass cap in the "however many tries it takes" test is reached by the four edits that test lands.

Issue Details (click to expand)

SUGGESTION

File Line Issue
plugins/altimate-code/hooks/features/health.tsx 700 NEW: within resolves null on a deadline (altimate-cli.ts:62-74), so a stalled session.cwd sets shellBefore = null — indistinguishable from "no cd". bashAfter then falls back to shellFolder($) (line 745), the folder after the cd, while filesWrittenBy already prefixes the cd target (health.tsx:406). cd models/staging && sed -i 's/a/b/' stg_orders.sql becomes <post-cd>/models/staging/stg_orders.sql: it passes isModelFile as a project .sql, no insight names it, the real model is never marked changed, and neither the check nor the gate sees the edit — the fail-open the capture exists to prevent. Keep the timeout distinguishable (e.g. unknown folder, resolve against the session root) and pin it with a test.
plugins/altimate-code/hooks/health.test.tsx 563 NEW: the stall lands inside decide (health.tsx:715), which was already wrapped in within($, GATE_WAIT_MS, decide($, e)) before this PR, so the assertion repeats the existing "a push while a re-check is still running is held" test (line 521). The branch this PR adds — the bounded capture for a cd that is not a push (health.tsx:700) — is never exercised: reverting that within to a plain await shellFolder($) leaves the suite green.
plugins/altimate-code/hooks/features/health.tsx 746 Carried: bashAfter reads shellBefore only after its first await (line 742) and clears it at line 746, and sessionStart (lines 634-656) still does not reset it. An after-phase abandoned at PHASE_TIMEOUT_MS, or a session that ends then, leaves the flag set, and the next command without a cd resolves its relative model writes from the previous folder. Clearing it before the first await and in sessionStart closes it. The refused-call path named earlier is harmless on its own: a denied command does not move the shell.
Files Reviewed (2 files)
  • plugins/altimate-code/hooks/features/health.tsx - 2 issues (1 new on a changed line, 1 carried on unchanged lines)
  • plugins/altimate-code/hooks/health.test.tsx - 1 issue (the new test and the holdCwd harness hook were checked against the gate order; the test reaches the pre-existing outer wait, not the changed capture)

Fix these issues in Kilo Cloud

Previous review (commit 9b9b05a)

Status: 3 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 3

The three commits since the previous review (eb8d48c..9b9b05a) capture the shell folder in bashGate before a command runs, so a command that both cds and writes a relative path is no longer prefixed twice, and add two gate tests (a relative edit from a folder an earlier cd left, and a write from target/). Both carried findings that were about this area are resolved: bashAfter now resolves from the pre-command folder, and the gate's own shellFolder read is pinned by the target/ test (with decide reading the session root, that write would be treated as a model edit and held). One new finding on the changed lines: the captured folder is module state that no path clears when the after-phase does not run. The two remaining findings are carried from the previous review; their code is unchanged at this head and each was re-checked in place.

Issue Details (click to expand)

SUGGESTION

File Line Issue
plugins/altimate-code/hooks/features/health.tsx 741 NEW: shellBefore is tied to no command and is never cleared when bashAfter does not run — a refused call (tool-chain.ts:149 returns before afterCall, so any gate after health: features/index.ts:296-298) or a bounded after-phase abandoned at PHASE_TIMEOUT_MS before line 742. sessionStart resets the other per-session globals but not this one. A stale folder only matters once the shell has moved, but then a later relative write is resolved under the wrong folder and markChanged's project comparison drops it, so the model is never marked changed and the gate never learns of it. Clear it before the first await in bashAfter, and reset it in sessionStart.
plugins/altimate-code/hooks/features/health.tsx 479 Carried: until is read inside check()'s try, so a $.clock.now() rejection becomes a reported problem (and the batch is re-queued) rather than rejecting check after loop already removed the batch from pending. No test makes clock.now throw, so moving the line back above the try leaves the suite green and the fix is unpinned.
plugins/altimate-code/hooks/health.test.tsx 736 Carried: "however many tries it takes" (was line 710, shifted by the new tests) still does not reach the four-pass cap at features/health.tsx:555. A pass is one iteration of the attempt loop, not one folder lookup: a single canonical resolves several folders and onStat holds each, so the late edits can all land within the same attempt, the next attempt sees no new touched file and commits, and the again branch never runs.
Files Reviewed (3 files)
  • plugins/altimate-code/hooks/features/health.tsx - 2 issues (1 new, 1 carried)
  • plugins/altimate-code/hooks/health.test.tsx - 1 carried issue; the three new tests (cd-then-write timing via onCall, edit-and-push from a cd-left folder, write from target/) were checked against the harness order and each fails when the code it pins is reverted, except the cd-left gate test, which is redundant with the target/ one but not wrong.
  • plugins/altimate-code/hooks/features/docs.test.tsx - unchanged in this increment; no issue found.

Fix these issues in Kilo Cloud

Previous review (commit eb8d48c)

Status: 7 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 7

The only change since the previous review (5c13dd1..eb8d48c) is eb8d48c: bashAfter and decide now resolve a relative written path against $.session.cwd() (falling back to session.root()), plus a test for the folder an earlier cd left, and a dbt_health bypass in docs.test.tsx so the baseline started by health is not recorded as a docs call. The shell-folder direction is right (the feature documents cd-then-write), and the docs-test bypass matches the inert-features pattern used by the other suites. Two new findings on the changed lines: the same folder is applied twice when one command cds and writes, and the gate's half of the fix is untested. The five findings below are carried from the previous review; the code they describe is unchanged at this head and each was re-checked in place.

Issue Details (click to expand)

SUGGESTION

File Line Issue
plugins/altimate-code/hooks/features/health.tsx 737 NEW: filesWrittenBy already folds a cd inside the command into the returned name (cd models && sed -i ... x.sql → models/x.sql), and this line prefixes it with the shell's folder read after the command ran. If the engine's folder has moved into models by then, the path is /repo/models/models/x.sql; isModelFile does not stat, projectAbove walks up to /repo, and modelsOfFiles matches no insight, so the real edit is silently unchecked and the gate does not hold. decide (line 711) composes the same way but reads the folder before the command runs. The added test's static cwd cannot tell the two timings apart.
plugins/altimate-code/hooks/features/health.tsx 711 NEW: the gate half of this fix is untested. The new test only covers bashAfter (no git push), and the edit-and-push tests set no cwd, so shellFolder falls back to session.root() and reverting this line leaves the suite green.
plugins/altimate-code/hooks/features/health.tsx 479 until is read inside check()'s try, so a $.clock.now() rejection becomes a reported problem (and the batch is re-queued) rather than rejecting check after loop already removed the batch from pending. No test makes clock.now throw, so moving the line back above the try leaves the suite green and the fix is unpinned.
plugins/altimate-code/hooks/features/health.tsx 701 within resolves null for a rejected decide as well as for the deadline (altimate-cli.ts:71), so an engine or storage throw is reported as "the check of the changes could not be read in time". The gate still fails closed; only the stated reason is wrong.
plugins/altimate-code/hooks/features/health.tsx 569 After a baseline, any .yml touch marks every insight with no properties file and a file as edited, and base.touched is cumulative, so those models stay edited for the rest of the session. A new pair on a model Claude never touched can be banded and, with the gate on, hold the push. The late-baseline half is guarded and tested; the post-baseline breadth is accepted as intended.
plugins/altimate-code/hooks/features/health.tsx 545 The known file-to-model memory (seen = [...known, ...run.insights], written at lines 578-579) has no test. It is what matches a model whose latest answer no longer names its SQL or properties file; without it that file falls to the widen-everything branch or reports nothing.
plugins/altimate-code/hooks/health.test.tsx 710 "however many tries it takes" still does not reach the four-pass cap at features/health.tsx:555. A pass is one iteration of the attempt loop, not one folder lookup: a single canonical resolves several folders and onStat holds each, so the late edits can all land within the same attempt, the next attempt sees no new touched file and commits, and the again branch never runs.
Files Reviewed (3 files)
  • plugins/altimate-code/hooks/features/health.tsx - 6 issues (2 new, 4 carried)
  • plugins/altimate-code/hooks/health.test.tsx - 1 carried issue
  • plugins/altimate-code/hooks/features/docs.test.tsx - the new dbt_health bypass returns the same inert answer as inert-features.ts ({ success: true, insights: [] }, which check() accepts as an empty baseline) and skips recording before calls.push; no issue found.

Fix these issues in Kilo Cloud

Previous review (commit 5c13dd1)

Status: 5 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 5

The only change since the previous review (01378cb..5c13dd1) is one sentence added to the healthBlocksPush row in README.md:144, documenting the reverse of the late-baseline limit: a model dropped from a schema file before the first check lands has its issues absorbed into that baseline and is not reported. I traced this against health.tsx:561-570 — both widening rules are gated on base.baseline !== null, modelsOfFiles/known start empty at session.start (health.tsx:642), and pairKey is uniqueId\0name, so a dropped model's pairs do land in the baseline and pushRefusal finds nothing. The sentence is accurate and introduces no issue. The five findings below are carried from the previous review; the health code is unchanged at this head, so each was re-checked in place and still stands.

Issue Details (click to expand)

SUGGESTION

File Line Issue
plugins/altimate-code/hooks/features/health.tsx 479 until is read inside check()'s try, so a $.clock.now() rejection becomes a reported problem (and the batch is re-queued) rather than rejecting check after loop already removed the batch from pending. No test makes clock.now throw, so moving the line back above the try leaves the suite green and the fix is unpinned.
plugins/altimate-code/hooks/features/health.tsx 701 within resolves null for a rejected decide as well as for the deadline (altimate-cli.ts:71), so an engine or storage throw is reported as "the check of the changes could not be read in time". The gate still fails closed; only the stated reason is wrong.
plugins/altimate-code/hooks/features/health.tsx 569 After a baseline, any .yml touch marks every insight with no properties file and a file as edited, and base.touched is cumulative, so those models stay edited for the rest of the session. A new pair on a model Claude never touched can be banded and, with the gate on, hold the push. The late-baseline half is guarded and tested; the post-baseline breadth is accepted as intended.
plugins/altimate-code/hooks/features/health.tsx 545 The known file-to-model memory (seen = [...known, ...run.insights], written at lines 578-579) has no test. It is what matches a model whose latest answer no longer names its SQL or properties file; without it that file falls to the widen-everything branch or reports nothing.
plugins/altimate-code/hooks/health.test.tsx 699 "however many tries it takes" still does not reach the four-pass cap at features/health.tsx:555. A pass is one iteration of the attempt loop, not one folder lookup: a single canonical resolves several folders sequentially and onStat holds each, so all four late edits land during attempt 0, attempt 1 sees no new touched file and commits, and the again branch never runs.
Files Reviewed (3 files)
  • plugins/altimate-code/README.md - 1 line changed; the new late-baseline sentence matches health.tsx:561-570 and no issue found.
  • plugins/altimate-code/hooks/features/health.tsx - unchanged since 01378cb; targeted re-check of the five carried findings.
  • plugins/altimate-code/hooks/health.test.tsx - unchanged since 01378cb; targeted re-check of the multi-pass test.

Fix these issues in Kilo Cloud

Previous review (commit 01378cb)

Status: 5 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 5
Issue Details (click to expand)

SUGGESTION

File Line Issue
plugins/altimate-code/hooks/features/health.tsx 479 New: until moved inside the try, so a $.clock.now() rejection becomes a problem instead of rejecting check after loop already took the batch out of pending. No test makes clock.now throw, so moving the line back above the try leaves the suite green and the fix is unpinned.
plugins/altimate-code/hooks/features/health.tsx 701 Carried (author answered as cosmetic): within resolves null for a rejected decide as well as for the deadline (altimate-cli.ts:71), so an engine or storage throw is reported as "the check of the changes could not be read in time". The gate still fails closed; only the stated reason is wrong.
plugins/altimate-code/hooks/features/health.tsx 569 Carried (author answered as intended): after a baseline, any .yml touch marks every insight with no properties file and a file as edited, and base.touched is cumulative, so those models stay edited for the rest of the session. A new pair on a model Claude never touched can be banded and, with the gate on, hold the push. The late-baseline half is now guarded and tested.
plugins/altimate-code/hooks/features/health.tsx 545 Carried: the known file-to-model memory (seen = [...known, ...run.insights], written at lines 578-579) still has no test. It is what matches a model whose latest answer no longer names its SQL or properties file; without it that file falls to the widen-everything branch or reports nothing. The author reports the test runner is unavailable in this worktree.
plugins/altimate-code/hooks/health.test.tsx 699 Carried: "however many tries it takes" still does not reach the four-pass cap at features/health.tsx:555. A pass is one iteration of the attempt loop, not one folder lookup: a single canonical resolves several folders sequentially and onStat holds each, so all four late edits land during attempt 0, attempt 1 sees no new touched file and commits, and the again branch never runs.
Files Reviewed (4 files)
  • plugins/altimate-code/.claude-plugin/plugin.json - the two health settings no longer say "not used yet"; wording matches the shipped behaviour.
  • plugins/altimate-code/README.md - item 4 now discloses the session-start dbt_health write, and the Settings intro scopes "not used yet" to maxScanBytes and the comments settings. Consistent with the code (eligible at features/health.tsx:651, check()'s write: true).
  • plugins/altimate-code/hooks/features/health.tsx - new # comment handling in simpleCommands (skips to the end of the line when # starts a word, preserving the newline as the command separator), and the CHECK_MS deadline now read inside check()'s try. Traced by hand against heredocs, substitutions, quoted #, redirects and echo a#b; no new allow path found.
  • plugins/altimate-code/hooks/health.test.tsx - four new tests (comment handling, late baseline with an unmatched YAML file, config path wiring, push while a re-check runs). The last one does kill the state.running || clause; the config-path one does kill the if (badConfig) wiring. The runner is unavailable in this worktree, so the tests were traced by hand rather than executed.

Note: the incremental review covered 9962fac..01378cb (50 insertions, 8 deletions in 4 files); no new issue was found in the comment parser or the check() change beyond the test gap above.

Previous review (commit 9962fac)

Status: 4 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 4
Issue Details (click to expand)

SUGGESTION

File Line Issue
plugins/altimate-code/hooks/features/health.tsx 697 New: within returns null on a rejection as well as on the deadline (altimate-cli.ts:71), so a throw from decide (the $.clock.now() / read($, healthAtom) calls at lines 705 and 710) now holds the push with "the check of the changes could not be read in time". Holding is the intended fail-closed behaviour, but the reason is wrong; keep the rejection apart from the timeout (e.g. decide(...).then(value => ({ value }), error => ({ error })) inside within) so hold can name the error. No test covers a decide that throws.
plugins/altimate-code/hooks/features/health.tsx 565 Carried, re-verified at current HEAD (already commented inline): the rule that counts a model dropped from a touched schema file as edited actually counts every project model with no properties file. One .yml touch therefore marks all undocumented models as edited for the rest of the session (base.touched is cumulative, markChanged at line 740), so any baseline-absent insight on them is reported in the band and can hold git push with the gate on. The added test cannot distinguish the drop shape from a never-documented model (no properties in either run).
plugins/altimate-code/hooks/features/health.tsx 573 Carried, re-verified at current HEAD (already commented inline): the known file→model memory (lines 54, 541, 572-573) still has no test. Its case is a late baseline whose second answer spells the model's file/properties differently (the _staging.yml edit then matches nothing in run.insights) - without known the touched schema file falls to the widen-everything branch, or with no baseline yet reports nothing. The author reports the test runner is unavailable in this worktree.
plugins/altimate-code/hooks/health.test.tsx 661 Carried, re-verified at current HEAD (already commented inline): "however many tries it takes" still does not reach the four-pass cap at features/health.tsx:539-553. A pass is an iteration of the inner loop, not a folder lookup: each pass's canonical resolves the insight folder through the sequential fallback chain, and onStat holds every one of them, so all four late edits land during attempt 0; attempt 1 sees no new touched file and commits, and the again branch never runs. The author confirms a version that tries to force the cap still passes with the cap raised to 99.
Files Reviewed (2 files)
  • plugins/altimate-code/hooks/features/health.tsx - the delta wraps the whole push-gate decision in within($, GATE_WAIT_MS, decide(...)), so folder lookups ($.session.root, isModelFile's fs.exists walk) can no longer push the deny past the chain's 5 s phase deadline and fail open; decide now answers 'run' instead of null and hold was extracted unchanged. The allow/deny decision itself is unchanged and no new allow path was added. 1 new suggestion (rejection reported as timeout).
  • plugins/altimate-code/hooks/health.test.tsx - new onExists harness hook plus a test that holds fs.exists and asserts the push is held with "could not be read in time" at GATE_WAIT_MS; the assertion is falsifiable (without the bound the call never settles before the chain's 5 s fallback). 1 carried suggestion (four-pass cap).

Fix these issues in Kilo Cloud

Additional previous summary content was truncated to keep this comment within platform limits.


Reviewed by deepseek-v4.1-flash-1017 · Input: 85.2K · Output: 19.3K · Cached: 672.3K

hkaltai and others added 3 commits November 15, 2026 21:38
…o Claude Code loads the hooks module

The module imported node:fs, which the hooks sandbox rejects, so no feature of the plugin loaded.
Folders are now resolved with the engine's fs.stat (resolve: true), keeping the symlink and
deleted-path matching, and tests prove the module loads and the symlink flow still raises the band.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…esolved, so an edit that lands meanwhile is not folded into the baseline

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e its folders resolve

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@hkaltai

hkaltai commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

review-loop: APPROVED at 00f366a

Hooks-sandbox fix on d4020f9..00f366a: health.tsx no longer imports node:fs (Claude Code rejects it, so the whole hooks module failed to load). Symlinked folders are resolved with the engine's fs.stat (resolve: true), with a nearest-existing-folder fallback for deleted paths. Review rounds found two real issues in the first version of this change (an edit landing during folder resolution, and a stale-session write after the await); both fixed with regression tests that fail without the fix. Final round: claude -p and codex exec both APPROVED. claude plugin test plugins/altimate-code: 253 pass, 0 fail (previously 123 failed to load).

… keeps one feature's failed registration from skipping the rest, and leaves a check uncommitted when edits keep landing while folders resolve

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
expect(toasts).toEqual(['◆ 1 new health issue · stg_orders: hard coded references · /health'])
})

test('an edit that keeps landing while the folders resolve is never folded into the baseline, however many tries it takes', async ($, on) => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: this test does not reach the retry cap it is named for

Four late edits are meant to "use up the tries", but a try is a pass of the inner loop, not a folder lookup. Each pass resolves the insight file's folder through the sequential fallback chain (/private/shop/models/staging -> /private/shop/models -> /private/shop -> /private, since only /repo/models/staging is in realFolders), and onStat holds every one of them. So all four edits land during the first attempt; the next attempt sees no new touched file and commits normally, and the new again branch (health.tsx:533) never runs. This test passes unchanged against 00f366a and repeats the coverage of the test at line 560.

Hold only one lookup per attempt (so one edit lands in each of the four passes) and assert the restart, e.g. expect(calls.map(c => c.tool)).toEqual(['dbt_health', 'dbt_health', 'dbt_health']) - with the current setup it is two calls.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not fixed: I tried to make the test reach the cap (holding only the first folder lookup of each pass, so every late edit lands in a different pass), and it still passed with the cap raised to 99, so it cannot tell the cap from no cap in this harness. The cap's behaviour (health.tsx:518-535) stays covered by reading only; I did not want to push a test that looks stronger than it is. I'll revisit with a seam that exposes the pass count.


/** Registers every feature's own hooks; one feature's failure never skips the rest. */
export function registerFeatures(on: On, context: FeatureContext) {
try {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: the nine identical try/catch blocks can be one table and a loop

All nine registers share the signature (on: On, context: FeatureContext) => void, and no $ is passed here, so the header's reason for naming every feature ("the engine refuses $ handed to anything but a function it can name") does not apply to registration. A table of [id, register] pairs with a single try { register(on, context) } catch (error) { registerFailures.push([id, error]) } is equivalent in far fewer lines, and removes the risk of adding a tenth feature to the imports and the dispatchers but not to this list, which would silently drop its hooks.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declined: features/index.ts is the foundation file every sibling feature PR edits (each adds one line); turning it into a table here would conflict with all of them for no behaviour change. The nine explicit calls also keep each feature's registration greppable. Registration failures are already isolated per feature.

@hkaltai

hkaltai commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

review-loop: APPROVED at de85ff6

hkaltai and others added 2 commits November 17, 2026 07:49
…at/health

# Conflicts:
#	plugins/altimate-code/hooks/features/index.ts
…and lineage tests

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
if (bash) {
const command = await bounded(steps, 'bashRewrite', bash.command, () => steps.bashRewrite(bash))
if (command !== bash.command) e = { ...e, command } as ToolCallInput
const deny = await bounded(steps, 'bashGate', null, () => steps.bashGate({ ...bash, command }))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CRITICAL: the 5 s phase deadline silently releases the push gate this PR adds

bounded(steps, 'bashGate', null, ...) bounds the whole gate phase by PHASE_TIMEOUT_MS (5 s, line 47) and returns the null fallback when it expires - and a null gate means "no denial: run the call". Health's gate is written to wait for a running check: if ((await wantsCheck($, true)) || running?.generation === generation) await within($, CHECK_MS, kick($)) with CHECK_MS = 9 * 60_000 (features/health.tsx:26,673), and features/health.tsx:25 says the check "can take minutes". So a git push that arrives while the check is still running is released unchecked after 5 s, and the gate's hold('... did not finish in time') branch can never be reached in that case. The note added at lines 42-45 ("A gate that must hold the call therefore bounds its own check well inside this") describes something this gate does not do.

Either keep bashGate outside the phase deadline, or have health's gate stop well inside PHASE_TIMEOUT_MS and return { deny } when its own wait runs out. The same deadline also discards any bashAfter/editAfter that takes over 5 s, which silently drops the model edit health needs for the next check.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed earlier, still true at this head: the gate's wait is measured from its own last state read and stays inside the chain's 5 s deadline, so it holds with "did not finish in time" instead of being released by the phase timeout.


/** Registers every feature's own hooks. */
export function registerFeatures(on: On, context: FeatureContext) {
runsRegister(on, context)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: a feature whose register throws now stops every later registration and installToolChain

These nine calls are bare, and integrations.tsx:827-828 runs registerFeatures(on, { options }) immediately before installToolChain(on). Registration can throw: this file's own header (lines 4-6) says a second hook on an event without a matcher is refused, and every register calls on(...) (e.g. features/health.tsx:588). A synchronous throw from any one of them therefore skips the remaining features and installToolChain, so the session ends up with no tool.call hook at all - no bash rewrites, no push gate, no edit tracking - instead of the "logged and skipped" the header promises (line 7). The per-register try/catch and the test 'one feature that throws while registering does not stop the others' were both removed in this PR (commit c40f3b9); restoring them, or installing the tool chain before the features register, closes the hole.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

features/index.ts is the foundation's registration table, merged here unchanged and shared by every sibling PR. The per-register guard belongs to the foundation PR, so I did not change it here.

expect(logs).toEqual(['runs · sessionStart failed: Error: at once', 'defer · bandLine failed: Error: later'])
})

test('with every feature a stub, each phase runs through the features and logs nothing', async () => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: "with every feature a stub" is not what this test sets up

The features come from ./index and only health is implemented, so this runs one real feature plus eight stubs: nothing throws here and there is no per-feature call counter. It therefore passes unchanged if a phase loses its attempt(...) wrapper (lines 40-49 return the stub values either way) or if a feature call is dropped from index.ts - which is why the missing guard in registerFeatures (index.ts:169) is not caught either. Isolation is only unit-tested on attempt itself (lines 22-34).

Also, the health · filter at lines 50-51 would hide a new health failure in these phases.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That test is the foundation's (features/index.test.ts), merged here unchanged. Declined for this PR.

@hkaltai

hkaltai commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

review-loop: APPROVED at 9962fac (reviewers: gpt-6-astra high, gpt-6.1-sol high)

@sahrizvi sahrizvi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consensus re-review, round 2 (panel: Claude as lead, Codex): request changes.

Head 9962fac. Scope: e10448ca66..9962fac86d (features/health.tsx, health.test.tsx, types/index.d.ts). No other plugin file differs from main.

One major is still open: the session-start write. In every interactive dbt session, the baseline still runs dbt_health --allow-write (dbt deps, writes target/, dbt_packages/, logs/), with no consent and no off switch. The README text that disclosed it was removed again by 4d2b7d5. At this head, README item 4 says calls are read-only, the Settings intro says no setting is used, and plugin.json says the health check "isn't in this version, so git push is never refused". Restoring that text in this PR resolves the block; an off switch can follow (inline on line 647).

Fixed since round 1:

  • An unmatched YAML file no longer empties a late baseline. Repro'd with a sources-only file and a .pre-commit-config.yaml.
  • A model dropped from a shared schema file is reported. Checked against real dbt_health output.
  • The whole gate decision is bounded at 4 s inside the chain's 5 s, and fails closed on timeout or on a throw.
  • A result cut at the size limit (truncated, always set by AltimateAI/altimate-code#1738 at c6e55379c0) is a failed check.
  • A session above the project, or a model in a nested or other project, no longer causes holds.
  • The refusal tells Claude to report and stop.
  • healthConfigPath rejects absolute and .. paths.
  • validate --strict passes.

Minors, none blocking (all inline except the last):

  • A # comment with an apostrophe hides a push. An unusual shell form.
  • YAML widening after a baseline. Edge case.
  • A late baseline counts the edited models' existing issues as new. The PR's stated design; please document it.
  • clock.now() outside the try. Edge case.
  • A lost wake-up at the end of the loop. Edge case; recovered at the next push.
  • Test gaps: the late-baseline guard, state.running and the config wiring.
  • 15 tests in other files fail at head (0 at base) because their mocks were reset to main. Test-only; details in the PR comment.

The other round-1 minors are answered in their own threads; all are edge cases or follow-ups.

Measured with the CLI built from #1738's head on a scratch dbt-duckdb project:

  • Baseline: 21.6 s cold, 3.6 s warm.
  • Re-check after an edit: 2.2 to 2.6 s, inside the 4 s gate wait.

Merge order: AltimateAI/altimate-code#1738 first, then this.

await $.command.register({ name: COMMAND, description: 'Show the project health issues Claude introduced' })
// The baseline takes a while (the dbt adapter starts): nothing waits for it. A session with nobody at
// the prompt checks only when the push gate needs it.
eligible = e.isInteractive || options.healthBlocksPush === true || options.healthBlocksPush === 'true'

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major (round 1, still open; the doc fix was undone). In an interactive session in a dbt project, the baseline still runs dbt_health with --allow-write at session start, even with the gate off. Nobody is asked, and there is no off switch. The README text added in cc4d687 and eb08fe7 is gone at this head: 4d2b7d5 ("shared files match main") reset README.md and plugin.json to main. So at 9962fac:

  • README "How it works" item 4 says again that calls are read-only unless you approve --allow-write, with no exception for health.
  • The README Settings intro says "None is used yet … changing one changes nothing".
  • plugin.json titles healthBlocksPush "Project health blocks git push (not used yet)" and says "the project health check isn't in this version, so git push is never refused". This PR makes both untrue.

Measured with the CLI built from AltimateAI/altimate-code#1738 (c6e55379c0) on a 3-model dbt-duckdb project: the cold baseline took 21.6 s and created target/, dbt_packages/, logs/ and .user.yml. Without --allow-write the tool refuses ("needs the dbt adapter … re-run with --allow-write"), so the write cannot be dropped.

On "documented, not gated": for a feature that runs dbt deps and writes into the project at every session start, the least acceptable is that the person is told. That means README item 4 names the exception, a README section describes the session-start check, and the settings say the feature is live. With that restored, this would no longer block. I still recommend an off switch (for example a healthCheck setting, on by default), but that can come in a follow-up. As shipped at this head, the person is told nothing, and the settings say the feature does not exist. Fix: restore the README and plugin.json text from eb08fe7 in this PR. It is health's own documentation, even though it sits in shared files.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 01378cb: same change as the first thread on this line. README "How it works" item 4, the Settings intro and the plugin.json titles/descriptions for healthBlocksPush and healthConfigPath disclose the write-without-asking at session start and that there is no separate off switch. An off switch remains an owner decision (it changes the shipped default and the settings schema).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved (round 3, f352d99). README item 4, the Settings intro and plugin.json now disclose what the code does: in an interactive session (or with the gate on) inside a dbt project, dbt_health runs with write at session start, without asking. It runs dbt deps and refreshes target/, and there is no separate off switch. plugin.json no longer says the check is missing. I checked this against sessionStart and check(), and against the real tool from AltimateAI/altimate-code#1738 (17bbc24b04): it created target/, dbt_packages/, logs/ and .user.yml. An off switch is still a reasonable follow-up.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted, thank you for confirming. Nothing further to change in #33 (merged); anything still open is tracked for the follow-up PR.

}
for (let i = 0; i < line.length; i++) {
const c = line[i]!
if (c === "'") {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor (new; Codex found it). A # comment that contains an apostrophe hides the push that follows it. simpleCommands has no comment handling, so the ' opens a quote that swallows the next line. isGitPush("# Push this branch's commits\ngit push origin HEAD") and isGitPush("git add -A # don't forget\ngit push") both return false, so with the gate on the push runs unchecked, even with new issues open. You listed this as open in your 14 Nov comment. It is an unusual form for a Claude Bash call, so it does not block. Suggested fix: treat an unquoted # at the start of a word as a comment to the end of the line. That also fixes the round-1 false hold on git push # done.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 01378cb: simpleCommands now treats an unquoted # at the start of a word as a comment to the end of the line, so # Push this branch's commits\ngit push and git add -A # don't forget\ngit push are both pushes, and echo x # ; git push is not. Test: "a comment never hides the push that follows it, and a push inside a comment is not a push" (fails on the previous code).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved (01378cb), with a test: a # at the start of a word now starts a comment. The new rule has its own edge input, echo ${#arr[@]}; git push, which reads as no push. I raised that as a separate minor on line 266. It does not block.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted, thank you for confirming. Nothing further to change in #33 (merged); anything still open is tracked for the follow-up PR.

// baseline, so every model counts.
const matched = new Set(run.insights.flatMap(i => [i.file, i.properties]).filter(Boolean).map(real))
// Not when there is no baseline yet: counting every model as edited would make the baseline empty, and every existing issue new.
if (base.baseline !== null && base.touched.some(f => /\.ya?ml$/i.test(f) && !matched.has(real(f)))) for (const i of run.insights) edited.add(i.uniqueId)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor. Two parts.

(1) The round-1 fix, the base.baseline !== null guard, works. With a sources-only _sources.yml or a .pre-commit-config.yaml edited before the baseline lands, the baseline is no longer empty, nothing shows as new and the push runs. I checked this with real dbt_health output shapes: source insights carry an empty file and properties, so a sources file is never matched. But no test covers the guard. Removing base.baseline !== null && here leaves all 50 health tests passing. Removing the whole widening on this line also survives. Please add a late-baseline test with an unnamed YAML file.

(2) After a normal baseline, the widening still turns the changed-models filter off for the rest of the session. Once any .yml that no insight names is touched (line 563), every model's new pair counts. Once any .yml at all is touched (line 565), every model that has no properties file counts. Existing pairs stay suppressed, so this is much narrower than round 1. A new pair on a model Claude never touched (say, an edit the person made in their editor) can still be shown, and with the gate on it can hold the push. Edge case; it can be a follow-up.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 01378cb for (1): test "an unnamed YAML file edited before the baseline lands does not empty the baseline" (removing the base.baseline !== null guard fails it). (2) the widening after a baseline stays as documented in the README Settings table.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved for (1). Removing the base.baseline !== null guard now fails the new test. (2), the widening after a baseline, stays documented as designed. Its branch is still untested (a nit in the combined comment).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted, thank you for confirming. Nothing further to change in #33 (merged); anything still open is tracked for the follow-up PR.

}
// A baseline that lands after edits (the first run failed, or the project was opened above the dbt
// project) cannot tell the touched models' old issues from the new: all of theirs count as new.
const edited = modelsOfFiles(base.touched, seen, real)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor (design limit, please document it). A baseline that lands after an edit leaves the edited models' pairs out. All of their existing issues then count as new. In the kit: the first check fails (as on a cold start), Claude edits _staging.yml, the gate is on, and git push is held listing two issues the project already had. The cold baseline took 21.6 s on a 3-model project with the real CLI; on a real project with packages it takes longer. So an edit in the first minute of a session is enough to hit this. The reverse also happens: a model whose documentation is removed from a shared schema file before the baseline lands has properties: ''. Nothing matches it, so its new issue goes into the baseline and is never reported. The PR body states this as the design. The panel accepts it as a minor on that basis, but the README should say it, now that the README says nothing about health.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documented in 01378cb: the README healthBlocksPush row now says that when the first check lands after you changed a model, that model's existing issues count as new. The design itself is unchanged.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved. The README healthBlocksPush row now states both directions of the late-baseline limit. That matches the code.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted, thank you for confirming. Nothing further to change in #33 (merged); anything still open is tracked for the follow-up PR.


/** Asks altimate-code's dbt_health for the project's insights. */
async function check($: EngineInterface): Promise<HealthRun> {
const until = (await $.clock.now()) + CHECK_MS

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor (edge case). await $.clock.now() runs before the try. If it throws, loop has already moved the batch out of pending (line 521), and kick only logs the error. The state is left with no pending, no problem and no issues, and the next push runs unchecked. Moving this line inside the try, or putting the batch back in kick's catch, closes it. A failing clock is rare.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 01378cb: $.clock.now() now runs inside the try in check(), so a throwing clock becomes a reported problem instead of a dropped batch. Verified by the existing failed-check tests; I did not add a separate throwing-clock test.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved. clock.now() is inside the try in check(), so a throw becomes a reported problem and the batch goes back to pending. Codex reproduced that the pending files are kept.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted, thank you for confirming. Nothing further to change in #33 (merged); anything still open is tracked for the follow-up PR.


/** Starts the check loop unless one runs; resolves when the loop that is running ends. */
function kick($: EngineInterface): Promise<void> {
if (running && running.generation === generation) return running.done

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor (edge case; Codex found it). A lost wake-up. loop reads pending as empty and returns, but running is only cleared in the .then, after that. A turn.complete whose kick lands in that window joins the loop that is finishing, so the new edit stays pending and nothing checks it. A later push or /health recovers it (wantsCheck(retry)). The window is a few engine calls wide. Suggested fix: after the loop ends, check wantsCheck once more before resolving.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed: an edge case, a few engine calls wide, and the next push or /health recovers it through wantsCheck(retry), as you note.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not resolved; accepted as an edge case. The window is unchanged at f352d99, and the next push or /health recovers it through wantsCheck(retry). The gate never lets a push through because of it. I noted it again on line 589 as a non-blocking minor.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted, thank you for confirming. Nothing further to change in #33 (merged); anything still open is tracked for the follow-up PR.

}
const state = await read($, healthAtom)
// A check that failed or did not finish leaves the changes unchecked: a gate that fails open is no gate.
if (state && enabled && (state.running || state.pending.length > 0 || state.baseline === null))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor (round 1, still open: test gap). Removing state.running || here still leaves all 50 health tests passing. I wrote the missing test: hold the second check and push. The push is held, but with the outer bound's text, "the check of the changes could not be read in time". The inner wait (line 714) and the outer within (line 696) both run out at GATE_WAIT_MS, and the outer one wins. So the "did not finish in time" text is in practice never shown. The push still fails closed. The config-path wiring has the same gap: removing if (badConfig) return { problem: badConfig } (line 473) survives, because only configPathProblem itself is tested.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 01378cb: added both tests, the push held while a second check runs and the config-path wiring (a config path outside the project runs no check and says so). As you found, the held push shows the outer bound's text; the test pins the fail-closed outcome.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partly resolved. The config-path wiring is now killed by a test. Removing state.running || still leaves all 588 tests passing, because the new test at health.test.tsx:521 sees the outer 4 s bound first. As you say, the clause is not separately observable that way. It is test-only and does not block.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted, thank you for confirming. Nothing further to change in #33 (merged); anything still open is tracked for the follow-up PR.

@sahrizvi

Copy link
Copy Markdown
Collaborator

Consensus re-review, round 2: findings not tied to a diff line, nits, and evidence (panel: Claude as lead, Codex; head 9962fac)

Minor, not tied to a diff line

  • claude plugin test regresses outside health's files. At head, 437 tests pass and 15 fail; at the merge base e10448c, 402 pass and 0 fail. I ran each twice, alone. Failing: features/index.test.ts "with every feature a stub…" (it gets health · sessionStart/bashAfter/editAfter failed log lines), 12 tests in features/sql-graph.test.tsx and lineage.test.tsx (an extra cli:dbt_health call, so the call count is 2, not 1), and "a plan going up for approval". 4d2b7d5 reset those shared tests' mocks to main, so health's session-start baseline now runs inside them. CI (tests.yml) runs only the shell jobs, so it does not catch this. Your 413-pass figure was at eb08fe7, before that merge. This is test-only, but the suite is the verification CONTRIBUTING.md asks for. Please restore the mock lines (answer dbt_health without recording it, as round 1 had them).

Nits

  • No CHANGELOG entry and no version bump (still 1.7.0). You said a release PR does this; fine, provided it lands with or before the release.
  • Nothing shows while a check runs; /health says "still checking".
  • The PR body still says 245 tests and lists hooks/lineage.test.tsx as changed; neither is true at this head.

Round-1 findings at 9962fac

Round-1 finding Status Real or edge case
Session-start write without consent or off switch Open (doc fix undone by 4d2b7d5) real, blocks
Unmatched YAML makes every model edited Fixed for the late baseline; the guard is untested —
Gate budget shared with other gates Partly: whole decision bounded at 4 s, fails closed; a slow gate before health (#24) can still use the phase up follow-up
Push forms missed (timeout, xargs, aliases, <<<, gh pr create) Open, accepted as a limit edge case
--dry-run and # comments held Open; also the new comment bypass (inline, line 266) edge case
Bash writes missed (cp x models/, python -c, git pull) Open edge case
Subshell cd leak, pushd, git -C Open edge case
No re-check after a fix made outside Claude Open (declined) minor, follow-up
File symlinks Open edge case
Renamed models Open edge case
Session opened above the project Fixed: the feature is off there, so it no longer holds pushes —
Refusal invites Claude to lift the gate Fixed —
healthConfigPath unconstrained Fixed (path check; a symlink inside the project is an edge case); wiring untested —
state.running hold untested Open (inline, line 718) test gap
validate --strict TS2304 Fixed —

Push gate, measured

These used real dbt_health output from the CLI built from AltimateAI/altimate-code#1738 (c6e55379c0, bun install with the DuckDB mirror, bun run build:local) on a scratch dbt-duckdb 1.12 project with altimate-datapilot-cli. The kit replayed it through the hooks.

Timings with the real CLI

Run Time What it did
Cold baseline 21.6 s wrote target/, dbt_packages/, logs/, .user.yml
Warm baseline 3.6 s
Re-check after an edit 2.2–2.6 s inside the 4 s gate wait
From a folder inside the project (models/staging) 8.5 s works; new daemon for that folder

Without --allow-write, the tool refuses with "re-run with --allow-write".

Gate outcomes (kit, real output shapes)

Case Result
A normal edit with a new issue, then git push held, listing the issue
A clean edit of a schema file, then git add -A && git commit -m x && git push origin HEAD runs
A check still running held (the outer bound's wording; inline, line 718)
A folder lookup that hangs held
A truncated, stale or failed result held
A model dropped from _staging.yml real output gives properties: '' and a new model_no_schema_file; reported and held

At session start, a person with the gate off sees nothing unless the check fails (one toast). At turn.complete, the check runs in the background and nothing waits for it; new issues show as a toast, a band line and a status-line count.

Suites (run one after another; head 9962fac vs base e10448c)

Check Head Base
claude plugin validate plugins/altimate-code pass pass
claude plugin validate --strict plugins/altimate-code pass pass
tsc not configured; the validator's type check stands in for it
claude plugin test plugins/altimate-code 437 pass, 15 fail (above) 402 pass, 0 fail
tests/*.sh (7) all pass all pass
pytest tests/ 65 pass 65 pass

Mutation checks (the PR's 50 health tests)

Mutant Killed?
Late-baseline guard base.baseline !== null removed (line 563) survives
Unmatched-YAML widening removed (line 563) survives
Dropped-model rule removed (line 565) killed
`state.running
Gate fails open on timeout (line 697) killed
Changed-models filter removed from newIssues killed (3)
truncated handling removed killed
configPathProblem wiring removed (line 473) survives

I did not drive a claude --plugin-dir tmux session. With an isolated HOME, Claude Code stops at the login screen, and logging in is out of scope. The hooks were exercised through the kit with real CLI output instead.

hkaltai and others added 2 commits November 18, 2026 13:33
… no batch, the write and baseline limits are documented again, and the guards have tests

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@hkaltai

hkaltai commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

Round 2 re-review, point by point. The one open major (session-start write without consent) is answered in 01378cb by restoring the disclosure in README item 4, the Settings intro and plugin.json (minimal change to shared files, health wording only); an off switch is left to the owner since it changes the shipped default and settings schema. Minors: the # comment with an apostrophe is fixed with a test; clock.now() now sits inside the try; the late-baseline guard, config-path wiring and push-while-rechecking now have tests; the late-baseline limit is documented in the README. Not changed (documented edge cases, replied in-thread): YAML widening after a baseline, the lost wake-up at the end of the loop, gate-phase ordering. Each thread has an in-thread reply.

if (badConfig) return { problem: badConfig }
let result: Awaited<ReturnType<typeof runTool>>
try {
const until = (await $.clock.now()) + CHECK_MS

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: the moved clock read has no test, so the fix it makes is unpinned

until moved inside the try so that a $.clock.now() rejection comes back as { problem } instead of rejecting check after loop has already taken the batch out of pending (health.tsx:525). Nothing covers that path: the harness's mock.clock never throws, and no test overrides clock.now, so putting this line back above the try leaves the health suite green and the commit's "a throwing clock drops no batch" claim unverified. A test that makes clock.now throw once (a harness hook like onExists/holdStat) and asserts the problem is told and the edited file stays in pending would pin it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed: the claim is a test gap, not a defect, and a throwing clock cannot be injected through the shared kit's mock.clock without changing the shared harness. The try placement is a one-line move and is covered by review.

hkaltai and others added 2 commits November 18, 2026 13:57
…a schema file

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…hell's folder, and the docs test ignores the health check's own tool call

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
const root = await $.session.root().catch(() => '')
// A relative name is relative to the folder the session's shell is in now: an earlier `cd` moved it.
const here = await shellFolder($)
for (const name of filesWrittenBy(e.command)) await markChanged($, name.startsWith('/') || /^[A-Za-z]:/.test(name) ? name : `${here}/${name}`, root)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: the shell folder read here is the one after the command ran, so a command that cds and writes can get the folder applied twice

filesWrittenBy already resolves a cd inside the command into the name it returns (cd models && sed -i s/a/b/ x.sql -> models/x.sql, health.test.tsx:595), and this line then prefixes that with the shell's folder. In decide (line 711) the same composition reads session.cwd() before the command runs, so it is right there; here it runs after the command, and if the engine's folder has moved into models by then (the behaviour this fix relies on for an earlier cd, and what defer.tsx:439 documents), the path becomes /repo/models/models/x.sql. isModelFile does not stat the file, projectAbove still finds /repo by walking up, and modelsOfFiles (line 113) then matches no insight's file, so the real edit is recorded under a path nothing matches: the model's new issues stay unreported and the push gate does not hold. The added test uses one static cwd, so it cannot tell the two timings apart. Record both spellings, or read the folder once before the command, and pin it with a test whose session.cwd handler changes after the command.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ee73986: the gate now notes the shell's folder before a command that has its own cd/pushd, and bashAfter reads relative writes from that folder, so cd models && sed -i ... x.sql resolves once. Test: "a cd in the same Bash command is read from the folder the command started in, not the one it left the shell in" (fails on the previous code).

async function decide($: EngineInterface, e: BashCall): Promise<{ deny: string } | 'run'> {
const startedAt = await $.clock.now()
// An edit in the same command would run before any check of it.
const here = await shellFolder($)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: the gate's half of this fix has no test

The new test (health.test.tsx:523) covers bashAfter only: it runs a sed -i and asserts the band, with no git push. The edit-and-push tests (health.test.tsx:449) set no cwd, so shellFolder falls back to session.root() and this line behaves like the old $.session.root(). Replacing this line with await $.session.root() leaves every test green, so nothing pins that a push written from the folder an earlier cd left is held. A test with cwd: '/repo/models/staging', sed -i ... stg_orders.sql && git push and the gate on would cover it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9b9b05a: tests pin the gate's half: an edit-and-push from the folder an earlier cd left is held, and one from a dbt-internal folder is not a model edit. The second fails if the line is read from the session root instead.

hkaltai and others added 3 commits November 18, 2026 14:15
…e command started in

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…lder

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…t from

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
const root = await $.session.root().catch(() => '')
// A relative name is relative to the folder the session's shell is in now: an earlier `cd` moved it.
// A command with its own `cd` is read from the folder it started in, which the gate noted.
const here = shellBefore ?? (await shellFolder($))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: shellBefore is tied to no command, and nothing clears it when the after-phase never runs

The gate sets this value for any command containing cd/pushd (line 699), but bashAfter is skipped whenever the call is denied: tool-chain.ts:149 returns the deny before afterCall, so a refusal from any gate that runs after health (sqlGraph, docs, cteProfile at features/index.ts:296-298) leaves the value set. It is also left set when the bounded after-phase is abandoned at PHASE_TIMEOUT_MS before line 742 (features/index.ts:310), since $.session.root() on line 738 is awaited first. And sessionStart (lines 638-643) resets generation, known, projects and drawn but not this value, so it can outlive the session that captured it.

A stale folder only changes the answer once the shell has moved since, but then a later relative write is resolved under the wrong folder: markChanged compares the two project folders (lines 751-756) and drops the edit, so the model is never marked changed and the push gate never learns of it.

Cheap fix: capture and clear before the first await in bashAfter (move lines 741-742 above line 738), and reset shellBefore = null in sessionStart with the rest of the per-session state. Storing { command, folder } and consuming it only for that command would make the coupling explicit.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f352d99: the note now carries its command and bashAfter uses it only for that same command, and sessionStart clears it, so a denied or abandoned call cannot leak a folder into a later command. No dedicated test: the chain-deadline and denied-by-a-later-gate paths cannot be driven through the harness's mock clock.

…ounded decision

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
return null
if (!(options.healthBlocksPush === true || options.healthBlocksPush === 'true') || !isGitPush(e.command)) {
// Where the shell is before this command runs: its own `cd`s are read from there, and `bashAfter` sees the folder after them.
if (CHANGES_FOLDER.test(e.command)) shellBefore = await within($, GATE_WAIT_MS, shellFolder($))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION]: On a deadline within yields null, so this assigns the same value as "no cd" and the write is then resolved from the folder the cd left

within resolves null when the wait expires (hooks/altimate-cli.ts:62-74), so a stalled session.cwd leaves shellBefore = null, which bashAfter cannot tell apart from a command without a cd. It then falls back to shellFolder($) (line 745) — the folder after the cd — while filesWrittenBy already keeps the cd target as a prefix (health.tsx:406). cd models/staging && sed -i 's/a/b/' stg_orders.sql is therefore recorded as <post-cd>/models/staging/stg_orders.sql: isModelFile accepts it as a project .sql, no insight names it, and the real model is never marked changed, so the check and the gate never see the edit — the fail-open this capture exists to prevent.

Keep the timeout distinguishable instead of nulling the value (e.g. mark the pre-command folder unknown and resolve those names against the session root), and pin it with a test: the new test stalls the lookup on a git push, which goes through decide, so replacing this within with a plain await shellFolder($) leaves the suite green.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f352d99: a timed-out lookup is recorded as unknown, and a command with its own cd whose start is unknown is read from the session's root (where it began) instead of the folder the cd left. No dedicated test for the timeout branch (the harness cannot drive the chain deadline).

expect(allowed?.deny).toBeUndefined()
})

test('with the push gate on, a push whose shell-folder lookup stalls is held, not let through', { options: { healthBlocksPush: true } }, async ($, on) => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION]: This stall is caught by the pre-existing outer wait, not by the line it was added for

The lookup stalls inside decide (health.tsx:715), and decide has been inside within($, GATE_WAIT_MS, decide($, e)) since before this PR, so the assertion is the same could not be read in time that the existing "a push while a re-check is still running is held" test (line 521) already makes. The branch this PR adds — the bounded capture for a cd that is not a push (health.tsx:700) — is never reached: no test runs a cd + non-push command against a stalled session.cwd, so reverting that within to a plain await keeps the suite green. Making this command a non-push one (e.g. cd models && sed -i 's/a/b/' models/staging/stg_orders.sql) would cover the new line instead.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged: that test pins the outer bounded decision, as the existing re-check test does. The cd + non-push capture bound is covered by review only, since a stalled session.cwd there needs the chain's real 5 s deadline, which the harness's mock clock does not drive.

…command, and an unknown start reads from the session root

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@hkaltai

hkaltai commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

review-loop: APPROVED at f352d99 (reviewers: gpt-6-astra high, gpt-6.1-sol high)

// A note belongs to its own command; a command with a `cd` whose start is unknown is read from the session's root, where it began.
const noted = shellBefore?.command === e.command ? shellBefore : null
shellBefore = null
const here = noted ? (noted.folder ?? root) : CHANGES_FOLDER.test(e.command) ? root : await shellFolder($)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: the unknown-start fallback assumes the shell began at the session root

Both fallbacks here resolve relative writes against $.session.root(), but cwd legitimately differs from the root: this feature's own test at health.test.tsx:534 runs with cwd = ${ROOT}/models/staging, and shellFolder falls back to the root only when the host does not say.

So in a session whose shell an earlier cd models/staging moved, a command with its own cd whose gate lookup is given up at GATE_WAIT_MS (note folder null, line 733) is read from the root: cd seeds && sed -i 's/a/b/' x.sql is recorded as <root>/seeds/x.sql instead of <root>/models/staging/seeds/x.sql. isModelFile rejects it, touched stays empty, and decide returns 'run' at line 720 — the edit is never checked and the push is not held. The same happens for the noted === null branch if the gate phase is dropped before the note is written.

This narrows the fail-open the previous round raised (root is right when the shell has not moved) but does not close it. Recording the last folder the shell was known to be in, and using that rather than the root when the start is unknown, would.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed: this needs a stalled session.cwd (an engine fault; it answers in milliseconds) on a command that has its own cd, in a session whose shell an earlier call had already moved off the root. Reading from the session root is the stated fallback for an unknown start, and it is strictly narrower than before (previously the folder after the cd). Edge case beyond the supported input.

return null
if (!(options.healthBlocksPush === true || options.healthBlocksPush === 'true') || !isGitPush(e.command)) {
// Where the shell is before this command runs: its own `cd`s are read from there, and `bashAfter` sees the folder after them.
if (CHANGES_FOLDER.test(e.command)) shellBefore = { command: e.command, folder: await within($, GATE_WAIT_MS, shellFolder($)) }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: neither new fallback is exercised by a test

The bounded capture added here and the two new root fallbacks at line 750 have no test. Every holdCwd case (health.test.tsx:563) is a git push, so it goes through decide and times out in the outer within($, GATE_WAIT_MS, decide($, e)) at line 705; it never reaches this line or bashAfter. Reverting this line to a plain await shellFolder($), or collapsing line 750 back to await shellFolder($), leaves the suite green.

A non-push case, e.g. cd models && sed -i 's/a/b/' x.sql with holdCwd stalling, would pin the bounded capture and the unknown-start fallback together.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged, not added: a stalled session.cwd on a non-push command only reaches this line past the chain's real 5 s deadline, which the harness's mock clock cannot drive, so a test would be a 5 s wall-clock test. The same limit applies to the earlier thread on this fallback.

@sahrizvi sahrizvi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consensus review, round 3 (panel: Claude as lead, Codex): approve.

Reviewed 100897ce7a..f352d99a59 (the 8 commits since round 2 and the merge of main). No other plugin file differs from main except docs.test.tsx, whose mock now answers health's own dbt_health call. Both panel members finished and agree on approve.

The round-2 block is resolved. README "How it works" item 4, the Settings intro and plugin.json now say what the code does. In an interactive session inside a dbt project, or with healthBlocksPush on, the check runs dbt_health with write at session start, without asking. That runs dbt deps and refreshes target/, and there is no separate off switch. plugin.json no longer says the check is missing. I checked this against the code (sessionStart at line 652 and check() at line 480) and against the real tool from AltimateAI/altimate-code#1738 (17bbc24b04). A cold run created target/, dbt_packages/, logs/ and .user.yml in 15 s. Without --allow-write the tool refuses. A re-check after an edit took 2 s and named the new pair.

The shell-folder logic works on the ordinary flows. I ran 8 kit probes at head, all as expected:

  • edit a model, then git push from the root: runs when clean, held with a new issue;
  • cd models && git push: runs when clean;
  • a push from a folder an earlier cd left: held with a new issue;
  • a relative echo >> stg_orders.sql from that folder: tracked;
  • cd staging && echo x >> stg_orders.sql with the gate off: tracked from the start folder;
  • no dbt project: nothing is checked and the push runs;
  • an engine without session.cwd: the session root is used.

The push decision, folder lookup included, is still bounded at 4 s and still fails closed. A stalled lookup gives a hold.

Remaining minors (inline). None blocks:

  • Overlapping Bash calls clear the folder note (line 748). Edge case: it needs parallel subagents, and the result is a missed check.
  • cd inside a subshell leaks, and pushd is ignored (line 406). Edge case, open since round 1.
  • ${#var} hides a push that follows it (line 266). Edge input that fails open.
  • The gate acts in a session opened above the dbt project (line 718). Unusual layout.
  • The lost wake-up (line 589). Edge case: the next push or /health recovers it.
  • Test gaps: f352d99's two rules and state.running (line 750). Test-only.
  • The healthBlocksPush text leaves out the holds on a failed or unfinished check (plugin.json). Wording only.

Nits are in the combined comment: no CHANGELOG entry, logs/ and .user.yml not named, and the unmatched-YAML widening still untested.

Checks at head vs base (run one after another):

  • claude plugin validate, --strict, and --strict .: pass at both.
  • claude plugin test: 588 pass, 0 fail at head (529 pass, 0 fail at base). The 15 failures from round 2 are gone.
  • tests/*.sh: 7 of 7 pass at both.
  • pytest: 65 pass at both.

No live claude --plugin-dir session: with an isolated HOME it stops at the login screen. The hooks ran through the kit, and the tool output came from the real CLI.

Merge order: AltimateAI/altimate-code#1738 first, then this PR.

// A relative name is relative to the folder the session's shell is in now: an earlier `cd` moved it.
// A command with its own `cd` is read from the folder it started in, which the gate noted.
// A note belongs to its own command; a command with a `cd` whose start is unknown is read from the session's root, where it began.
const noted = shellBefore?.command === e.command ? shellBefore : null

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor (edge case; Codex found it). shellBefore is one module-level slot, and any bashAfter clears it. Two Bash calls that overlap lose the start folder. For example, call A is cd staging && echo x > a.sql with the shell in /repo/models, and call B finishes in between (a subagent's call, say). A is then read from the session root, as /repo/staging/a.sql, and the real model goes unchecked. Claude Code runs a writing Bash command on its own, so this needs parallel subagents. The result is a missed check, not a wrong refusal. Safe in a follow-up: key the note by the tool call's id, if the chain has one.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. This is a real but non-blocking edge case; #33 is merged now, so it goes into a follow-up PR on health.tsx rather than this one. The start folder will be keyed by call instead of one module slot.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in #41: the start folder is kept per call (keyed by the call's tool_use_id, the command when it has none), so an overlapping Bash call no longer clears it. Test: 'a Bash call that finishes while another is running leaves the other its own start folder'.

const at = leaderEnd(words)
const bin = (words[at] ?? '').split('/').pop() ?? ''
const args = words.slice(at + 1)
if (bin === 'cd') cwd = args[0] && !args[0].startsWith('-') && !args[0].startsWith('~') ? from(args[0]) : ''

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor (edge case; open since round 1, raised again by Codex). filesWrittenBy keeps cwd after a subshell ends and ignores pushd. (cd models/staging && echo x > a.sql); echo y > models/marts/b.sql records models/staging/models/marts/b.sql, so b is never checked. This is a constructed cd chain that misses a check, and the author has acknowledged it as a known limit. It does not block.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. This is a real but non-blocking edge case; #33 is merged now, so it goes into a follow-up PR on health.tsx rather than this one. Until then it stays a documented limit of the constructed-cd-chain parser (subshell scope and pushd are outside the supported input).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open as a known limit (subshell cwd leak, pushd): not part of #41.

}
for (let i = 0; i < line.length; i++) {
const c = line[i]!
if (c === '#' && !inWord) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor (edge input; new with the comment rule). { ends the word (it is in OPERATORS), so the # of a parameter expansion starts a "comment". I ran it through the kit: isGitPush('echo ${#arr[@]}; git push') === false, so that push is not held. The gate fails open only on this unusual form, so it does not block. A fix is to treat # straight after ${ (and $#) as part of the word.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. This is a real but non-blocking edge case; #33 is merged now, so it goes into a follow-up PR on health.tsx rather than this one. The tokenizer will treat # after ${ and $# as part of the word.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in #41: the # of ${#name} is part of the word, not a comment; echo ${#arr[@]}; git push is a push. Test in 'a comment never hides the push...'.

// An edit in the same command would run before any check of it.
const here = await shellFolder($)
if (CHANGES_FOLDER.test(e.command)) shellBefore = { command: e.command, folder: here }
for (const name of filesWrittenBy(e.command)) if (await isModelFile($, name.startsWith('/') || /^[A-Za-z]:/.test(name) ? name : `${here}/${name}`)) return hold(`this command changes the model file ${name} and pushes in one go, so the change is not checked first. Run them as separate commands.`)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor (unusual layout; Codex found it). decide runs before it checks enabled. In a session opened at /repo with the dbt project at /repo/dbt, health is off for the session, yet cd dbt && echo x > models/a.sql && git push is held and told to run /health, which cannot check anything there. Separately, bashGate looks up the shell folder (bounded at 4 s) for every Bash command that contains cd, even with health off or outside any dbt project. The engine answers that lookup in milliseconds, so it costs nothing in practice. Neither blocks. Returning early when !enabled would cover both.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. This is a real but non-blocking edge case; #33 is merged now, so it goes into a follow-up PR on health.tsx rather than this one. decide will check enabled first and the cd lookup will be limited to commands that name a push.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in #41: bashGate and bashAfter return at once when the session is not enabled (no dbt project above the session root), so no shell-folder lookup and no hold. Test: 'a session opened above its dbt project holds nothing'.


/** Starts the check loop unless one runs; resolves when the loop that is running ends. */
function kick($: EngineInterface): Promise<void> {
if (running && running.generation === generation) return running.done

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor (edge case; unchanged since round 2). The lost wake-up is still there. An edit whose turn ends between the loop's last read and running = null gets the old done promise back from kick, and nothing re-checks it until the next push or /health. The gate recovers through wantsCheck(retry), so a push is never let through on it. The author has acknowledged it. It does not block.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, accepted as a documented edge case: the next push or /health recovers it through wantsCheck(retry), so a push is never let through on it. Any change to the wake-up goes into the follow-up PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, accepted as a documented edge case: the next push or /health recovers it through wantsCheck(retry), so a push is never let through on it. Any change to the wake-up goes into the follow-up PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open as a known limit (lost wake-up; the next push or /health recovers it): not part of #41.

// A note belongs to its own command; a command with a `cd` whose start is unknown is read from the session's root, where it began.
const noted = shellBefore?.command === e.command ? shellBefore : null
shellBefore = null
const here = noted ? (noted.folder ?? root) : CHANGES_FOLDER.test(e.command) ? root : await shellFolder($)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor (test-only). f352d99's two rules have no test. With const noted = shellBefore (no command match), or with this line's CHANGES_FOLDER.test(e.command) ? root : fallback removed, all 588 tests still pass. Removing state.running || at line 728 also still passes, because the test at health.test.tsx:521 sees the outer 4 s bound first. The unmatched-YAML widening after a baseline (line 567) is also untested. The author explained why the kit cannot drive the chain deadline. The note-matching rule can be pinned without the deadline, though: run a gated cd command, then a different command, and check which folder is used.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on the three missing tests (noted-folder fallback, state.running clause, post-baseline YAML widening); they go into the follow-up PR with the code they cover.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partly fixed in #41. Added: the per-call note rule (overlap test), the null start folder read from the session root, and the unnamed-YAML widening after a baseline (health.tsx:567). Declined: the state.running || clause at health.tsx:735 and the CHANGES_FOLDER.test(e.command) ? root fallback at health.tsx:770. The first is not observable on its own because the chain's outer 4 s bound reports first; the second needs a call whose gate never ran, which the gate's own 4 s bound (under the chain's 5 s) makes unreachable. Both stay as defensive code.

"title": "Project health blocks git push (not used yet)",
"description": "Not used yet: the project health check isn't in this version, so git push is never refused. When it ships, on refuses a push while the check finds new issues in the models you changed; off only shows them.",
"title": "Project health blocks git push",
"description": "On refuses a git push while the project health check finds new issues in the models you changed this session; off only shows them.",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor (wording; Codex found it). This says a push is refused "while the project health check finds new issues". After a model change, the gate also refuses when the check failed, did not finish within the 4 s wait, or has no baseline yet, even with zero issues (health.tsx:728). The README row at line 144 has the same gap. Adding one clause would match the code. It does not block, because the refusal text tells the user why.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed; the plugin.json and README wording will state that the push is also refused when the check failed, timed out or has no baseline. Follow-up PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already matches at main: plugin.json and the README row say the push is also held while the check cannot run or has not finished.

@sahrizvi

Copy link
Copy Markdown
Collaborator

Consensus review, round 3: findings not tied to a diff line, nits, and evidence (panel: Claude as lead, Codex)

Nits

  • CHANGELOG. plugins/altimate-code/CHANGELOG.md has no entry for project health, and the version is still 1.7.0. The author has said a release PR owns this.
  • What the write leaves behind. README item 4 names target/ and dbt deps. A cold run of the real tool also created dbt_packages/, logs/ and .user.yml in the project. Naming them would make the disclosure complete.
  • Unmatched-YAML widening untested. After a baseline, an edited YAML file that no insight names still makes every model count as edited (line 567). No test covers that branch.

Round-2 items: status at f352d99

Round-2 item Status Edge case or real
Major: the session-start write was not disclosed (README and plugin.json reverted) Resolved (01378cb): README item 4, the Settings intro and plugin.json match the code was real; now disclosed
A # comment with an apostrophe hid a push Resolved (01378cb), with a test. The new rule has its own edge, ${#var} (inline) edge
The late-baseline guard had no test Resolved: the guard mutant is now killed. The widening after a baseline is still untested (nit) test-only
Late baseline: document the limit Resolved: the README healthBlocksPush row states both directions design limit, documented
clock.now() ran outside the try Resolved: it is inside the try now edge
Lost wake-up at the end of loop Open, acknowledged by the author edge
state.running / configPathProblem tests Partly: the config-path wiring is now killed by a test. state.running still survives, because the outer bound wins first test-only
15 plugin tests failing from reset mocks Resolved: 588 pass, 0 fail —

Real CLI (AltimateAI/altimate-code#1738 at 17bbc24b04, built locally)

I ran it on a scratch dbt-duckdb 1.12.7 + altimate-datapilot-cli project with 3 models, with an isolated HOME.

Run Result
dbt_health --allow-write, cold 15 s, 8 insights. Created target/, dbt_packages/, logs/, .user.yml
without --allow-write refused: "re-run with --allow-write"
after adding a hard-coded reference to stg_orders 2 s. One new pair, stg_orders / hardcoded_refs, and none dropped
run from models/staging works (7 s)

Shell-folder probes (kit, gate on unless noted)

Flow Result
clean edit, then cd models && git push runs
clean edit, then git add -A && git commit -m "fix stg_orders.sql" && git push origin HEAD runs
new issue, git push with the shell in models/ held
echo x >> stg_orders.sql from models/staging, then cd ../.. && git push held
gate off: cd staging && echo x >> stg_orders.sql from models/ tracked and checked
no dbt project: cd sub && echo x > a.sql && git push runs, with no check
engine with no session.cwd session root used
gate off: cd command with a stalled folder lookup not held (bounded wait)

Mutation checks (588-test suite)

Mutant Result
# comment rule removed killed
late-baseline guard removed killed
configPathProblem wiring removed killed
the gate reads relative writes from the root, not the shell folder killed (2)
note used for any command (f352d99) survives
unknown-start fallback to root removed (f352d99) survives
state.running dropped from the hold survives

Suites (one after another; head f352d99 vs base 100897c)

Check Head Base
claude plugin validate plugins/altimate-code pass pass
claude plugin validate --strict plugins/altimate-code pass pass
claude plugin validate --strict . pass —
claude plugin test plugins/altimate-code 588 pass, 0 fail 529 pass, 0 fail
tests/*.sh (7) all pass all pass
pytest tests/ 65 pass 65 pass

tsc is not configured in the repo; the validator's type check stands in for it. There was no live claude --plugin-dir session, because an isolated HOME stops at the login screen.

Codex accepted every minor at the severity above. It first rated the overlapping-call and subshell-cd findings major, then accepted minor under this round's bar: both are missed checks on constructed or parallel cd chains, not wrong refusals or data loss. No finding was rejected.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@hkaltai
hkaltai merged commit b678c25 into main Nov 18, 2026
4 checks passed
@hkaltai

hkaltai commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the round-4 review. #33 is merged at ee9b55a, so the remaining minors (per-call start folder for the cd tracking, ${#arr} in the push scanner, decide before the enabled check, the three missing tests, and the gate wording in plugin.json/README) will go into a follow-up PR. The subshell/pushd cd chain and the lost wake-up window stay documented limits; the push is never let through on either.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants