Repository navigation
feat(runs): record each dbt run's results and show them in /runs - #27
Conversation
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…t wrote it, keep Re-run complete Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…is per project Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Review loop on this PR: two independent reviewers (
Left as known limits (round 2, not fixed): history is kept per session folder (50 runs shared by projects under it); a |
…ead per project folder Where a command ran is read only from plain cd statements; anything the shell could read differently (quotes, variables, subshells, pipes, comments, if, pushd, eval) records nothing instead of another project's run. A run written with --target-path or DBT_TARGET_PATH is read from that folder. A call that runs dbt in several projects records each; reads run together and one failing does not lose the others. Calls that overlap in one artifact folder are not recorded, since the file does not say which command wrote it. History keeps 50 runs per project, newest first by completion. Bare secret names (TOKEN, PASSWORD, KEY) are left out of Re-run. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Review-loop result (Claude + Codex, independent reviewers)Two independent external reviews, then three attack-the-fix gates. Replies to every finding raised against
The last gate's findings were all fixed after it ran and covered by tests, but that last round was not re-gated. 240 plugin tests pass. Demo video (real Claude Code session with this plugin and the altimate-code branch CLI, a dbt-duckdb project): |
|
Final-gate review (independent Claude + Codex CLI reviewers on HEAD). Open, NOT fixed (so this PR stays a draft):
I tried a redesign (any overlapping dbt call in the session blocks recording, no folder comparison, 30-minute staleness for never-ended calls) but could not run the plugin tests here ( |
|
Verify-only gate: no commits since the final-gate review, so nothing new to approve. The three MAJOR findings in the runs.tsx overlap guard stay open and this PR stays a draft until the owner picks the intended trade-off (block on any overlapping dbt call vs. compare per folder). |
…t, in any folder; otherwise Claude is told Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ll over an hour is not recorded Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Replies to the three open MAJOR findings in the
Review of the new commits (Claude + Codex, separate processes) found more, all fixed with tests that fail on the earlier code: bookkeeping ran after an await, so two calls ending together could both look alone; Gate status: the first verify pass on db4f1c1 returned REQUEST CHANGES from both reviewers. The second, on 040d424, returned APPROVED from Claude and REQUEST CHANGES from Codex (the hour-expiry case). That is fixed in 3fd6303 with a test (248 plugin tests pass), but no reviewer has seen it, and the gate's two-pass limit is used up. So this PR stays a draft and has no APPROVED comment. |
…ping it is not recorded Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
review-loop: APPROVED at 2283219 Gate on the overlap-guard fixes: Claude and Codex both APPROVED. A real MAJOR found on the way (a backgrounded dbt call dropped out of the overlap bookkeeping) was fixed in 2283219 with a regression test that fails without the fix; 249 plugin tests pass. Accepted MINOR limits: backgrounding forms not ending in '&' (e.g. '& disown'), 1-hour age-out, 50/500 caps. |
Code Review SummaryStatus: 10 Issues Found | Recommendation: Address before merge Incremental review of Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (3 files)
Previous Review Summaries (4 snapshots, latest commit 63043bb)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit 63043bb)Status: 11 Issues Found | Recommendation: Address before merge Incremental review of Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (3 files)
Fix these issues in Kilo Cloud Previous review (commit 51bcf35)Status: 11 Issues Found | Recommendation: Address before merge Incremental review of Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (4 files)
Fix these issues in Kilo Cloud Previous review (commit bec921d)Status: 16 Issues Found | Recommendation: Address before merge Incremental review of Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (4 files)
Fix these issues in Kilo Cloud Previous review (commit 087ab58)Status: 14 Issues Found | Recommendation: Address before merge Incremental review of Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (4 files)
[Snapshot truncated.] Additional previous summary content was truncated to keep this comment within platform limits. Reviewed by deepseek-v4.1-flash-1017 · Input: 72.1K · Output: 15.2K · Cached: 547.3K |
…ng feature register no longer drops the tool hook, doc and alias tidy Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
review-loop: APPROVED at e8f97f5 |
…ompletion time Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…p, skip ambiguous chained builds Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ar retries when it is skipped Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
| const varsHoldSecret = (body: string) => | ||
| [...body.replace(/\\\r?\n\s*/g, ' ').matchAll(VARS_VALUE)].some(match => holdsSecret(match[1]!.startsWith('"') ? match[1]!.replace(/\\(.)/g, '$1') : match[1]!)) | ||
| /** A password in the URL's user part, or a secret-named query parameter (`?access_token=`, `&api_key=`): the URL holds a credential whatever it is named. */ | ||
| const CREDENTIAL_URL = /:\/\/[^\s/:@"']+:[^\s@"']+@|:\/\/[^\s"']*[?&;][\w.-]*(?:secret|token|passw|pwd|credential|apikey|api_key|api-key|auth|signature|sig|key)[\w.-]*=/i |
There was a problem hiding this comment.
WARNING: The query-parameter test is a substring match, so a benign URL drops the run and clears every stored retry
The keyword alternatives are matched anywhere inside the parameter name ([\w.-]*(?:…|auth|sig|key)[\w.-]*=), so https://example.com/x?author=alice (auth), ?sort_key=name (key), ?assign=1 (sig) and ?monkey=1 (key) all count as a credential. resultsInvocations then returns nothing for the call (runs.tsx:243), and record reads the missing invocation as a folder it cannot tell (runs.tsx:511-513): the run is not stored and the retryLine of every run in the session is cleared. That is the same false-positive class the --vars key check was changed to avoid by splitting the key into words (keyIsSecret, runs.tsx:160).
Judging the parameter name with keyIsSecret (plus sig/signature as explicit names) instead of matching a substring keeps access_token dropped and author/sort_key recorded; asserting ?author= and ?sort_key= are recorded in the new test would pin it.
There was a problem hiding this comment.
Fixed in 51bcf35: the query-parameter test now matches strong words (secret, token, passw*, pwd, credential) anywhere in the name but auth/sig/key/apikey/signature only as a whole _ - . word, so author=, assign= and monkey= are no credentials. sort_key= still counts (key as the last word); that fails safe (the run is not recorded) and is a documented limit. Test: 'a query parameter is a credential only when its name is one'.
| const readEnded = await $.clock.now() | ||
| entry.ended = readEnded | ||
| // A call still running that started after this one ended and writes results may have replaced the artifact as well. | ||
| if (finished.some(other => other !== entry && other.at >= ended) || [...started.values()].some(other => other.writes && !other.background && other.at >= ended)) { |
There was a problem hiding this comment.
WARNING: The new still-running clause looks at writes only, so an artifact-touching operation slips through
The added clause catches a writer that started after this call ended and is still listed in started, but other.writes excludes exactly the commands this file already classifies as replacing or deleting run_results.json without writing results (isOperation, runs.tsx:385-387: dbt run-operation, dbt docs generate, dbt clean). A call removes its started entry only when its own record runs (runs.tsx:480), so an operation that started after this call ended and finished during the read is still in started while its bookkeeping has not run, yet the check skips it: the artifact it wrote is then read and stored under this call's command line and retry line.
[...started.values()].some(other => (other.writes || other.operation) && !other.background && other.at >= ended) closes that window without disturbing the dbt ls case the writes filter exists for.
There was a problem hiding this comment.
Fixed in 51bcf35: the still-running clause is now (other.writes || other.operation) && other.at >= ended (runs.tsx, the check after the read). Test: 'an operation that started while the artifact was being read and is still running leaves the run unrecorded' (fails on the previous code).
| if (isBackground(e) || hostBackgrounded(result)) { | ||
| // It keeps running and may replace the artifact any time: stored runs can no longer be retried from it. | ||
| const kept = started.get(startKey(e)) | ||
| if (kept) kept.background = true |
There was a problem hiding this comment.
SUGGESTION: began already holds this entry
const began = started.get(startKey(e)) (line 470) is the same map entry and nothing between the two awaits, so this second lookup is redundant. Using began here and on the next line (if (began) began.background = true, if (began?.writes || began?.operation)) reads the entry once and keeps the flag on the object the rest of record already uses.
There was a problem hiding this comment.
Fixed in 51bcf35: the backgrounded branch uses began (one lookup, one object); the second started.get is gone.
…ckgrounded call that started during the artifact read leaves the run unrecorded Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
| const readEnded = await $.clock.now() | ||
| entry.ended = readEnded | ||
| // A call still running that started after this one ended and writes results may have replaced the artifact as well. | ||
| if (finished.some(other => other !== entry && other.at >= ended) || [...started.values()].some(other => other.writes && other.at >= ended)) { |
There was a problem hiding this comment.
WARNING: The post-read overlap check cannot tell what the other call wrote or where
Both clauses drop this run and call clearRetries (which blanks every stored retryLine in the session) for calls that cannot have touched this artifact:
finishedentries hold only{ at, ended }(runs.tsx:374), so the first clause matches any dbt call that started after this one ended and finished during the read, whatever it ran and wherever:dbt build --project-dir /bstarted while/a's read is in flight drops/a's run, and a finisheddbt ls(no artifact at all) does the same.- The second clause still filters
other.writes, which excludes exactly the commands this file classifies as replacing or deletingrun_results.jsonwithout writing results (isOperation, runs.tsx:385-387:run-operation,docs generate,clean,show). Such a call leaves itsstartedentry only when its ownrecordruns (runs.tsx:479), so one that started after this call ended and finished during the read is skipped, and the artifact it wrote is stored under this call's command line.
Putting writes and the artifact folders on the finished entry, and testing (other.writes || other.operation) here, keeps the window this guard exists for without discarding runs whose file was untouched.
There was a problem hiding this comment.
Partly fixed in 51bcf35: the still-running clause now includes operations (run-operation, docs generate, clean, show), with a regression test. The finished-entry clause is unchanged on purpose: telling which folder or artifact another finished call touched needs the folders on every entry, and any overlap in time dropping the run (and clearing retries) is the conservative rule chosen for this guard (runs.tsx:374 finished): a wrongly attributed run is worse than a dropped one.
| if (isBackground(e) || hostBackgrounded(result)) { | ||
| // It keeps running and may replace the artifact any time: stored runs can no longer be retried from it. | ||
| const kept = started.get(startKey(e)) | ||
| if (kept?.writes || kept?.operation) await clearRetries($) |
There was a problem hiding this comment.
SUGGESTION: began already holds this entry
const began = started.get(startKey(e)) (runs.tsx:470) is the same map entry, and nothing awaits between the two lookups, so this second started.get is redundant. Using began here (if (began?.writes || began?.operation) await clearRetries($)) and dropping line 475 keeps one read and one object for the rest of record.
There was a problem hiding this comment.
Fixed in 51bcf35: same change as the sibling thread, began is read once and reused.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
| /** The words a `--vars` key is made of (`apiKey`, `db_password`, `auth-token`), lower case. */ | ||
| const keyWords = (key: string) => key.replace(/([a-z0-9])([A-Z])/g, '$1_$2').toLowerCase().split(/[^a-z0-9]+/).filter(Boolean) | ||
| /** Words that name a secret wherever they sit in a key, also run into a longer word (`clientsecret`, `accesstoken`). */ | ||
| const SECRET_PART = /secret|token|passw|pwd|credential|passphrase|authoriz|authoris/ |
There was a problem hiding this comment.
WARNING: The authoriz/authoris prefix also matches benign authorized_* / authorization_* keys, so the run is dropped and every stored Re-run failed is cleared
keyIsSecret tests SECRET_PART against the joined key (line 164), so any key containing those letters counts: --vars '{"authorized_schemas": "analytics"}', --vars '{"authorization_type": "rbac"}' and --vars '{"reauthorize": "daily"}' all count as holding a secret, and authorized_view_names is caught through VARS_PARENT too. varsHoldSecret then makes resultsInvocations return nothing for the call, and record reads the missing invocation as a folder it cannot tell (lines 510-512): the run is not stored and the retryLine of every run stored in the session is blanked.
authori[sz]ation keeps the case this commit adds (authorization, Authorization, http_authorization) without swallowing authorized_*/authorize, and a negative test would pin the boundary. The keyIsSecret doc at lines 152-158 still lists the secret words without mentioning authorization.
There was a problem hiding this comment.
Fixed in 51bcf35: SECRET_PART now names authorization/authorisation instead of the authoriz/authoris prefix, so authorized_schemas and reauthorize are no secrets while authorization / authorization_type still are (the latter fails safe).
| await running | ||
| }) | ||
|
|
||
| test('an authorization key in --vars is a secret', async ($, on) => { |
There was a problem hiding this comment.
SUGGESTION: Only the secret half of the new rule is pinned
Both assertions expect no invocation, so the test cannot see how far the new prefix reaches: a benign key from the same family (--vars '{"authorized_schemas": "analytics"}', {"authorization_type": "rbac"}) currently returns nothing as well, and no assertion would notice a fix that narrowed or widened the rule. Adding expect(resultsInvocations('dbt build --vars '{"authorized_schemas": "analytics"}', ROOT)).toHaveLength(1) would pin the non-secret side.
There was a problem hiding this comment.
Fixed in 51bcf35: the test now also pins the benign side (authorized_schemas, reauthorize are recorded; authorization_type and authorization are not). Test: 'authorized_* and similar keys are not secrets; authorization still is'.
… and a dbt call backgrounded before another command stays in flight Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
| /** Whether the command runs a dbt subcommand that writes run_results.json, whatever folder or project it names. */ | ||
| export const writesResults = (command: string) => parseDbtCommand(command).some(one => one.subcommand !== null && WRITES_RESULTS.has(one.subcommand)) | ||
| /** Whether the call hands dbt to the background: the tool returns at once, the command is still running. */ | ||
| const isBackground = (e: BashCall) => (e as { run_in_background?: unknown }).run_in_background === true || /(?<![&>])&(?![&>])/.test(e.command) |
There was a problem hiding this comment.
WARNING: Any & anywhere counts as backgrounding, including one inside a quoted value or in a |& pipe
The pattern now matches a single & anywhere in the command text rather than only a trailing control operator. dbt build --vars '{url: "https://api.example.com/v1?x=1&y=2"}' (or dbt build -s a |& tee build.log) therefore takes the isBackground branch at runs.tsx:473: nothing is recorded for the call and, because began.writes is true, its clearRetries blanks the retryLine of every run stored in the session. Stripping quoted spans before this test and treating & as a control operator only (/(?<![&>|])&(?![&>])/) keeps the ... & echo started spelling this commit adds while leaving a quoted & alone.
There was a problem hiding this comment.
| const varsHoldSecret = (body: string) => | ||
| [...body.replace(/\\\r?\n\s*/g, ' ').matchAll(VARS_VALUE)].some(match => holdsSecret(match[1]!.startsWith('"') ? match[1]!.replace(/\\(.)/g, '$1') : match[1]!)) | ||
| /** A password in the URL's user part, or a secret-named query parameter (`?access_token=`, `&api_key=`): the URL holds a credential whatever it is named. */ | ||
| const CREDENTIAL_URL = /:\/\/[^\s/:@"']*:[^\s@"']+@|:\/\/[^\s"']*[?&;][\w.-]*(?:secret|token|passw|pwd|credential|apikey|api_key|api-key|auth|signature|sig|key)[\w.-]*=/i |
There was a problem hiding this comment.
WARNING: The query-parameter keyword is a substring match, so a benign URL drops the run and clears every stored retry
[\w.-]*(?:…|auth|sig|key)[\w.-]*= matches the keyword anywhere inside the parameter name, so https://example.com/x?author=alice (auth), ?sort_key=name (key), ?assign=1 (sig) and ?monkey=1 (key) all count as credentials. resultsInvocations then returns nothing for the call (runs.tsx:243) and record reads the missing invocation as a folder it cannot tell (runs.tsx:510-512): the run is not stored and every stored retryLine is blanked. Judging the parameter name with keyIsSecret (plus explicit sig/signature) keeps access_token dropped and author/sort_key recorded.
There was a problem hiding this comment.
Fixed in 51bcf35: see the sibling thread; the credential keyword is matched on whole _ - . words for auth/sig/key, so author= and assign= and monkey= no longer count. sort_key= still does and that fails safe (documented limit).
| return SECRET_PART.test(joined) || words.includes('auth') || words.includes('oauth') || last === 'pass' || ((last === 'key' || last === 'keys') && !KEY_KINDS.has(words[words.length - 2] ?? '')) || KEY_COMPOUND.test(joined) | ||
| } | ||
| /** A key that names a secret, unless its value is a number, a boolean or null: `token_limit: 3` holds none. */ | ||
| const pairHoldsSecret = (key: string, value: string) => keyIsSecret(key) && !/^(?:-?\d+(?:\.\d+)?(?:[eE][+-]?\d+)?|true|false|null|~)$/i.test(value) |
There was a problem hiding this comment.
SUGGESTION: The number test never sees a quoted scalar, and the tightened pattern now rejects ISO dates
The value comes straight from VARS_PAIR, which keeps the surrounding quotes, so --vars '{"token_limit": "3"}' (or "true") still counts as a secret: the run is dropped (runs.tsx:236) and record clears every stored retryLine. The same now happens to a value the old \d[\d.eE+-]* accepted as a number, e.g. an unquoted date under a secret-named key (--vars '{token_expiry: 2026-01-01}'). Stripping the quotes before this test, and letting \d{4}-\d{2}-\d{2} through, keeps token_limit: 3 and a date non-secret while still treating 123-456-789 as one.
There was a problem hiding this comment.
Not changed: a quoted scalar ("3") or an ISO date under a secret-named key (token_expiry: 2026-01-01) is treated as a possible secret (runs.tsx:167 pairHoldsSecret exempts only bare numbers, booleans and null). Not recording such a run fails safe, and quoted or date-valued vars under secret-named keys are outside the supported-input contract of this heuristic.
…ber exemption Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
| /** A key run together from `api`, `private`... and `key` (`apikey`, `privatekey`). */ | ||
| const KEY_COMPOUND = /(?:api|private|access|secret|encryption|signing|ssh|master|auth)keys?$/ | ||
| /** `key: value` pairs of a flow-style `--vars` value (JSON or YAML); a nested mapping's own pairs are found in turn. */ | ||
| const VARS_PAIR = /["']?([\w.-]+)["']?\s*:\s*("(?:[^"\\]|\\.)*"|'[^']*'|[^,}\s{[][^,}]*)/g |
There was a problem hiding this comment.
WARNING: The unquoted value now runs past a newline, so a later secret key is swallowed and never judged
[^,}]* includes \r and \n, so a --vars argument holding a valid multi-line YAML mapping without braces is judged only by its first key:
dbt build --vars 'country: US
db_password: hunter2'
VARS_PAIR then yields one pair, country: US\ndb_password: hunter2'. country names no secret, and since pairs.length > 0 the whole-text SECRET_NAME fallback in holdsSecret (line 172) never runs, so the invocation is recorded with the password in commandLine, drawn in /runs and pasted by Re-run. Before this change the value stopped at whitespace ([^,\s]*), so db_password matched as its own pair and the run was dropped.
Keeping the unquoted branch on one line ([^,}\r\n]*) still reads {db_password: 123 correct horse battery staple} whole and keeps the new test green; adding an expect(resultsInvocations(..., ROOT)).toEqual([]) case for the block-style spelling would pin it.
There was a problem hiding this comment.
Fixed in 51bcf35: the unquoted value in VARS_PAIR now stops at a line break (any run without a comma, closing brace or line break), so every line of an unbraced multi-line mapping is judged and db_password: hunter2 on a later line is caught. Test: 'a secret on a later line of an unbraced multi-line --vars value is judged too' (fails on the previous code).
… dbt call a background one Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
| /** Whether the command runs a dbt subcommand that writes run_results.json, whatever folder or project it names. */ | ||
| export const writesResults = (command: string) => parseDbtCommand(command).some(one => one.subcommand !== null && WRITES_RESULTS.has(one.subcommand)) | ||
| /** Whether the call hands dbt to the background: the tool returns at once, the command is still running. */ | ||
| const isBackground = (e: BashCall) => (e as { run_in_background?: unknown }).run_in_background === true || /(?<![&>])&(?![&>])/.test(e.command.replace(/"(?:[^"\\]|\\.)*"|'[^']*'|\\./g, ' ')) |
There was a problem hiding this comment.
WARNING: A |& pipeline is still read as a background call
Masking the quoted spans fixes the quoted &, but an & is still matched wherever it sits, so dbt build -s a |& tee build.log - bash's pipe-both, a foreground pipeline that dbt-command.ts:94,646 tokenizes as an operator with dbt on both sides - makes isBackground true. record then takes the background branch (runs.tsx:473): the call is never recorded and, because began.writes is set, clearRetries blanks the retryLine of every run stored in the session. |& is the only ampersand-bearing operator left unprotected, since the lookbehind excludes only & and >. Excluding | as well (/(?<![&>|])&(?![&>])/) keeps ... & echo started and the 2>&1 & spelling working while leaving a pipe-both alone; a test on dbt build -s a |& tee log would pin it.
There was a problem hiding this comment.
Fixed in 51bcf35: |& is no longer matched as a background & (lookbehind adds |, runs.tsx:391). Test: 'a pipe-both ampersand is a foreground pipeline, not a background call' (fails on the previous code).
| /** Whether the command runs a dbt subcommand that writes run_results.json, whatever folder or project it names. */ | ||
| export const writesResults = (command: string) => parseDbtCommand(command).some(one => one.subcommand !== null && WRITES_RESULTS.has(one.subcommand)) | ||
| /** Whether the call hands dbt to the background: the tool returns at once, the command is still running. */ | ||
| const isBackground = (e: BashCall) => (e as { run_in_background?: unknown }).run_in_background === true || /(?<![&>])&(?![&>])/.test(e.command.replace(/"(?:[^"\\]|\\.)*"|'[^']*'|\\./g, ' ')) |
There was a problem hiding this comment.
SUGGESTION: The quoted-span grammar now has a seventh copy in this file, and the copies disagree
This masking pattern is also written in SECRET_ASSIGNMENT (line 138), VARS_VALUE (140), VARS_PAIR (149), VALUE (188) and withoutState (193), while VALUE's quoted branch is still the narrower "[^"]*". That divergence is exactly the earlier retry-line bug (--vars written in double quotes was cut at the first escaped quote), and any further fix to how the shell quotes a word has to be repeated in every copy. One shared constant plus a small masking helper would leave a single definition and let this line reuse it.
There was a problem hiding this comment.
Not changed: the copies differ on purpose (each is anchored to a different surrounding grammar: assignment value, --vars value, mapping pair, flag value, word mask), and unifying them is a behaviour-neutral refactor of heuristic parsers that this PR's tests already pin. The VALUE divergence the earlier retry-line bug came from is covered by its own regression test. Left as is to keep the PR to its feature.
| const { stored } = await start($, on, () => view('inv-q')) | ||
| const first: any = await $.tool.call({ tool: 'Bash', command: `dbt build --vars '{source_url: "https://example.com/d?page=2&limit=10"}'`, tool_use_id: 'one' }) | ||
| expect(own(first.context)).toEqual(['build: 1 ok']) | ||
| expect(kept(stored)).toHaveLength(1) |
There was a problem hiding this comment.
SUGGESTION: The new test pins only the single-quote half of the mask
The & here sits inside a '...' span, so the double-quoted alternative of the mask at runs.tsx:391 (a quoted argument, including an escaped quote inside it) is not exercised by this test or by any other: such commands pass only because of that branch, and removing it would leave every test green. A second case with a double-quoted argument holding the & (and one with an escaped quote) would pin the half this test leaves free - dbt build -s "a&b" is enough for the plain branch.
There was a problem hiding this comment.
Fixed in 51bcf35: added 'an ampersand inside a double-quoted argument, escaped quote included, does not background the call', which pins the double-quoted half of the mask.
|
review-loop: APPROVED at bec921d (reviewers: gpt-6-astra high, gpt-6.1-sol high) Gate run on the whole diff at head; fixes from the rounds: escaped double-quoted |
…eground, an operation still running during the read leaves the run unrecorded, narrower credential matching, and each line of a multi-line --vars value is judged Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
@saravmajestic, point by point on the requested changes: (1) a denied call stays in flight: fixed earlier in 70e6116 (the chain reports a gate refusal or a refusal result as dropped and |
| const varsHoldSecret = (body: string) => | ||
| [...body.replace(/\\\r?\n\s*/g, ' ').matchAll(VARS_VALUE)].some(match => holdsSecret(match[1]!.startsWith('"') ? match[1]!.replace(/\\(.)/g, '$1') : match[1]!)) | ||
| /** A password in the URL's user part, or a secret-named query parameter (`?access_token=`, `&api_key=`): the URL holds a credential whatever it is named. */ | ||
| const CREDENTIAL_URL = /:\/\/[^\s/:@"']*:[^\s@"']+@|:\/\/[^\s"']*[?&;](?:[\w.-]*(?:secret|token|passw|pwd|credential)[\w.-]*|(?:[\w.-]*[._-])?(?:apikey|auth|signature|sig|key)(?:[._-][\w.-]*)?)=/i |
There was a problem hiding this comment.
WARNING: The narrowed query-parameter match now misses ?authorization=, the word this commit adds to the secret vocabulary
The second alternative only matches apikey|auth|signature|sig|key (optionally after a prefix that ends in ./_/-), so https://example.com/x?authorization=Bearer%20x no longer matches: auth is followed by o, and authorization has no [._-] before it, where the previous [\w.-]*(?:...|auth|...)[\w.-]*= matched it. varsHoldSecret does not help either - the key that holds the URL is u, and keyIsSecret('authorization') (which this commit makes true, runs.tsx:145) is never consulted here. The call is therefore recorded, the credential lands in commandLine, is drawn in /runs and is pasted by Re-run. The same happens for ?myauth= / ?preauth_key-less spellings.
The opposite side is still a false positive: the (?:[\w.-]*[._-])? prefix lets ?unique_key=, ?sort_key=, ?primary_key= and ?partition_key= match (prefix unique_ + key), although runs.test.tsx:477 pins those keys as recordable vars. resultsInvocations then returns nothing (runs.tsx:236) and record reads the missing invocation as a folder it cannot tell (runs.tsx:509-511): the run is not stored and every stored retryLine in the session is blanked.
Judging the parameter name with keyIsSecret (plus explicit sig/signature) makes both sides agree with the --vars rule: authorization secret, unique_key/sort_key not. A case for ?sort_key= and one for ?authorization= in the new test would pin it.
There was a problem hiding this comment.
Fixed in 63043bb: a URL query parameter name is now judged with the same keyIsSecret rule as a --vars key (plus sig/signature), so ?authorization= is a credential and ?unique_key=, ?sort_key=, ?primary_key= and ?partition_key= are not (runs.tsx, queryHoldsSecret). Test: 'a query parameter is judged like a --vars key: authorization is a credential, unique_key and sort_key are not'.
|
review-loop: APPROVED at 51bcf35 (reviewers: gpt-6-astra high, gpt-6.1-sol high) |
…tion tests build their fake credentials at run time Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
| const varsHoldSecret = (body: string) => | ||
| [...body.replace(/\\\r?\n\s*/g, ' ').matchAll(VARS_VALUE)].some(match => holdsSecret(match[1]!.startsWith('"') ? match[1]!.replace(/\\(.)/g, '$1') : match[1]!)) | ||
| /** A password in the URL's user part, or a secret-named query parameter (`?access_token=`, `&api_key=`): the URL holds a credential whatever it is named. */ | ||
| const CREDENTIAL_URL = /:\/\/[^\s/:@"']*:[^\s@"']+@/ |
There was a problem hiding this comment.
SUGGESTION: The doc comment above still describes the query-parameter check that moved out of this regex
Line 177 reads "A password in the URL's user part, or a secret-named query parameter (?access_token=, &api_key=): the URL holds a credential whatever it is named", but CREDENTIAL_URL now matches only the user-part password; query parameters are judged by queryHoldsSecret on line 180, under different rules (authorization is a credential, unique_key/sort_key/author are not). Leaving the user-part sentence on this regex and keeping the query rule on queryHoldsSecret's own comment would keep the two matchers from drifting apart again.
There was a problem hiding this comment.
Fixed in da6a109: the comment on CREDENTIAL_URL now describes only the user-part password; the query-parameter rule lives on queryHoldsSecret's own comment.
|
review-loop: APPROVED at 63043bb (reviewers: gpt-6-astra high, gpt-6.1-sol high) |
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
review-loop: APPROVED at da6a109 (reviewers: gpt-6-astra high, gpt-6.1-sol high) |
saravmajestic
left a comment
There was a problem hiding this comment.
Approving at da6a109. My requested change is resolved: a call a gate refuses, or whose result is a refusal, now reaches bashDropped, which takes it out of started, so a held command no longer marks every dbt run for the next hour as overlapping. The new tool-chain test covers both paths. claude plugin test plugins/altimate-code: 619 pass.
GitGuardian fails on 4 "Generic Password" hits, all fake credentials in older commits of runs.test.tsx (1a6c253, 386f44c, a4db340, ada2f32); the head builds them at run time. They need marking as false positives in GitGuardian.
Before the second of #24 / #27 merges: each passes claude plugin test plugins/altimate-code alone (#24: 608 pass, #27: 619 pass), but a merge of the two heads gives 688 pass / 10 fail. The repo's CI runs only tests/*.sh, so it won't catch this.
cost.test.tsx(4): runs' note "run not recorded: overlapping dbt commands" lands where the cost tests expect their own line ("a selection that is not one plain model…", "a command that sets its own target…", "the lines under a command are the latest estimate…", "what leaves the query alone is still estimated…").runs.test.tsx(6): the overlap tests ("two dbt calls that overlap are not told apart…", "an older call still running in the same folder…", "two spellings of one project folder overlap…", etc.) expect the runs to stay unrecorded, but with the cost feature present they're recorded. Removing the await in cost'sbashAfterdidn't change it, so please check whether this is only test timing or a real gap in overlap detection with both features on.
|
@saravmajestic thanks for the combined-merge check. It is test timing and fake-engine isolation, not a gap in runs' overlap detection. The cost gate refuses a second concurrent |
What
Every dbt command Claude runs is recorded with its results, shown in
/runs, and can be re-run through the prompt.build,run,test,seed,snapshot,compile,retryorclone, the plugin calls altimate-code'sdbt_run_resultsand records the run once per invocation id (an unchangedrun_results.jsonis ignored). Up to 50 runs per project are kept in the plugin store; messages are cut to 300 characters, rows to 50 per run, and the whole store is held under 1 MB.build: 41 ok, 2 errors: stg_orders, fct_sales./runsopens a pane: one button per run (command, seconds, passed of total); inside a run the failures come first. Re-run fills the prompt with the recorded command; Re-run failed (shown on the newest run when it has failures) fills it withdbt retry.The feature fills in
hooks/features/runs.tsxandRunsStateintypes/index.d.ts, plus a small hook (bashDropped) intool-chain.tsandfeatures/index.tsthat tells the feature when a call was refused and never reachesbashAfter.Depends on
The companion altimate-code PR (AltimateAI/altimate-code#1729) that implements
dbt_run_results. Merge that one first.Verification
claude plugin test plugins/altimate-code: 619 pass, includinghooks/features/runs.test.tsx(a run recorded once, the same file ignored, the pane on terminal and desktop, Re-run and Re-run failed filling the prompt, the 50-run, row, message and store caps).claude plugin validatepasses. Recorded commands keep no secrets:NAME=secretassignments are replayed asNAME="$NAME", and a--varsmapping, a URL password or a secret-named URL query parameter makes the command unrecorded. A dbt call that overlaps another one (finished, still running or backgrounded) leaves its run unrecorded rather than attributing the wrong artifact.Demo video (hosted)
P-11.mp4 — watch on Notion (Altimate workspace; recorded from the approved head of this PR).
🤖 Generated with Claude Code
Demo video: P-11.mp4 (re-recorded; 58s: dbt_run_results tool on a dbt fixture, then dbt builds in Claude Code and the /runs pane).