Repository navigation
fix: hands-on findings — real reach in the task band, hardcoded refs, shell edits, older doc gaps, pane follow and fit, package repos, value changes - #44
Conversation
… shell edits, older doc gaps, pane follow and fit, package repos, value changes - The task's manifest copies sit in target/ itself: altimate-code reads a seed's header from the folder above the manifest's, so a copy one folder deeper traced nothing through a select * and the band said 0 columns. A reach of 0 beside readers that may pass the column on unnamed is "not traced", never "every downstream model was read". - Compile and validate flags a model, seed or source named by its table instead of ref() or source(). - A model a shell command rewrites gets the per-edit impact note. - Only columns an edit made undocumented are its gaps; older ones are said once, quietly. - The lineage pane follows the edited model as the edit starts; a graph wider than the pane opens as its ends, with its dashboards named. - A package repo is compiled and validated from the project below it that installs it (integration_tests); missing packages are a one-line note. - With no select item changed, the band names the columns whose values change and their reach, else says values may change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| for (const [id, node] of Object.entries<any>(manifest?.nodes ?? {})) { | ||
| if (['model', 'seed', 'snapshot'].includes(node?.resource_type) && typeof node?.name === 'string') add(id, node, `{{ ref('${node.name}') }}`) | ||
| } | ||
| for (const [id, node] of Object.entries<any>(manifest?.sources ?? {})) { |
There was a problem hiding this comment.
WARNING: The hardcoded-relation check goes blind on manifests over 4 MiB
hardcodedRelations can only flag relations present in the manifest it is given, but for a manifest larger than MAX_MANIFEST_BYTES (dbt-mods.tsx:81) compileAndValidate passes the output of MANIFEST_SLICE (dbt-mods.tsx:294). That slice (a) never includes a sources key, so this loop always sees {}, and (b) keeps only the checked node's own parent_map/child_map neighbourhood (keep in MANIFEST_SLICE). A table written by hand instead of ref() is by definition not a parent in the manifest, so its node is normally not in the slice either. On a large project the new warning therefore never fires for the exact case it was added for (swapping {{ ref('x') }} for x), while hardcodedRelations still returns [], which reads as no hardcoded relations rather than not checked. Returning the relation catalogue from the slice (name/package_name/resource_type/relation_name for every model, seed, snapshot and source, plus sources) or reading it separately when the manifest is sliced would keep the check honest.
|
|
||
| // An identifier part: double-quoted, backquoted, bracketed or bare. | ||
| const PART = String.raw`(?:"[^"]+"|` + '`[^`]+`' + String.raw`|\[[^\]]+\]|[A-Za-z_][\w$]*)` | ||
| const RELATION_AFTER = new RegExp(String.raw`\b(?:from|join)\s+(${PART}(?:\s*\.\s*${PART}){0,2})`, 'gi') |
There was a problem hiding this comment.
SUGGESTION: from inside a function call is read as a relation
RELATION_AFTER matches any from/join word, including the ones inside function arguments: extract(day from orders), trim(both ' ' from customers), substring(name from 1). When the identifier after it happens to equal a model, seed or source name, the edit gets a false hardcoded-relation warning and Claude's note tells it to put the ref() or source() back, i.e. to rewrite valid SQL. Blanking the argument lists of extract/trim/substring-style calls before matching, or requiring the keyword to start a clause, avoids that.
| if (!dirs.length) return null | ||
| // Files touched in the last minute, then each one's own time against the command's start. | ||
| const listed = await $.process.run(['find', ...dirs, '-name', '*.sql', '-type', 'f', '-mmin', '-1'], { timeoutMs: 10_000 }).catch(() => null) | ||
| const top = (await $.process.run(['git', '-C', root, 'rev-parse', '--show-toplevel'], { timeoutMs: 5_000 }).catch(() => null))?.stdout.trim() ?? '' |
There was a problem hiding this comment.
SUGGESTION: git rev-parse runs even when nothing changed
WRITES matches very common commands (python3, node, cp, mv, tee), so this runs after most shell calls in a dbt session. When find lists nothing the loop below is empty, but this git rev-parse still spawns a process; an early return (e.g. if (!listed?.stdout.trim()) return { root, files: [] }) skips that spawn and makes the empty path explicit. The returned root is also unused by the only caller.
| 'type_int type_float type_timestamp type_boolean quote_column int integer numeric decimal varchar text float double boolean').split(' '), | ||
| ) | ||
| // A changed line that filters or joins rather than computes a value. | ||
| const NOT_A_VALUE = /^\s*(?:where|and|or|on|from|join|left|right|inner|full|cross|group|order|having|limit|qualify|union|\)|\{%-?\s*(?:if|endif|for|endfor|else))\b/i |
There was a problem hiding this comment.
SUGGESTION: The ) alternative can never match
The whole group is followed by a word-boundary assertion, but ) is a non-word character, so the ) branch matches only when the parenthesis is immediately followed by a word character. Lines it was meant to skip, such as ) as total, or ) and status = ..., are followed by whitespace and so are scanned for column names anyway. Put the ) branch (and the Jinja {% branch) outside the word-boundary guard so lines starting with ) are really ignored.
Code Review SummaryStatus: 5 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (5 files)
Fix these issues in Kilo Cloud Previous Review Summaries (7 snapshots, latest commit c89d73d)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit c89d73d)Status: 4 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (7 files)
Resolved since the last review (no longer listed): the Fix these issues in Kilo Cloud Previous review (commit 298e866)Status: 11 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (5 files)
Fix these issues in Kilo Cloud Previous review (commit 0d142ff)Status: 8 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (6 files)
Fix these issues in Kilo Cloud Previous review (commit 6ab8041)Status: 8 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (6 files)
Fix these issues in Kilo Cloud Previous review (commit a8d0fc1)Status: 9 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (6 files)
Fix these issues in Kilo Cloud Previous review (commit 2f0ac68)Status: 7 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (7 files)
Fix these issues in Kilo Cloud Previous review (commit ef50c07)Status: 4 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (12 files)
Reviewed by deepseek-v4.1-flash-1017 · Input: 104.7K · Output: 35.9K · Cached: 1.8M |
…outputs not CTE columns, edit gaps kept, runner keeps the repo's dbt and the model's version, shell search window, value-change false positives Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| const ch = code[i]! | ||
| if (ch === '(') depth++ | ||
| else if (ch === ')') depth = Math.max(0, depth - 1) | ||
| else if (depth === 0 && /[Ss]/.test(ch) && /^select\b/i.test(code.slice(i, i + 7)) && !/\w/.test(code[i - 1] ?? '')) last = i |
There was a problem hiding this comment.
WARNING: Only the final top-level SELECT is read, so earlier union branches lose their outputs
last is overwritten by every top-level select, and selectItems(sql.slice(last, end)) then reads only the last row-producing branch. A model like select id from a union all select id from b (or the usual {% if is_incremental() %} … {% else %} … {% endif %} pair) therefore yields only the last branch's names. When an edit adds a column to an earlier branch, now does not contain it and old() in undocumentedByEdit returns true on !now.has(key), so the edit's own new undocumented column is reported as "not documented, from before this edit" and no gap warning is raised. The previous selectItems(sql) saw every scope, so this is a regression for set-operation and conditional models: collect the items of every top-level SELECT instead of only the last one, which keeps CTE and subquery scopes out without dropping branches.
| let filtering = false | ||
| const added = now.flatMap(line => { | ||
| const clause = CLAUSE.exec(line)?.[1]?.toLowerCase() | ||
| if (clause) filtering = FILTERS.has(clause) |
There was a problem hiding this comment.
WARNING: A select/from inside a parenthesised predicate resets the filter state
filtering changes only on a clause word that starts a line and is not tied to parenthesis depth, so a predicate split over lines around a subquery is read as a value change:
where claim_id in (
select claim_id from other
)
and paid_amount > 0
The select/from lines set filtering back to false, so editing > 0 makes paid_amount a candidate and the task summary claims its values change. The reverse also happens: a select item that wraps onto left(name, 3) or right(...) starts with a FILTERS word and is dropped. Track paren depth and only let a depth-0 clause keyword (one not followed by () start or end a filter clause.
| contextGiven.clear() | ||
| oldGapsSaid.clear() | ||
| lastReturned.clear() | ||
| editGaps.clear() |
There was a problem hiding this comment.
WARNING: editGaps is not cleared when a conversation ends
A /clear runs endDbtMods and no session.start follows it (see features/index.ts), yet only oldGapsSaid and lastReturned are cleared there. The previous conversation's editGaps entry is still passed as ours to undocumentedByEdit, so a column an earlier conversation's edit left undocumented is reported as this edit's fresh gap (band warning and Claude's note) instead of the quiet "from before this edit" mention. Clear editGaps in endDbtMods too (or key it by generation).
| for (let i = 0; i < parts.length; i++) known.set(parts.slice(i).join('.'), { relation: parts.slice(i).join('.'), use }) | ||
| } | ||
| // A manifest too big to read here comes as a slice of the model's graph, with every relation listed apart. | ||
| const nodes: [string, any][] = Array.isArray(manifest?.relations) ? manifest.relations.map((one: any) => [String(one?.id), one]) : Object.entries<any>(manifest?.nodes ?? {}) |
There was a problem hiding this comment.
SUGGESTION: The relations are enumerated twice for a sliced manifest
With manifest.relations present, the first loop visits every entry only to drop sources and the second visits the array again to keep them, repeating the Array.isArray(manifest?.relations) branch; a full manifest reads a third collection. One normalised list (const relations = Array.isArray(manifest?.relations) ? manifest.relations : [...Object.values(manifest?.nodes ?? {}), ...Object.values(manifest?.sources ?? {})]) followed by a single pass that picks the ref() or source() wording by resource_type visits each relation once and removes the duplicated guard.
… the newest check only, the runner asks dbt ls for the file's version, CTE column lists, predicates split over lines Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| */ | ||
| export function outputColumns(sql: string, depth = 0): { names: Set<string>; star: boolean } | null { | ||
| const code = masked(sql) | ||
| const select = topLevel(code, 'select', 0, true) |
There was a problem hiding this comment.
WARNING: Only the last top-level SELECT is read, so union and if/else branches lose their columns
topLevel(code, 'select', 0, true) keeps only the final top-level select, so select id from a union all select id, amount from b (and the common incremental {% if is_incremental() %} / {% else %} pair) yields only the last branch's names. When an edit adds a column to an earlier branch, now in undocumentedByEdit does not contain it, old() returns true on !now.has(key), and the edit's own new undocumented column is filed as 'from before this edit', so no gap warning is raised. The previous selectItems(sql) saw every scope; collect the items of every top-level SELECT instead of only the last one - a depth-0 scan already keeps CTE and subquery lists out.
| let star = false | ||
| for (const item of topItems(list)) { | ||
| const bare = item.replace(/\s+/g, ' ').trim() | ||
| if (/^(?:[\w"`]+\.)?\*$/.test(bare)) { |
There was a problem hiding this comment.
WARNING: A * with a modifier is no longer recognised as a star
Only an item that is exactly * or t.* sets star. * exclude (col) and * replace (x as y) (DuckDB, BigQuery, Snowflake) match neither this check nor the name regexes below, so outputColumns returns empty names with star: false. undocumentedByEdit then trusts that empty set and classifies every returned column as old through !now.has, so a model such as with src as (select id, amount from raw) select * exclude (extra) from src never warns about a newly added undocumented column. The earlier detector treated any top-level * as a star and fell back to selectItems; set star when an item starts with an optionally qualified * unless the item is exactly a plain * (the CTE resolution below handles that case).
| const added = now.flatMap(line => { | ||
| const opens = CLAUSE.exec(line)?.[1]?.toLowerCase() | ||
| const filtering = opens ? FILTERS.has(opens) : carried | ||
| const clauses = [...line.matchAll(CLAUSES)].map(one => one[1]!.toLowerCase()) |
There was a problem hiding this comment.
WARNING: carried is set by a clause word anywhere on the line, including inside a string literal
CLAUSES scans the raw line, so , 'order' as record_type (or a column literally named order) sets carried to true and every later line without a clause word is treated as a filter and dropped from the value candidates. In a {% set %} column list, for example, the changed line , med.paid_amount / 100 is no longer named and the summary falls back to 'values may change downstream'. The reverse also remains: the scan is not paren-depth aware, so select/from inside where claim_id in (select ... from other) still resets carried, and a change inside that filtering subquery is read as a value change. Strip string literals before splitting the words, and only take clause words at depth 0.
| } | ||
|
|
||
| /** The node names `dbt ls --output json --output-keys name` printed: one JSON object per line, among dbt's own log lines. */ | ||
| /** The models `dbt ls --output json --output-keys "name version original_file_path"` lists, each with its file and version. */ |
There was a problem hiding this comment.
SUGGESTION: A second copy of the dbt ls JSON-line parser
listedFiles repeats listedNames' loop (strip ANSI, one line at a time, skip non-{, guarded JSON.parse) with only the field test changed, and it was inserted between listedNames' doc comment and listedNames, so that comment now sits above the wrong function. One helper that parses each JSON line once and returns the node, with listedNames and listedFiles reading the fields they need, removes the duplicate and the misplaced JSDoc.
…the file's model, /clear drops edit gaps, versioned ref advice, depth-aware value candidates, shell edits below the session folder Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| const n = left.get(line) ?? 0 | ||
| if (n > 0) left.set(line, n - 1) | ||
| const changed = !!line && n === 0 | ||
| for (const token of line.matchAll(/\(|\)|(?<![\w.])(?:[A-Za-z_]\w*\.)?([A-Za-z_]\w*)\b(\s*\()?/g)) { |
There was a problem hiding this comment.
WARNING: A ( that follows a word is swallowed, so every later line is read at the wrong depth
The identifier branch ends in (\s*\()?, so in (, exists (, max( match as one token (group 2): the ( never reaches the clauses.push above, while the closing ) still runs clauses.pop(). The level's clause is therefore overwritten or popped early:
where id in (select id, x from t): the(afterinis swallowed, soselectoverwrites the level'swhereinstead of opening a subquery level. A changed line inside that filter subquery then namesid/xas value candidates and the summary claims their values change - the false positive this rewrite is meant to remove.select (select max(a) from t) as m, amount from t:max(is swallowed and its)pops the real grouping level, sofromstays in force and a later changedamountline is dropped.
Only a bare ( (not directly preceded by an identifier) is pushed. Matching ( separately from the word, or pushing whenever token[2] is set, keeps the depth balanced.
| const base = await $.session.root().catch(() => '.') | ||
| const own = await projectRoot($, commandDir(start.command, base), false).catch(() => null) | ||
| // Below it only once a dbt project has been seen this session: a session that never touches dbt never walks its folder. | ||
| const roots = own ? [own] : projectSeen ? (await projectsBelow($, base).catch(() => [] as string[])).slice(0, MAX_BASH_PROJECTS) : [] |
There was a problem hiding this comment.
SUGGESTION: The new projects-below search has no test
This branch (and MAX_BASH_PROJECTS) is what makes a shell edit in a project below the session folder get an impact note, but lineage.test.tsx only exercises the own path (start() makes the session root a project), so a wrong root, a note duplicated when two roots overlap, or the cap not being applied would go unnoticed. A test whose session root sits above two projects, with the shell command rewriting a model in the second, would cover it.
…package-qualified ref advice, value parts compared, shell edits below the session folder Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…h every CTE, macro lists compared by their Jinja, set-block column lists, ambiguous relation names left alone Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| continue | ||
| } | ||
| const word = token[2]?.toLowerCase() | ||
| if (word && !token[1] && CLAUSE_WORDS.has(word)) { |
There was a problem hiding this comment.
WARNING: A ( right after a clause keyword is still swallowed, so its ) pops a level that was never pushed
The new token[3] push below balances a function call such as max(, but this branch continues before reaching it: when a clause word itself is followed by (, the paren is part of the same token and no level is pushed. In from (select ...), join (select ...), on (...) or where (...) the inner select/from then overwrite this level's clause, and the matching ) pops nothing (or an outer level when one is open), so the clause state after the subquery is wrong. A changed line inside a from (/join ( subquery is read as a value part, so its columns are offered as value candidates, and any whose name the manifest also lists among the model's columns is traced as a value change. Push token[3] in this branch too (before continue), as the function-call path does.
| for (let at = topLevel(code, 'select'); at >= 0; at = topLevel(code, 'select', at + 6)) selects.push(at) | ||
| if (!selects.length) return null | ||
| let k = selects.length - 1 | ||
| while (k > 0 && /\b(?:union|intersect|except|minus)\b/i.test(code.slice(selects[k - 1]!, selects[k]!)) && topLevel(code.slice(selects[k - 1]!, selects[k]!), '(?:union|intersect|except|minus)') >= 0) k-- |
There was a problem hiding this comment.
WARNING: {% if %}/{% else %} branches still yield only the last SELECT's names
Walking back over set operations fixes union all, but two alternative top-level SELECTs ({% if ... %} select ... {% else %} select ... {% endif %}) are not joined by union, so k stays on the last one and only that branch's names are returned. A column added inside the earlier branch is missing from now.names, and undocumentedByEdit then falls through to jinjaOf(before) === jinjaOf(after), which is also true (the edit changed SQL text, not a Jinja tag), so the edit's own new undocumented column is filed as pre-existing and no gap warning is raised. Union the names of top-level SELECTs that no set operation joins, instead of keeping only the last.
| if (code[i] === '(') depth++ | ||
| else if (code[i] === ')') depth-- | ||
| } | ||
| out.set(unquote(match[1]!), { raw: sql.slice(open, i - 1), code: code.slice(open, i - 1) }) |
There was a problem hiding this comment.
SUGGESTION: ctesOf returns a code field that nothing reads
Only raw is used (body.raw in the recursion and one.raw in the source fingerprint; the recursion re-masks it anyway), so the masked code companion is a second full copy of every CTE body with no reader. Dropping it, or returning just the raw body, keeps the same behaviour with less copying.
| const dirs = models.map(one => (one === '.' ? root : `${root}/${one}`)) | ||
| if (!dirs.length) continue | ||
| const listed = await $.process.run(['find', ...dirs, '-name', '*.sql', '-type', 'f', '-mmin', `-${minutes}`], { timeoutMs: 10_000 }).catch(() => null) | ||
| const top = (await $.process.run(['git', '-C', root, 'rev-parse', '--show-toplevel'], { timeoutMs: 5_000 }).catch(() => null))?.stdout.trim() ?? '' |
There was a problem hiding this comment.
SUGGESTION: git rev-parse runs even when find listed nothing
With this change roots can hold up to five projects from projectsBelow, and WRITES matches very common commands (python3, node, cp, mv, tee), so each matching shell call spawns up to five git rev-parse processes even when find returned no files (the loop below then does nothing). Return early when listed?.stdout.trim() is empty, or move the git lookup inside the loop so it runs only once a candidate file is found.
| const base = await $.session.root().catch(() => '.') | ||
| const own = await projectRoot($, commandDir(start.command, base), false).catch(() => null) | ||
| // The walk below it is bounded (2000 folders) and kept for the session. | ||
| const roots = own ? [own] : (await projectsBelow($, base).catch(() => [] as string[])).slice(0, MAX_BASH_PROJECTS) |
There was a problem hiding this comment.
SUGGESTION: The now-ungated projects-below walk has no test
Removing the projectSeen gate means this branch runs for every writing shell command in a session whose folder is not a dbt project, not just after one has been seen. lineage.test.tsx only exercises the own path (its start() makes the session root a project), so a wrong root, a duplicate note when two roots overlap, or MAX_BASH_PROJECTS not being applied would go unnoticed. A test whose session root sits above two projects, with the command rewriting a model in the second, would cover it.
…et-block lists, the model in the ambiguity check, the pane's no-new-gaps heading Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| const read = /^\s*((?:"[^"]+"|`[^`]+`|[A-Za-z_]\w*))(?:\s+(?:as\s+)?(?!where\b|group\b|order\b|limit\b|qualify\b|having\b|window\b|union\b)[A-Za-z_]\w*)?\s*(?:;\s*)?(?:$|where\b|group\b|order\b|limit\b|qualify\b|having\b|window\b|union\b)/i.exec(tail) | ||
| const body = read ? ctes.get(unquote(read[1]!)) : undefined | ||
| const inner = body !== undefined ? outputColumns(body.raw, depth + 1, ctes) : null | ||
| if (inner) return { names: new Set([...names, ...inner.names]), star: inner.star, vars, ...(inner.source !== undefined && { source: inner.source }) } |
There was a problem hiding this comment.
WARNING: The {% set %} list a select * CTE renders is dropped, so a column added there is missed
inner is parsed from the CTE body, so inner.vars holds the variables that body renders (select {{ cols }}), but this return keeps only the outer level's vars. In {% set cols %}id, amount{% endset %} / with src as (select {{ cols }} from raw) / select * from src, names are inherited from the CTE while vars stays empty, so setBlockNames(after, now.vars) in undocumentedByEdit returns nothing. Adding paid_cents to the block then leaves now.names, nowSet and now.star all empty, so undocumentedByEdit falls back to jinjaOf(before) === jinjaOf(after) - true, because jinjaOf keeps only the {% set %}/{% endset %} tags and not the block's text - and the new column is filed as pre-existing, with no gap warning. Merge it here: vars: new Set([...vars, ...inner.vars]).
| const list = code.slice(select + 6, from < 0 || from > end ? end : from).replace(/^\s*(?:distinct|all)\b/i, '') | ||
| const names = new Set<string>() | ||
| // The Jinja variables the list renders (`{{ cols }}`): their `{% set %}` blocks name outputs. | ||
| const vars = new Set([...sql.slice(select + 6, from < 0 || from > end ? end : from).matchAll(/\{\{-?\s*([A-Za-z_]\w*)\s*-?\}\}/g)].map(one => one[1]!)) |
There was a problem hiding this comment.
SUGGESTION: Only a bare {{ name }} is recognised, so a filtered render loses its {% set %} list
The regex requires the whole expression to be one identifier, so {{ core | trim }} (or | join(', '), or any filter on a block-set column list) records no variable. setBlockNames(after, now.vars) then skips that block, and since jinjaOf keeps only the Jinja tags and not the block's text, a column newly added to the block is classified as "from before this edit" and no gap is warned - the false negative this round is meant to remove. Capture the leading identifier and ignore a trailing filter, e.g. /\{\{-?\s*([A-Za-z_]\w*)\b/g.
| expect(await pane.find({ type: 'Text', text: ' not documented, from before this edit' })).toBeDefined() | ||
| await pane.unmount() | ||
| // A check that found only older gaps does not claim the columns match. | ||
| show.rows = [{ order_id: 1, amount: 10, status: 'placed', amount_cents: 1000 }] |
There was a problem hiding this comment.
SUGGESTION: Dead reassignment
This array is identical to the initial show.rows in start() (line 808), so the line changes nothing, and the comment above it describes the assertion that actually depends on the rows set at line 835. Removing it (or setting the rows where they really change) keeps the test honest.
…red CTE columns, quoted identifiers, Jinja alternatives, named-item edits, runner always asks dbt ls, FROM in function calls, balanced subquery levels, fewer git calls Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (named items or column-less clause lines only, Jinja unchanged), quoted CTE names kept Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… lines only, top-level CTEs only Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| if (/^(?:[\w"`]+\.)?\*\s+\S/.test(bare)) modified = true | ||
| continue | ||
| } | ||
| const name = itemName(sql.slice(select + 6 + lead + a, select + 6 + lead + b)) |
There was a problem hiding this comment.
WARNING: A trailing comment is read as the select item's name
itemName is given the raw sql slice here, while the items were split on the masked list. A trailing line comment therefore becomes the name: amount -- in cents is captured as cents by the /[\s)](...)$/ fallback and amount itself is dropped, and amount /* cents */ gets no name at all. Before this change the name was read from the masked text, where comments were already blanked. The lost name defeats the rule added in this PR: onlyNamedItems measures the changed line with itemName too, so a new amount_cents -- cents select item resolves to cents, which outputColumns has already put in now.names, and old() files the new undocumented column as pre-existing (no gap warning). now.names also gains phantom names, which can make an unrelated undocumented column look like it was there before. Pass a comment-blanked slice (quotes kept, which is why the raw text was used) here, at setBlockNames (line 1090) and in onlyNamedItems (line 1117).
| // Named nowhere (through a `*` the SQL does not resolve, or a macro): there before only when no Jinja expression or | ||
| // tag changed and every line the edit changed either rewrote a named output's select item or sits in a clause that | ||
| // adds no column (WHERE, GROUP BY, HAVING, QUALIFY, ORDER BY, LIMIT). | ||
| return onlyNamedItems(before!, after!, new Set([...was.names, ...wasSet]), new Set([...now.names, ...nowSet])) |
There was a problem hiding this comment.
SUGGESTION: File-wide work runs once per undocumented column
old() is called twice for every undocumented column (filter for fresh, then again for old), and each call re-scans both whole texts with jinjaOf, rebuilds the two combined name sets, and - for any column no list names - runs onlyNamedItems, which re-diffs every line and calls columnlessLines twice more. Hoist jinjaOf(before) !== jinjaOf(after), the combined sets and the onlyNamedItems result (or at least its quietBefore/quietAfter sets) above old; a model with dozens of undocumented columns repeats all of it dozens of times per check.
| const listEnd = from < 0 || from > next ? next : from | ||
| const list = code.slice(select + 6, listEnd) | ||
| const lead = /^\s*(?:distinct|all)\b/i.exec(list)?.[0].length ?? 0 | ||
| for (const one of sql.slice(select + 6, listEnd).matchAll(/\{\{-?\s*([A-Za-z_]\w*)\s*-?\}\}/g)) vars.add(one[1]!) |
There was a problem hiding this comment.
SUGGESTION: A filtered {{ … }} render records no variable
The regex still matches only a bare {{ name }}, so {{ cols | trim }}, {{ cols|join(', ') }} or {{ env_var('COLS') }} add nothing to vars. setBlockNames then names no columns for that block, now.names/nowSet stay empty, and old() falls through to onlyNamedItems; the changed set-block line is not one of the named outputs, so every other undocumented column of the model is reported as this edit's fresh gap (a warning about columns the edit did not touch). Capture the leading identifier and ignore a trailing filter, e.g. /\{\{-?\s*([A-Za-z_]\w*)\b/g.
…ed diff within output lists, runner ls errors shown, nested projects searched Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| const afterSide = sides(now, nowSet) | ||
| // Named in the model's own list after the edit: there before only when that list named it before too (outside | ||
| // Jinja that may not run). | ||
| if (afterSide.names.has(key) || now.maybe.has(key) || nowSet.maybe.has(key)) return beforeSide.names.has(key) |
There was a problem hiding this comment.
WARNING: Conditional names are compared only against the unconditional "before" set
When the after side names the column only inside Jinja that may not run (now.maybe/nowSet.maybe), this returns beforeSide.names.has(key) and never consults was.maybe/wasSet.maybe. So a column that sits in the same unchanged {% if is_incremental() %} … {% else %} … {% endif %} branch on both sides is treated as fresh on any edit. On the first check of a model (lastReturned is null at dbt-mods.tsx:690) every undocumented column of an incremental model is reported as this edit's new gap, and editGaps.set(fileKey, gaps.fresh) (dbt-mods.tsx:692) then keeps them the edit's on later checks, since lastReturned excludes them (dbt-mods.tsx:696). The updated "an undocumented column is the edit's only when…" case encodes exactly this for a whitespace-only edit, which is the false warning that finding E was meant to remove. Return true when was.maybe.has(key) || wasSet.maybe.has(key) too: the same conditional branch named it before.
| } | ||
| // Where the item's own text starts (past whitespace and masked Jinja tags before it). | ||
| const start = select + 6 + lead + a + (/^\s*/.exec(list.slice(lead + a, lead + b))?.[0].length ?? 0) | ||
| const name = itemName(sql.slice(start, select + 6 + lead + b)) |
There was a problem hiding this comment.
WARNING: Item names are read from raw SQL, so a trailing comment becomes the name
itemName gets sql.slice(start, …), which still holds -- and /* */ comments (the items were split on masked code, but the slice is raw). amount -- in cents therefore resolves to cents and amount is lost, and amount /* cents */ gets no name at all. That defeats the rule this PR adds: a new undocumented amount_cents -- cents item resolves to cents, which is already in now.names, so onlyNamedItems treats the changed line as a named item and old() files the new column as pre-existing with no gap warning. The same raw text is passed to itemName in setBlocks (line 1125) and onlyNamedItems (line 1174). Pass a comment-blanked slice (quotes kept, which is why raw text was used) at all three.
| if (afterSide.names.has(key) || now.maybe.has(key) || nowSet.maybe.has(key)) return beforeSide.names.has(key) | ||
| // Named on neither side (a macro or a `*` fills it): there before only when every line the edit changed is a named | ||
| // output's item or a clause that adds no column. | ||
| return !was.maybe.has(key) && !wasSet.maybe.has(key) && !beforeSide.names.has(key) && onlyNamedItems(before!, after!, beforeSide, afterSide) |
There was a problem hiding this comment.
WARNING: An O(lines²) diff runs for every undocumented column
old() is evaluated twice per undocumented column (line 1110), and for every column the after list does not name it now runs onlyNamedItems: an LCS over every line of the model with one Uint32Array per line (up to 4M cells and roughly 16 MB allocated per call for a 2000-line model), plus columnlessLines, which masks the whole text twice. A model with dozens of undocumented columns (the PR's Tuva run reported 69) repeats all of it dozens of times per check on the hook's thread, and past the 4M-cell bail (line 1150) every column silently becomes fresh. Hoist the jinjaOf comparison, the combined sides(...) sets and one onlyNamedItems result (or at least its removed/added and quiet sets) above old.
| out.spans.push([base + select, base + listEnd]) | ||
| const list = code.slice(select + 6, listEnd) | ||
| const lead = /^\s*(?:distinct|all)\b/i.exec(list)?.[0].length ?? 0 | ||
| for (const one of sql.slice(select + 6, listEnd).matchAll(/\{\{-?\s*([A-Za-z_]\w*)\s*-?\}\}/g)) out.vars.add(one[1]!) |
There was a problem hiding this comment.
SUGGESTION: A filtered {{ … }} render still records no variable
The regex matches only a bare {{ name }}, so {{ cols | trim }}, {{ cols|join(', ') }} or {{ env_var('COLS') }} add nothing to vars. setBlocks(after, now.vars) then names no columns for that block, the changed set-block lines sit outside every span, and old() falls through to onlyNamedItems, so every other undocumented column of the model is reported as this edit's fresh gap. Capture the leading identifier and ignore a trailing filter, e.g. /\{\{-?\s*([A-Za-z_]\w*)\b/g.
Fixes the plugin behaviour seen in the 19 Nov hands-on testing (rows A, C–I) on dbt Labs' jaffle_shop_duckdb and Tuva Health's public tuva repo. One PR, one commit per round.
Merge order: this was cut from
fix/plugin-demo-bugs(#42). #42 merged before this was pushed, so the branch sits onmainwith #42 in it; nothing else to merge first.Live checks ran in real Claude Code 2.1.329 sessions (
claude --plugin-dir plugins/altimate-code) with altimate-code built from source (fix/tools-demo-bugs), dbt-core 1.12.7 and dbt-duckdb 1.11.1, using the steps in the report's section 5.A · band said "amount reaches 0 columns in 0 models · ✓ Every downstream model was read" on a compiled project
mainbefore fix: /clear keeps comments and health, plugin dbt runs keep the project's manifest, and five demo bugs #42 and onmainwith fix: /clear keeps comments and health, plugin dbt runs keep the project's manifest, and five demo bugs #42 (cents change instg_payments.amount, thendbt build): both said 0 columns in 0 models · 2 more may change · ✓ Every downstream model was read, while the per-edit note in the same session said 6 columns in 2 models. So the manifest overwrite was not the cause.target/.altimate-lineage/. altimate-code treats the folder above the manifest's folder as the project and reads a seed's CSV header from there.stg_paymentsdoesselect *from theraw_paymentsseed. From a copy one folder deeper, the header could not be found, soordersandcustomers(bothselect *) came back asmay_carrywith reasonstar, andunresolvedwas empty. The band read that as a real 0 with every model read. The same manifest intarget/traces 6 columns.target/itself (.altimate-manifest-before.<session>.json,.altimate-task-manifest.<session>.json). Copies left in the old folder are still cleaned up. If the reach is 0 while downstream models may pass the column on without naming it (star,natural_join,dynamic_name,row_reference), the band now says "not traced: N downstream models may read it without naming it (select *)" with a lower-bound footer, never "✓ Every downstream model was read".impact-report.test.ts› "a reach of 0 beside readers that select * is not traced…" uses altimate-code's real answer captured from the live run.task-impact.test.tsxchecks that the copy is intarget/and checks the cleanup of both folders.C ·
ref('stg_orders')replaced bymain.stg_orders, and compile & validate still said it matchedref()back unless the person asked for the table.dbt-mods.test.tsx› "a model, seed or source named by its table…" (spellings and false positives) and "an edit that swaps a ref() for the table it builds is flagged".D · no per-edit impact note when Claude edited through Bash
sed -i,perl -i, a redirect to a.sql,tee,python3/node/…,mv/cp/patch,git apply/checkout/restore), only the time is noted. Nothing is read or run before the command, so it never delays the command or the gates. Afterwards, model files under the project's model paths that changed since then (find -mmin -<minutes the command ran>, then each file's mtime) get the same impact-first note, diffed against the last commit. Withshow_edit_impactit goes to the transcript, and it always goes to Claude's context. At most 3 files per command.lineage.test.tsx› "a model a shell command rewrites gets the per-edit impact note too…" (a read-onlycatgets nothing).python3one-liner: "◆ impact-first · stg_payments.amount reaches 6 columns in 2 models" on screen, and "added to Claude's context: lineage for stg_payments.amount (changed by a shell command)".E · "undocumented order_id, amount" for columns that were already undocumented
editReadsBefore).dbt-mods.test.tsx› "an undocumented column is the edit's only when…" and "only the columns an edit left undocumented are its gaps…" (second edit: not said again).F · the pane showed the last model read while Claude edited another
lineage.test.tsx› "the pane moves to the model being edited as the edit starts…" (the lookup is held, and the pane already shows the model).G · at default size the graph ran off the right edge and dashboards weren't visible
vshows the whole graph, then cycles as before.lineage.test.tsx› "a graph wider than the pane opens as its ends…". Tests about the whole graph now pressvfirst.H · Tuva: compile & validate ran dbt in the package root and failed with "Run dbt deps"
dbt_project.yml, which is the package. Its packages aren't installed there, and its models are disabled there (claims_enabled).packages.yml/dependencies.ymlinstalls this package by a local path (local: ../). If that project's packages are installed, dbt runs there with the repo's own dbt (its.venvwhen that project has none) and selects the model bypackage:<pkg>,<name>(<name>.v<N>for a versioned model). The band says "ran from integration_tests". The doctor counts packages installed there as installed. If dbt still says packages are missing, the result is a one-line "not checked: dbt's packages are not installed in …" note instead of a failure.dbt-mods.test.tsx› "a package tested from integration_tests is compiled and validated from there…" and "a project whose packages are not installed gets a one-line note…"./…/tuva/integration_tests.I · "no output column changed: the model-level count stands" after a change to what paid_amount holds
core__medical_claimlists its columns in a{% set %}block that the select reads. The select-item diff saw no change.change_type modify: "values change in paid_amount → reaches N columns in M models". If none can be named: "values may change downstream; the model-level count stands" (or "no model reads it downstream" when the count is 0). It never shows 0 columns.impact-report.test.ts› "a change no select item shows (a value computed in a Jinja block)…".task-impact.test.tsxchecks the fallback text.Review loop
.venvdbt; a versioned model's other versions selected from the runner; the shell-edit search window under a long command; comment-only and multi-line predicate changes read as value changes; quoted CTEs flagged.select * from <cte>, are not outputs); gap bookkeeping written only by the newest check of the file in this conversation (an older or pre-/clearcheck ending late erases nothing); from the runner with no manifest there,dbt lsnames the file's model and version first; CTEs with a column list; a predicate whose WHERE ends the line before.*the SQL can't resolve, is set aside only if the edit left what the*reads unchanged (aselect * from <cte>resolves through an alias, a;and chainedselect *CTEs); from the runner,dbt lslists the whole package and the file's own model is used, and nothing runs if dbt lists no model for the file; edit gaps are cleared by/clear; replacement advice keeps a non-latest version (ref('orders', v=1)); value candidates follow the clause at each paren depth across lines, with strings and Jinja blanked; a shell edit in a project below the session folder is found once a dbt project has been seen.select *chain resolves through every CTE it reads, and an unresolved*compares what it reads together with the bodies of the CTEs it names; a UNION's names come from its first branch; replacement advice names another package's model with its package (ref('pkg', 'orders')); value candidates compare the value parts of the SQL (outside filters, joins, grouping and ordering at every depth, with function parentheses tracked), so a changed threshold or a reworded inline comment is not a value change; a shell edit in a project below the session folder is found without waiting for one to be seen.*the SQL can't resolve is fingerprinted with its own list, what it reads and every CTE of the model, so a column added anywhere upstream in the model stays the edit's;* exclude (…)counts as a*; a column no list names (a macro fills it) is set aside only when no Jinja expression or tag changed, and names in{% set %}column lists count as outputs; a bare relation name that two relations share is not flagged.*with a modifier (exclude,replace, …) stays unresolved rather than taking the CTE's list; only the{% set %}lists the model's own select renders count as outputs; the model's own relation takes part in the ambiguity check (and is never advice); the dbt check pane says "no new gaps" rather than "match" when older gaps remain.select *; a CTE's declared column names are whatselect *returns; quoted identifiers are masked so a keyword inside one does not end a list; Jinja alternatives ({% if %} select … {% else %} select … {% endif %}) each name outputs; a column no list names is set aside only when every line the edit changed is a named select item; from the runner,dbt lsalways gives the file's current model and version (a saved manifest can be stale); a FROM inside a function call (extract(day from x)) is not a relation; a subquery right after FROM keeps the value-candidate levels balanced; nogitcall when no model file changed; one pass over the relations; onedbt lsJSON-line parser; a test for the projects-below search.*or from a macro). It counts as from before the edit only when no Jinja expression or tag changed, and every line the edit changed is either a named output's select item or sits in a clause that adds no column (WHERE, GROUP BY, HAVING, QUALIFY, ORDER BY, LIMIT). This replaces the source fingerprints. Quoted CTE names keep their identity.{% if %},{% for %}) is no evidence that a column was there before the edit. The changed lines come from an ordered line diff, and a changed item line counts only when it sits inside the model's own output list (or a set-block list that list renders) on its side, so lines moved between CTEs keep the column the edit's. From the runner, a failingdbt lsshows dbt's error. A shell edit in a project inside the session's own project is found.Checks
claude plugin test plugins/altimate-code: 1286 pass, 1 fail at the round-10 fix. The failure is the known flake "a dbt call that starts while another call reads its artifact", which also fails onmain.mainat e36c74f): the lineage tests for D, F and G fail one by one.impact-report.test.ts,task-impact.test.tsxanddbt-mods.test.tsxdon't load on the base, because the functions under test don't exist there.claude plugin validate --strict .: passed.bash tests/*.sh: all 8 pass.🤖 Generated with Claude Code