Repository navigation
feat(review): 11 deterministic rules for macro, test and yml defects - #1770
RaghwendraSingh wants to merge 13 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
There was a problem hiding this comment.
5 issues found across 7 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/src/altimate/review/orchestrate.ts">
<violation number="1" location="packages/opencode/src/altimate/review/orchestrate.ts:2124">
P2: `dispatchVariantFindings` searches the repo and reads every hit file for each `adapter.dispatch` name in the macro, even names whose calls add no keyword. Those results are discarded. Limit the loop to the names `addedDispatchKwargs(file, newSql)` returns.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/orchestrate.ts:2206">
P3: For added YAML files, `oldContent` is `undefined`, so `yamlParseFindings` skips the "parsed before" gate. A new file with a syntax error then gets a finding that says it parsed before this change. Either skip added files here or change the message for files with no base.</violation>
</file>
<file name="packages/opencode/src/altimate/review/project-rules.ts">
<violation number="1" location="packages/opencode/src/altimate/review/project-rules.ts:740">
P2: This condition treats every omitted inline strategy as `merge`, even when `manifestConfig.incremental_strategy` is `insert_overwrite` or `microbatch`. Use the effective manifest strategy before applying the merge-only warning, otherwise project-level strategy configuration produces a false finding.</violation>
</file>
<file name="packages/opencode/src/altimate/review/dbt-patterns.ts">
<violation number="1" location="packages/opencode/src/altimate/review/dbt-patterns.ts:930">
P3: The join parser accepts only unquoted word aliases, so `AS "b"` makes this rule skip the entire join and miss repeated predicates. Parse quoted aliases before treating a join as having no analyzable `ON` clause.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/dbt-patterns.ts:972">
P2: `topLevelAnd` treats `AND` and parentheses inside quoted SQL text as syntax, so different literals containing `and` can yield the same malformed fragment and trigger a false redundant-predicate finding. Tokenize quoted literals before splitting the ON clause.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| const dispatchVariantFindings = async (file: ChangedFile, newSql: string): Promise<Finding[]> => { | ||
| if (!/\badapter\.dispatch\b/.test(newSql)) return [] | ||
| const texts = new Map<string, string>([[file.path, newSql]]) | ||
| for (const name of new Set([...newSql.matchAll(/\badapter\.dispatch\s*\(\s*['"](?:\w+\.)?(\w+)['"]/g)].map((m) => m[1]))) { |
There was a problem hiding this comment.
P2: dispatchVariantFindings searches the repo and reads every hit file for each adapter.dispatch name in the macro, even names whose calls add no keyword. Those results are discarded. Limit the loop to the names addedDispatchKwargs(file, newSql) returns.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 2124:
<comment>`dispatchVariantFindings` searches the repo and reads every hit file for each `adapter.dispatch` name in the macro, even names whose calls add no keyword. Those results are discarded. Limit the loop to the names `addedDispatchKwargs(file, newSql)` returns.</comment>
<file context>
@@ -2101,6 +2117,18 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
+ const dispatchVariantFindings = async (file: ChangedFile, newSql: string): Promise<Finding[]> => {
+ if (!/\badapter\.dispatch\b/.test(newSql)) return []
+ const texts = new Map<string, string>([[file.path, newSql]])
+ for (const name of new Set([...newSql.matchAll(/\badapter\.dispatch\s*\(\s*['"](?:\w+\.)?(\w+)['"]/g)].map((m) => m[1]))) {
+ const hits = (await input.grepProject?.(`__${name}(`).catch(() => [])) ?? []
+ for (const path of new Set(hits.map((h) => h.path)))
</file context>
There was a problem hiding this comment.
Not changing: nit (a small extra cost, no effect on correctness).
| const bigquery = dialectKnown ? /bigquery/i.test(dialect) : /['"]field['"]\s*:/.test(cfg.partition_by ?? "") | ||
| const predicates = cfg.incremental_predicates ?? manifestConfig?.incremental_predicates | ||
| const hasPredicates = Array.isArray(predicates) ? predicates.length > 0 : !!predicates && !/^\s*(none|\[\s*\])\s*$/i.test(String(predicates)) | ||
| if (bigquery && parts.length && uk.length && (strategy === "merge" || !strategy) && !hasPredicates) |
There was a problem hiding this comment.
P2: This condition treats every omitted inline strategy as merge, even when manifestConfig.incremental_strategy is insert_overwrite or microbatch. Use the effective manifest strategy before applying the merge-only warning, otherwise project-level strategy configuration produces a false finding.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/project-rules.ts, line 740:
<comment>This condition treats every omitted inline strategy as `merge`, even when `manifestConfig.incremental_strategy` is `insert_overwrite` or `microbatch`. Use the effective manifest strategy before applying the merge-only warning, otherwise project-level strategy configuration produces a false finding.</comment>
<file context>
@@ -411,6 +732,23 @@ export function modelConfigFindings(
+ const bigquery = dialectKnown ? /bigquery/i.test(dialect) : /['"]field['"]\s*:/.test(cfg.partition_by ?? "")
+ const predicates = cfg.incremental_predicates ?? manifestConfig?.incremental_predicates
+ const hasPredicates = Array.isArray(predicates) ? predicates.length > 0 : !!predicates && !/^\s*(none|\[\s*\])\s*$/i.test(String(predicates))
+ if (bigquery && parts.length && uk.length && (strategy === "merge" || !strategy) && !hasPredicates)
+ out.push(
+ finding(
</file context>
| if (bigquery && parts.length && uk.length && (strategy === "merge" || !strategy) && !hasPredicates) | |
| if (bigquery && parts.length && uk.length && (strategy === "merge" || (!strategy && String(manifestConfig?.incremental_strategy ?? "merge").toLowerCase() === "merge")) && !hasPredicates) |
There was a problem hiding this comment.
Fixed in 936b24a: incremental_strategy falls back to the manifest config, so an insert_overwrite set in dbt_project.yml isn't reported as an unpruned merge. Test added.
| const parts: string[] = [] | ||
| let depth = 0 | ||
| let last = 0 | ||
| for (const m of clause.matchAll(/[()]|\band\b/gi)) { |
There was a problem hiding this comment.
P2: topLevelAnd treats AND and parentheses inside quoted SQL text as syntax, so different literals containing and can yield the same malformed fragment and trigger a false redundant-predicate finding. Tokenize quoted literals before splitting the ON clause.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/dbt-patterns.ts, line 972:
<comment>`topLevelAnd` treats `AND` and parentheses inside quoted SQL text as syntax, so different literals containing `and` can yield the same malformed fragment and trigger a false redundant-predicate finding. Tokenize quoted literals before splitting the ON clause.</comment>
<file context>
@@ -891,6 +891,117 @@ const detectMultiCountDistinct: Detector = (c) => {
+ const parts: string[] = []
+ let depth = 0
+ let last = 0
+ for (const m of clause.matchAll(/[()]|\band\b/gi)) {
+ if (m[0] === "(") depth++
+ else if (m[0] === ")") depth--
</file context>
There was a problem hiding this comment.
Follow-up: low priority, tracked. We'll split on a literal-masked copy.
| file.status === "added" ? Promise.resolve(undefined) : read(file.oldPath ?? file.path, "old"), | ||
| ]) | ||
| if (file.kind === "other") checkedOther(file) | ||
| return yamlParseFindings(file, oldContent, newContent) |
There was a problem hiding this comment.
P3: For added YAML files, oldContent is undefined, so yamlParseFindings skips the "parsed before" gate. A new file with a syntax error then gets a finding that says it parsed before this change. Either skip added files here or change the message for files with no base.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 2206:
<comment>For added YAML files, `oldContent` is `undefined`, so `yamlParseFindings` skips the "parsed before" gate. A new file with a syntax error then gets a finding that says it parsed before this change. Either skip added files here or change the message for files with no base.</comment>
<file context>
@@ -2157,6 +2186,56 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
+ file.status === "added" ? Promise.resolve(undefined) : read(file.oldPath ?? file.path, "old"),
+ ])
+ if (file.kind === "other") checkedOther(file)
+ return yamlParseFindings(file, oldContent, newContent)
+ }),
+ )
</file context>
There was a problem hiding this comment.
Follow-up: low priority, tracked. Added files will get their own wording.
| i++ | ||
| rel = null | ||
| } else if (rel) i += rel[0].length | ||
| const tail = /^\s+(?:as\s+)?(\w+)\s+on\b/i.exec(s.slice(i)) ?? /^\s+on\b/i.exec(s.slice(i)) |
There was a problem hiding this comment.
P3: The join parser accepts only unquoted word aliases, so AS "b" makes this rule skip the entire join and miss repeated predicates. Parse quoted aliases before treating a join as having no analyzable ON clause.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/dbt-patterns.ts, line 930:
<comment>The join parser accepts only unquoted word aliases, so `AS "b"` makes this rule skip the entire join and miss repeated predicates. Parse quoted aliases before treating a join as having no analyzable `ON` clause.</comment>
<file context>
@@ -891,6 +891,117 @@ const detectMultiCountDistinct: Detector = (c) => {
+ i++
+ rel = null
+ } else if (rel) i += rel[0].length
+ const tail = /^\s+(?:as\s+)?(\w+)\s+on\b/i.exec(s.slice(i)) ?? /^\s+on\b/i.exec(s.slice(i))
+ if (!tail) {
+ joins.push({ preds: [], start: m.index!, end: i, inner })
</file context>
There was a problem hiding this comment.
Not changing: nit / recall gap.
| "A string of SQL is joined with `~` to a macro call or variable with no space at the seam, so the rendered SQL runs the two together (`ticket_id` + `, source_relation` works, `ticket_id` + `source_relation` does not). Put a space inside the string next to `~`.", | ||
| { | ||
| // A literal holding SQL (it has whitespace) that ends, or starts, with a word character right at the `~`. | ||
| added: /'[^'\n]*\s[^'\n]*[A-Za-z0-9]'\s*~\s*[A-Za-z_][\w.]*|[\w)]\s*~\s*'[A-Za-z0-9][^'\n]*\s[^'\n]*'/, |
There was a problem hiding this comment.
WARNING: ~ is also the Postgres/Redshift/DuckDB regex-match operator, so this fires on valid SQL
The second alternative ([\w)]\s*~\s*'…') matches a plain comparison such as where email ~ 'a b', and the first matches 'a b' ~ name, because added is tested with string literals kept and the rule runs on model SQL too. That reports "SQL text concatenated in Jinja without a space" for SQL that contains no Jinja ~ at all. Gate the rule on the line containing {{ or {% (or add an unless for a comparison context), so the regex operator form is not reported.
There was a problem hiding this comment.
Fixed in 936b24a: the rule now matches only a seam inside an open {{/{%, so col ~ 'a b' no longer fires. Tests added.
| for (const ch of body + ",") { | ||
| if ("([{".includes(ch)) depth++ | ||
| else if (")]}".includes(ch)) depth-- | ||
| if (ch === "," && depth === 0) { |
There was a problem hiding this comment.
WARNING: a comma inside a quoted default splits the parameter list into phantom parameters
cur is depth-tracked only for ([{ and )]}, not for quotes, so {% macro m(c, precision="10,2") %} parses as [c, precision(default), {name:"2", hasDefault:false}]. When that fragment is on an added signature line the rule emits new parameter 2ofm has no default; a ] inside a default drives depth negative and swallows every later parameter instead. Skip quoted spans while collecting cur (or tokenise with quote awareness) so string defaults cannot create or hide parameters.
There was a problem hiding this comment.
Follow-up: low priority, tracked. We'll skip quoted spans in the parameter split.
| const args = parenBody(sql, argsOpen) | ||
| const first = lineAt(argsOpen) | ||
| const last = first + args.split("\n").length - 1 | ||
| const kws = [...topLevel(args).matchAll(/(?:^|,)\s*(\w+)\s*=(?!=)/g)].map((k) => k[1]) |
There was a problem hiding this comment.
WARNING: keyword arguments are also read out of string arguments
topLevel(args) blanks nested parentheses but keeps string contents, so the (?:^|,)\s*(\w+)\s*=(?!=) scan treats = inside a quoted value as a keyword. For adapter.dispatch('ghost', 'pkg')(col=col, filter="live = true, backfill = false") it collects backfill, and any variant without a backfill parameter gets a false "takes no keyword argument" warning anchored to that string line. Mask literals before matching (as rule-catalog's stripStrings does) and keep the raw line only for the evidence text.
There was a problem hiding this comment.
Agreed, low priority (project-rules.ts:588, kwargs read out of string arguments). Leaving this for a follow-up PR rather than widening this one.
| const out: { macro: string; params: string[]; acceptsKwargs: boolean }[] = [] | ||
| for (const [macro, sig] of signatures(text)) { | ||
| if (!macro.endsWith(`__${name}`) || macro === name) continue | ||
| const at = new RegExp(`\\{%-?\\s*macro\\s+${escapeRe(macro)}\\s*\\(`).exec(text)!.index |
There was a problem hiding this comment.
WARNING: the non-null assertion throws for {% test %} signatures and loses every finding for the file
signatures(text) matches both macro and test, but this re-lookup regex accepts only macro. A generic test whose name ends in __<dispatched name> (tests legitimately live in macros/ and tests/, both searched by grepProject) makes exec() return null, so !.index throws. The throw propagates through dispatchVariantFindings into the macro lane's catch, which calls failedFile() and drops all findings for that macro file, not just this check. Guard the result (const at = …?.index; if (at === undefined) continue) or pass the kind through from signatures().
There was a problem hiding this comment.
Fixed in 936b24a: dispatchVariants skips {% test %} blocks and guards the re-lookup, so no findings are lost for the file. Test added.
| const bigquery = dialectKnown ? /bigquery/i.test(dialect) : /['"]field['"]\s*:/.test(cfg.partition_by ?? "") | ||
| const predicates = cfg.incremental_predicates ?? manifestConfig?.incremental_predicates | ||
| const hasPredicates = Array.isArray(predicates) ? predicates.length > 0 : !!predicates && !/^\s*(none|\[\s*\])\s*$/i.test(String(predicates)) | ||
| if (bigquery && parts.length && uk.length && (strategy === "merge" || !strategy) && !hasPredicates) |
There was a problem hiding this comment.
WARNING: incremental_strategy is read only from the inline config, so an insert_overwrite model is reported as an unpruned merge
strategy comes from unquote(cfg.incremental_strategy), while materialized and predicates fall back to the manifest. A BigQuery model with inline config(unique_key=…, partition_by=…) but +incremental_strategy: insert_overwrite in dbt_project.yml or the schema config has strategy === "", so (strategy === "merge" || !strategy) assumes merge and suggests incremental_predicates for a strategy that never merges. Fall back to manifestConfig?.incremental_strategy (and treat a known non-merge strategy as n/a), as modelSemantics already does.
There was a problem hiding this comment.
Fixed in 936b24a (same fix as the cubic thread): falls back to the manifest's incremental_strategy.
| const parts: string[] = [] | ||
| let depth = 0 | ||
| let last = 0 | ||
| for (const m of clause.matchAll(/[()]|\band\b/gi)) { |
There was a problem hiding this comment.
SUGGESTION: splitting on and inside literals creates a false "redundant" match
topLevelAnd sees the raw clause (literals are deliberately kept), so p.segment = 'smb and enterprise' and p.segment = 'smb and midmarket' both split into the fragment p.segment = 'smb, compare equal, and are reported as redundant even though the joins filter different values. Split on and in a literal-masked copy and keep the raw predicate for the message.
There was a problem hiding this comment.
Agreed, low priority (dbt-patterns.ts:972, and inside literals). Leaving this for a follow-up PR rather than widening this one. Same root cause as the other bot's thread.
| if (!missing.length) return [] | ||
| const others = [...new Set([...newContent.matchAll(DOC_REF)].map((m) => m[1]))].filter((n) => !missing.includes(n)).slice(0, 5) | ||
| let resolves = false | ||
| for (const n of others) if ((resolves = await docExists(n))) break |
There was a problem hiding this comment.
SUGGESTION: one locally-resolving doc() is not enough to prove the new one is local
The guard only requires any other doc() in the file to resolve in the repo. A package file that mixes a local doc with one shipped by an installed package (for example doc('hubspot__contact_id')) passes the guard, and the new name is then reported as "dbt fails to parse the project" although the block exists in that package — the exact case the docstring says it avoids. Require that the missing name's prefix (or every other doc() in the file) resolves locally before reporting.
There was a problem hiding this comment.
Agreed, low priority (project-rules.ts:499, mixed package/local doc()). Leaving this for a follow-up PR rather than widening this one.
| const ymls = reviewable.filter((f) => f.status !== "deleted" && /\.ya?ml$/i.test(f.path)) | ||
| const sets = await Promise.all( | ||
| ymls.map(async (file) => { | ||
| const [newContent, oldContent] = await Promise.all([ |
There was a problem hiding this comment.
SUGGESTION: every changed yml now costs two extra git shows before the schema lane re-reads the same file
getContent (makeContentResolver) has no cache, and ylms includes all schema_yml files, so each one is read old+new here and again in the schema block (around line 2282), on top of the contract lane's reads. The Promise.all over ylms also launches 2 × yml count subprocesses at once. On a schema-heavy PR this is a large share of the review's process time; memoising getContent per (path, side) or reusing these contents in the schema block would keep it bounded.
There was a problem hiding this comment.
Not changing: nit. The cost is small next to the schema lane.
|
|
||
| /** Each `{% macro name(…) %}` / `{% test name(…) %}` with its parameters (name and whether it has a default) | ||
| * and the head line range of its signature. Defaults may hold calls and lists. */ | ||
| function signatures(sql: string | undefined) { |
There was a problem hiding this comment.
SUGGESTION: signatures() duplicates macroSignatures() with different parsing — consolidate
macroSignatures() (line 217) still feeds macro-param-shifted and signatureAnchor(), but it splits on every comma and truncates at the first ), so defaults like cols=['x', 'y'] or ts=dbt.current_timestamp() produce phantom or falsely moved parameters that the new parser handles correctly. Now that signatures() exists, make it the single parser for both checks (returning sig.first/sig.last for the anchor) and delete the older helper so the two cannot drift.
There was a problem hiding this comment.
Follow-up: low priority, tracked. signatures() will become the single parser.
| return out | ||
| } | ||
|
|
||
| const DISPATCH_VARIANT = new RegExp(`^(default|${ADAPTERS.join("|")})__\\w`, "i") |
There was a problem hiding this comment.
SUGGESTION: the dispatch-variant skip list is limited to ADAPTERS
DISPATCH_VARIANT exempts default__ and the 16 adapters in ADAPTERS, so a new required parameter on a variant such as mysql__sat, sqlite__…, vertica__… or greenplum__… is still reported with "every call written before this change omits it" — false for a variant whose dispatcher passes every argument. Match the generic <prefix>__<same base name> shape instead of an adapter allow-list.
There was a problem hiding this comment.
Follow-up: low priority, tracked. We'll match the generic <prefix>__<base> shape.
Code Review SummaryStatus: 14 Issues Found | Recommendation: Address before merge Overview
The head has been rebased to Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (7 files)
Previous Review Summaries (3 snapshots, latest commit b234d43)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit b234d43)Status: 14 Issues Found | Recommendation: Address before merge Overview
Fix these issues in Kilo Cloud Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (7 files)
Note: the previously reported Previous review (commit 936b24a)Status: 11 Issues Found | Recommendation: Address before merge Overview
Fix these issues in Kilo Cloud Issue Details (click to expand)WARNING
SUGGESTION
Note: the fix commits resolve the Files Reviewed (7 files)
Previous review (commit a6823f9)Status: 11 Issues Found | Recommendation: Address before merge Overview
Fix these issues in Kilo Cloud Issue Details (click to expand)WARNING
SUGGESTION
Note: the new Files Reviewed (7 files)
Reviewed by deepseek-v4.1-flash-1017 · Input: 109.7K · Output: 22K · Cached: 1.2M Review guidance: REVIEW.md from base branch |
hkaltai
left a comment
There was a problem hiding this comment.
Verdict: request changes. Panel: Claude, Codex, lead. All three reviewers independently ran bun test test/altimate/review (711 pass); the findings below were reproduced.
Counts: CRITICAL 0, MAJOR 1, MINOR 6, NIT 6.
MINOR (not anchorable to a diff line), project-rules.ts:677: const strategy = unquote(cfg.incremental_strategy).toLowerCase() ignores manifest/project config, unlike materialized and incremental_predicates. A model whose strategy is insert_overwrite set in dbt_project.yml gets "incremental merge has no incremental_predicates". Fix: fall back to manifestConfig?.incremental_strategy.
The MAJOR is a crash: an unescaped, unaliased join relation (join unnest(a.arr) on ...) reaches new RegExp in redundantJoinPredicates and rejects the whole runReview. The rest are false positives or false negatives in summary-tier rules.
Nits
orchestrate.ts:2267:docs ${name}fixed-string search misses{% docs name %}(extra spaces/tab) and can emitdoc-block-missingfor an existing block. Grep forname, then filter with the existing whitespace-tolerant regex.project-rules.ts:304(macro-param-no-default): renaming a parameter (foo(a)->foo(relation)) fires "every call omits it"; positional callers still bind. Skip same-count renames or reword.project-rules.ts:456(isPackageProject): a project with noprofile:is treated as a package; dbt Cloud projects can omit it and may legitimately commitpackage-lock.yml. Require theintegration_tests/signal as well.project-rules.ts:379(yamlError): theyamllibrary reports two<<: *a/<<: *bkeys asDUPLICATE_KEY, which dbt's PyYAML accepts. Ignore duplicate<<.dbt-patterns.ts:957: a join inside a parenthesised relation starts beforeprev.end, so the slice is""and matchesJOIN_TYPE_GAP; requireprev.end <= start. Also the user message and// 46.comment say "an earlier join" but the logic is INNER only.orchestrate.ts:2205/2527: everyotherYAML file is marked reviewed after only a parse check, shrinking "not reviewed" without content coverage. Alsorule-catalog.tsmacro-group-by-ordinalmissesgroup by 1, customer_id;signaturesvsmacroSignaturesandcheckedOther/otherCheckedare confusingly named.
| if (!chained) seen = new Set() | ||
| const { alias, preds } = joins[k] | ||
| for (const p of preds) | ||
| if (chained && seen.has(p.key) && /=/.test(p.key) && !(alias && new RegExp(`\\b${alias}\\.`, "i").test(p.raw))) out.push(p.raw) |
There was a problem hiding this comment.
[MAJOR] Unescaped join relation text used in new RegExp crashes the whole review
!(alias && new RegExp(`\\b${alias}\\.`, "i").test(p.raw))When a join has no alias, alias falls back to rel[1].split(".").pop() (line 935), which is raw text. For ... join b on a.id = b.id join unnest(a.arr) on a.id = b.id (also Snowflake join table(flatten(a.arr)) on ...) redundantJoinPredicates throws SyntaxError: Invalid regular expression: unmatched parentheses. Reproduced directly. The model detectors run without a try/catch, so runReview rejects and the PR gets no verdict at all. Rare (needs a repeated predicate too), but the failure is total on valid SQL.
Fix: use escapeRe(alias) (or only accept the fallback when it matches /^\w+$/), and add a test with an unaliased unnest(...) join.
There was a problem hiding this comment.
Thank you for catching this; a whole-review crash on valid SQL is the worst outcome here. Fixed in "address review feedback: unaliased join crash, …" (b234d4333c): the alias fallback is only used when it is a plain identifier, and an unaliased table-function join (join unnest(a.arr) on …, join table(flatten(a.arr)) on …) now abstains for that join instead of building a regex from raw text. Tests cover both relation shapes and assert no throw.
There was a problem hiding this comment.
✅ Fixed — an unaliased relation whose fallback alias is not ^\w+$ now abstains (dbt-patterns.ts:996), and ownRef is only built from a word alias. join unnest(a.arr) on ... and join table(flatten(a.arr)) on ... return no finding and no throw; the new regression test fails against the old code.
| const JOIN_CLAUSE_END = /^(?:join|where|group\s+by|order\s+by|qualify|having|union|limit|window|left|right|inner|full|cross|natural|lateral|select|from)\b|^;/i | ||
|
|
||
| /** Normalised equality/comparison predicate: no whitespace, lowercase, `a=b` with sides sorted. */ | ||
| function predKey(p: string): string { |
There was a problem hiding this comment.
[MINOR] predKey lowercases and strips whitespace inside string literals, so different predicates compare equal
const t = p.replace(/\s+/g, "").toLowerCase()from a inner join b on a.flag = 'X' inner join c on b.id = c.id and a.flag = 'x' flags a.flag = 'x' as redundant (reproduced), but with 'X' filtering first it is not: removing it as suggested changes results. The doc comment says "Literals are compared as written", which this contradicts.
Fix: normalise only outside quoted literals, keep literal contents exact.
There was a problem hiding this comment.
Agreed, the code contradicted its own "literals are compared as written" comment. predKey now drops whitespace and lowercases only outside quoted literals, so a.flag = 'X' and a.flag = 'x' are distinct predicates (same commit, with a fire/silent pair for your exact example).
There was a problem hiding this comment.
'X' vs 'x' is covered by a test). Still wrong: double-quoted literals (BigQuery) are lowercased (a.flag = "X" then a.flag = "x" is flagged), and topLevelAnd splits on and inside a literal ('X AND Y' vs 'X AND Z' yields the truncated predicate a.flag = 'X). Both reproduced; see the new inline comment.
| if (!chained) seen = new Set() | ||
| const { alias, preds } = joins[k] | ||
| for (const p of preds) | ||
| if (chained && seen.has(p.key) && /=/.test(p.key) && !(alias && new RegExp(`\\b${alias}\\.`, "i").test(p.raw))) out.push(p.raw) |
There was a problem hiding this comment.
[MINOR] Predicates stay in seen across RIGHT/FULL joins
if (joins[k].inner) for (const p of preds) seen.add(p.key)from a inner join b on a.id = b.id right join c on b.id = c.id inner join d on c.id = d.id and a.id = b.id flags a.id = b.id (reproduced). The RIGHT join re-introduces rows where it is false, so removing it changes results ([1] vs [1, 2] in a SQLite check).
Fix: clear seen when a RIGHT/FULL join is crossed.
There was a problem hiding this comment.
Good find, and thanks for the SQLite check. Crossing a RIGHT or FULL join now clears the set of earlier predicates, since it brings back rows the inner join dropped. I also applied your nit from the review body: a join inside a parenthesised relation (starting before the previous join's end) no longer counts as a chain link, and the detector comment now says "INNER join". Tests use your four-join example for right and full outer, with the all-inner version still firing.
There was a problem hiding this comment.
✅ Fixed — RIGHT/FULL joins now clear seen (dbt-patterns.ts:973, :1023) and the new test covers RIGHT and FULL OUTER. Residual nit: the join modifier is read from a 20-character lookbehind, so a modifier separated from join by a lot of whitespace is missed (listed in the review body).
| "A string of SQL is joined with `~` to a macro call or variable with no space at the seam, so the rendered SQL runs the two together (`ticket_id` + `, source_relation` works, `ticket_id` + `source_relation` does not). Put a space inside the string next to `~`.", | ||
| { | ||
| // A literal holding SQL (it has whitespace) that ends, or starts, with a word character right at the `~`. | ||
| added: /'[^'\n]*\s[^'\n]*[A-Za-z0-9]'\s*~\s*[A-Za-z_][\w.]*|[\w)]\s*~\s*'[A-Za-z0-9][^'\n]*\s[^'\n]*'/, |
There was a problem hiding this comment.
[MINOR] jinja-concat-missing-space: catastrophic backtracking and ~ false positives
added: /'[^'\n]*\s[^'\n]*[A-Za-z0-9]'\s*~\s*[A-Za-z_][\w.]*|[\w)]\s*~\s*'[A-Za-z0-9][^'\n]*\s[^'\n]*'/,(1) The overlapping [^'\n]*\s[^'\n]* is super-linear on a long line with no closing quote: reported ~1.8 s for a 64 KB padded literal and several seconds at 100 KB+, synchronously. (2) The rule has no kind, so it also runs on model SQL, where the second branch matches the Postgres/Redshift regex operator (where memo ~ 'refund request') and log('Row count is' ~ n).
Fix: find the ~ first and inspect adjacent tokens linearly, or use a lookahead '(?=[^'\n]*\s)[^'\n]*[A-Za-z0-9]'; restrict to Jinja delimiters or kind: "macro".
There was a problem hiding this comment.
The ~ false positives were already addressed in 663b810 ("manifest incremental_strategy, {% test %} dispatch crash, Jinja seam gate"): the rule only matches inside an open {{/{% on the line and exempts log/print/raise_compiler_error/warn messages. The backtracking is fixed now with the lookahead form you suggested; a 64 KB unclosed literal went from about 0.9-1.3 s to under 1 ms, and a test guards a 100 KB line.
There was a problem hiding this comment.
{{/{% gate remove the original padded-literal blowup and the Postgres col ~ 'a b' false positive (tests added). The gate itself is still super-linear on a line with many unclosed openers: '{{ ''a'.repeat(n) takes ~120 ms at 10 KB and ~2.5 s at 20 KB. See the new inline comment.
| const at = new RegExp(`\\{%-?\\s*macro\\s+${escapeRe(macro)}\\s*\\(`).exec(text)!.index | ||
| const end = text.slice(at).search(/\{%-?\s*endmacro\b/) | ||
| const body = text.slice(at, end < 0 ? undefined : at + end) | ||
| out.push({ macro, params: sig.params.map((p) => p.name), acceptsKwargs: /\bkwargs\b/.test(body) || /\bvarargs\b/.test(body) }) |
There was a problem hiding this comment.
[MINOR] acceptsKwargs is true for varargs or the word kwargs in a comment
acceptsKwargs: /\bkwargs\b/.test(body) || /\bvarargs\b/.test(body)In Jinja, varargs captures extra positional args only; foo(x=1, extra=2) against default__foo(x) using varargs still raises takes no keyword argument. A {# no kwargs here #} comment also suppresses the finding. Any x__name macro is also treated as a variant (DISPATCH_VARIANT at line 251 could limit this to adapter prefixes).
Fix: only count real kwargs usage outside comments/strings.
There was a problem hiding this comment.
You're right that varargs only absorbs positionals. A variant now counts as accepting extra keywords only if it reads kwargs outside {# #} comments and string literals; tests cover varargs, a comment and a string mention. I left DISPATCH_VARIANT prefix matching as is for now: limiting it to known adapter prefixes would miss custom adapters, and a non-adapter x__name macro only matters if the dispatcher's keywords are passed to it.
There was a problem hiding this comment.
varargs, {# kwargs #} and a quoted 'kwargs' no longer count (project-rules.ts:643-644, test added). Still counted: kwargs in a SQL comment or plain template text, and any <x>__<name> macro is still treated as an adapter variant. See the new inline comment.
| export function testWindowFindings(file: ChangedFile): Finding[] { | ||
| if (file.status !== "modified" || !/\.sql$/i.test(file.path)) return [] | ||
| const rows = [...walkDiffRows(file.diff)] | ||
| const removed = rows.filter((r) => r.kind === "-").map((r) => windows(r.text)).filter((w) => w.secs.length) |
There was a problem hiding this comment.
[MINOR] test-window-shrunk ignores comparison direction
const removed = rows.filter((r) => r.kind === "-").map((r) => windows(r.text))ts > current_timestamp + interval '2 day' -> '1 day', or ts < dateadd('day', -30, ...) -> -7 make the test stricter, yet fire as a shortened lookback.
Fix: only fire for a window on the right of >/>=/between against the clock.
There was a problem hiding this comment.
Thanks, agreed. A window now counts only when it is a lookback: the clock minus the window (- interval, *_sub(…, interval …), or a negative dateadd count) on the right of >/>=/between, or on the left of </<=. Your + interval '2 day' → '1 day' and ts < dateadd('day', -30, …) → -7 cases are now silent, and current_date - interval 7 day <= d still fires.
There was a problem hiding this comment.
+ interval, ts < dateadd(-30)) are now silent with tests. The direction check still treats the upper bound of between and not between as a lookback (reproduced), and it only reads the current diff line, so a window on the line after the operator is no longer reported. See the new inline comment.
8aba994 to
f3c5e86
Compare
a6823f9 to
936b24a
Compare
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
| */ | ||
| export function redundantJoinPredicates(sql: string): string[] { | ||
| // Literal text stays (`= 'a'` and `= 'b'` differ); `and`/parens inside a literal are rare enough here. | ||
| const s = stripComments(sql).replace(/\{\{[\s\S]*?\}\}/g, "__jinja__").replace(/\{%[\s\S]*?%\}/g, " ") |
There was a problem hiding this comment.
SUGGESTION: literal text now feeds the structural join scan
stripLiterals was dropped here so = 'a' and = 'b' no longer compare equal, but the same s is also used as SQL structure: matchAll(/\bjoin\b/gi) (two lines below) and JOIN_CLAUSE_END now match keywords inside string literals. A model that adds a line such as {% set q = 'select * from a join b on a.id = b.id and y = 1' %} gets a phantom join parsed out of the string: when it precedes the real chain, chained is false at the next real join (its end lands mid-literal), so seen is reset and redundant predicates are missed; when it follows one, the phantom ON clause repeats the earlier predicate key and detectRedundantJoinPredicate finds the predicate text in the literal's own added line and reports "Remove it" against it. Mask literals in the copy used for the structural scan (e.g. replace each with a same-length placeholder) and keep the literal-preserving text only for predKey/the message.
| // A literal holding SQL (it has whitespace) that ends, or starts, with a word character right at the `~`, | ||
| // inside an open `{{`/`{%` on the line (a bare `col ~ 'a b'` is a Postgres regex match, not Jinja). | ||
| added: | ||
| /\{[{%](?:(?![}%]\}).)*?(?:'[^'\n]*\s[^'\n]*[A-Za-z0-9]'\s*~\s*[A-Za-z_][\w.]*|[\w)]\s*~\s*'[A-Za-z0-9][^'\n]*\s[^'\n]*')/, |
There was a problem hiding this comment.
SUGGESTION: the rewritten pattern still backtracks super-linearly on an unterminated quote
'[^'\n]*\s[^'\n]*[A-Za-z0-9]' and '[A-Za-z0-9][^'\n]*\s[^'\n]*' each join two unbounded [^'\n]* with only \s between them, and both stars can consume whitespace, so the split point is ambiguous. matchRule tests this against every added line synchronously, so a long added line with an open {{ and a ' that never closes makes the engine try every (star split, whitespace position) pair before failing, which is quadratic in line length on untrusted PR input. Find the ~ first and inspect the tokens next to it, or fold the whitespace test into a lookahead ('(?=[^'\n]*\s)[^'\n]*[A-Za-z0-9]') so only one unbounded star remains.
| // Lenient like PyYAML: a duplicate key elsewhere in the file must not hide the list. | ||
| let doc: any | ||
| try { | ||
| doc = YAML.parse(content ?? "", { uniqueKeys: false }) |
There was a problem hiding this comment.
SUGGESTION: this duplicates parseYaml
parseYaml (line 45) already wraps YAML.parse in the same try/catch and returns undefined on failure, and an undefined doc produces the same empty listed list below, so the inline copy adds a second YAML parse policy in this module that can drift from the helper. Give parseYaml an options argument (or a lenient flag) and call it here.
f3c5e86 to
a74e400
Compare
936b24a to
b234d43
Compare
| if (kind === 0 && !/-\s*$/.test(before) && !/\w*sub\s*\([^()]*(?:\([^()]*\)[^()]*)*$/i.test(before)) return false | ||
| const ops = [...line.matchAll(COMPARISON)] | ||
| const left = ops.filter((o) => o.index! < at).pop() | ||
| if (left) return /^(>=?|between)$/i.test(left[0]) |
There was a problem hiding this comment.
SUGGESTION: between treats its upper bound as a lookback too
ops.filter((o) => o.index < at).pop() returns between for every window after it, so the second window of event_at between current_date - interval 30 day and current_date - interval 1 hour is treated as a lower bound. Editing that upper bound to interval 30 minute makes the test check more rows (the bound moves later), yet shrunk is true and the finding claims rows "outside the new window are no longer tested". Only the window before the and that closes a between is a lookback — skip windows that sit after a between's and.
| } | ||
|
|
||
| const JOIN_TYPE_GAP = /^\s*(?:(?:natural|inner|left|right|full|cross)\s+)?(?:outer\s+)?$/i | ||
| const JOIN_CLAUSE_END = /^(?:join|where|group\s+by|order\s+by|qualify|having|union|limit|window|left|right|inner|full|cross|natural|lateral|select|from)\b|^;/i |
There was a problem hiding this comment.
SUGGESTION: third copy of the ON-clause helpers already in the repo
JOIN_CLAUSE_END duplicates the ON-clause terminator in rule-catalog.ts:182 (ON_END) and project-rules.ts:893 (ON_STOP), and topLevelAnd (line 1033) duplicates conjuncts in project-rules.ts:896; the copies have already drifted (group vs group\s+by, natural/lateral/from only here). Export one terminator and one top-level splitter and reuse them, so fixes reach every caller — including the literal-masking change the topLevelAnd thread asks for.
| const content = await read(qsPath, "new") | ||
| if (content) { | ||
| if (changed) checkedOther(changed) | ||
| all.push(quickstartModelFindings(changed ?? newModels[0], qsPath, content, newModels.map((f) => modelNameFromPath(f.path)))) |
There was a problem hiding this comment.
SUGGESTION: the finding can be anchored to a model it does not mention
When the PR does not change quickstart.yml, quickstartModelFindings targets newModels[0], but it only lists the new models whose names match the quickstart prefix. If an unrelated model is added first (an internal model sorted before the new public one), the file-level finding lands on that unrelated file. Pass the first missing model as the target, or fall back to the quickstart path.
| const t = performance.now() | ||
| expect(seam("{{ x ~ 'a " + "x ".repeat(50_000))).toBe(false) | ||
| expect(seam("{{ 'a " + "x ".repeat(50_000))).toBe(false) | ||
| expect(performance.now() - t).toBeLessThan(200) |
There was a problem hiding this comment.
SUGGESTION: wall-clock assertion is flaky under parallel CI
A 200 ms budget measured with performance.now() depends on host load, and bun test runs this file alongside the rest of test/altimate, so a busy runner can exceed it even though the scan is linear. Prefer a deterministic guard (e.g. run the scan on a much larger input inside a generous timeout, or drop the timing assertion), or raise the budget enough that only super-linear behavior fails.
|
Thanks for the thorough pass. The MAJOR (unaliased join crash) and all six inline MINORs are addressed in "address review feedback: unaliased join crash, literal-exact predicates, outer joins, concat regex, kwargs, lookback direction" ( |
hkaltai
left a comment
There was a problem hiding this comment.
Verdict: approve. Panel: Claude, Codex, lead. No CRITICAL or MAJOR remains. The prior MAJOR (unaliased unnest join crash) is fixed and has a regression test; both external reviewers ran bun test on the two touched test files (191 pass) and the four findings below were reproduced.
Counts: CRITICAL 0, MAJOR 0, MINOR 4 (all in summary-tier rules), NIT 4 new plus prior nits.
Prior findings
| Prior | Status |
|---|---|
MAJOR unescaped relation in new RegExp |
Fixed |
| MINOR predKey lowercases literals | Partially fixed (single quotes only; see inline) |
| MINOR seen across RIGHT/FULL | Fixed |
MINOR concat regex backtracking / ~ FPs |
Partially fixed (FP fixed; still super-linear on pathological lines; see inline) |
| MINOR acceptsKwargs | Partially fixed (see inline) |
| MINOR test-window direction | Partially fixed (see inline) |
| MINOR (body-only) incremental_strategy ignores manifest/project config | Fixed (project-rules.ts:714, test at review-project-rules.test.ts:712) |
Prior nits
| Nit | Status |
|---|---|
docs ${name} fixed-string search (orchestrate.ts:2369) |
Not fixed |
same-count param rename in macro-param-no-default |
Not fixed |
isPackageProject treats no profile: as package |
Not fixed |
duplicate << merge keys reported by yamlError |
Not fixed |
nested-relation join prev.end <= start |
Code fixed, no regression test (reverting the condition still passes); user-facing title still says "an earlier join" |
other yml marked reviewed; group by 1, customer_id; naming |
Not fixed |
New nits
dbt-patterns.ts:973:const outer = /\b(right|full)\s+(?:outer\s+)?$/i.test(before)reads the modifier from a 20-character lookbehind, sorightfollowed by many spaces thenjoinis not recognised as RIGHT and a predicate is flagged that changes results. Fix: tokenise the preceding word instead of a fixed window.dbt-patterns.ts:996-998: an abstaining table-function join is recorded withend: i(before its ON clause), so the next gap containson ...and the chain breaks, hiding later findings in that chain. Fix: scan the ON clause to setendbefore abstaining.dbt-patterns.ts:942-947:predKeylowercases quoted identifiers (a."Id"equalsa."ID") and mis-splits backslash-escaped literals;normalizeConditioninrule-catalog.tsalready exists for reuse.rule-catalog.ts:798:{{ "where memo ~ 'refund request'" }}(a quoted string inside Jinja) still matches the concat rule; the gate cannot tell code from a string inside the delimiters.
| function predKey(p: string): string { | ||
| const t = p | ||
| .split(/('(?:[^']|'')*')/) | ||
| .map((part, i) => (i % 2 ? part : part.replace(/\s+/g, "").toLowerCase())) |
There was a problem hiding this comment.
[MINOR] Literal handling in join-predicate comparison is still incomplete (double quotes, AND inside a literal)
.map((part, i) => (i % 2 ? part : part.replace(/\s+/g, "").toLowerCase()))Only single-quoted literals are protected. Reproduced:
... inner join b on a.flag = "X" inner join c on b.id = c.id and a.flag = "x"returnsa.flag = "x"as redundant (BigQuery double-quoted strings are lowercased).... join b on a.flag = 'X AND Y' join c on b.id = c.id and a.flag = 'X AND Z'returns the truncated predicatea.flag = 'X, becausetopLevelAnd(line 1037) splits onandinside the literal; the two full predicates differ.
Fix: split the clause with a quote-aware scanner (single and double quotes), and abstain on a predicate whose quotes are unbalanced. Add one test per case.
| if (kind === 0 && !/-\s*$/.test(before) && !/\w*sub\s*\([^()]*(?:\([^()]*\)[^()]*)*$/i.test(before)) return false | ||
| const ops = [...line.matchAll(COMPARISON)] | ||
| const left = ops.filter((o) => o.index! < at).pop() | ||
| if (left) return /^(>=?|between)$/i.test(left[0]) |
There was a problem hiding this comment.
[MINOR] test-window-shrunk: between upper bound, not between, and operator on a previous line
if (left) return /^(>=?|between)$/i.test(left[0])Reproduced with testWindowFindings:
ts between current_date - interval '30 day' and current_date - interval '7 day'->'3 day'fires, but that is the upper bound and the tested range grew.ts not between current_date - interval 30 day and current_date->7 dayfires, but a shorter excluded interval checks more rows.where event_at >=with the window on the next line is now silent, because only the current diff line is read.
Fix: for between, only count a window before the top-level and; bail out on not between; look back over the preceding diff rows for the last comparison operator. Add tests for each.
| // Only a real `kwargs` read takes extra keywords (`varargs` takes extra positionals only); a | ||
| // mention in a comment or string literal does not. | ||
| const code = body.replace(/\{#[\s\S]*?#\}/g, " ").replace(/'[^'\n]*'|"[^"\n]*"/g, "''") | ||
| out.push({ macro, params: sig.params.map((p) => p.name), acceptsKwargs: /\bkwargs\b/.test(code) }) |
There was a problem hiding this comment.
[MINOR] acceptsKwargs still counts kwargs in SQL comments/text, and any x__name macro is a variant
acceptsKwargs: /\bkwargs\b/.test(code)Only Jinja {# #} comments and quoted strings are stripped. select a -- extra kwargs are ignored or /* kwargs */ inside a variant body still gives acceptsKwargs: true, so a dispatcher passing an unsupported keyword is not reported. endsWith("__" + name) (line 636) also accepts helper__foo as a variant although DISPATCH_VARIANT already lists the adapter prefixes.
Fix: look for kwargs only inside {{ ... }} / {% ... %} spans, and restrict variants to the adapter prefixes.
| // inside an open `{{`/`{%` on the line (a bare `col ~ 'a b'` is a Postgres regex match, not Jinja). | ||
| added: | ||
| // The whitespace check is a lookahead so the literal is scanned once (no split backtracking on long lines). | ||
| /\{[{%](?:(?![}%]\}).)*?(?:'(?=[^'\n]*\s)[^'\n]*[A-Za-z0-9]'\s*~\s*[A-Za-z_][\w.]*|[\w)]\s*~\s*'[A-Za-z0-9](?=[^'\n]*\s)[^'\n]*')/, |
There was a problem hiding this comment.
[MINOR] jinja-concat-missing-space: delimiter gate is quadratic on lines with many unclosed openers
/\{[{%](?:(?![}%]\}).)*?(?:'(?=[^'\n]*\s)[^'\n]*[A-Za-z0-9]'\s*~ ...The lookahead fixed the single padded literal, but the lazy (?:(?!...).)*? prefix re-scans the line from every {{. "{{ 'a".repeat(n) took ~120 ms at 10 KB and ~2.5 s at 20 KB (reproduced). Needs an unusual line, but the match runs synchronously on every added line.
Fix: cap the gate span (e.g. .{0,300}?) or locate ~ first and inspect the adjacent tokens.
A macro builds its SELECT list from arguments and loops, so the column at a given position depends on the caller. `group-by-positional` needs two or more positions and so never saw `GROUP BY 2` in a high-water-mark query. - `rule-catalog:macro-group-by-ordinal`: a single `GROUP BY <n>` on an added line of a changed macro (new `macro` rule kind, so models' `group by 1` style is untouched). Anchors to the matched line; summary tier. - Tests: fires and records the line; twin with the column name is silent; two positions and model SQL are left to the existing rule. Dev evidence: 4 replayed reviewer comments asked for a column name instead of a single `GROUP BY` position in macro high-water-mark queries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…efault
Jinja binds an omitted macro parameter to Undefined, not `none`, so when a
parameter is added without a default every call written before the change
gets Undefined: an `is none` fallback downstream is skipped and the value
renders empty. `dbt-patterns:macro-var-no-default` covers `var()` reads only,
which is why it did not see this.
- `dbt-project-rules:macro-param-no-default`: a parameter present in the head
signature of a `{% macro %}` / `{% test %}` that existed at base, absent
from the base signature, with no default, on an added signature line.
`model` / `column_name` of a generic test are skipped. Anchored to the
parameter line; summary tier.
- Signature parsing keeps defaults that hold calls and lists intact.
- Tests: fires on the added line of a test block; `=none` twin is silent;
new macros and unchanged parameters are silent.
Dev evidence: a replayed reviewer comment on a generic test whose new
parameter lacked `=none` while its sibling test had it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sn't accept
When a dispatcher starts forwarding a new keyword, every `<adapter>__` variant
must take it; a variant that doesn't fails on that adapter with "takes no
keyword argument".
- `dbt-project-rules:dispatch-kwarg-not-in-variant`: for each keyword on an
added line inside an `adapter.dispatch('name', …)(…)` argument list, find
the repo's `<prefix>__name` variants (same file plus `grepProject` at head)
and report those whose signature lacks the keyword and whose body doesn't
read `kwargs`. Anchored to the added keyword line; summary tier.
- Uses the existing `grepProject`/`getContent` plumbing; no new inputs.
- Tests: fires on the keyword line naming only the variant that lacks it;
twin where every variant takes it is silent; `kwargs` catch-all and
unchanged lines are silent.
Dev evidence: a replayed reviewer comment on a dispatcher that began
forwarding two new keywords to its adapter variants.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…shorter A data test that checks "rows in the last day" and is edited to check the last 30 minutes silently stops testing everything outside the new window. - `dbt-project-rules:test-window-shrunk`: in a modified singular test, pair each added line with a removed line of the same shape (window literals masked) and report when an `interval N unit` or negative `dateadd` count got shorter. Anchored to the added line; summary tier. - Equivalent rewrites (1 day -> 24 hour), longer windows and filters that replace the literal with a var are silent (tests cover each). Dev evidence: two replayed reviewer comments on singular tests whose lookback shrank to the scheduler interval. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…any changed yml `.quickstart/quickstart.yml` has no dbt role, so the review listed it as not reviewed and never looked at it. Two structural defects live there and in other yml files. - `dbt-project-rules:yaml-parse-error`: a changed `.yml`/`.yaml` (any role) that parsed at base and no longer parses at head. A key indented under the previous entry is reported as "YAML key defined twice". Anchored to the parser's error line; summary tier. File-level by nature. - `dbt-project-rules:quickstart-public-model-missing`: a new model named with the prefix most `public_models` entries share (>=80%, >=3 entries) and not listed there. Goes on the quickstart file when the PR changes it, else on the new model; summary tier. - "Other" files these checks read are marked reviewed in the coverage ledger. - Tests: duplicate-key error line; correctly indented twin and an already broken base are silent; end-to-end run on a quickstart file; listed model twin is silent; staging/intermediate names are not public candidates. Dev evidence: two replayed reviewer comments on quickstart files (a mis-indented variable key; a new model left out of `public_models`). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dbt fails to parse a project when a yml description calls `doc('x')` and no
`{% docs x %}` block exists, which breaks every build of the PR.
- `dbt-project-rules:doc-block-missing`: single-argument `doc('x')` names on
added lines of a schema yml, not referenced at base, with no
`{% docs x %}` block in the project's `.md` files at head (`grepProject`,
cached per name; a failed search counts as found). Fires only when another
`doc()` in the same file resolves locally, so docs kept in an installed
package are not reported. Anchored to the added line; summary tier.
- Uses the existing `grepProject` plumbing; no new inputs.
- Tests: missing name fires with its line; a defined name is silent; a file
whose docs all live elsewhere is silent; end-to-end search of `*.md`.
Deferred: "any `dbt parse` error on head that is absent at base" needs both
sides' parse results, which the review CLI does not receive.
Dev evidence: a replayed reviewer comment on a yml `doc()` whose block was
never written, which failed the PR's CI build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Projects that install a package resolve their own dependencies and never read the package's lock file; committing it only pins the package's own development installs and drifts from its `packages.yml` ranges. In an application project the lock file belongs in git, so the rule needs the package signal. - `dbt-project-rules:package-lock-in-package`: `package-lock.yml` with status added, next to a `dbt_project.yml` whose repo has an `integration_tests/dbt_project.yml` or whose project names no `profile:`. File-level by nature; anchored to the file's first line; summary tier. - The file is marked reviewed in the coverage ledger when checked. - Tests: package with integration tests and package without a profile fire; application project twin and an already-committed (modified) lock file are silent. Deferred: `require-dbt-version` upper bound excluding the current dbt minor needs the running dbt version, which the review does not receive. Dev evidence: a replayed reviewer comment asking not to commit the lock file in a package repository. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s and Jinja SQL seams Three line-level defects reviewers raised on model SQL that no rule saw. - `dbt-project-rules:merge-no-incremental-predicates`: on BigQuery (known dialect, or a `partition_by` dict when unknown), an incremental model that merges (explicit or default strategy) on a `unique_key` into a partitioned table with no `incremental_predicates` in the file or the manifest config. Only when the PR touches the config or the incremental filter (existing gate). Anchored to the config/filter line; summary tier. - `dbt-patterns:redundant-join-predicate`: an equality in a JOIN's ON clause that an earlier join of the same join chain already states and that does not reference the newly joined relation. Chains break at any clause other than a join, so separate queries/CTEs never pair. Anchored to the added line; summary tier. - `rule-catalog:jinja-concat-missing-space`: a string literal holding SQL (it contains whitespace) joined with `~` to a call or variable with a word character at the seam. Identifier building (`'stg_' ~ name`) and quote wrapping are silent. Models and macros; summary tier. - Tests: each rule fires on its defect shape and its twin (`incremental_ predicates` set; predicate on the joined relation; space inside the literal) is silent, plus the cross-CTE and other-engine cases. Dev evidence: replayed reviewer comments asking for `incremental_predicates` on a partitioned merge, removing join predicates "included in join above", and adding a space around a macro call in a concatenated `datediff` string. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion-test projects Two gaps found replaying the dev target PRs (not the noise sample): - `dbt-project-rules:quickstart-public-model-missing` returned nothing when the quickstart file had an unrelated duplicate key elsewhere: the strict YAML parse threw. Read it leniently, as PyYAML (which dbt uses) does. - `dbt-project-rules:package-lock-in-package` looked for `integration_tests/dbt_project.yml` only; packages often nest it as `integration_tests/<name>/dbt_project.yml`. Also accept a nested project found through `grepProject`. - Tests: a quickstart file with a duplicate key still reports the missing model (listed-model twin silent); a nested integration project marks the repo as a package. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…te redundant Found on dev target files: `dbt-patterns:redundant-join-predicate` reported `x.platform = ''` repeated across two LEFT JOINs. - After a LEFT JOIN the same predicate is not redundant: the left join keeps the rows it rejects, so a later join's copy still filters. Only predicates seen in an earlier inner (or plain) join of the chain count. - Literals are compared as written; they were blanked, so `= 'web'` and `= 'app'` looked identical. - Tests: LEFT JOIN twin and different-literal twin are silent; equal literals after an inner join still fire. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Per-rule judging on dev target PRs: 5 of 6 non-target findings of `dbt-project-rules:macro-param-no-default` were on `<adapter>__x` variants (`default__`, `snowflake__`, ...). A variant is called only by its dispatcher, which the same change updates and which passes every argument, so "existing callers omit it" does not hold there. - Skip macros named `<adapter or default>__…`. - Test: a new parameter on an adapter variant is silent; the same on the dispatcher-level macro still fires. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…`{% test %}` dispatch crash, Jinja seam gate
- `project-rules.ts`: `modelConfigFindings` falls back to the manifest's
`incremental_strategy` when the file doesn't set one, so a BigQuery
`insert_overwrite` set in `dbt_project.yml` is no longer reported as an
unpruned merge by `merge-no-incremental-predicates`.
- `project-rules.ts`: `dispatchVariants` skips `{% test %}` blocks and guards the
macro re-lookup, which threw on a `{% test x__name %}` and dropped every
finding for the file.
- `rule-catalog.ts`: `jinja-concat-missing-space` only matches a seam inside an
open `{{`/`{%` on the line, so a Postgres/Redshift/DuckDB `col ~ 'a b'` regex
match no longer fires, and log/print/compiler-error messages are exempt.
- Tests for each.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…xact predicates, outer joins, concat regex, kwargs, lookback direction - `dbt-patterns.ts`: `redundantJoinPredicates` no longer builds a `RegExp` from raw relation text; an unaliased table-function join (`join unnest(a.arr) on`, `join table(flatten(...)) on`) abstains instead of throwing and rejecting the whole review. - `dbt-patterns.ts`: `predKey` normalises whitespace and case only outside quoted literals, so `a.flag = 'X'` and `a.flag = 'x'` stay different. - `dbt-patterns.ts`: a RIGHT/FULL join clears the predicates seen so far (it re-introduces rows an earlier inner join dropped); a join nested inside a parenthesised relation is no longer treated as a chain link. - `rule-catalog.ts`: `jinja-concat-missing-space` checks for whitespace with a lookahead, so a long unclosed literal is scanned linearly (was ~1 s at 64 KB). - `project-rules.ts`: a dispatch variant accepts extra keywords only when it reads `kwargs` outside comments and strings; `varargs` no longer counts. - `project-rules.ts`: `test-window-shrunk` only counts lookback windows (clock minus the window on the right of `>`/`>=`/`between`, or left of `<`/`<=`), so a stricter future bound or a column-below-clock filter is silent. - Tests for each. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
a74e400 to
08af665
Compare
b234d43 to
3d47982
Compare
Issue for this PR
Part of #1792
Type of change
What does this PR do?
Stacked on #1769. Adds 11 deterministic rules for macro, test and yml defects that the replay showed every reviewer missing. They are written from defects in the tuning repos and checked on repos never used for tuning. All start at summary tier.
Results (rules only, no AI)
New rules
GROUP BY <n>with a single position inside macro SQL.adapter.dispatchkeywords an adapter variant doesn't accept.public_modelsmissing a new model.doc()reference with no docs block.package-lock.ymladded to a package repository.How did you verify your code works?
bun test test/altimate/review: 711 pass.bunx tsc --noEmit: clean.review-dbt-patterns/review-project-rulestests.Cost side effects
None. The 11 rules are deterministic and local: no model calls, warehouse queries or cloud resources.
Screenshots / recordings
Not a UI change.
Checklist
Changes since review:
b234d4333caddresses the second-round review feedback: unaliased table-function joins abstain instead of crashing, literals kept exact in join predicates, RIGHT/FULL joins reset the chain, linearjinja-concat-missing-spaceregex, dispatch-variant kwargs, lookback direction.🤖 Generated with Claude Code