Skip to content

feat(review): close remaining review gaps: action dbt_prepare, data-test adequacy, parse-only column changes - #1726

Open
RaghwendraSingh wants to merge 4 commits into
feat/review-incremental-semanticsfrom
feat/review-ac-gaps
Open

RaghwendraSingh wants to merge 4 commits into
feat/review-incremental-semanticsfrom
feat/review-ac-gaps

Conversation

@RaghwendraSingh

@RaghwendraSingh RaghwendraSingh commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Issue for this PR

Part of #1792
Stacked on #1725 (← #1724 ← #1723 ← … ← #1719); merge in that order. Draft.

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

This closes the remaining acceptance-criteria gaps in the review stack. Two of the fixes came from replaying the PRs behind known escaped defects.

  • Optional artifact preparation in the action (dbt_prepare). Default none, so nothing changes for existing users.
    • compile runs dbt compile on the head, then compiles the PR's merge-base into target-base/ in a temporary worktree. A shallow checkout is fetched and deepened when needed.
    • full also runs dbt docs generate.
    • A failed base compile is a warning, and those models are marked unverified.
    • Inputs reach the script only through env.
    • The manual docs snippet now compiles the merge-base instead of the base branch's tip.
    • The example workflow gets a generic name.
  • Data tests that can't do their job (yml only, any adapter):
    • a test comparing a model with itself (equality / equal_rowcount against its own ref, or a relationships test to its own column) passes by construction;
    • new tests that inherit severity: warn or an error_if threshold from dbt_project.yml are reported;
    • a dbt_project.yml change that loosens those thresholds for a folder is reported.
  • Column-set rules work in dbt parse-only CI. Without compiled SQL, output columns come from the raw model's final SELECT; a Jinja expression in the list makes it unknown. In a parse-only replay, on_schema_change='fail' with new columns now fires on the two real escapes it targets.
  • Dedup windows split across lines are read whole. "Dedup has no ORDER BY" no longer fires falsely when order by is on its own line, and the load-timestamp rule sees such windows.

How did you verify your code works?

  • New tests:
    • review-action.test.ts: dbt_prepare is opt-in, reads inputs only from env, and builds into target-base/.
    • review-test-adequacy.test.ts: self-comparison, inherited warn/error_if, loosened thresholds, dbt 1.8+ arguments:.
    • Additions to review-incremental-semantics.test.ts: multi-line windows, raw-SQL column delta.
  • Suites: 506 review tests and 1,367 skill and release-gate tests pass, bun typecheck is clean, and the real-world corpus is unchanged (15/15 caught, 0/5 false positives).
  • dbt_prepare run locally against a real dbt 1.10 DuckDB project. The head compiled into target/, the merge-base compiled into target-base/ (the base still has the removed column), and the temporary worktree was cleaned up.
  • Replay of the PRs behind six known escapes in a private dbt monorepo (parse-only, as their CI runs):
    • on_schema_change now catches two of them (two escapes each).
    • One escape lived in Python loader code, outside dbt.
    • One targeted the removed microbatch rule.
    • One needs the opt-in slice var.
    • One is pre-existing code the PR didn't touch, so no rule reports it by design.
  • The earlier 7-PR replay is unchanged: 0 findings on files the PR didn't change.

Cost side effects

dbt_prepare: compile adds one dbt compile per side of the PR in CI, and full adds dbt docs generate (warehouse queries for the catalog). Both are opt-in; the default none adds nothing. No model calls.

Screenshots / recordings

N/A (CLI / action).

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR
  • I checked for cost side effects (model calls, queries or cloud resources that grow with user data) and described them, or wrote "none"

🤖 Generated with Claude Code


Summary by cubic

Closes the remaining acceptance-criteria gaps in the dbt review stack: adds an opt-in dbt_prepare action input for building the artifacts the full review needs, and fixes two classes of missed findings — data tests that can't do their job, and column-set rules that skipped work in parse-only CI.

New dbt_prepare action input

  • Default none; compile runs dbt compile on the head and the PR's merge-base into target-base/, and full also runs dbt docs generate.
  • Inputs reach the script only through env; a failed base compile, docs generate, or checkout is a warning that keeps the review running.
  • A stale target-base/ is removed before the base compile, docs now compile the merge-base, and the example workflow gets a generic name.

Review gap fixes

  • Reports data tests that pass by construction: self-comparisons with equality/equal_rowcount/relationships or expect_table_aggregation_to_equal_other_table with matching expressions, plus tests that inherit severity: warn or an error_if threshold from dbt_project.yml.
  • Reports dbt_project.yml changes that loosen those thresholds, counting only bounds that tolerate more failing rows.
  • Without compiled SQL, output columns come from the raw model's final SELECT, so on_schema_change='fail' fires in parse-only CI; a Jinja expression or {% if %} tag in the list leaves it unknown.
  • Dedup windows are read from the whole file and only count when an added line falls inside them, so a split ORDER BY no longer triggers a false "no ORDER BY".

Written for commit e77d121. Summary will update on new commits.

View guided diff Turn on auto-fix

@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 30a65ef6-eeb3-47f2-883f-7f1612c1191b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions

Copy link
Copy Markdown
- - - - - - - - - - - - - - - - - - - - - - - - -
                    AIRECEIPTS                    
             1 session behind this PR             

claude-opus-5-5 100% · <s…...221,036,213 tokens
  SUBAGENTS (19).................63,001,251 tokens
--------------------------------------------------
TOTAL unpriced................≥ 284,037,464 tokens
  counted: 1 session + 19 subagents
  cache served 97% of input tokens

1 candidate session not attributed
(in repo + branch window, no branch commit)
  full receipts + session ids: section below
- - - - - - - - - - - - - - - - - - - - - - - - -
                npx aireceipts-cli                
         github.com/anandgupta42/receipts         
- - - - - - - - - - - - - - - - - - - - - - - - -
full receipts (1 session)
session id scope turns time tokens in / out cached
orchestrator 6fe8be4d entire session (slice unavailable) 700 189h 52m 1.6k / 696k 98%

orchestrator · 6fe8be4d

- - - - - - - - - - - - - - - - - - - - - - - - -
                    AIRECEIPTS                    
         “data-eng-code-review-landscape”         
  Claude Code · Nov 05 2026 10:10 UTC · 189h 52m  
      claude-opus-5-5 100% · <synthetic> 0%       
         cache served 98% of input tokens         

pre-edit: 1% of tokens (12/698 turns)
  (share before the first named edit tool)

Bash..................164,495,680 tok  (495 calls)
(thinking/reply)........19,821,384 tok  (67 turns)
Write....................8,521,992 tok  (31 calls)
ToolSearch...............5,032,526 tok  (25 calls)
Read.....................4,036,974 tok  (29 calls)
mcp__claude_ai_Atlass…...3,292,204 tok  (28 calls)
Edit.....................3,095,292 tok  (28 calls)
mcp__claude_ai_Atlass…...2,143,864 tok  (12 calls)
mcp__claude_ai_Atlass…...1,725,016 tok  (17 calls)
Agent....................1,704,960 tok  (22 calls)
mcp__claude_ai_Atlassian…...947,323 tok  (7 calls)
ExitPlanMode................758,820 tok  (4 calls)
mcp__notion__notion-upda…...607,506 tok  (4 calls)
mcp__claude_ai_Atlassian…...489,926 tok  (5 calls)
mcp__claude_ai_Atlassia…...489,318 tok  (13 calls)
mcp__claude_ai_Atlassian_…...485,105 tok  (1 call)
mcp__claude_ai_Atlassia…...479,160 tok  (17 calls)
mcp__atlassian__createJi…...419,313 tok  (2 calls)
WebSearch...................384,482 tok  (4 calls)
AskUserQuestion.............366,689 tok  (2 calls)
SendMessage.................358,309 tok  (7 calls)
mcp__notion__notion-fetch...327,408 tok  (2 calls)
mcp__notion__notion-creat…...248,897 tok  (1 call)
mcp__atlassian__searchJir…...222,754 tok  (1 call)
WebFetch.....................217,960 tok  (1 call)
mcp__notion__notion-search...106,809 tok  (1 call)
mcp__atlassian__atlassianU…...94,821 tok  (1 call)
mcp__atlassian__getVisible…...94,821 tok  (1 call)
TaskStop......................66,901 tok  (1 call)
--------------------------------------------------
TOTAL..............................221,036,213 tok
no price table matched
- - - - - - - - - - - - - - - - - - - - - - - - -
                npx aireceipts-cli                
         github.com/anandgupta42/receipts         
- - - - - - - - - - - - - - - - - - - - - - - - -
subagents (19)
subagent cost
Goal: an evidence-backed, cited landscape of (a) dbt PR review tools, (b) data-… 2,134,743 tokens
Goal: determine what gaps altimate-code's dbt PR review has, by comparing what… 5,303,299 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 2,794,842 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 3,244,430 tokens
Label escaped-defect candidates by READING them (no keyword bucketing). Directo… 2,570,116 tokens
Repo: /Users/raghwendrasingh/code/altimateai/altimate-code (current branch feat… 4,836,719 tokens
You are doing an ADVERSARIAL review of a set of Jira tickets that plan improvem… 763,761 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 3,351,198 tokens
Label escaped-defect candidates by READING them (no keyword bucketing). Directo… 4,214,327 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 2,821,134 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 1,795,004 tokens
You are doing an ADVERSARIAL review of a stack of changes to an open-source dbt… 4,084,798 tokens
Repo: /Users/raghwendrasingh/code/altimateai/altimate-code. Module: packages/op… 8,815,483 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 2,592,769 tokens
Research task, read-only (do not push, comment, or open PRs anywhere).Goal: dec… 1,198,872 tokens
Judge each of altimate-code's own inline PR review comments by reading it. Inpu… 1,673,212 tokens
Label escaped-defect candidates by READING them (no keyword bucketing). Directo… 2,862,492 tokens
Goal: read a Notion doc tree about an "agentic data engineering software factor… 1,335,636 tokens
Goal: test whether the bug classes that real reviewers catch on dbt PRs at one… 6,608,416 tokens

Generated by aireceipts

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10 issues found across 10 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/opencode/test/altimate/review-test-adequacy.test.ts">

<violation number="1" location="packages/opencode/test/altimate/review-test-adequacy.test.ts:56">
P3: The test name claims 'deeper wins', but no case sets the same key at two levels. Add a fixture where a folder overrides the project-level `+severity` (for example, `shop: +severity: error` with `marts: +severity: warn`) and assert the deeper value and its path.</violation>
</file>

<file name="packages/opencode/src/altimate/review/test-adequacy.ts">

<violation number="1" location="packages/opencode/src/altimate/review/test-adequacy.ts:28">
P2: `fewer_rows_than` is in `SELF_COMPARE`, but a self-comparison makes it fail every time, not pass. The message "passes by construction" is wrong for this test. Drop it from the set and report it separately, or change the message to cover both outcomes.</violation>

<violation number="2" location="packages/opencode/src/altimate/review/test-adequacy.ts:69">
P2: `refsEntity` matches `ref(` inside names such as `my_ref(`, creating false self-comparison findings. Require a word boundary before `ref` and allow normal whitespace before `(`.</violation>

<violation number="3" location="packages/opencode/src/altimate/review/test-adequacy.ts:117">
P3: `path` records only the last node containing either setting, so findings misidentify the source when `severity` and `error_if` are inherited from different levels. Track a path per threshold.</violation>

<violation number="4" location="packages/opencode/src/altimate/review/test-adequacy.ts:128">
P2: `segmentsOf` fails to strip nested configured model paths, so inherited test thresholds are missed under paths such as `src/dbt/models/marts`. Strip the full configured model directory, not just its first segment.</violation>

<violation number="5" location="packages/opencode/src/altimate/review/test-adequacy.ts:164">
P2: `own` treats any explicit threshold as overriding every inherited setting, so tests can inherit an ineffective `severity` or `error_if` without being reported. Track severity and error thresholds independently.</violation>
</file>

<file name="docs/docs/usage/dbt-pr-review.md">

<violation number="1" location="docs/docs/usage/dbt-pr-review.md:215">
P3: This snippet pins the action to `@main`, but the rest of the page pins `@v0.8.5`. Pin to the same release tag so users get a reviewed, reproducible action version.</violation>
</file>

<file name="packages/opencode/src/altimate/review/orchestrate.ts">

<violation number="1" location="packages/opencode/src/altimate/review/orchestrate.ts:664">
P2: When only the base compile is missing, `columnDelta` now falls through to the raw-SQL path and can emit removed-column warnings. Before this change that case returned no deltas. Confirm this is intended for models marked unverified, or keep returning no deltas when the head is compiled and the base is not.</violation>
</file>

<file name="packages/opencode/src/altimate/review/dbt-patterns.ts">

<violation number="1" location="packages/opencode/src/altimate/review/dbt-patterns.ts:235">
P2: The `length >= 3` filter drops valid added SQL lines such as `id` or `x`. A multiline window changed only by adding such a two-character expression is then ignored, so the no-`ORDER BY` detector misses the change; filter only empty normalized lines.</violation>

<violation number="2" location="packages/opencode/src/altimate/review/dbt-patterns.ts:243">
P2: The parenthesis scan counts parentheses inside SQL string literals. A value like `status = ')'` closes the window early, truncating `spec` before a real `ORDER BY`; mask quoted literals or use a quote-aware scanner while finding the matching window close.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

"equality",
"equal_rowcount",
"cardinality_equality",
"fewer_rows_than",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: fewer_rows_than is in SELF_COMPARE, but a self-comparison makes it fail every time, not pass. The message "passes by construction" is wrong for this test. Drop it from the set and report it separately, or change the message to cover both outcomes.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/test-adequacy.ts, line 28:

<comment>`fewer_rows_than` is in `SELF_COMPARE`, but a self-comparison makes it fail every time, not pass. The message "passes by construction" is wrong for this test. Drop it from the set and report it separately, or change the message to cover both outcomes.</comment>

<file context>
@@ -0,0 +1,214 @@
+  "equality",
+  "equal_rowcount",
+  "cardinality_equality",
+  "fewer_rows_than",
+  "expect_table_row_count_to_equal_other_table",
+  "expect_table_aggregation_to_equal_other_table",
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. fewer_rows_than is dropped (against itself it always fails), and an aggregation comparison on the same model is flagged only when both expressions match.

current,
}
// Compiled SQL on both sides: column lineage plus the final SELECT's names.
if (engineNewSql && (!withOld || engineOldSql)) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When only the base compile is missing, columnDelta now falls through to the raw-SQL path and can emit removed-column warnings. Before this change that case returned no deltas. Confirm this is intended for models marked unverified, or keep returning no deltas when the head is compiled and the base is not.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 664:

<comment>When only the base compile is missing, `columnDelta` now falls through to the raw-SQL path and can emit removed-column warnings. Before this change that case returned no deltas. Confirm this is intended for models marked unverified, or keep returning no deltas when the head is compiled and the base is not.</comment>

<file context>
@@ -648,19 +649,41 @@ async function columnDelta(
+          current,
+        }
+  // Compiled SQL on both sides: column lineage plus the final SELECT's names.
+  if (engineNewSql && (!withOld || engineOldSql)) {
+    const current = await outputColumns(runner, engineNewSql, dialect)
+    const before = withOld ? await outputColumns(runner, engineOldSql!, dialect) : undefined
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. A {% if %} or {% for %} tag in the raw select list now makes the list unknown, so conditional columns are no longer reported as added or removed. We kept the raw fallback when only one side is compiled: both sides are then read from raw SQL, so the comparison stays like for like.

Comment thread github/review/action.yml Resolved
*/
function dedupWindows(c: Ctx): Array<{ spec: string; text: string }> {
const sql = stripComments(c.newSql)
const added = c.added.map((l) => stripComments(l).trim()).filter((l) => l.length >= 3)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The length >= 3 filter drops valid added SQL lines such as id or x. A multiline window changed only by adding such a two-character expression is then ignored, so the no-ORDER BY detector misses the change; filter only empty normalized lines.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/dbt-patterns.ts, line 235:

<comment>The `length >= 3` filter drops valid added SQL lines such as `id` or `x`. A multiline window changed only by adding such a two-character expression is then ignored, so the no-`ORDER BY` detector misses the change; filter only empty normalized lines.</comment>

<file context>
@@ -225,15 +225,42 @@ const detectNotIn: Detector = (c) => {
+ */
+function dedupWindows(c: Ctx): Array<{ spec: string; text: string }> {
+  const sql = stripComments(c.newSql)
+  const added = c.added.map((l) => stripComments(l).trim()).filter((l) => l.length >= 3)
+  const out: Array<{ spec: string; text: string }> = []
+  for (const m of sql.matchAll(/row_number\s*\(\s*\)\s*over\s*\(/gi)) {
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. When the diff has hunk headers (the PR API always sends them), matching now uses line numbers and no longer depends on the length >= 3 text filter.

const arg = (t: TestRef, name: string) => t.args?.[name] ?? t.args?.arguments?.[name]
const shortName = (test: string) => test.split(".").pop()!.toLowerCase()
const refsEntity = (v: unknown, entity: string) =>
typeof v === "string" && new RegExp(`ref\\(\\s*['"]${entity.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}['"]\\s*\\)`).test(v)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: refsEntity matches ref( inside names such as my_ref(, creating false self-comparison findings. Require a word boundary before ref and allow normal whitespace before (.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/test-adequacy.ts, line 69:

<comment>`refsEntity` matches `ref(` inside names such as `my_ref(`, creating false self-comparison findings. Require a word boundary before `ref` and allow normal whitespace before `(`.</comment>

<file context>
@@ -0,0 +1,214 @@
+const arg = (t: TestRef, name: string) => t.args?.[name] ?? t.args?.arguments?.[name]
+const shortName = (test: string) => test.split(".").pop()!.toLowerCase()
+const refsEntity = (v: unknown, entity: string) =>
+  typeof v === "string" && new RegExp(`ref\\(\\s*['"]${entity.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}['"]\\s*\\)`).test(v)
+
+function finding(file: ChangedFile, model: string | undefined, rule: string, severity: Finding["severity"], title: string, body: string, extra = ""): Finding {
</file context>
Suggested change
typeof v === "string" && new RegExp(`ref\\(\\s*['"]${entity.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}['"]\\s*\\)`).test(v)
typeof v === "string" && new RegExp(`\\bref\\s*\\(\\s*['"]${entity.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}['"]\\s*\\)`).test(v)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked (\bref).

const inherited = inheritedThresholds(project, segmentsOf(file.path, input.layout))
const own = (t: TestRef) => {
const c = t.args?.config ?? {}
return [c.severity, c.error_if, c.warn_if, t.args?.severity, t.args?.error_if].some((v) => v !== undefined)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: own treats any explicit threshold as overriding every inherited setting, so tests can inherit an ineffective severity or error_if without being reported. Track severity and error thresholds independently.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/test-adequacy.ts, line 164:

<comment>`own` treats any explicit threshold as overriding every inherited setting, so tests can inherit an ineffective `severity` or `error_if` without being reported. Track severity and error thresholds independently.</comment>

<file context>
@@ -0,0 +1,214 @@
+    const inherited = inheritedThresholds(project, segmentsOf(file.path, input.layout))
+    const own = (t: TestRef) => {
+      const c = t.args?.config ?? {}
+      return [c.severity, c.error_if, c.warn_if, t.args?.severity, t.args?.error_if].some((v) => v !== undefined)
+    }
+    const loose = added.filter((t) => !own(t))
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. We'll check each own setting separately.

data_tests:
- not_null${own}
`
test("the effective value walks project → folders, deeper wins", () => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The test name claims 'deeper wins', but no case sets the same key at two levels. Add a fixture where a folder overrides the project-level +severity (for example, shop: +severity: error with marts: +severity: warn) and assert the deeper value and its path.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/review-test-adequacy.test.ts, line 56:

<comment>The test name claims 'deeper wins', but no case sets the same key at two levels. Add a fixture where a folder overrides the project-level `+severity` (for example, `shop: +severity: error` with `marts: +severity: warn`) and assert the deeper value and its path.</comment>

<file context>
@@ -0,0 +1,83 @@
+        data_tests:
+          - not_null${own}
+`
+  test("the effective value walks project → folders, deeper wins", () => {
+    const doc = YAML.parse(project)
+    expect(inheritedThresholds(doc, ["marts"])).toEqual({ severity: "warn", path: ["shop", "marts"] })
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: nit, tracked. We'll add a fixture where the deeper level wins.

The action can build these for you with `dbt_prepare` (off by default):

```yaml
- uses: AltimateAI/altimate-code/github/review@main

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This snippet pins the action to @main, but the rest of the page pins @v0.8.5. Pin to the same release tag so users get a reviewed, reproducible action version.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/docs/usage/dbt-pr-review.md, line 215:

<comment>This snippet pins the action to `@main`, but the rest of the page pins `@v0.8.5`. Pin to the same release tag so users get a reviewed, reproducible action version.</comment>

<file context>
@@ -209,12 +209,30 @@ jobs:
+The action can build these for you with `dbt_prepare` (off by default):
+
+```yaml
+      - uses: AltimateAI/altimate-code/github/review@main
+        with:
+          dbt_prepare: compile        # none | compile | full
</file context>
Suggested change
- uses: AltimateAI/altimate-code/github/review@main
- uses: AltimateAI/altimate-code/github/review@v0.8.5

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing yet: @main is intentional until a release that includes dbt_prepare ships, and we'll re-pin it then.

const t = thresholds(n)
if (t.severity) eff.severity = t.severity
if (t.error_if) eff.error_if = t.error_if
if (t.severity || t.error_if) path = at.length ? at : ["(all tests)"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: path records only the last node containing either setting, so findings misidentify the source when severity and error_if are inherited from different levels. Track a path per threshold.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/test-adequacy.ts, line 117:

<comment>`path` records only the last node containing either setting, so findings misidentify the source when `severity` and `error_if` are inherited from different levels. Track a path per threshold.</comment>

<file context>
@@ -0,0 +1,214 @@
+    const t = thresholds(n)
+    if (t.severity) eff.severity = t.severity
+    if (t.error_if) eff.error_if = t.error_if
+    if (t.severity || t.error_if) path = at.length ? at : ["(all tests)"]
+  }
+  return { ...eff, path }
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: nit, tracked. Each threshold will track its own path.

function rawSelectList(sql: string): Set<string> {
const plain = sql
.replace(/\{#[\s\S]*?#\}/g, " ")
.replace(/\{%[\s\S]*?%\}/g, " ")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: rawSelectList blanks Jinja tags but keeps block bodies, so conditional columns become unconditional

{% if %}/{% for %} bodies stay in the text, so select id, amount {% if target.name == 'prod' %}, _loaded_at{% endif %} from t parses as {id, amount, _loaded_at} even when the compiled head renders the false branch. Line 664 now falls back to this raw path whenever either compiled side is missing (e.g. the common "compile the head only, no target-base" setup), so this yields a false columnDelta — a spurious new column that drives on_schema_change, contract, and consumer-list findings, where the old code returned none. Consider treating a model whose select list contains any {% if/for/set %} block as unknown (empty set), and/or using raw only when neither side has compiled SQL.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. Same fix as r4626962305.

}
if (close < 0) continue
const text = sql.slice(m.index!, close + 1)
if (!added.some((l) => text.includes(l) || l.includes(text.replace(/\s+/g, " ")))) continue

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: "Added line falls inside the window" is a raw substring match, so untouched windows are flagged

text.includes(l) has no line anchoring, so a short or clause-shaped added line elsewhere in the file matches any window that merely contains that text. Example: a pre-existing row_number() over (partition by country, id) (no ORDER BY) is untouched, and the PR adds , id to the select list — text.includes(", id") is true and dedup_no_tiebreaker fires on code the PR never touched, with evidence.line pointing at the pre-existing window. Same for a shared partition by customer_id / order by _loaded_at line. The "a window the PR doesn't touch is not reported" test only covers a line that matches nothing. Match by line range (compute the window's line span and check an added line number falls in it) instead of substring containment.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. Same fix as r4626962384.

Comment thread github/review/action.yml Outdated
fi
if [[ -z "$MB" ]]; then echo "::warning::Could not find the merge-base with $BASE; skipping the base compile."; exit 0; fi
WT="$RUNNER_TEMP/altimate-review-base"
git worktree add --detach "$WT" "$MB" >/dev/null

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Unguarded git worktree add aborts the whole review

The base compile at line 227 is best-effort (|| echo "::warning::…"), but git worktree add is a plain command under set -e. If it fails — e.g. a stale $RUNNER_TEMP/altimate-review-base on a reused/self-hosted runner, or a lazy-fetch error for $MB — the step fails and the following "Run dbt PR review" step is skipped, contradicting the documented "a base compile that fails is a warning: the review still runs". Clean $WT first and guard the add (|| { echo "::warning::…"; exit 0; }), and register the cleanup trap before the add.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. The cleanup trap is set first, and a failed git worktree add now warns and skips the base compile.

for (const [path, t] of flat(newDoc)) {
const prev = before.get(path) ?? {}
const toWarn = t.severity === "warn" && prev.severity !== "warn"
const newThreshold = t.error_if && t.error_if !== "!= 0" && t.error_if !== prev.error_if

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: error_if "loosening" has no direction check and matches the default by exact string

newThreshold = t.error_if && t.error_if !== "!= 0" && t.error_if !== prev.error_if fires for any change, including a tightening: >100 → >10 is reported as "test thresholds loosened … so those tests stop failing the build", the opposite of what happened. It also treats only the exact spelling "!= 0" as the strict default — "!=0", ">0", " > 0 " are all flagged as custom thresholds. Parse the bound and compare direction (only report when the new bound allows more failures), and normalize the strict-default forms.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. Same fix as r4626962408.

"equality",
"equal_rowcount",
"cardinality_equality",
"fewer_rows_than",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Some SELF_COMPARE entries are not "passes by construction"

fewer_rows_than compares row counts with <; against its own ref it always fails (count < count is false), so the emitted body ("passes by construction") is wrong. And expect_table_aggregation_to_equal_other_table is designed to compare two different aggregations: the same model with expression: sum(subtotal) vs compare_expression: sum(net) + sum(tax) is a valid cross-column invariant, yet it is flagged. Only treat aggregation tests as self-comparisons when the expressions match, and drop/adjust fewer_rows_than.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. Same fix as r4626962291.

Comment thread github/review/action.yml Outdated
build "$PROJECT" "$PROJECT/target"
if [[ "$IN_PREPARE" == "full" ]]; then
echo "Generating catalog.json…"
( cd "$PROJECT" && "${DBT[@]}" docs generate --target-path "$PROJECT/target" ${PROFILES[@]+"${PROFILES[@]}"} )

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: dbt docs generate is unguarded in full mode

In full mode this plain command under set -e aborts the step (and skips the review) if the warehouse is unreachable or credentials are missing — exactly the failure the input description warns about. dbt compile at line 207 already produced a usable target/, so degrade instead: … docs generate … || echo "::warning::docs generate failed; continuing without catalog.json.".

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. A failed dbt docs generate now warns and the review continues without catalog.json.

const rel = root && file.startsWith(root) ? file.slice(root.length) : file
const parts = rel.split("/").slice(0, -1)
const modelDirs = layout?.dirs.model ?? ["models"]
return modelDirs.includes(parts[0]) ? parts.slice(1) : parts

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: segmentsOf doesn't normalize multi-segment/relative model-paths

modelDirs.includes(parts[0]) matches only the first segment exactly. A valid model-paths: ["./models"] or ["transform/models"] isn't stripped (parts[0] is models/transform), so inheritedThresholds walks keys that don't exist and misses inherited severity/error_if. roleFromLayout in diff-filter.ts already normalizes these dirs; reuse that (normalize and strip the longest matching configured prefix).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. Same as r4626962396.

Comment thread github/review/action.yml
[[ -n "${IN_PROFILES:-}" ]] && PROFILES=(--profiles-dir "$(cd "$PROJECT" && cd "$IN_PROFILES" && pwd)")
build() { # $1 = project dir, $2 = target path
( cd "$1"
[[ -f packages.yml || -f dependencies.yml ]] && "${DBT[@]}" deps ${PROFILES[@]+"${PROFILES[@]}"}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: "${DBT[@]}" expands an array that can be empty

read -ra DBT <<< "$IN_DBT" produces an empty array when dbt_command is empty/whitespace. Under set -u on bash < 4.4 (e.g. macOS 3.2) "${DBT[@]}" is an unbound-variable error; on newer bash the command word vanishes and deps/compile fails obscurely. Validate it ([[ ${#DBT[@]} -gt 0 ]] || { echo "::error::dbt_command is empty"; exit 1; }) and use the file's own ${DBT[@]+"${DBT[@]}"} idiom for consistency.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing: dbt_command defaults to dbt, and an empty value is a configuration error that fails loudly.

// A dbt_project.yml change that loosens test thresholds for a whole path.
for (const file of reviewable.filter((f) => f.kind === "project_config" && /(^|\/)dbt_project\.ya?ml$/.test(f.path))) {
if (file.status !== "modified") continue
const [oldContent, newContent] = await Promise.all([getContent?.(file.path, "old"), getContent?.(file.path, "new")])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Re-fetch of dbt_project.yml is redundant and unguarded

When the PR modifies dbt_project.yml, this re-reads the head content that projectYml already holds (fetched with .catch at line 1671), and unlike that read it has no .catch. A rejecting getContent (valid custom/unit-test resolver) aborts the entire runReview instead of degrading this lane. Reuse projectYml when file.path === projectYmlPath, and guard the old-side read.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. We'll reuse projectYml and add a .catch.

const out = detectTestAdequacy({ file, newContent: newTests(), projectYml: project })
expect(rule(out)).toEqual(["test-inherits-warn"])
expect(out[0].body).toContain("`shop` → `marts`")
const own = newTests(":\n config:\n severity: error").replace("- not_null:", "- not_null:")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: No-op .replace and a silent negative assertion

.replace("- not_null:", "- not_null:") replaces a string with itself (dead code — the interpolated argument already supplies the colon). Also, if a future indentation slip made this YAML invalid, parse() returns undefined, detectTestAdequacy returns [], and line 68 would pass silently. Drop the .replace and assert the parsed shape so the negative assertion is trustworthy.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: nit, tracked. We'll remove the no-op .replace.

@kilo-code-bot

kilo-code-bot Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Code Review Summary

Status: 16 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 5
SUGGESTION 11
Issue Details (click to expand)

WARNING

File Line Issue
github/review/action.yml 228 The stale target-base/ cleanup runs only after the guards that skip the base compile, so on a persisted/cached workspace the previous run's base SQL is still consumed (and not reported unverified); the new unguarded rm -rf can also abort the step under set -e. Move the cleanup above the BASE/MB/worktree guards and make it non-fatal.
github/review/action.yml 207 The head dbt compile/deps is unguarded under set -e, so a head compile failure skips the review step entirely (even in comment mode) while the docs (line 224) say compile "usually works offline". Degrade to a warning like the base side, or document that it fails the job.
packages/opencode/src/altimate/review/test-adequacy.ts 27 cardinality_equality on the same model with a different field is a real cross-column check but is still reported as a self-comparison; the aggregation case is now guarded, this one is not.
packages/opencode/src/altimate/review/test-adequacy.ts 225 Loosening is decided per config node rather than on the effective value: deleting an override under a looser parent goes unreported, a redundant override is reported, and a non-simple error_if change is called "loosened" in either direction.
packages/opencode/src/altimate/review/orchestrate.ts 685 The parse-only select list is treated as complete when an item yields no name (select id, amount::numeric(18,2) → {id}), so a cast change reads as a removed column and drives a high-confidence warning.

SUGGESTION

File Line Issue
packages/opencode/src/altimate/review/dbt-patterns.ts 229 addedLineNumbers re-implements the exported commentableLines hunk walker (same regex/counter, already diverging); extract a shared helper in diff-filter.ts or pass the set in through Ctx.
packages/opencode/src/altimate/review/dbt-patterns.ts 257 The inline comment blanking is not quote-aware ('--' in a literal blanks the rest of the line and the dedup window disappears); reuse the existing length-preserving blankComments() from select-list.ts.
packages/opencode/src/altimate/review/dbt-patterns.ts 260 The length >= 3 added-line filter drops valid two-character lines (id, x) on the header-less fallback path, so a window changed only by such a line is ignored.
packages/opencode/src/altimate/review/dbt-patterns.ts 266 The parenthesis-depth scan still runs on comment-stripped but not literal-stripped SQL, so a (/) inside a literal truncates or over-extends the window.
packages/opencode/src/altimate/review/test-adequacy.ts 140 segmentsOf strips only an exact first model-paths segment, so nested or relative model dirs (transform/models, ./models) miss inherited thresholds.
packages/opencode/src/altimate/review/test-adequacy.ts 182 own() treats any one local config field as overriding every inherited setting, so a test can still inherit an ineffective severity or error_if unreported.
github/review/action.yml 203 "${DBT[@]}" is an unbound-variable error for an empty dbt_command under set -u on bash < 4.4; validate the array and use the file's ${DBT[@]+...} idiom.
packages/opencode/src/altimate/review/orchestrate.ts 1784 dbt_project.yml is re-fetched (redundant with line 1756) without a .catch, so a rejecting resolver aborts the whole runReview.
packages/opencode/test/altimate/review-test-adequacy.test.ts 83 The no-op .replace("- not_null:", "- not_null:") is still there, and the negative assertion would pass silently if the YAML stopped parsing.
packages/opencode/test/altimate/review-action.test.ts 31 The ordering assertion passes when the cleanup line is absent (indexOf = -1 compares less than the build index); assert the line exists before comparing indexes.
docs/docs/usage/dbt-pr-review.md 215 The new snippet pins the action to @main while the rest of the page pins @v0.8.5.
Files Reviewed (10 files)
  • docs/docs/usage/dbt-pr-review.md - 1 issue
  • github/review/action.yml - 3 issues
  • github/review/examples/dbt-pr-review.yml - 0 issues
  • packages/opencode/src/altimate/review/dbt-patterns.ts - 4 issues
  • packages/opencode/src/altimate/review/orchestrate.ts - 2 issues
  • packages/opencode/src/altimate/review/test-adequacy.ts - 4 issues
  • packages/opencode/test/altimate/review-action.test.ts - 1 issue
  • packages/opencode/test/altimate/review-incremental-semantics.test.ts - 0 issues
  • packages/opencode/test/altimate/review-test-adequacy.test.ts - 1 issue
  • packages/opencode/test/skill/release-v0.8.5-adversarial.test.ts - 0 issues

Fix these issues in Kilo Cloud

Previous Review Summary (commit 7da6850)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 7da6850)

Status: 12 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 5
SUGGESTION 7
Issue Details (click to expand)

WARNING

File Line Issue
packages/opencode/src/altimate/review/orchestrate.ts 681 rawSelectList keeps {% if %} block bodies, so conditional columns are read as unconditional; this raw path is now also used when only the head is compiled (line 664), producing false columnDelta findings where it previously returned none.
packages/opencode/src/altimate/review/dbt-patterns.ts 250 The "added line falls inside the window" test is an unanchored substring match, so an unrelated short/clause-shaped added line can flag a pre-existing, untouched dedup window.
github/review/action.yml 224 git worktree add is unguarded under set -e; a stale worktree or fetch failure aborts the action and skips the review, contradicting the documented "failed base compile is a warning".
packages/opencode/src/altimate/review/test-adequacy.ts 204 error_if loosening has no direction check: tightening (>100 → >10) is reported as loosened; only the exact "!= 0" spelling is treated as the strict default.
packages/opencode/src/altimate/review/test-adequacy.ts 28 fewer_rows_than against its own ref always fails (not "passes by construction"), and expect_table_aggregation_to_equal_other_table with different expressions is a valid cross-column invariant that gets flagged.

SUGGESTION

File Line Issue
packages/opencode/src/altimate/review/dbt-patterns.ts 243 Paren-depth scan uses comment-stripped but not literal-stripped SQL; a (/) in a string literal corrupts the window boundary.
packages/opencode/src/altimate/review/test-adequacy.ts 164 own() lets any one config field suppress all inherited-warn/threshold findings, though the settings are independent.
github/review/action.yml 210 dbt docs generate is unguarded in full mode; a warehouse/credentials failure aborts the review instead of degrading.
packages/opencode/src/altimate/review/test-adequacy.ts 128 segmentsOf doesn't normalize multi-segment/relative model-paths, so inherited thresholds are mis-scoped.
github/review/action.yml 203 "${DBT[@]}" on an empty array errors under set -u on bash < 4.4; validate and use the file's ${DBT[@]+"${DBT[@]}"} idiom.
packages/opencode/src/altimate/review/orchestrate.ts 1693 dbt_project.yml is re-fetched (redundant with line 1671) without a .catch; a rejecting resolver aborts runReview.
packages/opencode/test/altimate/review-test-adequacy.test.ts 67 No-op .replace and a negative assertion that would silently pass if the YAML failed to parse.
Files Reviewed (10 files)
  • docs/docs/usage/dbt-pr-review.md - 0 issues
  • github/review/action.yml - 3 issues
  • github/review/examples/dbt-pr-review.yml - 0 issues
  • packages/opencode/src/altimate/review/dbt-patterns.ts - 2 issues
  • packages/opencode/src/altimate/review/orchestrate.ts - 2 issues
  • packages/opencode/src/altimate/review/test-adequacy.ts - 4 issues
  • packages/opencode/test/altimate/review-action.test.ts - 0 issues
  • packages/opencode/test/altimate/review-incremental-semantics.test.ts - 0 issues
  • packages/opencode/test/altimate/review-test-adequacy.test.ts - 1 issue
  • packages/opencode/test/skill/release-v0.8.5-adversarial.test.ts - 0 issues

Fix these issues in Kilo Cloud


Reviewed by deepseek-v4.1-flash-1017 · Input: 140.6K · Output: 46.1K · Cached: 2.3M

Review guidance: REVIEW.md from base branch feat/review-incremental-semantics

@hkaltai hkaltai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: changes requested (one MAJOR). Panel: Claude, Codex, and a lead review. Counts: 0 CRITICAL, 1 MAJOR, 5 MINOR, 4 NIT.

The opt-in dbt_prepare, test-adequacy rules, parse-only column sets and multi-line dedup windows are well structured and tested (targeted suites pass, inputs reach the script only through env, worktree cleanup is trapped). The MAJOR is that the new dedup-window attribution uses a substring match and can flag windows the PR never touched (a regression versus the previous added-line-only behaviour). The MINORs are precision edges in the new threshold and select-list rules and in the action's head handling.

Scope: no linked ticket, so scope vs ticket cannot be verified; the diff matches the PR description. The example workflow rename is a small related cleanup.

Nits

  • packages/opencode/src/altimate/review/orchestrate.ts (comment before rawSelectList use): says "CI that only runs dbt parse" but the fallback also runs when the head is compiled and the base is not. Update the comment.
  • orchestrate.ts:684: the comment "A Jinja expression inside an item ... leaves its alias intact" sits after the guard it explains; move it above if (!names || names.includes("__jinja__")).
  • test-adequacy.ts:162 const own = ... / loose: loose means "tests without their own threshold"; inheriting is clearer. thresholds() also returns severity, so testConfigLevels would fit; rule id test-thresholds-loosened is also emitted for tightenings.
  • github/review/action.yml:196 read -ra DBT <<< "$IN_DBT": ignores quoting and an empty dbt_command yields an empty array (script would run compile). Validate non-empty. Also test-adequacy.ts own() treats a lone warn_if as overriding an inherited severity: warn.

}
if (close < 0) continue
const text = sql.slice(m.index!, close + 1)
if (!added.some((l) => text.includes(l) || l.includes(text.replace(/\s+/g, " ")))) continue

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] Dedup window is attributed to the PR by substring match, so untouched windows are flagged

if (!added.some((l) => text.includes(l) || l.includes(text.replace(/\s+/g, " ")))) continue

text.includes(l) is a substring test, not a position test. Scenario (reproduced by two reviewers): the model already has an unchanged row_number() over (partition by customer_id) as rn; the PR only adds a customer_id to a group by elsewhere. The added line customer_id is a substring of the window, so dedup_no_tiebreaker fires (high confidence) on code the PR did not touch. Before this PR only added lines that themselves contained row_number were considered. The existing "untouched window" test uses select 1 as unrelated, which can never match.

Fix: map each window's [m.index, close] to new-file line numbers and intersect with the added hunk line numbers (or at minimum require the added line to contain row_number/partition by/order by). Add a test with an unrelated added bare identifier that also appears in the window. Also normalise whitespace on both sides of the second comparison.

// No compiled SQL (CI that only runs `dbt parse`): the raw model's own SELECT
// list, when it names its columns. Columns produced inside a macro stay unknown.
if (!newSql) return none
const current = rawSelectList(newSql)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Parse-only select list treated as complete when items are dropped

const current = rawSelectList(newSql)

selectListNames is deliberately conservative (items it cannot name contribute nothing). Previously that only supplemented lineage; here it is the whole answer. select id, amount::numeric(18,2) yields ["id"], so going from amount to a cast is seen as a removed column, and with exposures/contracts this produces a high-confidence "removed columns" warning. The branch also now runs when the head is compiled but the base is not.

Fix: return unknown (empty set) when any non-empty select item produced no name. Related: stripping {% if %} / {% for %} tags keeps their bodies, so {% if var('x', false) %}, region {% endif %} is counted as an unconditional added column and can trigger the on_schema_change='fail' warning; treat projection-affecting control blocks as unknown.

for (const [path, t] of flat(newDoc)) {
const prev = before.get(path) ?? {}
const toWarn = t.severity === "warn" && prev.severity !== "warn"
const newThreshold = t.error_if && t.error_if !== "!= 0" && t.error_if !== prev.error_if

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Any changed error_if other than != 0 is reported as "loosened"

const newThreshold = t.error_if && t.error_if !== "!= 0" && t.error_if !== prev.error_if

Changing '>1000' to '>10' (a tightening) or adding the default-equivalent '>0' / '!=0' is reported as "test thresholds loosened". The same literal comparison is used for the inherited-threshold rule. Also, the comparison is per node rather than effective value: with root +severity: warn, deleting marts: +severity: error loosens every marts test but reports nothing, and a redundant +severity: warn under an already-warn parent is reported.

Fix: normalise op N and compare numerically, and compare inheritedThresholds(old, path) with inheritedThresholds(new, path) across the union of paths.

const project = parse(input.projectYml)
if (project && added.length) {
const inherited = inheritedThresholds(project, segmentsOf(file.path, input.layout))
const own = (t: TestRef) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Any local threshold suppresses all inherited checks

const own = (t: TestRef) => {

With project-level severity: warn, adding only config: {warn_if: '>10'} or error_if: '!= 0' returns no finding, yet severity is still inherited (dbt resolves these as separate properties).

Fix: merge local and inherited properties individually and inspect the effective config.

const SELF_COMPARE = new Set([
"equality",
"equal_rowcount",
"cardinality_equality",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] cardinality_equality / aggregation tests against the same model are not always self-comparisons

"cardinality_equality",

cardinality_equality with to: ref('m'), field: b on column a compares two different columns' distinct counts, a real check, but is reported as comparing the model with itself. Same for expect_table_aggregation_to_equal_other_table with different expressions. (fewer_rows_than against itself fails rather than passes, so "passes by construction" is also inaccurate there.)

Fix: for these tests require the field/expression to match as the relationships branch does, and drop fewer_rows_than. Separately, segmentsOf strips only model dirs, so seeds/_s.yml walks data_tests: shop: seeds: as if seeds were a folder; strip seed/snapshot dirs too.

Comment thread github/review/action.yml
"${DBT[@]}" compile --target-path "$2" ${PROFILES[@]+"${PROFILES[@]}"} )
}
echo "Compiling the head…"
build "$PROJECT" "$PROJECT/target"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] dbt_prepare compiles the checkout, ignoring the head input; head-side failures abort the job

build "$PROJECT" "$PROJECT/target"
  1. The merge-base is computed against literal HEAD and the head compile uses the checkout, but the review step passes --head "$IN_HEAD" when set. With head: A while B is checked out, A's diff is reviewed using B's compiled SQL and equivalence can be wrongly reported. Prepare against the selected head or reject a mismatch.
  2. Under set -e, a failed head deps/compile/docs generate fails the step so the review never runs (even in mode: comment), while the input description and docs (docs/docs/usage/dbt-pr-review.md:224) say compile "usually works offline". Many adapters need a connection to compile. Either degrade to a warning like the base side, or document that it fails the job and drop the offline claim.
  3. Minor: manifest_path is not honoured (head always goes to $PROJECT/target), and a runtime-generated profiles.yml in the project dir is absent from the base worktree, so the base compile quietly fails.

RaghwendraSingh and others added 4 commits November 20, 2026 12:13
…erge-base and catalog artifacts

- new action inputs: `dbt_prepare` (`none` default | `compile` | `full`), `dbt_command`, `dbt_profiles_dir`
- `compile` runs `dbt compile` on the head and compiles the PR's merge-base into `target-base/` in a temporary worktree (fetching and deepening a shallow checkout when needed); `full` adds `dbt docs generate`
- a base compile that fails is a warning: the review still runs and marks those models unverified
- inputs reach the script only through env, never interpolated
- docs: the manual base-compile snippet now compiles the merge-base, not the base branch's tip
- the example workflow is renamed `examples/dbt-pr-review.yml` (was named after one internal repo)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ows split across lines are read whole

Found by replaying the PRs behind known escaped defects:

- without compiled SQL, output columns come from the raw model's final SELECT (Jinja blanked; a Jinja expression in the list makes it unknown). `on_schema_change='fail'` with new columns now fires on the two real escapes it targets, in a parse-only CI setup
- `row_number() over (…)` windows are read from the whole file, so an `ORDER BY` on its own line is seen: the "dedup has no ORDER BY" rule no longer fires falsely on multi-line windows, and the load-timestamp rule sees them. A window counts only when an added line falls inside it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nherited warn/thresholds, loosened project thresholds

- a test added or changed in the PR that compares a model with itself (`dbt_utils.equality`/`equal_rowcount`/… against its own `ref`, or a `relationships` test to its own column) passes by construction (warning)
- new tests that inherit `severity: warn` or an `error_if` threshold from `dbt_project.yml` (walked from the top through the project name and folders, deeper wins) are reported once per yml (suggestion)
- a `dbt_project.yml` change that turns a folder's tests to `warn` or sets an `error_if` threshold is reported (warning)
- reads dbt 1.8+ `arguments:` too; all findings are catalog origin (never block)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…window touch, self-compare tests, `error_if` loosening, prepare-step failures

- `orchestrate.ts`: a `{% if %}` / `{% for %}` tag inside a raw model's select
  list makes the list unknown, so conditional columns no longer read as added
  or removed in the parse-only column delta
- `dbt-patterns.ts`: a dedup window counts as changed only when an added line
  falls inside its line span (head line numbers from the hunk headers), not
  when an added line's text happens to occur in it
- `test-adequacy.ts`: drop `fewer_rows_than` (it fails against itself, it
  doesn't pass by construction); an aggregation comparison on the same model
  is flagged only when both expressions match
- `test-adequacy.ts`: `error_if` is reported as loosened only when the new bound
  tolerates more failing rows; `> 0` / `!=0` / `>= 1` count as the strict default
- `action.yml`: a failed `dbt docs generate` or base checkout warns instead of
  failing the step; a stale `target-base/` is removed before the base compile
- tests for each

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@RaghwendraSingh
RaghwendraSingh force-pushed the feat/review-incremental-semantics branch from fe31dc6 to 9625cfd Compare November 20, 2026 06:53
Comment thread github/review/action.yml
trap 'git worktree remove --force "$WT" >/dev/null 2>&1 || true' EXIT
git worktree add --detach "$WT" "$MB" >/dev/null || { echo "::warning::Could not check out the base; skipping the base compile."; exit 0; }
# A stale target-base/ from an earlier run must not stand in for a failed compile.
rm -rf "$PROJECT/target-base"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: The stale target-base/ cleanup runs only after the guards that can skip the base compile

When BASE is empty, the merge-base can't be found, or git worktree add fails, the script exits 0 before this line, so a target-base/ left by an earlier run survives. makeCompiledResolver reads target-base/compiled whenever it exists (compiled.ts:97), so on a cached or self-hosted workspace the head is compared against the previous run's base, and because the old side still resolves, those models are not even listed as unverified. Move the cleanup above the BASE/MB/worktree guards, and make it non-fatal (rm -rf ... || { echo "::warning::..."; exit 0; }) so an rm failure cannot abort the step and skip the review.

}

/** Head-side line numbers of the added lines; undefined when the diff has no hunk headers. */
function addedLineNumbers(diff: string | undefined): Set<number> | undefined {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: This hunk walker duplicates the exported commentableLines

commentableLines (orchestrate.ts:268) already walks @@ hunks with the same regex and the same right counter to compute head-side line numbers; this copy already differs (it does not drop the diff's trailing-newline row) and the two will drift. dbt-patterns cannot import orchestrate without a cycle (the file already keeps a local modelNameFromPath for exactly that reason), so extract the walker into a shared module (e.g. diff-filter.ts) and take the added-line set from it, or pass the numbers in through Ctx.

*/
function dedupWindows(c: Ctx): Array<{ spec: string; text: string }> {
// Comments blanked with newlines kept, so offsets map to head line numbers.
const sql = c.newSql.replace(/\/\*[\s\S]*?\*\/|--[^\n]*|\{#[\s\S]*?#\}/g, (m) => m.replace(/[^\n]/g, " "))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: This comment blanking is not quote-aware; blankComments() already does this

A -- inside a string literal blanks the rest of the line: select '--' as sep, row_number() over (partition by id) as rn loses the window entirely (a missed dedup finding), and a /* in a literal can blank a multi-line span. blankComments() in select-list.ts is the existing length-preserving, quote-aware helper (it skips '...', "..." and backticked identifiers), so reuse it here and keep only the {# ... #} alternative.

const run = prep.run!
expect(run).toMatch(/docs generate[^\n]*\) \|\|\n\s*echo "::warning::/)
expect(run).toMatch(/git worktree add[^\n]*\|\| \{ echo "::warning::/)
expect(run.indexOf('rm -rf "$PROJECT/target-base"')).toBeLessThan(run.indexOf('build "$WT/$REL"'))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: This ordering assertion passes when the cleanup line is absent

run.indexOf('rm -rf ...') is -1 when the line is deleted, and -1 < indexOf('build ...') is true, so the test still passes without the rm -rf "$PROJECT/target-base" line it is meant to guard. Assert the cleanup line is present (e.g. expect(run).toContain('rm -rf "$PROJECT/target-base"')) before comparing indexes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants