Skip to content

feat(review): close remaining review gaps: action dbt_prepare, data-test adequacy, parse-only column changes - #1726

Open
RaghwendraSingh wants to merge 5 commits into
feat/review-incremental-semanticsfrom
feat/review-ac-gaps
Open

RaghwendraSingh wants to merge 5 commits into
feat/review-incremental-semanticsfrom
feat/review-ac-gaps

Conversation

@RaghwendraSingh

@RaghwendraSingh RaghwendraSingh commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Issue for this PR

Part of #1792
Stacked on #1725 (← #1724 ← #1723 ← … ← #1719); merge in that order. Draft.

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

This closes the remaining acceptance-criteria gaps in the review stack. Two of the fixes came from replaying the PRs behind known escaped defects.

  • Optional artifact preparation in the action (dbt_prepare). Default none, so nothing changes for existing users.
    • compile runs dbt compile on the head, then compiles the PR's merge-base into target-base/ in a temporary worktree. A shallow checkout is fetched and deepened when needed.
    • full also runs dbt docs generate.
    • A failed base compile is a warning, and those models are marked unverified.
    • Inputs reach the script only through env.
    • The manual docs snippet now compiles the merge-base instead of the base branch's tip.
    • The example workflow gets a generic name.
  • Data tests that can't do their job (yml only, any adapter):
    • a test comparing a model with itself (equality / equal_rowcount against its own ref, or a relationships test to its own column) passes by construction;
    • new tests that inherit severity: warn or an error_if threshold from dbt_project.yml are reported;
    • a dbt_project.yml change that loosens those thresholds for a folder is reported.
  • Column-set rules work in dbt parse-only CI. Without compiled SQL, output columns come from the raw model's final SELECT; a Jinja expression in the list makes it unknown. In a parse-only replay, on_schema_change='fail' with new columns now fires on the two real escapes it targets.
  • Dedup windows split across lines are read whole. "Dedup has no ORDER BY" no longer fires falsely when order by is on its own line, and the load-timestamp rule sees such windows.

How did you verify your code works?

  • New tests:
    • review-action.test.ts: dbt_prepare is opt-in, reads inputs only from env, and builds into target-base/.
    • review-test-adequacy.test.ts: self-comparison, inherited warn/error_if, loosened thresholds, dbt 1.8+ arguments:.
    • Additions to review-incremental-semantics.test.ts: multi-line windows, raw-SQL column delta.
  • Suites: 506 review tests and 1,367 skill and release-gate tests pass, bun typecheck is clean, and the real-world corpus is unchanged (15/15 caught, 0/5 false positives).
  • dbt_prepare run locally against a real dbt 1.10 DuckDB project. The head compiled into target/, the merge-base compiled into target-base/ (the base still has the removed column), and the temporary worktree was cleaned up.
  • Replay of the PRs behind six known escapes in a private dbt monorepo (parse-only, as their CI runs):
    • on_schema_change now catches two of them (two escapes each).
    • One escape lived in Python loader code, outside dbt.
    • One targeted the removed microbatch rule.
    • One needs the opt-in slice var.
    • One is pre-existing code the PR didn't touch, so no rule reports it by design.
  • The earlier 7-PR replay is unchanged: 0 findings on files the PR didn't change.

Cost side effects

dbt_prepare: compile adds one dbt compile per side of the PR in CI, and full adds dbt docs generate (warehouse queries for the catalog). Both are opt-in; the default none adds nothing. No model calls.

Screenshots / recordings

N/A (CLI / action).

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR
  • I checked for cost side effects (model calls, queries or cloud resources that grow with user data) and described them, or wrote "none"

Changes since review: d9caa1597c addresses the second-round review feedback: dedup-window touch needs window syntax on an added line, unnamed raw select items make the column set unknown, severity/error_if resolved separately, threshold loosening compared on effective values, cardinality_equality field, prepare-step head handling.

🤖 Generated with Claude Code


Summary by cubic

Closes the remaining acceptance-criteria gaps in the dbt review stack: adds an opt-in dbt_prepare action input for building the artifacts the full review needs, and fixes two classes of missed findings — data tests that can't do their job, and column-set rules that skipped work in parse-only CI.

New dbt_prepare action input

  • Default none; compile runs dbt compile on the head and the PR's merge-base into target-base/, and full also runs dbt docs generate.
  • Inputs reach the script only through env; a failed compile, docs generate, or checkout is a warning that keeps the review running, and a head that isn't the checkout skips preparation.
  • A stale target-base/ is removed before the base compile, and the runtime profiles.yml is copied into the base worktree.

Review gap fixes

  • Reports data tests that pass by construction: self-comparisons via equality/equal_rowcount/relationships, cardinality_equality on its own column, or expect_table_aggregation_to_equal_other_table with matching expressions, plus tests that inherit severity: warn or an error_if threshold from dbt_project.yml.
  • Reports dbt_project.yml changes that loosen those thresholds, counting only bounds that tolerate more failing rows.
  • Without compiled SQL, output columns come from the raw model's final SELECT, so on_schema_change='fail' fires in parse-only CI; a {% if %} tag, Jinja expression, or unnamed item (e.g. a cast) in the list leaves it unknown.
  • Dedup windows are read from the whole file and only count when an added line falls inside them, so a split ORDER BY no longer triggers a false "no ORDER BY".

Written for commit d9caa15. Summary will update on new commits.

View guided diff Turn on auto-fix

@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a54c7098-9863-4275-87fb-dc9c7b914b55

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions

Copy link
Copy Markdown
- - - - - - - - - - - - - - - - - - - - - - - - -
                    AIRECEIPTS                    
             1 session behind this PR             

claude-opus-5-5 100% · <s…...221,036,213 tokens
  SUBAGENTS (19).................63,001,251 tokens
--------------------------------------------------
TOTAL unpriced................≥ 284,037,464 tokens
  counted: 1 session + 19 subagents
  cache served 97% of input tokens

1 candidate session not attributed
(in repo + branch window, no branch commit)
  full receipts + session ids: section below
- - - - - - - - - - - - - - - - - - - - - - - - -
                npx aireceipts-cli                
         github.com/anandgupta42/receipts         
- - - - - - - - - - - - - - - - - - - - - - - - -
full receipts (1 session)
session id scope turns time tokens in / out cached
orchestrator 6fe8be4d entire session (slice unavailable) 700 189h 52m 1.6k / 696k 98%

orchestrator · 6fe8be4d

- - - - - - - - - - - - - - - - - - - - - - - - -
                    AIRECEIPTS                    
         “data-eng-code-review-landscape”         
  Claude Code · Nov 05 2026 10:10 UTC · 189h 52m  
      claude-opus-5-5 100% · <synthetic> 0%       
         cache served 98% of input tokens         

pre-edit: 1% of tokens (12/698 turns)
  (share before the first named edit tool)

Bash..................164,495,680 tok  (495 calls)
(thinking/reply)........19,821,384 tok  (67 turns)
Write....................8,521,992 tok  (31 calls)
ToolSearch...............5,032,526 tok  (25 calls)
Read.....................4,036,974 tok  (29 calls)
mcp__claude_ai_Atlass…...3,292,204 tok  (28 calls)
Edit.....................3,095,292 tok  (28 calls)
mcp__claude_ai_Atlass…...2,143,864 tok  (12 calls)
mcp__claude_ai_Atlass…...1,725,016 tok  (17 calls)
Agent....................1,704,960 tok  (22 calls)
mcp__claude_ai_Atlassian…...947,323 tok  (7 calls)
ExitPlanMode................758,820 tok  (4 calls)
mcp__notion__notion-upda…...607,506 tok  (4 calls)
mcp__claude_ai_Atlassian…...489,926 tok  (5 calls)
mcp__claude_ai_Atlassia…...489,318 tok  (13 calls)
mcp__claude_ai_Atlassian_…...485,105 tok  (1 call)
mcp__claude_ai_Atlassia…...479,160 tok  (17 calls)
mcp__atlassian__createJi…...419,313 tok  (2 calls)
WebSearch...................384,482 tok  (4 calls)
AskUserQuestion.............366,689 tok  (2 calls)
SendMessage.................358,309 tok  (7 calls)
mcp__notion__notion-fetch...327,408 tok  (2 calls)
mcp__notion__notion-creat…...248,897 tok  (1 call)
mcp__atlassian__searchJir…...222,754 tok  (1 call)
WebFetch.....................217,960 tok  (1 call)
mcp__notion__notion-search...106,809 tok  (1 call)
mcp__atlassian__atlassianU…...94,821 tok  (1 call)
mcp__atlassian__getVisible…...94,821 tok  (1 call)
TaskStop......................66,901 tok  (1 call)
--------------------------------------------------
TOTAL..............................221,036,213 tok
no price table matched
- - - - - - - - - - - - - - - - - - - - - - - - -
                npx aireceipts-cli                
         github.com/anandgupta42/receipts         
- - - - - - - - - - - - - - - - - - - - - - - - -
subagents (19)
subagent cost
Goal: an evidence-backed, cited landscape of (a) dbt PR review tools, (b) data-… 2,134,743 tokens
Goal: determine what gaps altimate-code's dbt PR review has, by comparing what… 5,303,299 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 2,794,842 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 3,244,430 tokens
Label escaped-defect candidates by READING them (no keyword bucketing). Directo… 2,570,116 tokens
Repo: /Users/raghwendrasingh/code/altimateai/altimate-code (current branch feat… 4,836,719 tokens
You are doing an ADVERSARIAL review of a set of Jira tickets that plan improvem… 763,761 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 3,351,198 tokens
Label escaped-defect candidates by READING them (no keyword bucketing). Directo… 4,214,327 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 2,821,134 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 1,795,004 tokens
You are doing an ADVERSARIAL review of a stack of changes to an open-source dbt… 4,084,798 tokens
Repo: /Users/raghwendrasingh/code/altimateai/altimate-code. Module: packages/op… 8,815,483 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 2,592,769 tokens
Research task, read-only (do not push, comment, or open PRs anywhere).Goal: dec… 1,198,872 tokens
Judge each of altimate-code's own inline PR review comments by reading it. Inpu… 1,673,212 tokens
Label escaped-defect candidates by READING them (no keyword bucketing). Directo… 2,862,492 tokens
Goal: read a Notion doc tree about an "agentic data engineering software factor… 1,335,636 tokens
Goal: test whether the bug classes that real reviewers catch on dbt PRs at one… 6,608,416 tokens

Generated by aireceipts

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

9 issues found across 10 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/opencode/test/altimate/review-test-adequacy.test.ts">

<violation number="1" location="packages/opencode/test/altimate/review-test-adequacy.test.ts:56">
P3: The test name claims 'deeper wins', but no case sets the same key at two levels. Add a fixture where a folder overrides the project-level `+severity` (for example, `shop: +severity: error` with `marts: +severity: warn`) and assert the deeper value and its path.</violation>
</file>

<file name="packages/opencode/src/altimate/review/test-adequacy.ts">

<violation number="1" location="packages/opencode/src/altimate/review/test-adequacy.ts:28">
P2: `fewer_rows_than` is in `SELF_COMPARE`, but a self-comparison makes it fail every time, not pass. The message "passes by construction" is wrong for this test. Drop it from the set and report it separately, or change the message to cover both outcomes.</violation>

<violation number="2" location="packages/opencode/src/altimate/review/test-adequacy.ts:69">
P2: `refsEntity` matches `ref(` inside names such as `my_ref(`, creating false self-comparison findings. Require a word boundary before `ref` and allow normal whitespace before `(`.</violation>

<violation number="3" location="packages/opencode/src/altimate/review/test-adequacy.ts:117">
P3: `path` records only the last node containing either setting, so findings misidentify the source when `severity` and `error_if` are inherited from different levels. Track a path per threshold.</violation>

<violation number="4" location="packages/opencode/src/altimate/review/test-adequacy.ts:128">
P2: `segmentsOf` fails to strip nested configured model paths, so inherited test thresholds are missed under paths such as `src/dbt/models/marts`. Strip the full configured model directory, not just its first segment.</violation>
</file>

<file name="docs/docs/usage/dbt-pr-review.md">

<violation number="1" location="docs/docs/usage/dbt-pr-review.md:215">
P3: This snippet pins the action to `@main`, but the rest of the page pins `@v0.8.5`. Pin to the same release tag so users get a reviewed, reproducible action version.</violation>
</file>

<file name="packages/opencode/src/altimate/review/orchestrate.ts">

<violation number="1" location="packages/opencode/src/altimate/review/orchestrate.ts:664">
P2: When only the base compile is missing, `columnDelta` now falls through to the raw-SQL path and can emit removed-column warnings. Before this change that case returned no deltas. Confirm this is intended for models marked unverified, or keep returning no deltas when the head is compiled and the base is not.</violation>
</file>

<file name="packages/opencode/src/altimate/review/dbt-patterns.ts">

<violation number="1" location="packages/opencode/src/altimate/review/dbt-patterns.ts:235">
P2: The `length >= 3` filter drops valid added SQL lines such as `id` or `x`. A multiline window changed only by adding such a two-character expression is then ignored, so the no-`ORDER BY` detector misses the change; filter only empty normalized lines.</violation>

<violation number="2" location="packages/opencode/src/altimate/review/dbt-patterns.ts:243">
P2: The parenthesis scan counts parentheses inside SQL string literals. A value like `status = ')'` closes the window early, truncating `spec` before a real `ORDER BY`; mask quoted literals or use a quote-aware scanner while finding the matching window close.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

"equality",
"equal_rowcount",
"cardinality_equality",
"fewer_rows_than",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: fewer_rows_than is in SELF_COMPARE, but a self-comparison makes it fail every time, not pass. The message "passes by construction" is wrong for this test. Drop it from the set and report it separately, or change the message to cover both outcomes.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/test-adequacy.ts, line 28:

<comment>`fewer_rows_than` is in `SELF_COMPARE`, but a self-comparison makes it fail every time, not pass. The message "passes by construction" is wrong for this test. Drop it from the set and report it separately, or change the message to cover both outcomes.</comment>

<file context>
@@ -0,0 +1,214 @@
+  "equality",
+  "equal_rowcount",
+  "cardinality_equality",
+  "fewer_rows_than",
+  "expect_table_row_count_to_equal_other_table",
+  "expect_table_aggregation_to_equal_other_table",
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. fewer_rows_than is dropped (against itself it always fails), and an aggregation comparison on the same model is flagged only when both expressions match.

current,
}
// Compiled SQL on both sides: column lineage plus the final SELECT's names.
if (engineNewSql && (!withOld || engineOldSql)) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When only the base compile is missing, columnDelta now falls through to the raw-SQL path and can emit removed-column warnings. Before this change that case returned no deltas. Confirm this is intended for models marked unverified, or keep returning no deltas when the head is compiled and the base is not.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 664:

<comment>When only the base compile is missing, `columnDelta` now falls through to the raw-SQL path and can emit removed-column warnings. Before this change that case returned no deltas. Confirm this is intended for models marked unverified, or keep returning no deltas when the head is compiled and the base is not.</comment>

<file context>
@@ -648,19 +649,41 @@ async function columnDelta(
+          current,
+        }
+  // Compiled SQL on both sides: column lineage plus the final SELECT's names.
+  if (engineNewSql && (!withOld || engineOldSql)) {
+    const current = await outputColumns(runner, engineNewSql, dialect)
+    const before = withOld ? await outputColumns(runner, engineOldSql!, dialect) : undefined
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. A {% if %} or {% for %} tag in the raw select list now makes the list unknown, so conditional columns are no longer reported as added or removed. We kept the raw fallback when only one side is compiled: both sides are then read from raw SQL, so the comparison stays like for like.

Comment thread github/review/action.yml Resolved
*/
function dedupWindows(c: Ctx): Array<{ spec: string; text: string }> {
const sql = stripComments(c.newSql)
const added = c.added.map((l) => stripComments(l).trim()).filter((l) => l.length >= 3)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The length >= 3 filter drops valid added SQL lines such as id or x. A multiline window changed only by adding such a two-character expression is then ignored, so the no-ORDER BY detector misses the change; filter only empty normalized lines.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/dbt-patterns.ts, line 235:

<comment>The `length >= 3` filter drops valid added SQL lines such as `id` or `x`. A multiline window changed only by adding such a two-character expression is then ignored, so the no-`ORDER BY` detector misses the change; filter only empty normalized lines.</comment>

<file context>
@@ -225,15 +225,42 @@ const detectNotIn: Detector = (c) => {
+ */
+function dedupWindows(c: Ctx): Array<{ spec: string; text: string }> {
+  const sql = stripComments(c.newSql)
+  const added = c.added.map((l) => stripComments(l).trim()).filter((l) => l.length >= 3)
+  const out: Array<{ spec: string; text: string }> = []
+  for (const m of sql.matchAll(/row_number\s*\(\s*\)\s*over\s*\(/gi)) {
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. When the diff has hunk headers (the PR API always sends them), matching now uses line numbers and no longer depends on the length >= 3 text filter.

const arg = (t: TestRef, name: string) => t.args?.[name] ?? t.args?.arguments?.[name]
const shortName = (test: string) => test.split(".").pop()!.toLowerCase()
const refsEntity = (v: unknown, entity: string) =>
typeof v === "string" && new RegExp(`ref\\(\\s*['"]${entity.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}['"]\\s*\\)`).test(v)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: refsEntity matches ref( inside names such as my_ref(, creating false self-comparison findings. Require a word boundary before ref and allow normal whitespace before (.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/test-adequacy.ts, line 69:

<comment>`refsEntity` matches `ref(` inside names such as `my_ref(`, creating false self-comparison findings. Require a word boundary before `ref` and allow normal whitespace before `(`.</comment>

<file context>
@@ -0,0 +1,214 @@
+const arg = (t: TestRef, name: string) => t.args?.[name] ?? t.args?.arguments?.[name]
+const shortName = (test: string) => test.split(".").pop()!.toLowerCase()
+const refsEntity = (v: unknown, entity: string) =>
+  typeof v === "string" && new RegExp(`ref\\(\\s*['"]${entity.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}['"]\\s*\\)`).test(v)
+
+function finding(file: ChangedFile, model: string | undefined, rule: string, severity: Finding["severity"], title: string, body: string, extra = ""): Finding {
</file context>
Suggested change
typeof v === "string" && new RegExp(`ref\\(\\s*['"]${entity.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}['"]\\s*\\)`).test(v)
typeof v === "string" && new RegExp(`\\bref\\s*\\(\\s*['"]${entity.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}['"]\\s*\\)`).test(v)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked (\bref).

data_tests:
- not_null${own}
`
test("the effective value walks project → folders, deeper wins", () => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The test name claims 'deeper wins', but no case sets the same key at two levels. Add a fixture where a folder overrides the project-level +severity (for example, shop: +severity: error with marts: +severity: warn) and assert the deeper value and its path.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/review-test-adequacy.test.ts, line 56:

<comment>The test name claims 'deeper wins', but no case sets the same key at two levels. Add a fixture where a folder overrides the project-level `+severity` (for example, `shop: +severity: error` with `marts: +severity: warn`) and assert the deeper value and its path.</comment>

<file context>
@@ -0,0 +1,83 @@
+        data_tests:
+          - not_null${own}
+`
+  test("the effective value walks project → folders, deeper wins", () => {
+    const doc = YAML.parse(project)
+    expect(inheritedThresholds(doc, ["marts"])).toEqual({ severity: "warn", path: ["shop", "marts"] })
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: nit, tracked. We'll add a fixture where the deeper level wins.

The action can build these for you with `dbt_prepare` (off by default):

```yaml
- uses: AltimateAI/altimate-code/github/review@main

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This snippet pins the action to @main, but the rest of the page pins @v0.8.5. Pin to the same release tag so users get a reviewed, reproducible action version.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/docs/usage/dbt-pr-review.md, line 215:

<comment>This snippet pins the action to `@main`, but the rest of the page pins `@v0.8.5`. Pin to the same release tag so users get a reviewed, reproducible action version.</comment>

<file context>
@@ -209,12 +209,30 @@ jobs:
+The action can build these for you with `dbt_prepare` (off by default):
+
+```yaml
+      - uses: AltimateAI/altimate-code/github/review@main
+        with:
+          dbt_prepare: compile        # none | compile | full
</file context>
Suggested change
- uses: AltimateAI/altimate-code/github/review@main
- uses: AltimateAI/altimate-code/github/review@v0.8.5

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing yet: @main is intentional until a release that includes dbt_prepare ships, and we'll re-pin it then.

const t = thresholds(n)
if (t.severity) eff.severity = t.severity
if (t.error_if) eff.error_if = t.error_if
if (t.severity || t.error_if) path = at.length ? at : ["(all tests)"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: path records only the last node containing either setting, so findings misidentify the source when severity and error_if are inherited from different levels. Track a path per threshold.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/test-adequacy.ts, line 117:

<comment>`path` records only the last node containing either setting, so findings misidentify the source when `severity` and `error_if` are inherited from different levels. Track a path per threshold.</comment>

<file context>
@@ -0,0 +1,214 @@
+    const t = thresholds(n)
+    if (t.severity) eff.severity = t.severity
+    if (t.error_if) eff.error_if = t.error_if
+    if (t.severity || t.error_if) path = at.length ? at : ["(all tests)"]
+  }
+  return { ...eff, path }
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: nit, tracked. Each threshold will track its own path.

function rawSelectList(sql: string): Set<string> {
const plain = sql
.replace(/\{#[\s\S]*?#\}/g, " ")
.replace(/\{%[\s\S]*?%\}/g, " ")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: rawSelectList blanks Jinja tags but keeps block bodies, so conditional columns become unconditional

{% if %}/{% for %} bodies stay in the text, so select id, amount {% if target.name == 'prod' %}, _loaded_at{% endif %} from t parses as {id, amount, _loaded_at} even when the compiled head renders the false branch. Line 664 now falls back to this raw path whenever either compiled side is missing (e.g. the common "compile the head only, no target-base" setup), so this yields a false columnDelta — a spurious new column that drives on_schema_change, contract, and consumer-list findings, where the old code returned none. Consider treating a model whose select list contains any {% if/for/set %} block as unknown (empty set), and/or using raw only when neither side has compiled SQL.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. Same fix as r4626962305.

}
if (close < 0) continue
const text = sql.slice(m.index!, close + 1)
if (!added.some((l) => text.includes(l) || l.includes(text.replace(/\s+/g, " ")))) continue

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: "Added line falls inside the window" is a raw substring match, so untouched windows are flagged

text.includes(l) has no line anchoring, so a short or clause-shaped added line elsewhere in the file matches any window that merely contains that text. Example: a pre-existing row_number() over (partition by country, id) (no ORDER BY) is untouched, and the PR adds , id to the select list — text.includes(", id") is true and dedup_no_tiebreaker fires on code the PR never touched, with evidence.line pointing at the pre-existing window. Same for a shared partition by customer_id / order by _loaded_at line. The "a window the PR doesn't touch is not reported" test only covers a line that matches nothing. Match by line range (compute the window's line span and check an added line number falls in it) instead of substring containment.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. Same fix as r4626962384.

Comment thread github/review/action.yml Outdated
fi
if [[ -z "$MB" ]]; then echo "::warning::Could not find the merge-base with $BASE; skipping the base compile."; exit 0; fi
WT="$RUNNER_TEMP/altimate-review-base"
git worktree add --detach "$WT" "$MB" >/dev/null

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Unguarded git worktree add aborts the whole review

The base compile at line 227 is best-effort (|| echo "::warning::…"), but git worktree add is a plain command under set -e. If it fails — e.g. a stale $RUNNER_TEMP/altimate-review-base on a reused/self-hosted runner, or a lazy-fetch error for $MB — the step fails and the following "Run dbt PR review" step is skipped, contradicting the documented "a base compile that fails is a warning: the review still runs". Clean $WT first and guard the add (|| { echo "::warning::…"; exit 0; }), and register the cleanup trap before the add.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. The cleanup trap is set first, and a failed git worktree add now warns and skips the base compile.

for (const [path, t] of flat(newDoc)) {
const prev = before.get(path) ?? {}
const toWarn = t.severity === "warn" && prev.severity !== "warn"
const newThreshold = t.error_if && t.error_if !== "!= 0" && t.error_if !== prev.error_if

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: error_if "loosening" has no direction check and matches the default by exact string

newThreshold = t.error_if && t.error_if !== "!= 0" && t.error_if !== prev.error_if fires for any change, including a tightening: >100 → >10 is reported as "test thresholds loosened … so those tests stop failing the build", the opposite of what happened. It also treats only the exact spelling "!= 0" as the strict default — "!=0", ">0", " > 0 " are all flagged as custom thresholds. Parse the bound and compare direction (only report when the new bound allows more failures), and normalize the strict-default forms.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. Same fix as r4626962408.

"equality",
"equal_rowcount",
"cardinality_equality",
"fewer_rows_than",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Some SELF_COMPARE entries are not "passes by construction"

fewer_rows_than compares row counts with <; against its own ref it always fails (count < count is false), so the emitted body ("passes by construction") is wrong. And expect_table_aggregation_to_equal_other_table is designed to compare two different aggregations: the same model with expression: sum(subtotal) vs compare_expression: sum(net) + sum(tax) is a valid cross-column invariant, yet it is flagged. Only treat aggregation tests as self-comparisons when the expressions match, and drop/adjust fewer_rows_than.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. Same fix as r4626962291.

Comment thread github/review/action.yml Outdated
build "$PROJECT" "$PROJECT/target"
if [[ "$IN_PREPARE" == "full" ]]; then
echo "Generating catalog.json…"
( cd "$PROJECT" && "${DBT[@]}" docs generate --target-path "$PROJECT/target" ${PROFILES[@]+"${PROFILES[@]}"} )

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: dbt docs generate is unguarded in full mode

In full mode this plain command under set -e aborts the step (and skips the review) if the warehouse is unreachable or credentials are missing — exactly the failure the input description warns about. dbt compile at line 207 already produced a usable target/, so degrade instead: … docs generate … || echo "::warning::docs generate failed; continuing without catalog.json.".

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e77d121. A failed dbt docs generate now warns and the review continues without catalog.json.

const rel = root && file.startsWith(root) ? file.slice(root.length) : file
const parts = rel.split("/").slice(0, -1)
const modelDirs = layout?.dirs.model ?? ["models"]
return modelDirs.includes(parts[0]) ? parts.slice(1) : parts

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: segmentsOf doesn't normalize multi-segment/relative model-paths

modelDirs.includes(parts[0]) matches only the first segment exactly. A valid model-paths: ["./models"] or ["transform/models"] isn't stripped (parts[0] is models/transform), so inheritedThresholds walks keys that don't exist and misses inherited severity/error_if. roleFromLayout in diff-filter.ts already normalizes these dirs; reuse that (normalize and strip the longest matching configured prefix).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. Same as r4626962396.

Comment thread github/review/action.yml
[[ -n "${IN_PROFILES:-}" ]] && PROFILES=(--profiles-dir "$(cd "$PROJECT" && cd "$IN_PROFILES" && pwd)")
build() { # $1 = project dir, $2 = target path
( cd "$1"
[[ -f packages.yml || -f dependencies.yml ]] && "${DBT[@]}" deps ${PROFILES[@]+"${PROFILES[@]}"}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: "${DBT[@]}" expands an array that can be empty

read -ra DBT <<< "$IN_DBT" produces an empty array when dbt_command is empty/whitespace. Under set -u on bash < 4.4 (e.g. macOS 3.2) "${DBT[@]}" is an unbound-variable error; on newer bash the command word vanishes and deps/compile fails obscurely. Validate it ([[ ${#DBT[@]} -gt 0 ]] || { echo "::error::dbt_command is empty"; exit 1; }) and use the file's own ${DBT[@]+"${DBT[@]}"} idiom for consistency.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing: dbt_command defaults to dbt, and an empty value is a configuration error that fails loudly.

// A dbt_project.yml change that loosens test thresholds for a whole path.
for (const file of reviewable.filter((f) => f.kind === "project_config" && /(^|\/)dbt_project\.ya?ml$/.test(f.path))) {
if (file.status !== "modified") continue
const [oldContent, newContent] = await Promise.all([getContent?.(file.path, "old"), getContent?.(file.path, "new")])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Re-fetch of dbt_project.yml is redundant and unguarded

When the PR modifies dbt_project.yml, this re-reads the head content that projectYml already holds (fetched with .catch at line 1671), and unlike that read it has no .catch. A rejecting getContent (valid custom/unit-test resolver) aborts the entire runReview instead of degrading this lane. Reuse projectYml when file.path === projectYmlPath, and guard the old-side read.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. We'll reuse projectYml and add a .catch.

const out = detectTestAdequacy({ file, newContent: newTests(), projectYml: project })
expect(rule(out)).toEqual(["test-inherits-warn"])
expect(out[0].body).toContain("`shop` → `marts`")
const own = newTests(":\n config:\n severity: error").replace("- not_null:", "- not_null:")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: No-op .replace and a silent negative assertion

.replace("- not_null:", "- not_null:") replaces a string with itself (dead code — the interpolated argument already supplies the colon). Also, if a future indentation slip made this YAML invalid, parse() returns undefined, detectTestAdequacy returns [], and line 68 would pass silently. Drop the .replace and assert the parsed shape so the negative assertion is trustworthy.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: nit, tracked. We'll remove the no-op .replace.

@kilo-code-bot

kilo-code-bot Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Code Review Summary

Status: 14 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 3
SUGGESTION 11
Issue Details (click to expand)

WARNING

File Line Issue
github/review/action.yml 248 Unguarded cp ... profiles.yml under set -e aborts the prepare step (skipping the review entirely) when the project dir is absent at the merge-base; every other failure on this path degrades to a warning.
packages/opencode/src/altimate/review/test-adequacy.ts 254 Reporting only the topmost loosened path hides an independent error_if loosening in a deeper folder that resets severity: error, and the emitted "every data test under X" wording is then wrong.
github/review/action.yml 244 (carried) The stale target-base/ cleanup still runs only after the guards that skip the base compile (BASE/MB empty, failed worktree add, and the new head-compile exit at 222), so a persisted workspace can compare against the previous run's base SQL.

SUGGESTION

File Line Issue
github/review/action.yml 218 dbt compile now writes to dirname(manifest_path), but the review resolves head compiled SQL from target/compiled (compiled.ts:96, run.ts:407), so a custom manifest_path silently loses the compiled SQL.
packages/opencode/src/altimate/review/test-adequacy.ts 198 The else if still suppresses the inherited-error_if finding whenever another added test inherits severity: warn; the two sets are independent.
packages/opencode/src/altimate/review/select-list.ts 98 complete turns a single unparsed name (backticked columns, quoted table prefixes) into an unknown list, disabling the parse-only column rule for those models.
packages/opencode/src/altimate/review/dbt-patterns.ts 229 (carried) addedLineNumbers re-implements the exported commentableLines (orchestrate.ts:268); extract a shared hunk walker.
packages/opencode/src/altimate/review/dbt-patterns.ts 258 (carried) The comment blanking regex is not quote-aware; reuse blankComments() from select-list.ts ('--' in a literal hides a window).
packages/opencode/src/altimate/review/dbt-patterns.ts 266 (carried) The length >= 3 filter still drops valid two-character added lines on the no-hunk-header fallback.
packages/opencode/src/altimate/review/dbt-patterns.ts 274 (carried) The paren-depth scan runs on comment-stripped but not literal-stripped SQL, so (/) inside a literal truncates or over-extends the window.
packages/opencode/src/altimate/review/test-adequacy.ts 140 (carried) segmentsOf still compares only the first path segment, so ./models or nested model-paths miss inherited thresholds.
packages/opencode/src/altimate/review/orchestrate.ts 1787 (carried) dbt_project.yml is re-fetched without .catch, so a rejecting resolver aborts the whole runReview.
packages/opencode/test/altimate/review-test-adequacy.test.ts 83 (carried) The no-op .replace("- not_null:", "- not_null:") remains, and the negative assertion passes silently if the YAML stops parsing.
packages/opencode/test/altimate/review-action.test.ts 31 (carried) The ordering assertion passes when the cleanup line is absent (indexOf = -1 compares less than the build index).
Files Reviewed (9 files)
  • docs/docs/usage/dbt-pr-review.md - 0 issues
  • github/review/action.yml - 3 issues
  • packages/opencode/src/altimate/review/dbt-patterns.ts - 4 issues
  • packages/opencode/src/altimate/review/orchestrate.ts - 1 issue
  • packages/opencode/src/altimate/review/select-list.ts - 1 issue
  • packages/opencode/src/altimate/review/test-adequacy.ts - 3 issues
  • packages/opencode/test/altimate/review-action.test.ts - 1 issue
  • packages/opencode/test/altimate/review-incremental-semantics.test.ts - 0 issues
  • packages/opencode/test/altimate/review-test-adequacy.test.ts - 1 issue

Fix these issues in Kilo Cloud

Previous Review Summaries (2 snapshots, latest commit e77d121)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit e77d121)

Status: 16 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 5
SUGGESTION 11
Issue Details (click to expand)

WARNING

File Line Issue
github/review/action.yml 228 The stale target-base/ cleanup runs only after the guards that skip the base compile, so on a persisted/cached workspace the previous run's base SQL is still consumed (and not reported unverified); the new unguarded rm -rf can also abort the step under set -e. Move the cleanup above the BASE/MB/worktree guards and make it non-fatal.
github/review/action.yml 207 The head dbt compile/deps is unguarded under set -e, so a head compile failure skips the review step entirely (even in comment mode) while the docs (line 224) say compile "usually works offline". Degrade to a warning like the base side, or document that it fails the job.
packages/opencode/src/altimate/review/test-adequacy.ts 27 cardinality_equality on the same model with a different field is a real cross-column check but is still reported as a self-comparison; the aggregation case is now guarded, this one is not.
packages/opencode/src/altimate/review/test-adequacy.ts 225 Loosening is decided per config node rather than on the effective value: deleting an override under a looser parent goes unreported, a redundant override is reported, and a non-simple error_if change is called "loosened" in either direction.
packages/opencode/src/altimate/review/orchestrate.ts 685 The parse-only select list is treated as complete when an item yields no name (select id, amount::numeric(18,2) → {id}), so a cast change reads as a removed column and drives a high-confidence warning.

SUGGESTION

File Line Issue
packages/opencode/src/altimate/review/dbt-patterns.ts 229 addedLineNumbers re-implements the exported commentableLines hunk walker (same regex/counter, already diverging); extract a shared helper in diff-filter.ts or pass the set in through Ctx.
packages/opencode/src/altimate/review/dbt-patterns.ts 257 The inline comment blanking is not quote-aware ('--' in a literal blanks the rest of the line and the dedup window disappears); reuse the existing length-preserving blankComments() from select-list.ts.
packages/opencode/src/altimate/review/dbt-patterns.ts 260 The length >= 3 added-line filter drops valid two-character lines (id, x) on the header-less fallback path, so a window changed only by such a line is ignored.
packages/opencode/src/altimate/review/dbt-patterns.ts 266 The parenthesis-depth scan still runs on comment-stripped but not literal-stripped SQL, so a (/) inside a literal truncates or over-extends the window.
packages/opencode/src/altimate/review/test-adequacy.ts 140 segmentsOf strips only an exact first model-paths segment, so nested or relative model dirs (transform/models, ./models) miss inherited thresholds.
packages/opencode/src/altimate/review/test-adequacy.ts 182 own() treats any one local config field as overriding every inherited setting, so a test can still inherit an ineffective severity or error_if unreported.
github/review/action.yml 203 "${DBT[@]}" is an unbound-variable error for an empty dbt_command under set -u on bash < 4.4; validate the array and use the file's ${DBT[@]+...} idiom.
packages/opencode/src/altimate/review/orchestrate.ts 1784 dbt_project.yml is re-fetched (redundant with line 1756) without a .catch, so a rejecting resolver aborts the whole runReview.
packages/opencode/test/altimate/review-test-adequacy.test.ts 83 The no-op .replace("- not_null:", "- not_null:") is still there, and the negative assertion would pass silently if the YAML stopped parsing.
packages/opencode/test/altimate/review-action.test.ts 31 The ordering assertion passes when the cleanup line is absent (indexOf = -1 compares less than the build index); assert the line exists before comparing indexes.
docs/docs/usage/dbt-pr-review.md 215 The new snippet pins the action to @main while the rest of the page pins @v0.8.5.
Files Reviewed (10 files)
  • docs/docs/usage/dbt-pr-review.md - 1 issue
  • github/review/action.yml - 3 issues
  • github/review/examples/dbt-pr-review.yml - 0 issues
  • packages/opencode/src/altimate/review/dbt-patterns.ts - 4 issues
  • packages/opencode/src/altimate/review/orchestrate.ts - 2 issues
  • packages/opencode/src/altimate/review/test-adequacy.ts - 4 issues
  • packages/opencode/test/altimate/review-action.test.ts - 1 issue
  • packages/opencode/test/altimate/review-incremental-semantics.test.ts - 0 issues
  • packages/opencode/test/altimate/review-test-adequacy.test.ts - 1 issue
  • packages/opencode/test/skill/release-v0.8.5-adversarial.test.ts - 0 issues

Fix these issues in Kilo Cloud

Previous review (commit 7da6850)

Status: 12 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 5
SUGGESTION 7
Issue Details (click to expand)

WARNING

File Line Issue
packages/opencode/src/altimate/review/orchestrate.ts 681 rawSelectList keeps {% if %} block bodies, so conditional columns are read as unconditional; this raw path is now also used when only the head is compiled (line 664), producing false columnDelta findings where it previously returned none.
packages/opencode/src/altimate/review/dbt-patterns.ts 250 The "added line falls inside the window" test is an unanchored substring match, so an unrelated short/clause-shaped added line can flag a pre-existing, untouched dedup window.
github/review/action.yml 224 git worktree add is unguarded under set -e; a stale worktree or fetch failure aborts the action and skips the review, contradicting the documented "failed base compile is a warning".
packages/opencode/src/altimate/review/test-adequacy.ts 204 error_if loosening has no direction check: tightening (>100 → >10) is reported as loosened; only the exact "!= 0" spelling is treated as the strict default.
packages/opencode/src/altimate/review/test-adequacy.ts 28 fewer_rows_than against its own ref always fails (not "passes by construction"), and expect_table_aggregation_to_equal_other_table with different expressions is a valid cross-column invariant that gets flagged.

SUGGESTION

File Line Issue
packages/opencode/src/altimate/review/dbt-patterns.ts 243 Paren-depth scan uses comment-stripped but not literal-stripped SQL; a (/) in a string literal corrupts the window boundary.
packages/opencode/src/altimate/review/test-adequacy.ts 164 own() lets any one config field suppress all inherited-warn/threshold findings, though the settings are independent.
github/review/action.yml 210 dbt docs generate is unguarded in full mode; a warehouse/credentials failure aborts the review instead of degrading.
packages/opencode/src/altimate/review/test-adequacy.ts 128 segmentsOf doesn't normalize multi-segment/relative model-paths, so inherited thresholds are mis-scoped.
github/review/action.yml 203 "${DBT[@]}" on an empty array errors under set -u on bash < 4.4; validate and use the file's ${DBT[@]+"${DBT[@]}"} idiom.
packages/opencode/src/altimate/review/orchestrate.ts 1693 dbt_project.yml is re-fetched (redundant with line 1671) without a .catch; a rejecting resolver aborts runReview.
packages/opencode/test/altimate/review-test-adequacy.test.ts 67 No-op .replace and a negative assertion that would silently pass if the YAML failed to parse.
Files Reviewed (10 files)
  • docs/docs/usage/dbt-pr-review.md - 0 issues
  • github/review/action.yml - 3 issues
  • github/review/examples/dbt-pr-review.yml - 0 issues
  • packages/opencode/src/altimate/review/dbt-patterns.ts - 2 issues
  • packages/opencode/src/altimate/review/orchestrate.ts - 2 issues
  • packages/opencode/src/altimate/review/test-adequacy.ts - 4 issues
  • packages/opencode/test/altimate/review-action.test.ts - 0 issues
  • packages/opencode/test/altimate/review-incremental-semantics.test.ts - 0 issues
  • packages/opencode/test/altimate/review-test-adequacy.test.ts - 1 issue
  • packages/opencode/test/skill/release-v0.8.5-adversarial.test.ts - 0 issues

Fix these issues in Kilo Cloud


Reviewed by deepseek-v4.1-flash-1017 · Input: 170.3K · Output: 60.4K · Cached: 3M

Review guidance: REVIEW.md from base branch feat/review-incremental-semantics

@hkaltai hkaltai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: changes requested (one MAJOR). Panel: Claude, Codex, and a lead review. Counts: 0 CRITICAL, 1 MAJOR, 5 MINOR, 4 NIT.

The opt-in dbt_prepare, test-adequacy rules, parse-only column sets and multi-line dedup windows are well structured and tested (targeted suites pass, inputs reach the script only through env, worktree cleanup is trapped). The MAJOR is that the new dedup-window attribution uses a substring match and can flag windows the PR never touched (a regression versus the previous added-line-only behaviour). The MINORs are precision edges in the new threshold and select-list rules and in the action's head handling.

Scope: no linked ticket, so scope vs ticket cannot be verified; the diff matches the PR description. The example workflow rename is a small related cleanup.

Nits

  • packages/opencode/src/altimate/review/orchestrate.ts (comment before rawSelectList use): says "CI that only runs dbt parse" but the fallback also runs when the head is compiled and the base is not. Update the comment.
  • orchestrate.ts:684: the comment "A Jinja expression inside an item ... leaves its alias intact" sits after the guard it explains; move it above if (!names || names.includes("__jinja__")).
  • test-adequacy.ts:162 const own = ... / loose: loose means "tests without their own threshold"; inheriting is clearer. thresholds() also returns severity, so testConfigLevels would fit; rule id test-thresholds-loosened is also emitted for tightenings.
  • github/review/action.yml:196 read -ra DBT <<< "$IN_DBT": ignores quoting and an empty dbt_command yields an empty array (script would run compile). Validate non-empty. Also test-adequacy.ts own() treats a lone warn_if as overriding an inherited severity: warn.

}
if (close < 0) continue
const text = sql.slice(m.index!, close + 1)
if (!added.some((l) => text.includes(l) || l.includes(text.replace(/\s+/g, " ")))) continue

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] Dedup window is attributed to the PR by substring match, so untouched windows are flagged

if (!added.some((l) => text.includes(l) || l.includes(text.replace(/\s+/g, " ")))) continue

text.includes(l) is a substring test, not a position test. Scenario (reproduced by two reviewers): the model already has an unchanged row_number() over (partition by customer_id) as rn; the PR only adds a customer_id to a group by elsewhere. The added line customer_id is a substring of the window, so dedup_no_tiebreaker fires (high confidence) on code the PR did not touch. Before this PR only added lines that themselves contained row_number were considered. The existing "untouched window" test uses select 1 as unrelated, which can never match.

Fix: map each window's [m.index, close] to new-file line numbers and intersect with the added hunk line numbers (or at minimum require the added line to contain row_number/partition by/order by). Add a test with an unrelated added bare identifier that also appears in the window. Also normalise whitespace on both sides of the second comparison.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this was a real regression. The main path was already moved to positions in e77d121 ("dedup window touch"): each window's [m.index, close] span is mapped to head line numbers and intersected with the added lines from the hunk headers, so your group by customer_id scenario no longer fires when the diff has headers. The text fallback (diffs without hunk headers) still had the substring test, so the new commit (d9caa1597c) "address review feedback: dedup window text fallback, …" restricts it to added lines that themselves carry window syntax (row_number, over, partition by, order by) and normalises whitespace on both sides. Added a test where the added line is a bare customer_id that also appears in the window (not flagged), plus the positive case with differing whitespace.

// No compiled SQL (CI that only runs `dbt parse`): the raw model's own SELECT
// list, when it names its columns. Columns produced inside a macro stay unknown.
if (!newSql) return none
const current = rawSelectList(newSql)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Parse-only select list treated as complete when items are dropped

const current = rawSelectList(newSql)

selectListNames is deliberately conservative (items it cannot name contribute nothing). Previously that only supplemented lineage; here it is the whole answer. select id, amount::numeric(18,2) yields ["id"], so going from amount to a cast is seen as a removed column, and with exposures/contracts this produces a high-confidence "removed columns" warning. The branch also now runs when the head is compiled but the base is not.

Fix: return unknown (empty set) when any non-empty select item produced no name. Related: stripping {% if %} / {% for %} tags keeps their bodies, so {% if var('x', false) %}, region {% endif %} is counted as an unconditional added column and can trigger the on_schema_change='fail' warning; treat projection-affecting control blocks as unknown.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that the raw list is the whole answer here, not a supplement. selectListNames now takes a complete option, and the parse-only path uses it: any non-empty item without a name of its own (your amount::numeric(18,2)) makes the list unknown, so no removed/added columns are reported. The {% if %} / {% for %} case was handled in e77d121 (a tag becomes its own __jinja__ item, which marks the list unknown). I also reworded the comment to cover the compiled-head/uncompiled-base case and moved the Jinja note above the guard it explains. Test: a column turned into a cast plus an added column produces no on_schema_change finding.

for (const [path, t] of flat(newDoc)) {
const prev = before.get(path) ?? {}
const toWarn = t.severity === "warn" && prev.severity !== "warn"
const newThreshold = t.error_if && t.error_if !== "!= 0" && t.error_if !== prev.error_if

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Any changed error_if other than != 0 is reported as "loosened"

const newThreshold = t.error_if && t.error_if !== "!= 0" && t.error_if !== prev.error_if

Changing '>1000' to '>10' (a tightening) or adding the default-equivalent '>0' / '!=0' is reported as "test thresholds loosened". The same literal comparison is used for the inherited-threshold rule. Also, the comparison is per node rather than effective value: with root +severity: warn, deleting marts: +severity: error loosens every marts test but reports nothing, and a redundant +severity: warn under an already-warn parent is reported.

Fix: normalise op N and compare numerically, and compare inheritedThresholds(old, path) with inheritedThresholds(new, path) across the union of paths.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The numeric part landed in e77d121: error_if is normalised to the number of tolerated failing rows, so '>1000' to '>10' is a tightening and '>0' / '!=0' / '>=1' count as the strict default. The per-node comparison you pointed out is fixed in the new commit (d9caa1597c): loosening is now judged on effective values, walking both versions of data_tests: across the union of their paths, and only the topmost loosened path is reported. Deleting marts: +severity: error under a root +severity: warn is now reported for shop → marts, and a redundant +severity: warn under a warn parent is not. Tests cover both, plus a root-level change reported once rather than per folder.

const project = parse(input.projectYml)
if (project && added.length) {
const inherited = inheritedThresholds(project, segmentsOf(file.path, input.layout))
const own = (t: TestRef) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Any local threshold suppresses all inherited checks

const own = (t: TestRef) => {

With project-level severity: warn, adding only config: {warn_if: '>10'} or error_if: '!= 0' returns no finding, yet severity is still inherited (dbt resolves these as separate properties).

Fix: merge local and inherited properties individually and inspect the effective config.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, dbt resolves these as separate properties. The new commit (d9caa1597c) checks severity and error_if individually: a test inherits severity: warn unless it sets its own severity, so a local warn_if: '>10' or error_if: '!= 0' alone is still reported. It inherits a loose error_if unless it sets its own error_if (and its effective severity isn't warn). This also covers the nit about a lone warn_if overriding an inherited severity: warn. I renamed loose to inheritingWarn / inheritingThreshold following your naming suggestion. Tests cover each combination.

const SELF_COMPARE = new Set([
"equality",
"equal_rowcount",
"cardinality_equality",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] cardinality_equality / aggregation tests against the same model are not always self-comparisons

"cardinality_equality",

cardinality_equality with to: ref('m'), field: b on column a compares two different columns' distinct counts, a real check, but is reported as comparing the model with itself. Same for expect_table_aggregation_to_equal_other_table with different expressions. (fewer_rows_than against itself fails rather than passes, so "passes by construction" is also inaccurate there.)

Fix: for these tests require the field/expression to match as the relationships branch does, and drop fewer_rows_than. Separately, segmentsOf strips only model dirs, so seeds/_s.yml walks data_tests: shop: seeds: as if seeds were a folder; strip seed/snapshot dirs too.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. e77d121 dropped fewer_rows_than and limited expect_table_aggregation_to_equal_other_table to matching expressions. The new commit (d9caa1597c) adds the missing cardinality_equality case: it is flagged as a self-comparison only when field is the same column, so column a against field: b on the same model counts as a real check. segmentsOf now strips any resource dir (seeds, snapshots, tests as well as models, from the layout when one is known), so seeds/_s.yml no longer walks data_tests: shop: seeds: as if seeds were a folder. Tests added for both.

Comment thread github/review/action.yml Outdated
"${DBT[@]}" compile --target-path "$2" ${PROFILES[@]+"${PROFILES[@]}"} )
}
echo "Compiling the head…"
build "$PROJECT" "$PROJECT/target"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] dbt_prepare compiles the checkout, ignoring the head input; head-side failures abort the job

build "$PROJECT" "$PROJECT/target"
  1. The merge-base is computed against literal HEAD and the head compile uses the checkout, but the review step passes --head "$IN_HEAD" when set. With head: A while B is checked out, A's diff is reviewed using B's compiled SQL and equivalence can be wrongly reported. Prepare against the selected head or reject a mismatch.
  2. Under set -e, a failed head deps/compile/docs generate fails the step so the review never runs (even in mode: comment), while the input description and docs (docs/docs/usage/dbt-pr-review.md:224) say compile "usually works offline". Many adapters need a connection to compile. Either degrade to a warning like the base side, or document that it fails the job and drop the offline claim.
  3. Minor: manifest_path is not honoured (head always goes to $PROJECT/target), and a runtime-generated profiles.yml in the project dir is absent from the base worktree, so the base compile quietly fails.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three points are addressed in the new commit's prepare-step changes (d9caa1597c). (1) When head is set and doesn't resolve to the checked-out commit, preparation is skipped with a warning, so we never review one commit against another commit's compiled SQL. (2) A failed head compile is now a warning and the review still runs (e77d121 had already done this for docs generate and the base checkout). The input description and the docs no longer claim compile "usually works offline"; they now say many adapters connect while compiling. (3) The head is compiled into the directory of manifest_path, and a runtime-generated profiles.yml in the project dir is copied into the base worktree when no --profiles-dir is given. An empty dbt_command is also rejected now (from your nit). The action test asserts each of these.

RaghwendraSingh and others added 4 commits November 20, 2026 12:13
…erge-base and catalog artifacts

- new action inputs: `dbt_prepare` (`none` default | `compile` | `full`), `dbt_command`, `dbt_profiles_dir`
- `compile` runs `dbt compile` on the head and compiles the PR's merge-base into `target-base/` in a temporary worktree (fetching and deepening a shallow checkout when needed); `full` adds `dbt docs generate`
- a base compile that fails is a warning: the review still runs and marks those models unverified
- inputs reach the script only through env, never interpolated
- docs: the manual base-compile snippet now compiles the merge-base, not the base branch's tip
- the example workflow is renamed `examples/dbt-pr-review.yml` (was named after one internal repo)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ows split across lines are read whole

Found by replaying the PRs behind known escaped defects:

- without compiled SQL, output columns come from the raw model's final SELECT (Jinja blanked; a Jinja expression in the list makes it unknown). `on_schema_change='fail'` with new columns now fires on the two real escapes it targets, in a parse-only CI setup
- `row_number() over (…)` windows are read from the whole file, so an `ORDER BY` on its own line is seen: the "dedup has no ORDER BY" rule no longer fires falsely on multi-line windows, and the load-timestamp rule sees them. A window counts only when an added line falls inside it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nherited warn/thresholds, loosened project thresholds

- a test added or changed in the PR that compares a model with itself (`dbt_utils.equality`/`equal_rowcount`/… against its own `ref`, or a `relationships` test to its own column) passes by construction (warning)
- new tests that inherit `severity: warn` or an `error_if` threshold from `dbt_project.yml` (walked from the top through the project name and folders, deeper wins) are reported once per yml (suggestion)
- a `dbt_project.yml` change that turns a folder's tests to `warn` or sets an `error_if` threshold is reported (warning)
- reads dbt 1.8+ `arguments:` too; all findings are catalog origin (never block)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…window touch, self-compare tests, `error_if` loosening, prepare-step failures

- `orchestrate.ts`: a `{% if %}` / `{% for %}` tag inside a raw model's select
  list makes the list unknown, so conditional columns no longer read as added
  or removed in the parse-only column delta
- `dbt-patterns.ts`: a dedup window counts as changed only when an added line
  falls inside its line span (head line numbers from the hunk headers), not
  when an added line's text happens to occur in it
- `test-adequacy.ts`: drop `fewer_rows_than` (it fails against itself, it
  doesn't pass by construction); an aggregation comparison on the same model
  is flagged only when both expressions match
- `test-adequacy.ts`: `error_if` is reported as loosened only when the new bound
  tolerates more failing rows; `> 0` / `!=0` / `>= 1` count as the strict default
- `action.yml`: a failed `dbt docs generate` or base checkout warns instead of
  failing the step; a stale `target-base/` is removed before the base compile
- tests for each

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@RaghwendraSingh
RaghwendraSingh force-pushed the feat/review-incremental-semantics branch from fe31dc6 to 9625cfd Compare November 20, 2026 06:53
Comment thread github/review/action.yml
trap 'git worktree remove --force "$WT" >/dev/null 2>&1 || true' EXIT
git worktree add --detach "$WT" "$MB" >/dev/null || { echo "::warning::Could not check out the base; skipping the base compile."; exit 0; }
# A stale target-base/ from an earlier run must not stand in for a failed compile.
rm -rf "$PROJECT/target-base"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: The stale target-base/ cleanup runs only after the guards that can skip the base compile

When BASE is empty, the merge-base can't be found, or git worktree add fails, the script exits 0 before this line, so a target-base/ left by an earlier run survives. makeCompiledResolver reads target-base/compiled whenever it exists (compiled.ts:97), so on a cached or self-hosted workspace the head is compared against the previous run's base, and because the old side still resolves, those models are not even listed as unverified. Move the cleanup above the BASE/MB/worktree guards, and make it non-fatal (rm -rf ... || { echo "::warning::..."; exit 0; }) so an rm failure cannot abort the step and skip the review.

}

/** Head-side line numbers of the added lines; undefined when the diff has no hunk headers. */
function addedLineNumbers(diff: string | undefined): Set<number> | undefined {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: This hunk walker duplicates the exported commentableLines

commentableLines (orchestrate.ts:268) already walks @@ hunks with the same regex and the same right counter to compute head-side line numbers; this copy already differs (it does not drop the diff's trailing-newline row) and the two will drift. dbt-patterns cannot import orchestrate without a cycle (the file already keeps a local modelNameFromPath for exactly that reason), so extract the walker into a shared module (e.g. diff-filter.ts) and take the added-line set from it, or pass the numbers in through Ctx.

*/
function dedupWindows(c: Ctx): Array<{ spec: string; text: string }> {
// Comments blanked with newlines kept, so offsets map to head line numbers.
const sql = c.newSql.replace(/\/\*[\s\S]*?\*\/|--[^\n]*|\{#[\s\S]*?#\}/g, (m) => m.replace(/[^\n]/g, " "))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: This comment blanking is not quote-aware; blankComments() already does this

A -- inside a string literal blanks the rest of the line: select '--' as sep, row_number() over (partition by id) as rn loses the window entirely (a missed dedup finding), and a /* in a literal can blank a multi-line span. blankComments() in select-list.ts is the existing length-preserving, quote-aware helper (it skips '...', "..." and backticked identifiers), so reuse it here and keep only the {# ... #} alternative.

const run = prep.run!
expect(run).toMatch(/docs generate[^\n]*\) \|\|\n\s*echo "::warning::/)
expect(run).toMatch(/git worktree add[^\n]*\|\| \{ echo "::warning::/)
expect(run.indexOf('rm -rf "$PROJECT/target-base"')).toBeLessThan(run.indexOf('build "$WT/$REL"'))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: This ordering assertion passes when the cleanup line is absent

run.indexOf('rm -rf ...') is -1 when the line is deleted, and -1 < indexOf('build ...') is true, so the test still passes without the rm -rf "$PROJECT/target-base" line it is meant to guard. Assert the cleanup line is present (e.g. expect(run).toContain('rm -rf "$PROJECT/target-base"')) before comparing indexes.

…plete raw select lists, effective test thresholds, `cardinality_equality` field, prepare-step head handling

- `dbt-patterns.ts`: without hunk headers, a dedup window counts as touched
  only by an added line that holds window syntax (`row_number`, `over`,
  `partition by`, `order by`), with whitespace normalised on both sides; a bare
  identifier that also occurs in the window no longer flags it
- `select-list.ts` / `orchestrate.ts`: the parse-only column delta treats the
  raw select list as unknown when any item has no name of its own (e.g.
  `amount::numeric(18,2)`); comments updated for the compiled-on-one-side case
- `test-adequacy.ts`: `severity` and `error_if` are resolved separately, so a
  local `warn_if` / `error_if` no longer hides an inherited `severity: warn`
- `test-adequacy.ts`: threshold loosening compares effective values across the
  union of config paths in both versions, reports only the topmost loosened
  path, ignores redundant overrides and catches deleted stricter overrides
- `test-adequacy.ts`: `cardinality_equality` against the same model is flagged
  only when `field` is the same column; seed/snapshot yml paths strip their
  resource dir before walking `data_tests:`
- `action.yml`: `dbt_prepare` skips when `head` is not the checkout, rejects an
  empty `dbt_command`, warns instead of failing on a head compile error, builds
  the head into the `manifest_path` directory, and copies a runtime
  `profiles.yml` into the base worktree; docs drop the offline claim
- tests for each

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@RaghwendraSingh

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough pass, Harish. The MAJOR (dedup attribution) is fixed: positions since e77d121, plus a syntax-gated text fallback in the new commit (d9caa1597c). All five MINORs are addressed in "fix(review): address review feedback: dedup window text fallback, complete raw select lists, effective test thresholds, cardinality_equality field, prepare-step head handling", with a test for each. On the nits: the dbt parse comment now covers the one-side-compiled case and the Jinja comment sits above its guard. loose is now inheritingWarn / inheritingThreshold. test-thresholds-loosened is no longer emitted for tightenings. An empty dbt_command fails fast and the input now documents whitespace splitting without shell quoting. A lone warn_if no longer hides an inherited severity: warn. I kept the thresholds() name to keep this diff focused; happy to rename it in a follow-up. Review suite: 467 pass; tsc is clean outside /tui/.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 existing issue remains and 5 new issues found across 9 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/opencode/test/altimate/review-action.test.ts">

<violation number="1" location="packages/opencode/test/altimate/review-action.test.ts:44">
P2: This ordering check passes when the head check is missing: `indexOf` returns -1 for an absent string, and -1 is less than any build index. Assert presence first so a removed `$IN_HEAD^{commit}` check fails the test.</violation>
</file>

<file name="packages/opencode/src/altimate/review/select-list.ts">

<violation number="1" location="packages/opencode/src/altimate/review/select-list.ts:98">
P3: `complete` treats valid quoted aliases such as `1 AS "null"` as unnamed and makes the whole raw select list unknown, so parse-only column checks silently skip it. Preserve explicitly written aliases even when they match the reserved-word guard.</violation>
</file>

<file name="github/review/action.yml">

<violation number="1" location="github/review/action.yml:204">
P2: A failed head compile leaves existing or partial artifacts in place before `exit 0`, so the next review can use stale SQL and manifest data instead of treating the head as unverified. Clear the head artifact set safely on this failure path before continuing.</violation>

<violation number="2" location="github/review/action.yml:220">
P2: Custom `manifest_path` values compile SQL into the manifest’s directory, but the review still searches the hard-coded `target/compiled` directory. A setting such as `build/manifest.json` therefore causes `dbt_prepare` to produce artifacts the engine cannot find; pass the prepared target directory through to the resolver or keep preparation and resolver paths aligned.</violation>

<violation number="3" location="github/review/action.yml:248">
P2: This unguarded `cp` can fail when `project_dir` does not exist at the merge-base, causing the whole preparation step to fail before the base compile’s warning fallback. Guard the copy or turn its failure into a warning so the review still runs.</violation>
</file>

Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.

View guided diff | Turn on auto-fix | Re-trigger cubic

expect(prep.env?.IN_MANIFEST).toBe("${{ inputs.manifest_path }}")
expect(run).toMatch(/\$\{#DBT\[@\]\} -eq 0 \]\]; then echo "::error::/)
// The head check runs before anything is compiled.
expect(run.indexOf('"$IN_HEAD^{commit}"')).toBeLessThan(run.indexOf('build "$PROJECT"'))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This ordering check passes when the head check is missing: indexOf returns -1 for an absent string, and -1 is less than any build index. Assert presence first so a removed $IN_HEAD^{commit} check fails the test.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/review-action.test.ts, line 44:

<comment>This ordering check passes when the head check is missing: `indexOf` returns -1 for an absent string, and -1 is less than any build index. Assert presence first so a removed `$IN_HEAD^{commit}` check fails the test.</comment>

<file context>
@@ -30,4 +30,21 @@ describe("action: dbt_prepare", () => {
+    expect(prep.env?.IN_MANIFEST).toBe("${{ inputs.manifest_path }}")
+    expect(run).toMatch(/\$\{#DBT\[@\]\} -eq 0 \]\]; then echo "::error::/)
+    // The head check runs before anything is compiled.
+    expect(run.indexOf('"$IN_HEAD^{commit}"')).toBeLessThan(run.indexOf('build "$PROJECT"'))
+    expect(run).toMatch(/if ! build "\$PROJECT" "\$TARGET"; then\n\s*echo "::warning::/)
+    expect(run).toContain('TARGET="$(dirname "$MANIFEST")"')
</file context>
Suggested change
expect(run.indexOf('"$IN_HEAD^{commit}"')).toBeLessThan(run.indexOf('build "$PROJECT"'))
const headCheck = run.indexOf('"$IN_HEAD^{commit}"')
expect(headCheck).toBeGreaterThanOrEqual(0)
expect(headCheck).toBeLessThan(run.indexOf('build "$PROJECT"'))

Comment thread github/review/action.yml
# A profiles.yml generated into the project dir at runtime is untracked, so
# the base worktree lacks it.
if [[ ${#PROFILES[@]} -eq 0 && -f "$PROJECT/profiles.yml" && ! -f "$WT/$REL/profiles.yml" ]]; then
cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This unguarded cp can fail when project_dir does not exist at the merge-base, causing the whole preparation step to fail before the base compile’s warning fallback. Guard the copy or turn its failure into a warning so the review still runs.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At github/review/action.yml, line 248:

<comment>This unguarded `cp` can fail when `project_dir` does not exist at the merge-base, causing the whole preparation step to fail before the base compile’s warning fallback. Guard the copy or turn its failure into a warning so the review still runs.</comment>

<file context>
@@ -226,6 +242,11 @@ runs:
+        # A profiles.yml generated into the project dir at runtime is untracked, so
+        # the base worktree lacks it.
+        if [[ ${#PROFILES[@]} -eq 0 && -f "$PROJECT/profiles.yml" && ! -f "$WT/$REL/profiles.yml" ]]; then
+          cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml"
+        fi
         echo "Compiling the base ($BASE @ ${MB:0:12})…"
</file context>
Suggested change
cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml"
cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml" || echo "::warning::Could not copy profiles.yml to the base worktree; base compile may fail."

Comment thread github/review/action.yml
# against another commit's compiled SQL, so skip preparing instead.
if [[ -n "${IN_HEAD:-}" && "$(git rev-parse --verify -q "$IN_HEAD^{commit}" || true)" != "$(git rev-parse HEAD)" ]]; then
echo "::warning::head ($IN_HEAD) is not the checked-out commit; skipping dbt_prepare. Check out the head to build its artifacts."
exit 0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: A failed head compile leaves existing or partial artifacts in place before exit 0, so the next review can use stale SQL and manifest data instead of treating the head as unverified. Clear the head artifact set safely on this failure path before continuing.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At github/review/action.yml, line 204:

<comment>A failed head compile leaves existing or partial artifacts in place before `exit 0`, so the next review can use stale SQL and manifest data instead of treating the head as unverified. Clear the head artifact set safely on this failure path before continuing.</comment>

<file context>
@@ -188,12 +188,21 @@ runs:
+        # against another commit's compiled SQL, so skip preparing instead.
+        if [[ -n "${IN_HEAD:-}" && "$(git rev-parse --verify -q "$IN_HEAD^{commit}" || true)" != "$(git rev-parse HEAD)" ]]; then
+          echo "::warning::head ($IN_HEAD) is not the checked-out commit; skipping dbt_prepare. Check out the head to build its artifacts."
+          exit 0
+        fi
         PROJECT="$(cd "${IN_PROJECT_DIR:-.}" && pwd)"
</file context>

Comment thread github/review/action.yml
[[ "$MANIFEST" == /* ]] || MANIFEST="$PROJECT/$MANIFEST"
TARGET="$(dirname "$MANIFEST")"
echo "Compiling the head…"
if ! build "$PROJECT" "$TARGET"; then

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Custom manifest_path values compile SQL into the manifest’s directory, but the review still searches the hard-coded target/compiled directory. A setting such as build/manifest.json therefore causes dbt_prepare to produce artifacts the engine cannot find; pass the prepared target directory through to the resolver or keep preparation and resolver paths aligned.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At github/review/action.yml, line 220:

<comment>Custom `manifest_path` values compile SQL into the manifest’s directory, but the review still searches the hard-coded `target/compiled` directory. A setting such as `build/manifest.json` therefore causes `dbt_prepare` to produce artifacts the engine cannot find; pass the prepared target directory through to the resolver or keep preparation and resolver paths aligned.</comment>

<file context>
@@ -203,11 +212,18 @@ runs:
+        TARGET="$(dirname "$MANIFEST")"
         echo "Compiling the head…"
-        build "$PROJECT" "$PROJECT/target"
+        if ! build "$PROJECT" "$TARGET"; then
+          echo "::warning::The head compile failed (dbt may need a warehouse connection to compile); the review runs without compiled artifacts."
+          exit 0
</file context>

if (m && before && (m[1] || /([A-Za-z0-9_$"`\]')]|\bend)$/i.test(before))) name = unquote(m[2])
}
if (name && (IDENT.test(name) || /^[^"`]+$/.test(name)) && !/^(end|null|true|false)$/i.test(name)) names.push(name.toLowerCase())
else if (opts.complete) return null

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: complete treats valid quoted aliases such as 1 AS "null" as unnamed and makes the whole raw select list unknown, so parse-only column checks silently skip it. Preserve explicitly written aliases even when they match the reserved-word guard.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/select-list.ts, line 98:

<comment>`complete` treats valid quoted aliases such as `1 AS "null"` as unnamed and makes the whole raw select list unknown, so parse-only column checks silently skip it. Preserve explicitly written aliases even when they match the reserved-word guard.</comment>

<file context>
@@ -93,6 +95,7 @@ export function selectListNames(sql: string): string[] | null {
       if (m && before && (m[1] || /([A-Za-z0-9_$"`\]')]|\bend)$/i.test(before))) name = unquote(m[2])
     }
     if (name && (IDENT.test(name) || /^[^"`]+$/.test(name)) && !/^(end|null|true|false)$/i.test(name)) names.push(name.toLowerCase())
+    else if (opts.complete) return null
   }
   return names
</file context>

Comment thread github/review/action.yml
# A profiles.yml generated into the project dir at runtime is untracked, so
# the base worktree lacks it.
if [[ ${#PROFILES[@]} -eq 0 && -f "$PROJECT/profiles.yml" && ! -f "$WT/$REL/profiles.yml" ]]; then
cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Unguarded cp under set -e can fail the whole prepare step and skip the review

Every other failure on this path degrades to a warning, but this cp does not. If the dbt project directory does not exist at the merge-base (a PR that adds or renames project_dir, e.g. a new analytics/ folder, or a rename), or the copy fails for any reason, set -e exits the step non-zero and the following "Run dbt PR review" step never runs - contradicting the input description and docs/docs/usage/dbt-pr-review.md ("A head or base compile that fails is a warning: the review still runs"). Add a -d "$WT/$REL" guard to the condition and make the copy non-fatal:

Suggested change
cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml"
cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml" || echo "::warning::Could not copy profiles.yml into the base worktree."

for (const path of paths.values()) {
const hit = loosened(path)
// Report only the topmost loosened path, not each folder below it.
if (!hit || (path.length && loosened(path.slice(0, -1)))) continue

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Ancestor suppression hides an independent loosening deeper in the tree

loosened(path.slice(0, -1)) skips a child whenever any ancestor loosened, even when the child loosens for a different reason and its tests still gate the build. Example: old shop -> marts -> core has +severity: error, +error_if: ">10"; the PR adds +severity: warn under shop and raises core's +error_if to ">100" while core keeps +severity: error. loosened(["shop", "marts"]) is true because warn is inherited, so the core error_if loosening (10 -> 100 tolerated failures) is dropped, and the emitted shop finding claims warn now "applies to every data test under shop" - false for core. Suppress a child only when its effective severity did not reset the ancestor's loosened value (or when its why matches the ancestor's).

`They inherit \`severity: warn\` from \`dbt_project.yml\`${where}, so failures are reported but never fail \`dbt build\`. ` +
"If a test should gate the build, set `config: {severity: error}` on it."),
)
else if (inheritingThreshold.length && inherited.error_if && allowedFailures(inherited.error_if) !== 0)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: The else if still drops the inherited-threshold finding when any other added test inherits warn

inheritingWarn and inheritingThreshold are now independent sets, but only one finding can be emitted. With data_tests: shop -> marts: {+severity: warn, +error_if: ">100"} and a PR that adds not_null (inherits warn) plus unique with config: {severity: error}, inheritingWarn is non-empty so this branch is skipped and unique's inherited error_if: >100 - which still gates the build - is never reported. Two independent ifs (instead of else if) would cover both sets.

if (m && before && (m[1] || /([A-Za-z0-9_$"`\]')]|\bend)$/i.test(before))) name = unquote(m[2])
}
if (name && (IDENT.test(name) || /^[^"`]+$/.test(name)) && !/^(end|null|true|false)$/i.test(name)) names.push(name.toLowerCase())
else if (opts.complete) return null

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: complete lets one unparsed identifier hide the whole list (backticked columns, quoted prefixes)

The bare matcher at line 88 accepts only "quoted" or unquoted identifiers, and the fallback alias matcher at line 91 requires whitespace before the token, so select `order_id`, `amount` from {{ ref('src') }} and select "t"."amount" from t produce no name for those items. Under the new complete mode that turns the entire list into null, so rawSelectList returns an empty set and the parse-only on_schema_change rule silently goes dark for BigQuery/Databricks models that backtick-quote columns. Allow backticked/bracketed identifiers (and quoted qualifiers) in the bare matcher - unquote already handles them.

Comment thread github/review/action.yml
# The head goes where the review reads the manifest (`manifest_path`).
MANIFEST="${IN_MANIFEST:-target/manifest.json}"
[[ "$MANIFEST" == /* ]] || MANIFEST="$PROJECT/$MANIFEST"
TARGET="$(dirname "$MANIFEST")"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: manifest_path now sets the head compile target, but the review still reads compiled SQL from target/compiled

TARGET moves dbt compile's manifest and compiled/ to dirname(manifest_path), but makeCompiledResolver defaults headDir to "target/compiled" (packages/opencode/src/altimate/review/compiled.ts:96) and run.ts:407 never passes headDir. With manifest_path: build/manifest.json the manifest and catalog are found, yet every compiled-SQL lookup misses - the review falls back to raw Jinja, or reads a target/compiled left by an earlier run, instead of the SQL this step just built. Either derive the resolver's headDir from dirname(manifestPath) in run.ts, or document that dbt_prepare requires the default target/manifest.json.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants