Repository navigation
feat(review): close remaining review gaps: action dbt_prepare, data-test adequacy, parse-only column changes - #1726
RaghwendraSingh wants to merge 5 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
full receipts (1 session)
orchestrator ·
|
| subagent | cost |
|---|---|
| Goal: an evidence-backed, cited landscape of (a) dbt PR review tools, (b) data-… | 2,134,743 tokens |
| Goal: determine what gaps altimate-code's dbt PR review has, by comparing what… | 5,303,299 tokens |
| Label dbt PR review comments by reading each one. Instructions: read /private/t… | 2,794,842 tokens |
| Label dbt PR review comments by reading each one. Instructions: read /private/t… | 3,244,430 tokens |
| Label escaped-defect candidates by READING them (no keyword bucketing). Directo… | 2,570,116 tokens |
| Repo: /Users/raghwendrasingh/code/altimateai/altimate-code (current branch feat… | 4,836,719 tokens |
| You are doing an ADVERSARIAL review of a set of Jira tickets that plan improvem… | 763,761 tokens |
| Label dbt PR review comments by reading each one. Instructions: read /private/t… | 3,351,198 tokens |
| Label escaped-defect candidates by READING them (no keyword bucketing). Directo… | 4,214,327 tokens |
| Label dbt PR review comments by reading each one. Instructions: read /private/t… | 2,821,134 tokens |
| Label dbt PR review comments by reading each one. Instructions: read /private/t… | 1,795,004 tokens |
| You are doing an ADVERSARIAL review of a stack of changes to an open-source dbt… | 4,084,798 tokens |
| Repo: /Users/raghwendrasingh/code/altimateai/altimate-code. Module: packages/op… | 8,815,483 tokens |
| Label dbt PR review comments by reading each one. Instructions: read /private/t… | 2,592,769 tokens |
| Research task, read-only (do not push, comment, or open PRs anywhere).Goal: dec… | 1,198,872 tokens |
| Judge each of altimate-code's own inline PR review comments by reading it. Inpu… | 1,673,212 tokens |
| Label escaped-defect candidates by READING them (no keyword bucketing). Directo… | 2,862,492 tokens |
| Goal: read a Notion doc tree about an "agentic data engineering software factor… | 1,335,636 tokens |
| Goal: test whether the bug classes that real reviewers catch on dbt PRs at one… | 6,608,416 tokens |
Generated by aireceipts
There was a problem hiding this comment.
9 issues found across 10 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/test/altimate/review-test-adequacy.test.ts">
<violation number="1" location="packages/opencode/test/altimate/review-test-adequacy.test.ts:56">
P3: The test name claims 'deeper wins', but no case sets the same key at two levels. Add a fixture where a folder overrides the project-level `+severity` (for example, `shop: +severity: error` with `marts: +severity: warn`) and assert the deeper value and its path.</violation>
</file>
<file name="packages/opencode/src/altimate/review/test-adequacy.ts">
<violation number="1" location="packages/opencode/src/altimate/review/test-adequacy.ts:28">
P2: `fewer_rows_than` is in `SELF_COMPARE`, but a self-comparison makes it fail every time, not pass. The message "passes by construction" is wrong for this test. Drop it from the set and report it separately, or change the message to cover both outcomes.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/test-adequacy.ts:69">
P2: `refsEntity` matches `ref(` inside names such as `my_ref(`, creating false self-comparison findings. Require a word boundary before `ref` and allow normal whitespace before `(`.</violation>
<violation number="3" location="packages/opencode/src/altimate/review/test-adequacy.ts:117">
P3: `path` records only the last node containing either setting, so findings misidentify the source when `severity` and `error_if` are inherited from different levels. Track a path per threshold.</violation>
<violation number="4" location="packages/opencode/src/altimate/review/test-adequacy.ts:128">
P2: `segmentsOf` fails to strip nested configured model paths, so inherited test thresholds are missed under paths such as `src/dbt/models/marts`. Strip the full configured model directory, not just its first segment.</violation>
</file>
<file name="docs/docs/usage/dbt-pr-review.md">
<violation number="1" location="docs/docs/usage/dbt-pr-review.md:215">
P3: This snippet pins the action to `@main`, but the rest of the page pins `@v0.8.5`. Pin to the same release tag so users get a reviewed, reproducible action version.</violation>
</file>
<file name="packages/opencode/src/altimate/review/orchestrate.ts">
<violation number="1" location="packages/opencode/src/altimate/review/orchestrate.ts:664">
P2: When only the base compile is missing, `columnDelta` now falls through to the raw-SQL path and can emit removed-column warnings. Before this change that case returned no deltas. Confirm this is intended for models marked unverified, or keep returning no deltas when the head is compiled and the base is not.</violation>
</file>
<file name="packages/opencode/src/altimate/review/dbt-patterns.ts">
<violation number="1" location="packages/opencode/src/altimate/review/dbt-patterns.ts:235">
P2: The `length >= 3` filter drops valid added SQL lines such as `id` or `x`. A multiline window changed only by adding such a two-character expression is then ignored, so the no-`ORDER BY` detector misses the change; filter only empty normalized lines.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/dbt-patterns.ts:243">
P2: The parenthesis scan counts parentheses inside SQL string literals. A value like `status = ')'` closes the window early, truncating `spec` before a real `ORDER BY`; mask quoted literals or use a quote-aware scanner while finding the matching window close.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| "equality", | ||
| "equal_rowcount", | ||
| "cardinality_equality", | ||
| "fewer_rows_than", |
There was a problem hiding this comment.
P2: fewer_rows_than is in SELF_COMPARE, but a self-comparison makes it fail every time, not pass. The message "passes by construction" is wrong for this test. Drop it from the set and report it separately, or change the message to cover both outcomes.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/test-adequacy.ts, line 28:
<comment>`fewer_rows_than` is in `SELF_COMPARE`, but a self-comparison makes it fail every time, not pass. The message "passes by construction" is wrong for this test. Drop it from the set and report it separately, or change the message to cover both outcomes.</comment>
<file context>
@@ -0,0 +1,214 @@
+ "equality",
+ "equal_rowcount",
+ "cardinality_equality",
+ "fewer_rows_than",
+ "expect_table_row_count_to_equal_other_table",
+ "expect_table_aggregation_to_equal_other_table",
</file context>
There was a problem hiding this comment.
Fixed in e77d121. fewer_rows_than is dropped (against itself it always fails), and an aggregation comparison on the same model is flagged only when both expressions match.
| current, | ||
| } | ||
| // Compiled SQL on both sides: column lineage plus the final SELECT's names. | ||
| if (engineNewSql && (!withOld || engineOldSql)) { |
There was a problem hiding this comment.
P2: When only the base compile is missing, columnDelta now falls through to the raw-SQL path and can emit removed-column warnings. Before this change that case returned no deltas. Confirm this is intended for models marked unverified, or keep returning no deltas when the head is compiled and the base is not.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 664:
<comment>When only the base compile is missing, `columnDelta` now falls through to the raw-SQL path and can emit removed-column warnings. Before this change that case returned no deltas. Confirm this is intended for models marked unverified, or keep returning no deltas when the head is compiled and the base is not.</comment>
<file context>
@@ -648,19 +649,41 @@ async function columnDelta(
+ current,
+ }
+ // Compiled SQL on both sides: column lineage plus the final SELECT's names.
+ if (engineNewSql && (!withOld || engineOldSql)) {
+ const current = await outputColumns(runner, engineNewSql, dialect)
+ const before = withOld ? await outputColumns(runner, engineOldSql!, dialect) : undefined
</file context>
There was a problem hiding this comment.
Fixed in e77d121. A {% if %} or {% for %} tag in the raw select list now makes the list unknown, so conditional columns are no longer reported as added or removed. We kept the raw fallback when only one side is compiled: both sides are then read from raw SQL, so the comparison stays like for like.
| */ | ||
| function dedupWindows(c: Ctx): Array<{ spec: string; text: string }> { | ||
| const sql = stripComments(c.newSql) | ||
| const added = c.added.map((l) => stripComments(l).trim()).filter((l) => l.length >= 3) |
There was a problem hiding this comment.
P2: The length >= 3 filter drops valid added SQL lines such as id or x. A multiline window changed only by adding such a two-character expression is then ignored, so the no-ORDER BY detector misses the change; filter only empty normalized lines.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/dbt-patterns.ts, line 235:
<comment>The `length >= 3` filter drops valid added SQL lines such as `id` or `x`. A multiline window changed only by adding such a two-character expression is then ignored, so the no-`ORDER BY` detector misses the change; filter only empty normalized lines.</comment>
<file context>
@@ -225,15 +225,42 @@ const detectNotIn: Detector = (c) => {
+ */
+function dedupWindows(c: Ctx): Array<{ spec: string; text: string }> {
+ const sql = stripComments(c.newSql)
+ const added = c.added.map((l) => stripComments(l).trim()).filter((l) => l.length >= 3)
+ const out: Array<{ spec: string; text: string }> = []
+ for (const m of sql.matchAll(/row_number\s*\(\s*\)\s*over\s*\(/gi)) {
</file context>
There was a problem hiding this comment.
Follow-up: low priority, tracked. When the diff has hunk headers (the PR API always sends them), matching now uses line numbers and no longer depends on the length >= 3 text filter.
| const arg = (t: TestRef, name: string) => t.args?.[name] ?? t.args?.arguments?.[name] | ||
| const shortName = (test: string) => test.split(".").pop()!.toLowerCase() | ||
| const refsEntity = (v: unknown, entity: string) => | ||
| typeof v === "string" && new RegExp(`ref\\(\\s*['"]${entity.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}['"]\\s*\\)`).test(v) |
There was a problem hiding this comment.
P2: refsEntity matches ref( inside names such as my_ref(, creating false self-comparison findings. Require a word boundary before ref and allow normal whitespace before (.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/test-adequacy.ts, line 69:
<comment>`refsEntity` matches `ref(` inside names such as `my_ref(`, creating false self-comparison findings. Require a word boundary before `ref` and allow normal whitespace before `(`.</comment>
<file context>
@@ -0,0 +1,214 @@
+const arg = (t: TestRef, name: string) => t.args?.[name] ?? t.args?.arguments?.[name]
+const shortName = (test: string) => test.split(".").pop()!.toLowerCase()
+const refsEntity = (v: unknown, entity: string) =>
+ typeof v === "string" && new RegExp(`ref\\(\\s*['"]${entity.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}['"]\\s*\\)`).test(v)
+
+function finding(file: ChangedFile, model: string | undefined, rule: string, severity: Finding["severity"], title: string, body: string, extra = ""): Finding {
</file context>
| typeof v === "string" && new RegExp(`ref\\(\\s*['"]${entity.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}['"]\\s*\\)`).test(v) | |
| typeof v === "string" && new RegExp(`\\bref\\s*\\(\\s*['"]${entity.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}['"]\\s*\\)`).test(v) |
There was a problem hiding this comment.
Follow-up: low priority, tracked (\bref).
| data_tests: | ||
| - not_null${own} | ||
| ` | ||
| test("the effective value walks project → folders, deeper wins", () => { |
There was a problem hiding this comment.
P3: The test name claims 'deeper wins', but no case sets the same key at two levels. Add a fixture where a folder overrides the project-level +severity (for example, shop: +severity: error with marts: +severity: warn) and assert the deeper value and its path.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/review-test-adequacy.test.ts, line 56:
<comment>The test name claims 'deeper wins', but no case sets the same key at two levels. Add a fixture where a folder overrides the project-level `+severity` (for example, `shop: +severity: error` with `marts: +severity: warn`) and assert the deeper value and its path.</comment>
<file context>
@@ -0,0 +1,83 @@
+ data_tests:
+ - not_null${own}
+`
+ test("the effective value walks project → folders, deeper wins", () => {
+ const doc = YAML.parse(project)
+ expect(inheritedThresholds(doc, ["marts"])).toEqual({ severity: "warn", path: ["shop", "marts"] })
</file context>
There was a problem hiding this comment.
Follow-up: nit, tracked. We'll add a fixture where the deeper level wins.
| The action can build these for you with `dbt_prepare` (off by default): | ||
|
|
||
| ```yaml | ||
| - uses: AltimateAI/altimate-code/github/review@main |
There was a problem hiding this comment.
P3: This snippet pins the action to @main, but the rest of the page pins @v0.8.5. Pin to the same release tag so users get a reviewed, reproducible action version.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/docs/usage/dbt-pr-review.md, line 215:
<comment>This snippet pins the action to `@main`, but the rest of the page pins `@v0.8.5`. Pin to the same release tag so users get a reviewed, reproducible action version.</comment>
<file context>
@@ -209,12 +209,30 @@ jobs:
+The action can build these for you with `dbt_prepare` (off by default):
+
+```yaml
+ - uses: AltimateAI/altimate-code/github/review@main
+ with:
+ dbt_prepare: compile # none | compile | full
</file context>
| - uses: AltimateAI/altimate-code/github/review@main | |
| - uses: AltimateAI/altimate-code/github/review@v0.8.5 |
There was a problem hiding this comment.
Not changing yet: @main is intentional until a release that includes dbt_prepare ships, and we'll re-pin it then.
| const t = thresholds(n) | ||
| if (t.severity) eff.severity = t.severity | ||
| if (t.error_if) eff.error_if = t.error_if | ||
| if (t.severity || t.error_if) path = at.length ? at : ["(all tests)"] |
There was a problem hiding this comment.
P3: path records only the last node containing either setting, so findings misidentify the source when severity and error_if are inherited from different levels. Track a path per threshold.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/test-adequacy.ts, line 117:
<comment>`path` records only the last node containing either setting, so findings misidentify the source when `severity` and `error_if` are inherited from different levels. Track a path per threshold.</comment>
<file context>
@@ -0,0 +1,214 @@
+ const t = thresholds(n)
+ if (t.severity) eff.severity = t.severity
+ if (t.error_if) eff.error_if = t.error_if
+ if (t.severity || t.error_if) path = at.length ? at : ["(all tests)"]
+ }
+ return { ...eff, path }
</file context>
There was a problem hiding this comment.
Follow-up: nit, tracked. Each threshold will track its own path.
| function rawSelectList(sql: string): Set<string> { | ||
| const plain = sql | ||
| .replace(/\{#[\s\S]*?#\}/g, " ") | ||
| .replace(/\{%[\s\S]*?%\}/g, " ") |
There was a problem hiding this comment.
WARNING: rawSelectList blanks Jinja tags but keeps block bodies, so conditional columns become unconditional
{% if %}/{% for %} bodies stay in the text, so select id, amount {% if target.name == 'prod' %}, _loaded_at{% endif %} from t parses as {id, amount, _loaded_at} even when the compiled head renders the false branch. Line 664 now falls back to this raw path whenever either compiled side is missing (e.g. the common "compile the head only, no target-base" setup), so this yields a false columnDelta — a spurious new column that drives on_schema_change, contract, and consumer-list findings, where the old code returned none. Consider treating a model whose select list contains any {% if/for/set %} block as unknown (empty set), and/or using raw only when neither side has compiled SQL.
There was a problem hiding this comment.
Fixed in e77d121. Same fix as r4626962305.
| } | ||
| if (close < 0) continue | ||
| const text = sql.slice(m.index!, close + 1) | ||
| if (!added.some((l) => text.includes(l) || l.includes(text.replace(/\s+/g, " ")))) continue |
There was a problem hiding this comment.
WARNING: "Added line falls inside the window" is a raw substring match, so untouched windows are flagged
text.includes(l) has no line anchoring, so a short or clause-shaped added line elsewhere in the file matches any window that merely contains that text. Example: a pre-existing row_number() over (partition by country, id) (no ORDER BY) is untouched, and the PR adds , id to the select list — text.includes(", id") is true and dedup_no_tiebreaker fires on code the PR never touched, with evidence.line pointing at the pre-existing window. Same for a shared partition by customer_id / order by _loaded_at line. The "a window the PR doesn't touch is not reported" test only covers a line that matches nothing. Match by line range (compute the window's line span and check an added line number falls in it) instead of substring containment.
There was a problem hiding this comment.
Fixed in e77d121. Same fix as r4626962384.
| fi | ||
| if [[ -z "$MB" ]]; then echo "::warning::Could not find the merge-base with $BASE; skipping the base compile."; exit 0; fi | ||
| WT="$RUNNER_TEMP/altimate-review-base" | ||
| git worktree add --detach "$WT" "$MB" >/dev/null |
There was a problem hiding this comment.
WARNING: Unguarded git worktree add aborts the whole review
The base compile at line 227 is best-effort (|| echo "::warning::…"), but git worktree add is a plain command under set -e. If it fails — e.g. a stale $RUNNER_TEMP/altimate-review-base on a reused/self-hosted runner, or a lazy-fetch error for $MB — the step fails and the following "Run dbt PR review" step is skipped, contradicting the documented "a base compile that fails is a warning: the review still runs". Clean $WT first and guard the add (|| { echo "::warning::…"; exit 0; }), and register the cleanup trap before the add.
There was a problem hiding this comment.
Fixed in e77d121. The cleanup trap is set first, and a failed git worktree add now warns and skips the base compile.
| for (const [path, t] of flat(newDoc)) { | ||
| const prev = before.get(path) ?? {} | ||
| const toWarn = t.severity === "warn" && prev.severity !== "warn" | ||
| const newThreshold = t.error_if && t.error_if !== "!= 0" && t.error_if !== prev.error_if |
There was a problem hiding this comment.
WARNING: error_if "loosening" has no direction check and matches the default by exact string
newThreshold = t.error_if && t.error_if !== "!= 0" && t.error_if !== prev.error_if fires for any change, including a tightening: >100 → >10 is reported as "test thresholds loosened … so those tests stop failing the build", the opposite of what happened. It also treats only the exact spelling "!= 0" as the strict default — "!=0", ">0", " > 0 " are all flagged as custom thresholds. Parse the bound and compare direction (only report when the new bound allows more failures), and normalize the strict-default forms.
There was a problem hiding this comment.
Fixed in e77d121. Same fix as r4626962408.
| "equality", | ||
| "equal_rowcount", | ||
| "cardinality_equality", | ||
| "fewer_rows_than", |
There was a problem hiding this comment.
WARNING: Some SELF_COMPARE entries are not "passes by construction"
fewer_rows_than compares row counts with <; against its own ref it always fails (count < count is false), so the emitted body ("passes by construction") is wrong. And expect_table_aggregation_to_equal_other_table is designed to compare two different aggregations: the same model with expression: sum(subtotal) vs compare_expression: sum(net) + sum(tax) is a valid cross-column invariant, yet it is flagged. Only treat aggregation tests as self-comparisons when the expressions match, and drop/adjust fewer_rows_than.
There was a problem hiding this comment.
Fixed in e77d121. Same fix as r4626962291.
| build "$PROJECT" "$PROJECT/target" | ||
| if [[ "$IN_PREPARE" == "full" ]]; then | ||
| echo "Generating catalog.json…" | ||
| ( cd "$PROJECT" && "${DBT[@]}" docs generate --target-path "$PROJECT/target" ${PROFILES[@]+"${PROFILES[@]}"} ) |
There was a problem hiding this comment.
SUGGESTION: dbt docs generate is unguarded in full mode
In full mode this plain command under set -e aborts the step (and skips the review) if the warehouse is unreachable or credentials are missing — exactly the failure the input description warns about. dbt compile at line 207 already produced a usable target/, so degrade instead: … docs generate … || echo "::warning::docs generate failed; continuing without catalog.json.".
There was a problem hiding this comment.
Fixed in e77d121. A failed dbt docs generate now warns and the review continues without catalog.json.
| const rel = root && file.startsWith(root) ? file.slice(root.length) : file | ||
| const parts = rel.split("/").slice(0, -1) | ||
| const modelDirs = layout?.dirs.model ?? ["models"] | ||
| return modelDirs.includes(parts[0]) ? parts.slice(1) : parts |
There was a problem hiding this comment.
SUGGESTION: segmentsOf doesn't normalize multi-segment/relative model-paths
modelDirs.includes(parts[0]) matches only the first segment exactly. A valid model-paths: ["./models"] or ["transform/models"] isn't stripped (parts[0] is models/transform), so inheritedThresholds walks keys that don't exist and misses inherited severity/error_if. roleFromLayout in diff-filter.ts already normalizes these dirs; reuse that (normalize and strip the longest matching configured prefix).
There was a problem hiding this comment.
Follow-up: low priority, tracked. Same as r4626962396.
| [[ -n "${IN_PROFILES:-}" ]] && PROFILES=(--profiles-dir "$(cd "$PROJECT" && cd "$IN_PROFILES" && pwd)") | ||
| build() { # $1 = project dir, $2 = target path | ||
| ( cd "$1" | ||
| [[ -f packages.yml || -f dependencies.yml ]] && "${DBT[@]}" deps ${PROFILES[@]+"${PROFILES[@]}"} |
There was a problem hiding this comment.
SUGGESTION: "${DBT[@]}" expands an array that can be empty
read -ra DBT <<< "$IN_DBT" produces an empty array when dbt_command is empty/whitespace. Under set -u on bash < 4.4 (e.g. macOS 3.2) "${DBT[@]}" is an unbound-variable error; on newer bash the command word vanishes and deps/compile fails obscurely. Validate it ([[ ${#DBT[@]} -gt 0 ]] || { echo "::error::dbt_command is empty"; exit 1; }) and use the file's own ${DBT[@]+"${DBT[@]}"} idiom for consistency.
There was a problem hiding this comment.
Not changing: dbt_command defaults to dbt, and an empty value is a configuration error that fails loudly.
| // A dbt_project.yml change that loosens test thresholds for a whole path. | ||
| for (const file of reviewable.filter((f) => f.kind === "project_config" && /(^|\/)dbt_project\.ya?ml$/.test(f.path))) { | ||
| if (file.status !== "modified") continue | ||
| const [oldContent, newContent] = await Promise.all([getContent?.(file.path, "old"), getContent?.(file.path, "new")]) |
There was a problem hiding this comment.
SUGGESTION: Re-fetch of dbt_project.yml is redundant and unguarded
When the PR modifies dbt_project.yml, this re-reads the head content that projectYml already holds (fetched with .catch at line 1671), and unlike that read it has no .catch. A rejecting getContent (valid custom/unit-test resolver) aborts the entire runReview instead of degrading this lane. Reuse projectYml when file.path === projectYmlPath, and guard the old-side read.
There was a problem hiding this comment.
Follow-up: low priority, tracked. We'll reuse projectYml and add a .catch.
| const out = detectTestAdequacy({ file, newContent: newTests(), projectYml: project }) | ||
| expect(rule(out)).toEqual(["test-inherits-warn"]) | ||
| expect(out[0].body).toContain("`shop` → `marts`") | ||
| const own = newTests(":\n config:\n severity: error").replace("- not_null:", "- not_null:") |
There was a problem hiding this comment.
SUGGESTION: No-op .replace and a silent negative assertion
.replace("- not_null:", "- not_null:") replaces a string with itself (dead code — the interpolated argument already supplies the colon). Also, if a future indentation slip made this YAML invalid, parse() returns undefined, detectTestAdequacy returns [], and line 68 would pass silently. Drop the .replace and assert the parsed shape so the negative assertion is trustworthy.
There was a problem hiding this comment.
Follow-up: nit, tracked. We'll remove the no-op .replace.
Code Review SummaryStatus: 14 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (9 files)
Fix these issues in Kilo Cloud Previous Review Summaries (2 snapshots, latest commit e77d121)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit e77d121)Status: 16 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (10 files)
Fix these issues in Kilo Cloud Previous review (commit 7da6850)Status: 12 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (10 files)
Reviewed by deepseek-v4.1-flash-1017 · Input: 170.3K · Output: 60.4K · Cached: 3M Review guidance: REVIEW.md from base branch |
hkaltai
left a comment
There was a problem hiding this comment.
Verdict: changes requested (one MAJOR). Panel: Claude, Codex, and a lead review. Counts: 0 CRITICAL, 1 MAJOR, 5 MINOR, 4 NIT.
The opt-in dbt_prepare, test-adequacy rules, parse-only column sets and multi-line dedup windows are well structured and tested (targeted suites pass, inputs reach the script only through env, worktree cleanup is trapped). The MAJOR is that the new dedup-window attribution uses a substring match and can flag windows the PR never touched (a regression versus the previous added-line-only behaviour). The MINORs are precision edges in the new threshold and select-list rules and in the action's head handling.
Scope: no linked ticket, so scope vs ticket cannot be verified; the diff matches the PR description. The example workflow rename is a small related cleanup.
Nits
packages/opencode/src/altimate/review/orchestrate.ts(comment beforerawSelectListuse): says "CI that only runsdbt parse" but the fallback also runs when the head is compiled and the base is not. Update the comment.orchestrate.ts:684: the comment "A Jinja expression inside an item ... leaves its alias intact" sits after the guard it explains; move it aboveif (!names || names.includes("__jinja__")).test-adequacy.ts:162const own = .../loose:loosemeans "tests without their own threshold";inheritingis clearer.thresholds()also returnsseverity, sotestConfigLevelswould fit; rule idtest-thresholds-loosenedis also emitted for tightenings.github/review/action.yml:196read -ra DBT <<< "$IN_DBT": ignores quoting and an emptydbt_commandyields an empty array (script would runcompile). Validate non-empty. Alsotest-adequacy.tsown()treats a lonewarn_ifas overriding an inheritedseverity: warn.
| } | ||
| if (close < 0) continue | ||
| const text = sql.slice(m.index!, close + 1) | ||
| if (!added.some((l) => text.includes(l) || l.includes(text.replace(/\s+/g, " ")))) continue |
There was a problem hiding this comment.
[MAJOR] Dedup window is attributed to the PR by substring match, so untouched windows are flagged
if (!added.some((l) => text.includes(l) || l.includes(text.replace(/\s+/g, " ")))) continuetext.includes(l) is a substring test, not a position test. Scenario (reproduced by two reviewers): the model already has an unchanged row_number() over (partition by customer_id) as rn; the PR only adds a customer_id to a group by elsewhere. The added line customer_id is a substring of the window, so dedup_no_tiebreaker fires (high confidence) on code the PR did not touch. Before this PR only added lines that themselves contained row_number were considered. The existing "untouched window" test uses select 1 as unrelated, which can never match.
Fix: map each window's [m.index, close] to new-file line numbers and intersect with the added hunk line numbers (or at minimum require the added line to contain row_number/partition by/order by). Add a test with an unrelated added bare identifier that also appears in the window. Also normalise whitespace on both sides of the second comparison.
There was a problem hiding this comment.
Thanks, this was a real regression. The main path was already moved to positions in e77d121 ("dedup window touch"): each window's [m.index, close] span is mapped to head line numbers and intersected with the added lines from the hunk headers, so your group by customer_id scenario no longer fires when the diff has headers. The text fallback (diffs without hunk headers) still had the substring test, so the new commit (d9caa1597c) "address review feedback: dedup window text fallback, …" restricts it to added lines that themselves carry window syntax (row_number, over, partition by, order by) and normalises whitespace on both sides. Added a test where the added line is a bare customer_id that also appears in the window (not flagged), plus the positive case with differing whitespace.
| // No compiled SQL (CI that only runs `dbt parse`): the raw model's own SELECT | ||
| // list, when it names its columns. Columns produced inside a macro stay unknown. | ||
| if (!newSql) return none | ||
| const current = rawSelectList(newSql) |
There was a problem hiding this comment.
[MINOR] Parse-only select list treated as complete when items are dropped
const current = rawSelectList(newSql)selectListNames is deliberately conservative (items it cannot name contribute nothing). Previously that only supplemented lineage; here it is the whole answer. select id, amount::numeric(18,2) yields ["id"], so going from amount to a cast is seen as a removed column, and with exposures/contracts this produces a high-confidence "removed columns" warning. The branch also now runs when the head is compiled but the base is not.
Fix: return unknown (empty set) when any non-empty select item produced no name. Related: stripping {% if %} / {% for %} tags keeps their bodies, so {% if var('x', false) %}, region {% endif %} is counted as an unconditional added column and can trigger the on_schema_change='fail' warning; treat projection-affecting control blocks as unknown.
There was a problem hiding this comment.
Agreed that the raw list is the whole answer here, not a supplement. selectListNames now takes a complete option, and the parse-only path uses it: any non-empty item without a name of its own (your amount::numeric(18,2)) makes the list unknown, so no removed/added columns are reported. The {% if %} / {% for %} case was handled in e77d121 (a tag becomes its own __jinja__ item, which marks the list unknown). I also reworded the comment to cover the compiled-head/uncompiled-base case and moved the Jinja note above the guard it explains. Test: a column turned into a cast plus an added column produces no on_schema_change finding.
| for (const [path, t] of flat(newDoc)) { | ||
| const prev = before.get(path) ?? {} | ||
| const toWarn = t.severity === "warn" && prev.severity !== "warn" | ||
| const newThreshold = t.error_if && t.error_if !== "!= 0" && t.error_if !== prev.error_if |
There was a problem hiding this comment.
[MINOR] Any changed error_if other than != 0 is reported as "loosened"
const newThreshold = t.error_if && t.error_if !== "!= 0" && t.error_if !== prev.error_ifChanging '>1000' to '>10' (a tightening) or adding the default-equivalent '>0' / '!=0' is reported as "test thresholds loosened". The same literal comparison is used for the inherited-threshold rule. Also, the comparison is per node rather than effective value: with root +severity: warn, deleting marts: +severity: error loosens every marts test but reports nothing, and a redundant +severity: warn under an already-warn parent is reported.
Fix: normalise op N and compare numerically, and compare inheritedThresholds(old, path) with inheritedThresholds(new, path) across the union of paths.
There was a problem hiding this comment.
The numeric part landed in e77d121: error_if is normalised to the number of tolerated failing rows, so '>1000' to '>10' is a tightening and '>0' / '!=0' / '>=1' count as the strict default. The per-node comparison you pointed out is fixed in the new commit (d9caa1597c): loosening is now judged on effective values, walking both versions of data_tests: across the union of their paths, and only the topmost loosened path is reported. Deleting marts: +severity: error under a root +severity: warn is now reported for shop → marts, and a redundant +severity: warn under a warn parent is not. Tests cover both, plus a root-level change reported once rather than per folder.
| const project = parse(input.projectYml) | ||
| if (project && added.length) { | ||
| const inherited = inheritedThresholds(project, segmentsOf(file.path, input.layout)) | ||
| const own = (t: TestRef) => { |
There was a problem hiding this comment.
[MINOR] Any local threshold suppresses all inherited checks
const own = (t: TestRef) => {With project-level severity: warn, adding only config: {warn_if: '>10'} or error_if: '!= 0' returns no finding, yet severity is still inherited (dbt resolves these as separate properties).
Fix: merge local and inherited properties individually and inspect the effective config.
There was a problem hiding this comment.
Right, dbt resolves these as separate properties. The new commit (d9caa1597c) checks severity and error_if individually: a test inherits severity: warn unless it sets its own severity, so a local warn_if: '>10' or error_if: '!= 0' alone is still reported. It inherits a loose error_if unless it sets its own error_if (and its effective severity isn't warn). This also covers the nit about a lone warn_if overriding an inherited severity: warn. I renamed loose to inheritingWarn / inheritingThreshold following your naming suggestion. Tests cover each combination.
| const SELF_COMPARE = new Set([ | ||
| "equality", | ||
| "equal_rowcount", | ||
| "cardinality_equality", |
There was a problem hiding this comment.
[MINOR] cardinality_equality / aggregation tests against the same model are not always self-comparisons
"cardinality_equality",cardinality_equality with to: ref('m'), field: b on column a compares two different columns' distinct counts, a real check, but is reported as comparing the model with itself. Same for expect_table_aggregation_to_equal_other_table with different expressions. (fewer_rows_than against itself fails rather than passes, so "passes by construction" is also inaccurate there.)
Fix: for these tests require the field/expression to match as the relationships branch does, and drop fewer_rows_than. Separately, segmentsOf strips only model dirs, so seeds/_s.yml walks data_tests: shop: seeds: as if seeds were a folder; strip seed/snapshot dirs too.
There was a problem hiding this comment.
Thanks. e77d121 dropped fewer_rows_than and limited expect_table_aggregation_to_equal_other_table to matching expressions. The new commit (d9caa1597c) adds the missing cardinality_equality case: it is flagged as a self-comparison only when field is the same column, so column a against field: b on the same model counts as a real check. segmentsOf now strips any resource dir (seeds, snapshots, tests as well as models, from the layout when one is known), so seeds/_s.yml no longer walks data_tests: shop: seeds: as if seeds were a folder. Tests added for both.
| "${DBT[@]}" compile --target-path "$2" ${PROFILES[@]+"${PROFILES[@]}"} ) | ||
| } | ||
| echo "Compiling the head…" | ||
| build "$PROJECT" "$PROJECT/target" |
There was a problem hiding this comment.
[MINOR] dbt_prepare compiles the checkout, ignoring the head input; head-side failures abort the job
build "$PROJECT" "$PROJECT/target"- The merge-base is computed against literal
HEADand the head compile uses the checkout, but the review step passes--head "$IN_HEAD"when set. Withhead: Awhile B is checked out, A's diff is reviewed using B's compiled SQL and equivalence can be wrongly reported. Prepare against the selected head or reject a mismatch. - Under
set -e, a failed headdeps/compile/docs generatefails the step so the review never runs (even inmode: comment), while the input description and docs (docs/docs/usage/dbt-pr-review.md:224) say compile "usually works offline". Many adapters need a connection to compile. Either degrade to a warning like the base side, or document that it fails the job and drop the offline claim. - Minor:
manifest_pathis not honoured (head always goes to$PROJECT/target), and a runtime-generatedprofiles.ymlin the project dir is absent from the base worktree, so the base compile quietly fails.
There was a problem hiding this comment.
All three points are addressed in the new commit's prepare-step changes (d9caa1597c). (1) When head is set and doesn't resolve to the checked-out commit, preparation is skipped with a warning, so we never review one commit against another commit's compiled SQL. (2) A failed head compile is now a warning and the review still runs (e77d121 had already done this for docs generate and the base checkout). The input description and the docs no longer claim compile "usually works offline"; they now say many adapters connect while compiling. (3) The head is compiled into the directory of manifest_path, and a runtime-generated profiles.yml in the project dir is copied into the base worktree when no --profiles-dir is given. An empty dbt_command is also rejected now (from your nit). The action test asserts each of these.
…erge-base and catalog artifacts - new action inputs: `dbt_prepare` (`none` default | `compile` | `full`), `dbt_command`, `dbt_profiles_dir` - `compile` runs `dbt compile` on the head and compiles the PR's merge-base into `target-base/` in a temporary worktree (fetching and deepening a shallow checkout when needed); `full` adds `dbt docs generate` - a base compile that fails is a warning: the review still runs and marks those models unverified - inputs reach the script only through env, never interpolated - docs: the manual base-compile snippet now compiles the merge-base, not the base branch's tip - the example workflow is renamed `examples/dbt-pr-review.yml` (was named after one internal repo) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ows split across lines are read whole Found by replaying the PRs behind known escaped defects: - without compiled SQL, output columns come from the raw model's final SELECT (Jinja blanked; a Jinja expression in the list makes it unknown). `on_schema_change='fail'` with new columns now fires on the two real escapes it targets, in a parse-only CI setup - `row_number() over (…)` windows are read from the whole file, so an `ORDER BY` on its own line is seen: the "dedup has no ORDER BY" rule no longer fires falsely on multi-line windows, and the load-timestamp rule sees them. A window counts only when an added line falls inside it Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nherited warn/thresholds, loosened project thresholds - a test added or changed in the PR that compares a model with itself (`dbt_utils.equality`/`equal_rowcount`/… against its own `ref`, or a `relationships` test to its own column) passes by construction (warning) - new tests that inherit `severity: warn` or an `error_if` threshold from `dbt_project.yml` (walked from the top through the project name and folders, deeper wins) are reported once per yml (suggestion) - a `dbt_project.yml` change that turns a folder's tests to `warn` or sets an `error_if` threshold is reported (warning) - reads dbt 1.8+ `arguments:` too; all findings are catalog origin (never block) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…window touch, self-compare tests, `error_if` loosening, prepare-step failures
- `orchestrate.ts`: a `{% if %}` / `{% for %}` tag inside a raw model's select
list makes the list unknown, so conditional columns no longer read as added
or removed in the parse-only column delta
- `dbt-patterns.ts`: a dedup window counts as changed only when an added line
falls inside its line span (head line numbers from the hunk headers), not
when an added line's text happens to occur in it
- `test-adequacy.ts`: drop `fewer_rows_than` (it fails against itself, it
doesn't pass by construction); an aggregation comparison on the same model
is flagged only when both expressions match
- `test-adequacy.ts`: `error_if` is reported as loosened only when the new bound
tolerates more failing rows; `> 0` / `!=0` / `>= 1` count as the strict default
- `action.yml`: a failed `dbt docs generate` or base checkout warns instead of
failing the step; a stale `target-base/` is removed before the base compile
- tests for each
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fe31dc6 to
9625cfd
Compare
7da6850 to
e77d121
Compare
| trap 'git worktree remove --force "$WT" >/dev/null 2>&1 || true' EXIT | ||
| git worktree add --detach "$WT" "$MB" >/dev/null || { echo "::warning::Could not check out the base; skipping the base compile."; exit 0; } | ||
| # A stale target-base/ from an earlier run must not stand in for a failed compile. | ||
| rm -rf "$PROJECT/target-base" |
There was a problem hiding this comment.
WARNING: The stale target-base/ cleanup runs only after the guards that can skip the base compile
When BASE is empty, the merge-base can't be found, or git worktree add fails, the script exits 0 before this line, so a target-base/ left by an earlier run survives. makeCompiledResolver reads target-base/compiled whenever it exists (compiled.ts:97), so on a cached or self-hosted workspace the head is compared against the previous run's base, and because the old side still resolves, those models are not even listed as unverified. Move the cleanup above the BASE/MB/worktree guards, and make it non-fatal (rm -rf ... || { echo "::warning::..."; exit 0; }) so an rm failure cannot abort the step and skip the review.
| } | ||
|
|
||
| /** Head-side line numbers of the added lines; undefined when the diff has no hunk headers. */ | ||
| function addedLineNumbers(diff: string | undefined): Set<number> | undefined { |
There was a problem hiding this comment.
SUGGESTION: This hunk walker duplicates the exported commentableLines
commentableLines (orchestrate.ts:268) already walks @@ hunks with the same regex and the same right counter to compute head-side line numbers; this copy already differs (it does not drop the diff's trailing-newline row) and the two will drift. dbt-patterns cannot import orchestrate without a cycle (the file already keeps a local modelNameFromPath for exactly that reason), so extract the walker into a shared module (e.g. diff-filter.ts) and take the added-line set from it, or pass the numbers in through Ctx.
| */ | ||
| function dedupWindows(c: Ctx): Array<{ spec: string; text: string }> { | ||
| // Comments blanked with newlines kept, so offsets map to head line numbers. | ||
| const sql = c.newSql.replace(/\/\*[\s\S]*?\*\/|--[^\n]*|\{#[\s\S]*?#\}/g, (m) => m.replace(/[^\n]/g, " ")) |
There was a problem hiding this comment.
SUGGESTION: This comment blanking is not quote-aware; blankComments() already does this
A -- inside a string literal blanks the rest of the line: select '--' as sep, row_number() over (partition by id) as rn loses the window entirely (a missed dedup finding), and a /* in a literal can blank a multi-line span. blankComments() in select-list.ts is the existing length-preserving, quote-aware helper (it skips '...', "..." and backticked identifiers), so reuse it here and keep only the {# ... #} alternative.
| const run = prep.run! | ||
| expect(run).toMatch(/docs generate[^\n]*\) \|\|\n\s*echo "::warning::/) | ||
| expect(run).toMatch(/git worktree add[^\n]*\|\| \{ echo "::warning::/) | ||
| expect(run.indexOf('rm -rf "$PROJECT/target-base"')).toBeLessThan(run.indexOf('build "$WT/$REL"')) |
There was a problem hiding this comment.
SUGGESTION: This ordering assertion passes when the cleanup line is absent
run.indexOf('rm -rf ...') is -1 when the line is deleted, and -1 < indexOf('build ...') is true, so the test still passes without the rm -rf "$PROJECT/target-base" line it is meant to guard. Assert the cleanup line is present (e.g. expect(run).toContain('rm -rf "$PROJECT/target-base"')) before comparing indexes.
…plete raw select lists, effective test thresholds, `cardinality_equality` field, prepare-step head handling - `dbt-patterns.ts`: without hunk headers, a dedup window counts as touched only by an added line that holds window syntax (`row_number`, `over`, `partition by`, `order by`), with whitespace normalised on both sides; a bare identifier that also occurs in the window no longer flags it - `select-list.ts` / `orchestrate.ts`: the parse-only column delta treats the raw select list as unknown when any item has no name of its own (e.g. `amount::numeric(18,2)`); comments updated for the compiled-on-one-side case - `test-adequacy.ts`: `severity` and `error_if` are resolved separately, so a local `warn_if` / `error_if` no longer hides an inherited `severity: warn` - `test-adequacy.ts`: threshold loosening compares effective values across the union of config paths in both versions, reports only the topmost loosened path, ignores redundant overrides and catches deleted stricter overrides - `test-adequacy.ts`: `cardinality_equality` against the same model is flagged only when `field` is the same column; seed/snapshot yml paths strip their resource dir before walking `data_tests:` - `action.yml`: `dbt_prepare` skips when `head` is not the checkout, rejects an empty `dbt_command`, warns instead of failing on a head compile error, builds the head into the `manifest_path` directory, and copies a runtime `profiles.yml` into the base worktree; docs drop the offline claim - tests for each Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for the thorough pass, Harish. The MAJOR (dedup attribution) is fixed: positions since e77d121, plus a syntax-gated text fallback in the new commit ( |
There was a problem hiding this comment.
1 existing issue remains and 5 new issues found across 9 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/test/altimate/review-action.test.ts">
<violation number="1" location="packages/opencode/test/altimate/review-action.test.ts:44">
P2: This ordering check passes when the head check is missing: `indexOf` returns -1 for an absent string, and -1 is less than any build index. Assert presence first so a removed `$IN_HEAD^{commit}` check fails the test.</violation>
</file>
<file name="packages/opencode/src/altimate/review/select-list.ts">
<violation number="1" location="packages/opencode/src/altimate/review/select-list.ts:98">
P3: `complete` treats valid quoted aliases such as `1 AS "null"` as unnamed and makes the whole raw select list unknown, so parse-only column checks silently skip it. Preserve explicitly written aliases even when they match the reserved-word guard.</violation>
</file>
<file name="github/review/action.yml">
<violation number="1" location="github/review/action.yml:204">
P2: A failed head compile leaves existing or partial artifacts in place before `exit 0`, so the next review can use stale SQL and manifest data instead of treating the head as unverified. Clear the head artifact set safely on this failure path before continuing.</violation>
<violation number="2" location="github/review/action.yml:220">
P2: Custom `manifest_path` values compile SQL into the manifest’s directory, but the review still searches the hard-coded `target/compiled` directory. A setting such as `build/manifest.json` therefore causes `dbt_prepare` to produce artifacts the engine cannot find; pass the prepared target directory through to the resolver or keep preparation and resolver paths aligned.</violation>
<violation number="3" location="github/review/action.yml:248">
P2: This unguarded `cp` can fail when `project_dir` does not exist at the merge-base, causing the whole preparation step to fail before the base compile’s warning fallback. Guard the copy or turn its failure into a warning so the review still runs.</violation>
</file>
Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.
View guided diff | Turn on auto-fix | Re-trigger cubic
| expect(prep.env?.IN_MANIFEST).toBe("${{ inputs.manifest_path }}") | ||
| expect(run).toMatch(/\$\{#DBT\[@\]\} -eq 0 \]\]; then echo "::error::/) | ||
| // The head check runs before anything is compiled. | ||
| expect(run.indexOf('"$IN_HEAD^{commit}"')).toBeLessThan(run.indexOf('build "$PROJECT"')) |
There was a problem hiding this comment.
P2: This ordering check passes when the head check is missing: indexOf returns -1 for an absent string, and -1 is less than any build index. Assert presence first so a removed $IN_HEAD^{commit} check fails the test.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/review-action.test.ts, line 44:
<comment>This ordering check passes when the head check is missing: `indexOf` returns -1 for an absent string, and -1 is less than any build index. Assert presence first so a removed `$IN_HEAD^{commit}` check fails the test.</comment>
<file context>
@@ -30,4 +30,21 @@ describe("action: dbt_prepare", () => {
+ expect(prep.env?.IN_MANIFEST).toBe("${{ inputs.manifest_path }}")
+ expect(run).toMatch(/\$\{#DBT\[@\]\} -eq 0 \]\]; then echo "::error::/)
+ // The head check runs before anything is compiled.
+ expect(run.indexOf('"$IN_HEAD^{commit}"')).toBeLessThan(run.indexOf('build "$PROJECT"'))
+ expect(run).toMatch(/if ! build "\$PROJECT" "\$TARGET"; then\n\s*echo "::warning::/)
+ expect(run).toContain('TARGET="$(dirname "$MANIFEST")"')
</file context>
| expect(run.indexOf('"$IN_HEAD^{commit}"')).toBeLessThan(run.indexOf('build "$PROJECT"')) | |
| const headCheck = run.indexOf('"$IN_HEAD^{commit}"') | |
| expect(headCheck).toBeGreaterThanOrEqual(0) | |
| expect(headCheck).toBeLessThan(run.indexOf('build "$PROJECT"')) |
| # A profiles.yml generated into the project dir at runtime is untracked, so | ||
| # the base worktree lacks it. | ||
| if [[ ${#PROFILES[@]} -eq 0 && -f "$PROJECT/profiles.yml" && ! -f "$WT/$REL/profiles.yml" ]]; then | ||
| cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml" |
There was a problem hiding this comment.
P2: This unguarded cp can fail when project_dir does not exist at the merge-base, causing the whole preparation step to fail before the base compile’s warning fallback. Guard the copy or turn its failure into a warning so the review still runs.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At github/review/action.yml, line 248:
<comment>This unguarded `cp` can fail when `project_dir` does not exist at the merge-base, causing the whole preparation step to fail before the base compile’s warning fallback. Guard the copy or turn its failure into a warning so the review still runs.</comment>
<file context>
@@ -226,6 +242,11 @@ runs:
+ # A profiles.yml generated into the project dir at runtime is untracked, so
+ # the base worktree lacks it.
+ if [[ ${#PROFILES[@]} -eq 0 && -f "$PROJECT/profiles.yml" && ! -f "$WT/$REL/profiles.yml" ]]; then
+ cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml"
+ fi
echo "Compiling the base ($BASE @ ${MB:0:12})…"
</file context>
| cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml" | |
| cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml" || echo "::warning::Could not copy profiles.yml to the base worktree; base compile may fail." |
| # against another commit's compiled SQL, so skip preparing instead. | ||
| if [[ -n "${IN_HEAD:-}" && "$(git rev-parse --verify -q "$IN_HEAD^{commit}" || true)" != "$(git rev-parse HEAD)" ]]; then | ||
| echo "::warning::head ($IN_HEAD) is not the checked-out commit; skipping dbt_prepare. Check out the head to build its artifacts." | ||
| exit 0 |
There was a problem hiding this comment.
P2: A failed head compile leaves existing or partial artifacts in place before exit 0, so the next review can use stale SQL and manifest data instead of treating the head as unverified. Clear the head artifact set safely on this failure path before continuing.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At github/review/action.yml, line 204:
<comment>A failed head compile leaves existing or partial artifacts in place before `exit 0`, so the next review can use stale SQL and manifest data instead of treating the head as unverified. Clear the head artifact set safely on this failure path before continuing.</comment>
<file context>
@@ -188,12 +188,21 @@ runs:
+ # against another commit's compiled SQL, so skip preparing instead.
+ if [[ -n "${IN_HEAD:-}" && "$(git rev-parse --verify -q "$IN_HEAD^{commit}" || true)" != "$(git rev-parse HEAD)" ]]; then
+ echo "::warning::head ($IN_HEAD) is not the checked-out commit; skipping dbt_prepare. Check out the head to build its artifacts."
+ exit 0
+ fi
PROJECT="$(cd "${IN_PROJECT_DIR:-.}" && pwd)"
</file context>
| [[ "$MANIFEST" == /* ]] || MANIFEST="$PROJECT/$MANIFEST" | ||
| TARGET="$(dirname "$MANIFEST")" | ||
| echo "Compiling the head…" | ||
| if ! build "$PROJECT" "$TARGET"; then |
There was a problem hiding this comment.
P2: Custom manifest_path values compile SQL into the manifest’s directory, but the review still searches the hard-coded target/compiled directory. A setting such as build/manifest.json therefore causes dbt_prepare to produce artifacts the engine cannot find; pass the prepared target directory through to the resolver or keep preparation and resolver paths aligned.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At github/review/action.yml, line 220:
<comment>Custom `manifest_path` values compile SQL into the manifest’s directory, but the review still searches the hard-coded `target/compiled` directory. A setting such as `build/manifest.json` therefore causes `dbt_prepare` to produce artifacts the engine cannot find; pass the prepared target directory through to the resolver or keep preparation and resolver paths aligned.</comment>
<file context>
@@ -203,11 +212,18 @@ runs:
+ TARGET="$(dirname "$MANIFEST")"
echo "Compiling the head…"
- build "$PROJECT" "$PROJECT/target"
+ if ! build "$PROJECT" "$TARGET"; then
+ echo "::warning::The head compile failed (dbt may need a warehouse connection to compile); the review runs without compiled artifacts."
+ exit 0
</file context>
| if (m && before && (m[1] || /([A-Za-z0-9_$"`\]')]|\bend)$/i.test(before))) name = unquote(m[2]) | ||
| } | ||
| if (name && (IDENT.test(name) || /^[^"`]+$/.test(name)) && !/^(end|null|true|false)$/i.test(name)) names.push(name.toLowerCase()) | ||
| else if (opts.complete) return null |
There was a problem hiding this comment.
P3: complete treats valid quoted aliases such as 1 AS "null" as unnamed and makes the whole raw select list unknown, so parse-only column checks silently skip it. Preserve explicitly written aliases even when they match the reserved-word guard.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/select-list.ts, line 98:
<comment>`complete` treats valid quoted aliases such as `1 AS "null"` as unnamed and makes the whole raw select list unknown, so parse-only column checks silently skip it. Preserve explicitly written aliases even when they match the reserved-word guard.</comment>
<file context>
@@ -93,6 +95,7 @@ export function selectListNames(sql: string): string[] | null {
if (m && before && (m[1] || /([A-Za-z0-9_$"`\]')]|\bend)$/i.test(before))) name = unquote(m[2])
}
if (name && (IDENT.test(name) || /^[^"`]+$/.test(name)) && !/^(end|null|true|false)$/i.test(name)) names.push(name.toLowerCase())
+ else if (opts.complete) return null
}
return names
</file context>
| # A profiles.yml generated into the project dir at runtime is untracked, so | ||
| # the base worktree lacks it. | ||
| if [[ ${#PROFILES[@]} -eq 0 && -f "$PROJECT/profiles.yml" && ! -f "$WT/$REL/profiles.yml" ]]; then | ||
| cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml" |
There was a problem hiding this comment.
WARNING: Unguarded cp under set -e can fail the whole prepare step and skip the review
Every other failure on this path degrades to a warning, but this cp does not. If the dbt project directory does not exist at the merge-base (a PR that adds or renames project_dir, e.g. a new analytics/ folder, or a rename), or the copy fails for any reason, set -e exits the step non-zero and the following "Run dbt PR review" step never runs - contradicting the input description and docs/docs/usage/dbt-pr-review.md ("A head or base compile that fails is a warning: the review still runs"). Add a -d "$WT/$REL" guard to the condition and make the copy non-fatal:
| cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml" | |
| cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml" || echo "::warning::Could not copy profiles.yml into the base worktree." |
| for (const path of paths.values()) { | ||
| const hit = loosened(path) | ||
| // Report only the topmost loosened path, not each folder below it. | ||
| if (!hit || (path.length && loosened(path.slice(0, -1)))) continue |
There was a problem hiding this comment.
WARNING: Ancestor suppression hides an independent loosening deeper in the tree
loosened(path.slice(0, -1)) skips a child whenever any ancestor loosened, even when the child loosens for a different reason and its tests still gate the build. Example: old shop -> marts -> core has +severity: error, +error_if: ">10"; the PR adds +severity: warn under shop and raises core's +error_if to ">100" while core keeps +severity: error. loosened(["shop", "marts"]) is true because warn is inherited, so the core error_if loosening (10 -> 100 tolerated failures) is dropped, and the emitted shop finding claims warn now "applies to every data test under shop" - false for core. Suppress a child only when its effective severity did not reset the ancestor's loosened value (or when its why matches the ancestor's).
| `They inherit \`severity: warn\` from \`dbt_project.yml\`${where}, so failures are reported but never fail \`dbt build\`. ` + | ||
| "If a test should gate the build, set `config: {severity: error}` on it."), | ||
| ) | ||
| else if (inheritingThreshold.length && inherited.error_if && allowedFailures(inherited.error_if) !== 0) |
There was a problem hiding this comment.
SUGGESTION: The else if still drops the inherited-threshold finding when any other added test inherits warn
inheritingWarn and inheritingThreshold are now independent sets, but only one finding can be emitted. With data_tests: shop -> marts: {+severity: warn, +error_if: ">100"} and a PR that adds not_null (inherits warn) plus unique with config: {severity: error}, inheritingWarn is non-empty so this branch is skipped and unique's inherited error_if: >100 - which still gates the build - is never reported. Two independent ifs (instead of else if) would cover both sets.
| if (m && before && (m[1] || /([A-Za-z0-9_$"`\]')]|\bend)$/i.test(before))) name = unquote(m[2]) | ||
| } | ||
| if (name && (IDENT.test(name) || /^[^"`]+$/.test(name)) && !/^(end|null|true|false)$/i.test(name)) names.push(name.toLowerCase()) | ||
| else if (opts.complete) return null |
There was a problem hiding this comment.
SUGGESTION: complete lets one unparsed identifier hide the whole list (backticked columns, quoted prefixes)
The bare matcher at line 88 accepts only "quoted" or unquoted identifiers, and the fallback alias matcher at line 91 requires whitespace before the token, so select `order_id`, `amount` from {{ ref('src') }} and select "t"."amount" from t produce no name for those items. Under the new complete mode that turns the entire list into null, so rawSelectList returns an empty set and the parse-only on_schema_change rule silently goes dark for BigQuery/Databricks models that backtick-quote columns. Allow backticked/bracketed identifiers (and quoted qualifiers) in the bare matcher - unquote already handles them.
| # The head goes where the review reads the manifest (`manifest_path`). | ||
| MANIFEST="${IN_MANIFEST:-target/manifest.json}" | ||
| [[ "$MANIFEST" == /* ]] || MANIFEST="$PROJECT/$MANIFEST" | ||
| TARGET="$(dirname "$MANIFEST")" |
There was a problem hiding this comment.
SUGGESTION: manifest_path now sets the head compile target, but the review still reads compiled SQL from target/compiled
TARGET moves dbt compile's manifest and compiled/ to dirname(manifest_path), but makeCompiledResolver defaults headDir to "target/compiled" (packages/opencode/src/altimate/review/compiled.ts:96) and run.ts:407 never passes headDir. With manifest_path: build/manifest.json the manifest and catalog are found, yet every compiled-SQL lookup misses - the review falls back to raw Jinja, or reads a target/compiled left by an earlier run, instead of the SQL this step just built. Either derive the resolver's headDir from dirname(manifestPath) in run.ts, or document that dbt_prepare requires the default target/manifest.json.
Issue for this PR
Part of #1792
Stacked on #1725 (← #1724 ← #1723 ← … ← #1719); merge in that order. Draft.
Type of change
What does this PR do?
This closes the remaining acceptance-criteria gaps in the review stack. Two of the fixes came from replaying the PRs behind known escaped defects.
dbt_prepare). Defaultnone, so nothing changes for existing users.compilerunsdbt compileon the head, then compiles the PR's merge-base intotarget-base/in a temporary worktree. A shallow checkout is fetched and deepened when needed.fullalso runsdbt docs generate.equality/equal_rowcountagainst its ownref, or arelationshipstest to its own column) passes by construction;severity: warnor anerror_ifthreshold fromdbt_project.ymlare reported;dbt_project.ymlchange that loosens those thresholds for a folder is reported.dbt parse-only CI. Without compiled SQL, output columns come from the raw model's final SELECT; a Jinja expression in the list makes it unknown. In a parse-only replay,on_schema_change='fail'with new columns now fires on the two real escapes it targets.order byis on its own line, and the load-timestamp rule sees such windows.How did you verify your code works?
review-action.test.ts:dbt_prepareis opt-in, reads inputs only from env, and builds intotarget-base/.review-test-adequacy.test.ts: self-comparison, inheritedwarn/error_if, loosened thresholds, dbt 1.8+arguments:.review-incremental-semantics.test.ts: multi-line windows, raw-SQL column delta.bun typecheckis clean, and the real-world corpus is unchanged (15/15 caught, 0/5 false positives).dbt_preparerun locally against a real dbt 1.10 DuckDB project. The head compiled intotarget/, the merge-base compiled intotarget-base/(the base still has the removed column), and the temporary worktree was cleaned up.on_schema_changenow catches two of them (two escapes each).Cost side effects
dbt_prepare: compileadds onedbt compileper side of the PR in CI, andfulladdsdbt docs generate(warehouse queries for the catalog). Both are opt-in; the defaultnoneadds nothing. No model calls.Screenshots / recordings
N/A (CLI / action).
Checklist
Changes since review:
d9caa1597caddresses the second-round review feedback: dedup-window touch needs window syntax on an added line, unnamed raw select items make the column set unknown,severity/error_ifresolved separately, threshold loosening compared on effective values,cardinality_equalityfield, prepare-step head handling.🤖 Generated with Claude Code
Summary by cubic
Closes the remaining acceptance-criteria gaps in the dbt review stack: adds an opt-in
dbt_prepareaction input for building the artifacts the full review needs, and fixes two classes of missed findings — data tests that can't do their job, and column-set rules that skipped work in parse-only CI.New
dbt_prepareaction inputnone;compilerunsdbt compileon the head and the PR's merge-base intotarget-base/, andfullalso runsdbt docs generate.headthat isn't the checkout skips preparation.target-base/is removed before the base compile, and the runtimeprofiles.ymlis copied into the base worktree.Review gap fixes
equality/equal_rowcount/relationships,cardinality_equalityon its own column, orexpect_table_aggregation_to_equal_other_tablewith matching expressions, plus tests that inheritseverity: warnor anerror_ifthreshold fromdbt_project.yml.dbt_project.ymlchanges that loosen those thresholds, counting only bounds that tolerate more failing rows.on_schema_change='fail'fires in parse-only CI; a{% if %}tag, Jinja expression, or unnamed item (e.g. a cast) in the list leaves it unknown.ORDER BYno longer triggers a false "no ORDER BY".Written for commit d9caa15. Summary will update on new commits.