Repository navigation
feat(review): consumers outside dbt, engine-aware integer division, opt-in contract and column-list checks - #1724
RaghwendraSingh wants to merge 4 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
There was a problem hiding this comment.
11 issues found across 12 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/src/altimate/review/adapter-capabilities.ts">
<violation number="1" location="packages/opencode/src/altimate/review/adapter-capabilities.ts:75">
P3: `capabilityFor` reads from a plain object, so inherited keys like `constructor` or `toString` return a non-undefined value for an unmapped dialect. That breaks the "unmapped adapters get no capability" contract. Use an own-property check.</violation>
</file>
<file name="packages/opencode/test/altimate/review-contracts-dialect.test.ts">
<violation number="1" location="packages/opencode/test/altimate/review-contracts-dialect.test.ts:207">
P3: This test does not exercise the documented head-reading path: omitting `head` makes `loadConsumerLists` call `git ls-files`, not `git ls-tree`. Commit the fixture and pass its commit ref as `head` so the head branch is actually covered.</violation>
</file>
<file name="packages/opencode/src/altimate/review/unit-tests.ts">
<violation number="1" location="packages/opencode/src/altimate/review/unit-tests.ts:51">
P2: A syntactically valid YAML `given` entry such as `- null` makes this detector throw when it reads `g.format`, aborting the entire review because the caller awaits it in `Promise.all`. Guard each entry before reading `format` and `rows`.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/unit-tests.ts:58">
P2: Expecting rows from all-empty inputs is not always a defect: scalar aggregates and default-row models legitimately emit rows from empty sources. Flagging these as `warning` with 'nothing to transform' will misfire on valid tests. Lower this to a `suggestion`, or soften the title and body so they don't assert the model can only pass by emitting constants.</violation>
</file>
<file name="packages/opencode/src/altimate/review/config.ts">
<violation number="1" location="packages/opencode/src/altimate/review/config.ts:51">
P2: An invalid `pattern` passes config parsing and only produces a stderr warning later, so the check is silently disabled. Validate the regex in the schema so `parseReviewConfig` rejects it, matching the fail-fast rule documented on `loadReviewConfig`.</violation>
</file>
<file name="packages/opencode/src/altimate/review/rule-catalog.ts">
<violation number="1" location="packages/opencode/src/altimate/review/rule-catalog.ts:1878">
P3: This body hardcodes the list of truncating engines that `TRUNCATING_DIVISION_ENGINES` already defines in adapter-capabilities.ts. Import the constant and interpolate it so a capability change updates the message too.</violation>
</file>
<file name="packages/opencode/src/altimate/review/orchestrate.ts">
<violation number="1" location="packages/opencode/src/altimate/review/orchestrate.ts:675">
P3: `columnDelta` is recomputed in both `consumerContractLane` and `consumerListLane` for the same model, and `runner.columnLineage` is not cached. Compute the old/new output-column sets once per context and share them between the lanes.</violation>
</file>
<file name="packages/opencode/src/altimate/review/consumer-lists.ts">
<violation number="1" location="packages/opencode/src/altimate/review/consumer-lists.ts:39">
P2: Whitespace splitting turns trailing comment words into column names, so `surrogate` and `key` are treated as list columns. A dotted entry such as `o.email` is dropped without notice. Strip comments and split on commas, or document that the capture group must hold bare names only.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/consumer-lists.ts:41">
P2: Quoted column names containing characters outside `IDENT` are silently dropped, so consumer-list checks miss columns such as `"order-id"`. Preserve valid quoted tokens, including non-identifier names, when parsing the configured list.</violation>
</file>
<file name="docs/docs/usage/dbt-pr-review.md">
<violation number="1" location="docs/docs/usage/dbt-pr-review.md:487">
P3: This says every empty-input unit test is reported, but the rule only checks newly added or changed tests with inline `dict` fixtures; unchanged and CSV/SQL-fixture tests are skipped. Document that scope to avoid promising findings the reviewer will not emit.</violation>
</file>
<file name="packages/opencode/src/altimate/review/runner.ts">
<violation number="1" location="packages/opencode/src/altimate/review/runner.ts:526">
P2: The retry is skipped whenever partial-schema lineage returns any edge, even if it omitted columns from undocumented relations. Retry for manifest-derived schemas regardless of edge count, otherwise column-breakage checks can compare incomplete output sets.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| const passesAnyway = expectRows.length === 0 | ||
| out.push( | ||
| makeFinding({ | ||
| severity: passesAnyway ? "suggestion" : "warning", |
There was a problem hiding this comment.
P2: Expecting rows from all-empty inputs is not always a defect: scalar aggregates and default-row models legitimately emit rows from empty sources. Flagging these as warning with 'nothing to transform' will misfire on valid tests. Lower this to a suggestion, or soften the title and body so they don't assert the model can only pass by emitting constants.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/unit-tests.ts, line 58:
<comment>Expecting rows from all-empty inputs is not always a defect: scalar aggregates and default-row models legitimately emit rows from empty sources. Flagging these as `warning` with 'nothing to transform' will misfire on valid tests. Lower this to a `suggestion`, or soften the title and body so they don't assert the model can only pass by emitting constants.</comment>
<file context>
@@ -0,0 +1,77 @@
+ const passesAnyway = expectRows.length === 0
+ out.push(
+ makeFinding({
+ severity: passesAnyway ? "suggestion" : "warning",
+ category: "test_coverage",
+ title: passesAnyway
</file context>
There was a problem hiding this comment.
Follow-up: low priority, tracked. A scalar aggregate over empty inputs can legitimately return a row, so we'll soften this to a suggestion.
| * (omitted = files that mention the model by name). | ||
| */ | ||
| consumerColumnLists: z | ||
| .array(z.object({ files: z.string(), pattern: z.string(), model: z.string().optional() })) |
There was a problem hiding this comment.
P2: An invalid pattern passes config parsing and only produces a stderr warning later, so the check is silently disabled. Validate the regex in the schema so parseReviewConfig rejects it, matching the fail-fast rule documented on loadReviewConfig.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/config.ts, line 51:
<comment>An invalid `pattern` passes config parsing and only produces a stderr warning later, so the check is silently disabled. Validate the regex in the schema so `parseReviewConfig` rejects it, matching the fail-fast rule documented on `loadReviewConfig`.</comment>
<file context>
@@ -35,6 +35,21 @@ export const ReviewConfig = z.object({
+ * (omitted = files that mention the model by name).
+ */
+ consumerColumnLists: z
+ .array(z.object({ files: z.string(), pattern: z.string(), model: z.string().optional() }))
+ .default([]),
/** Allow `--force-tier` (a debug bypass of the risk classifier). Set false
</file context>
| .array(z.object({ files: z.string(), pattern: z.string(), model: z.string().optional() })) | |
| .array(z.object({ files: z.string().min(1), pattern: z.string().refine((p) => { try { new RegExp(p); return true } catch { return false } }, "invalid regex"), model: z.string().optional() })) |
There was a problem hiding this comment.
Not changing: a bad consumer-list pattern is skipped with a warning on purpose, so an optional list never fails the whole review. Validating it in the schema is a possible follow-up.
| } | ||
|
|
||
| export function capabilityFor(dialect: string | undefined): AdapterCapability | undefined { | ||
| return dialect ? ADAPTER_CAPABILITIES[dialect.toLowerCase()] : undefined |
There was a problem hiding this comment.
P3: capabilityFor reads from a plain object, so inherited keys like constructor or toString return a non-undefined value for an unmapped dialect. That breaks the "unmapped adapters get no capability" contract. Use an own-property check.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/adapter-capabilities.ts, line 75:
<comment>`capabilityFor` reads from a plain object, so inherited keys like `constructor` or `toString` return a non-undefined value for an unmapped dialect. That breaks the "unmapped adapters get no capability" contract. Use an own-property check.</comment>
<file context>
@@ -0,0 +1,79 @@
+}
+
+export function capabilityFor(dialect: string | undefined): AdapterCapability | undefined {
+ return dialect ? ADAPTER_CAPABILITIES[dialect.toLowerCase()] : undefined
+}
+
</file context>
There was a problem hiding this comment.
Follow-up: nit, tracked (Object.hasOwn guard).
| await fs.writeFile(path.join(root, "loaders", "fct.py"), 'COLS = ["id", "amount"]\n') | ||
| await fs.writeFile(path.join(root, "loaders", "other.txt"), 'COLS = ["x"]\n') | ||
| await g("add", "-A") | ||
| const lists = await loadConsumerLists({ |
There was a problem hiding this comment.
P3: This test does not exercise the documented head-reading path: omitting head makes loadConsumerLists call git ls-files, not git ls-tree. Commit the fixture and pass its commit ref as head so the head branch is actually covered.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/review-contracts-dialect.test.ts, line 207:
<comment>This test does not exercise the documented head-reading path: omitting `head` makes `loadConsumerLists` call `git ls-files`, not `git ls-tree`. Commit the fixture and pass its commit ref as `head` so the head branch is actually covered.</comment>
<file context>
@@ -0,0 +1,260 @@
+ await fs.writeFile(path.join(root, "loaders", "fct.py"), 'COLS = ["id", "amount"]\n')
+ await fs.writeFile(path.join(root, "loaders", "other.txt"), 'COLS = ["x"]\n')
+ await g("add", "-A")
+ const lists = await loadConsumerLists({
+ configs: [
+ { files: "loaders/**/*.py", pattern: "COLS\\s*=\\s*\\[([^\\]]*)\\]", model: "fct" },
</file context>
There was a problem hiding this comment.
Fixed in 41e2bd8. A test now commits the fixture and passes head.
| { added: /(?<![\d.])\d+\s*\/\s*\d+(?![\d.])/ }, | ||
| { add: "select 7 / 2 as r" }, | ||
| { add: "select 7.0 / 2 as r" }, | ||
| "`7 / 2` returns 3, not 3.5, on Postgres, Redshift, Trino/Athena and SQL Server (`count(…) / count(…)` too): " + |
There was a problem hiding this comment.
P3: This body hardcodes the list of truncating engines that TRUNCATING_DIVISION_ENGINES already defines in adapter-capabilities.ts. Import the constant and interpolate it so a capability change updates the message too.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/rule-catalog.ts, line 1878:
<comment>This body hardcodes the list of truncating engines that `TRUNCATING_DIVISION_ENGINES` already defines in adapter-capabilities.ts. Import the constant and interpolate it so a capability change updates the message too.</comment>
<file context>
@@ -1824,10 +1875,18 @@ const MISC: Rule[] = [
- { added: /(?<![\d.])\d+\s*\/\s*\d+(?![\d.])/ },
- { add: "select 7 / 2 as r" },
- { add: "select 7.0 / 2 as r" },
+ "`7 / 2` returns 3, not 3.5, on Postgres, Redshift, Trino/Athena and SQL Server (`count(…) / count(…)` too): " +
+ "integer division drops the fraction. Cast one side to numeric for a real ratio.",
+ {
</file context>
There was a problem hiding this comment.
Follow-up: nit, tracked. We'll use TRUNCATING_DIVISION_ENGINES or remove it.
| const model = modelNameFromPath(ctx.file.path) | ||
| const consumers = await runner.modelConsumers(model) | ||
| if (!consumers || (!consumers.exposures.length && !consumers.contractEnforced)) return [] | ||
| const { added, removed } = await columnDelta(ctx, runner, dialect) |
There was a problem hiding this comment.
P3: columnDelta is recomputed in both consumerContractLane and consumerListLane for the same model, and runner.columnLineage is not cached. Compute the old/new output-column sets once per context and share them between the lanes.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 675:
<comment>`columnDelta` is recomputed in both `consumerContractLane` and `consumerListLane` for the same model, and `runner.columnLineage` is not cached. Compute the old/new output-column sets once per context and share them between the lanes.</comment>
<file context>
@@ -606,6 +622,156 @@ async function columnBreakageLane(
+ const model = modelNameFromPath(ctx.file.path)
+ const consumers = await runner.modelConsumers(model)
+ if (!consumers || (!consumers.exposures.length && !consumers.contractEnforced)) return []
+ const { added, removed } = await columnDelta(ctx, runner, dialect)
+ const out: Finding[] = []
+
</file context>
There was a problem hiding this comment.
Follow-up: low priority, tracked. We'll memoize columnLineage per (sql, dialect).
| correctness finding there. On Snowflake, BigQuery, Databricks and DuckDB it is | ||
| only portability advice, shown for packages (`portability: true`). | ||
| - **dbt unit tests.** A unit test whose `given` inputs are all `rows: []` is | ||
| reported: it either never tests the model's logic, or expects rows the model has nothing to |
There was a problem hiding this comment.
P3: This says every empty-input unit test is reported, but the rule only checks newly added or changed tests with inline dict fixtures; unchanged and CSV/SQL-fixture tests are skipped. Document that scope to avoid promising findings the reviewer will not emit.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/docs/usage/dbt-pr-review.md, line 487:
<comment>This says every empty-input unit test is reported, but the rule only checks newly added or changed tests with inline `dict` fixtures; unchanged and CSV/SQL-fixture tests are skipped. Document that scope to avoid promising findings the reviewer will not emit.</comment>
<file context>
@@ -448,6 +477,15 @@ The deterministic rules group into four families:
+ correctness finding there. On Snowflake, BigQuery, Databricks and DuckDB it is
+ only portability advice, shown for packages (`portability: true`).
+- **dbt unit tests.** A unit test whose `given` inputs are all `rows: []` is
+ reported: it either never tests the model's logic, or expects rows the model has nothing to
+ build from.
- **dbt config / Jinja (`DBT0xx`).** A core **minijinja** parser reads
</file context>
There was a problem hiding this comment.
Follow-up: nit, tracked. The docs will narrow the unit-test scope.
| const model = modelNameFromPath(ctx.file.path) | ||
| const consumers = await runner.modelConsumers(model) | ||
| if (!consumers || (!consumers.exposures.length && !consumers.contractEnforced)) return [] | ||
| const { added, removed } = await columnDelta(ctx, runner, dialect) |
There was a problem hiding this comment.
WARNING: Duplicate column-lineage work on every changed model
columnDelta calls runner.columnLineage for the old and the new compiled SQL, but columnBreakageLane (lines 582/584) already ran those same two calls for this exact ctx and dialect — and consumerListLane (line 745) recomputes the identical delta again whenever lists are configured. On the default lite/full tiers those lanes all run, so each changed model now costs 2-4 extra native lineage calls, and each of those can itself retry without a schema in runner.columnLineage. Compute columnDelta once per model and pass the result to both lanes, or memoize columnLineage per (sql, dialect) in the runner.
There was a problem hiding this comment.
Follow-up: low priority, tracked. Same as the cubic memoization comment.
| } | ||
|
|
||
| /** Display names of the engines where `int / int` truncates. */ | ||
| export const TRUNCATING_DIVISION_ENGINES = "Postgres, Redshift, Trino/Athena and SQL Server" |
There was a problem hiding this comment.
SUGGESTION: Unused export duplicates the engine list
TRUNCATING_DIVISION_ENGINES is never imported anywhere, while the same engine list is hard-coded twice in rule-catalog.ts (the rule body at 1878 and portabilityBody at 1885-1886). Either use this constant in those bodies or delete it; as written the list can drift from the behavior the table documents.
There was a problem hiding this comment.
Follow-up: nit, tracked. Same as the cubic engine-list comment.
| added: /(?<![\w.])\d[\d_]*\s*\/\s*\d[\d_]*(?![\d.])|\bcount\s*\([^()]*\)\s*\/\s*(nullif\s*\(\s*)?count\s*\(/i, | ||
| dialectGate: "integerDivisionTruncates", | ||
| ignoreStrings: true, | ||
| portabilityBody: |
There was a problem hiding this comment.
SUGGESTION: Portability finding keeps a truncation headline
gatedBody swaps only body, and the title is built from rule.title (rule-catalog.ts:161). So on Snowflake/BigQuery/Databricks/DuckDB the comment reads "integer division truncates" directly above a body saying it "returns a decimal on snowflake". Since the point of the gate is engine-accurate messaging, add a gated title (e.g. portabilityTitle) or make this rule's title engine-neutral.
There was a problem hiding this comment.
Follow-up: low priority, tracked. We'll add a portability title.
|
|
||
| /** Replace quoted string contents so text inside literals can't match. */ | ||
| function stripStrings(s: string): string { | ||
| return s.replace(/'(?:[^']|'')*'/g, "''").replace(/"(?:[^"\\]|\\.)*"/g, '""') |
There was a problem hiding this comment.
SUGGESTION: stripStrings misses backslash-escaped quotes
The single-quote branch only handles doubled quotes (''). On engines where \' is a valid escape (Snowflake, BigQuery), 'it\'s 7 / 2' is stripped only up to the escaped quote, leaving 7 / 2 exposed and producing a false "integer division" finding. Handle escapes as well, e.g. /'(?:[^'\\]|\\.|'')*'/g.
There was a problem hiding this comment.
Follow-up: low priority, tracked (\' escapes in stripStrings).
| let newDoc: unknown | ||
| let oldDoc: unknown | ||
| try { | ||
| newDoc = YAML.parse(newContent) |
There was a problem hiding this comment.
SUGGESTION: The same YAML is parsed twice per changed schema file
orchestrate.ts calls detectSchemaYmlPatterns and detectUnitTestFixtures with the same oldContent/newContent, and detectSchemaYmlPatterns already parses both sides (dbt-patterns.ts:1338 / dbt-patterns.ts:1352). That doubles the parse cost per changed yml and lets the two error paths disagree. Parse once in the caller and pass the parsed documents (or share one parse helper).
There was a problem hiding this comment.
Not changing for now: the double YAML parse is cheap at schema-file sizes.
| ? undefined | ||
| : makeCompiledResolver({ cwd: dbtRootReal, projectName, pathPrefix }) | ||
|
|
||
| const consumerLists = config.consumerColumnLists.length |
There was a problem hiding this comment.
SUGGESTION: Consumer lists are loaded before lane selection
loadConsumerLists runs for every configured glob: one git ls-files/ls-tree plus a git show per matching file (read → getContent(p, "new")). runReview only consumes them when lineage_breakage or contract_violation is in the selected lanes, so a trivial-tier PR or a restricted reviewers list pays the full scan for nothing. Gate the load on those lanes, or pass a lazy loader into runReview.
There was a problem hiding this comment.
Not changing for now: lists load only when configured, and the cost is one git ls-files per entry.
| const raw = loadRawManifest(opts.manifestPath) | ||
| if (!raw) return undefined | ||
| const nodes = Object.entries<any>(raw.nodes ?? {}) | ||
| const hit = nodes.find(([id, n]) => id.startsWith("model.") && n?.name === model) |
There was a problem hiding this comment.
SUGGESTION: Reimplements the manifest node lookup
helpers.ts already exports findModel/getUniqueId for this (both also check resource_type === "model"); this inline nodes.find duplicates them. It also takes the first name match across the whole manifest, so when a package model shares the name, exposures and contract are read from the wrong node. Reuse findModel, or resolve the id via getUniqueId and look the node up by unique_id.
There was a problem hiding this comment.
Follow-up: low priority, tracked. We'll resolve by unique_id or path so a package model with the same name can't shadow the project model.
Code Review SummaryStatus: 10 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (12 files)
Checked and cleared
Fix these issues in Kilo Cloud Previous Review Summary (commit 1522b5a)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit 1522b5a)Status: 7 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (12 files)
Findings Checked And Cleared
Reviewed by deepseek-v4.1-flash-1017 · Input: 601.1K · Output: 73.3K · Cached: 8.8M Review guidance: REVIEW.md from base branch |
hkaltai
left a comment
There was a problem hiding this comment.
Verdict: request changes. Panel: Claude, Codex, and lead review. Both external reviewers independently reported the consumer-list --head defect, and I reproduced both MAJOR findings against the code.
Counts: CRITICAL 0, MAJOR 2, MINOR 2, NIT 8.
The design is sound (cited per-engine table, opt-ins off by default, no new finding can block), but two advertised behaviours do nothing in production: the "Table not found" suppression (wrong error field) and consumer lists under --head (unsupported git pathspec). Neither is caught by the new tests because the fixtures use shapes or paths that bypass the defect.
Nits
packages/opencode/src/altimate/review/orchestrate.ts(~1474):if (lanes.has("lineage_breakage") || lanes.has("contract_violation"))also runs the list lane when onlycontract_violationis enabled; gate each lane on its own category.runner.ts(~519-526): the no-schema lineage retry can leave old and new SQL analysed under different schema modes;columnBreakageLanehas no star check likeoutputColumns. Retry only when both sides are empty, or apply the star check there.adapter-capabilities.ts:79:export const TRUNCATING_DIVISION_ENGINES = ...is unused while the rule bodies hard-code the same list; use it or delete it.consumer-lists.ts:39:(m[1] ?? "")silently yields nothing for a pattern with no capture group; warn on stderr.orchestrate.ts(consumerContractLane/consumerListLane):columnDeltais recomputed per lane; compute once per ctx.orchestrate.tsconsumerContractLane doc: "reported unless ... opts in" should read "reported only when ... opts in".unit-tests.ts:passesAnywayis misleading for "cannot fail"; considercannotFail.ConsumerColumnListConfigduplicates the zod schema inconfig.ts; derive viaz.infer.- Missing tests: versioned models in
modelConsumers(n.namevsfoo_v2),dialectKnownfalse for unmapped adapter,head-path list loading.
| hasTables | ||
| ? asArray(data.validation?.errors).map((e: any) => ({ ...e, rule: "validate", severity: "error" })) | ||
| ? asArray(data.validation?.errors) | ||
| .filter((e: any) => opts.schemaContext || e?.category?.type !== "TableNotFound") |
There was a problem hiding this comment.
[MAJOR] TableNotFound filter reads the wrong field, so the false-warning fix never fires
.filter((e: any) => opts.schemaContext || e?.category?.type !== "TableNotFound")The engine reports these errors as { code: "E001", kind: { type: "TableNotFound" } }; nothing renames kind to category. Existing code reads kind.type (isExistenceError in native/altimate-core.ts, and validate-identifier-case.test.ts). The new test mocks category: { type: ... }, a shape the engine does not produce, so it passes anyway.
Failure scenario: manifest present, no catalog.json; a model selects from an undocumented model stg. The engine returns kind.type = "TableNotFound", the filter keeps it, and the review still posts the false "Table 'stg' not found" error and can block the gate.
Fix: filter on e?.code === "E001" || e?.kind?.type === "TableNotFound" (or reuse isExistenceError narrowed to tables) and change the test fixture to the real { code, kind } shape.
There was a problem hiding this comment.
Thanks, you're right. It was only corrected later in the stack. Now fixed in this PR in fix(review): address review feedback: TableNotFoundread fromkind, numeric count ratios, per-model contract yml, separate consumer lists: the filter reads kind.type === "TableNotFound", and the runner test uses the engine's real { code: "E001", kind: { type, table } } shape. #1739 later widens it to isExistenceError (tables and columns).
| continue | ||
| } | ||
| const pathspec = `:(glob)${cfg.files}` | ||
| const args = opts.head ? ["ls-tree", "-r", "--name-only", opts.head, "--", pathspec] : ["ls-files", "--", pathspec] |
There was a problem hiding this comment.
[MAJOR] git ls-tree rejects glob pathspecs, so explicit-head reviews silently load no consumer lists
const args = opts.head ? ["ls-tree", "-r", "--name-only", opts.head, "--", pathspec] : ["ls-files", "--", pathspec]git ls-tree does not support pathspec magic. Reproduced: fatal: :(glob)packages/*.ts: pathspec magic not supported by this command: 'glob'. The catch then sets files = [] with no message.
Failure scenario: CI runs review --base main --head <sha> with consumerColumnLists configured. No list loads, the lane stays silent, and a PR that adds a column while leaving an export list stale gets no warning. The only test covers the working-tree path (no head).
Fix: run git ls-tree -r --name-only <head> without a pathspec and match cfg.files in JS with the same glob matcher used elsewhere; write a stderr warning when git fails; add a test that passes head.
There was a problem hiding this comment.
Thanks for the repro. Fixed in fix(review): address review comments: consumer lists with --head, null unit-test inputs: both paths list all files (ls-tree -r -z --name-only <head> / ls-files -z) and match cfg.files with Bun.Glob. There's a test that passes head and ignores an untracked file. The new commit also reports a failed listing on stderr instead of silently loading nothing.
| "`7 / 2` returns 3, not 3.5, on Postgres, Redshift, Trino/Athena and SQL Server (`count(…) / count(…)` too): " + | ||
| "integer division drops the fraction. Cast one side to numeric for a real ratio.", | ||
| { | ||
| added: /(?<![\w.])\d[\d_]*\s*\/\s*\d[\d_]*(?![\d.])|\bcount\s*\([^()]*\)\s*\/\s*(nullif\s*\(\s*)?count\s*\(/i, |
There was a problem hiding this comment.
[MINOR] count(...) / count(...) branch flags code that already applied the fix
\bcount\s*\([^()]*\)\s*\/\s*(nullif\s*\(\s*)?count\s*\(It ignores what precedes or follows the pair. select 100.0 * count(*) / nullif(count(x), 0), 1.0 * count(*) / count(x) and count(*) / count(x)::float all match, yet evaluate as numeric on Postgres. The common percentage idiom would be flagged as a truncation bug on Postgres, Redshift, Trino and SQL Server projects. The counter-example only covers count(*) * 1.0 / count(...).
Fix: add an unless for a preceding \d+\.\d* *\* and a trailing :: cast, and add counter-examples for both.
There was a problem hiding this comment.
Agreed, thanks. Fixed in the new commit: the count-ratio branch skips a decimal multiplier before it (100.0 * count(*) / nullif(count(x), 0), 1.0 * …) and a :: cast on the divisor. There are counter-examples for each, and 100 * count(*) / count(x) still fires.
| const declared = new Set(consumers.contractColumns) | ||
| const yml = await opts.changedYmlText() | ||
| // A column the PR adds to a changed yml is declared even when the manifest predates the PR. | ||
| const inChangedYml = (c: string) => |
There was a problem hiding this comment.
[MINOR] Contract reconciliation accepts a column declared under any model in any changed yml
const inChangedYml = (c: string) =>The regex looks for name: <col> anywhere in any changed schema file. If another model's yml declares region, a genuinely missing fct.region contract entry is suppressed (reproduced by Codex). The same over-matching exists in consumer-lists.ts: all matches in one file are merged into one column set, so two lists in a file ([id, amount, region] and [id, amount]) mask a stale list.
Fix: parse the yml and reconcile within the matching model (and version); keep each regex match as its own list. Opt-in features, so MINOR.
There was a problem hiding this comment.
Thanks. Fixed in the new commit. A column counts as declared only under this model's own models: entry (or its version, orders_v2 → orders v2) in a changed yml, parsed with yaml (ymlModelColumns). Each consumer-list regex match is now its own list, so [id, amount] next to [id, amount, region] is reported. Tests cover another model's yml, versioned yml, and two lists in one file.
…re integer division, empty unit-test fixtures - `adapter-capabilities.ts`: per-adapter behavior table with doc citations; engine-dependent rules read it instead of assuming one engine - integer division: correctness finding on Postgres/Redshift/Trino/Athena/SQL Server; portability advice (packages only) on Snowflake/BigQuery/Databricks/DuckDB; also matches `count(…) / count(…)` and underscore literals - a column removed from a model with `exposures` or an enforced contract names those consumers (`dbt.manifest` evidence, via a new `modelConsumers` runner method) - a new column missing from an enforced contract is reported unless `ciRunsDbtBuild: true`; a column declared in a changed yml counts - opt-in `consumerColumnLists` in `review.yml`: explicit column lists in loaders/exports must follow a model's added and removed columns - dbt unit tests whose `given` inputs are all `rows: []` are reported (cannot fail, or expect rows from nothing) - column sets come from column lineage; a `*` in the output skips the checks rather than guessing - column lineage retries without the schema when a manifest-only (partial) schema yields nothing: undocumented upstream tables made it return no columns, silently disabling column-breakage checks on projects without `catalog.json` - a manifest-only schema no longer reports `validate: Table '<x>' not found` for undocumented models (an engine warning that could gate a PR); kept when a complete `catalog.json` schema is loaded Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hes inside strings, generic docs From an adversarial generality review: - the contract-vs-`SELECT` check restates what `dbt build` enforces, so it is now opt-in (`checkContractColumns`, replaces `ciRunsDbtBuild`) and comments only (catalog origin) - integer division ignores text inside string literals (`'2024/01/15'` is not a division) - the empty-inputs unit-test finding no longer claims the test "cannot fail" (models emitting constant rows still produce output) - an adapter the review doesn't map (a community adapter) no longer counts as a known dialect, so generic advice isn't suppressed on a guess - docs: the column-list example is a generic export script, not one project's loader convention Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ull unit-test inputs - `consumer-lists.ts`: `git ls-tree` rejects `:(glob)` pathspec magic, so with a head ref the lists were never loaded; list all files (`ls-tree -r -z` / `ls-files -z`) and match the configured glob with `Bun.Glob` - `unit-tests.ts`: a `given: [null]` entry no longer throws on `g.format` - tests: the head-ref path and the null `given` entry Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…`, numeric count ratios, per-model contract yml, separate consumer lists
- Runner: the partial-schema filter reads the engine's real error shape
(`kind.type === "TableNotFound"`, not `category`), so an undocumented
upstream model is no longer reported as a missing table against a
manifest-only schema. The runner test uses the engine's `{ code, kind }`
shape. (The same change was previously only in the next PR.)
- `integer-division`: a count ratio already made numeric (`100.0 * count(…) /
…`, `1.0 * …`, a `::` cast on the divisor) no longer matches.
- Contract reconciliation: a column counts as declared only under this
model's own `models:` entry (or its version) in a changed yml, parsed with
`yaml`, not any `name: <col>` in any changed file (`ymlModelColumns`).
- Consumer lists: each regex match is its own list, so a stale list next to an
up-to-date one in the same file is reported; a pattern without a capture
group and a failed git listing are reported on stderr.
- Doc fix: the contract-columns check is reported only when opted in.
- Tests: real `TableNotFound` shape, numeric-ratio counter-examples, another
model's yml, versioned yml, two lists in one file.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6736c7c to
27a9a99
Compare
1522b5a to
9fa035e
Compare
|
@hkaltai, re your review: Thanks for the review and the repros. Changes since then:
|
| sources: ["https://docs.aws.amazon.com/redshift/latest/dg/r_numeric_computations201.html"], | ||
| }, | ||
| trino: { | ||
| integerDivisionTruncates: true, |
There was a problem hiding this comment.
WARNING: Trino/Athena / returns a floating-point quotient, so this entry (and the rule body) inverts the behavior
Trino's / operator returns a floating-point result for integer operands (7 / 2 -> 3.5); the integer-division forms are divide() (what the cited functions/math.html page documents) and the // operator. With integerDivisionTruncates: true here, the integer-division catalog rule fires as a correctness finding (title "integer division truncates", body "returns 3, not 3.5") on every int / int / count(...) / count(...) in a Trino or Athena project - exactly the false positive the adapter gate exists to prevent. Flipping trino and athena (line 33) to false makes them the portability variant like Snowflake/BigQuery/DuckDB. Worth confirming on a live Trino/Athena before merge.
| // Known = set in config, or an adapter the review maps to a dialect. An | ||
| // unmapped adapter (a community adapter) is treated as unknown, so generic | ||
| // advice isn't suppressed on a guess. | ||
| const dialectKnown = !!config.dialect || (!!detectedDialect && Object.values(ADAPTER_DIALECT).includes(detectedDialect)) |
There was a problem hiding this comment.
WARNING: dialectKnown ignores the capability table, so the clickhouse entry is unreachable
adapter-capabilities.ts lists clickhouse (integerDivisionTruncates: false), but clickhouse is not a value of ADAPTER_DIALECT, so detectDialect() returns the raw adapter name and this predicate sets dialectKnown = false. gatedBody() then short-circuits to rule.body ("integer division truncates") and the finding is not tagged portability, so a dbt-clickhouse project gets a false positive on 7 / 2; suppressPortability is false for the same reason, so it is not dropped either. Deriving known-ness from the capability table (!!capabilityFor(detectedDialect)) instead of Object.values(ADAPTER_DIALECT) makes that entry reachable.
| model, | ||
| confidence: "high", | ||
| evidence: { tool: "consumer-column-list", result: { list: list.path, missing, stale } }, | ||
| ruleKey: `lineage_breakage:consumer-list:${list.path}`, |
There was a problem hiding this comment.
WARNING: Two lists in the same file collapse into one finding
makeFinding derives the id from category + file + model + column + ruleKey, and this ruleKey carries only list.path. Now that extractColumnLists returns one list per match, two matches in the same file produce two findings with the same fingerprint, so dedupe() drops one and the second (stale) list is masked in the output - the same masking this PR fixes at the extraction layer. Add something list-specific to the ruleKey, e.g. the match index or the missing/stale columns.
Changes since review: Review feedback commit:
TableNotFoundread from the engine'skind, numeric count ratios, per-model contract yml, and separate consumer lists.Issue for this PR
Part of #1792
Stacked on #1723 (← #1722 ← #1721 ← #1720 ← #1719); merge in that order. Draft: the new rules are not yet measured on a public-repo benchmark.
Type of change
What does this PR do?
Deterministic checks that work on any dbt project, from the manifest and the diff, plus three fixes found by an end-to-end run on a real dbt project.
New checks
adapter-capabilities.tsis a per-adapter table, with each entry citing the engine's docs. Rules whose correctness depends on the engine read the table instead of assuming one engine.int / int(andcount(…) / count(…)) is a correctness finding on Postgres, Redshift, Trino/Athena and SQL Server.exposuresor an enforced contract produces a finding that names those consumers (newmodelConsumersrunner method).SELECT, opt-in (checkContractColumns). A new column missing from an enforced contract is reported. It's off by default becausedbt buildalready fails on it, and it never blocks.consumerColumnLists). For export or reverse-ETL scripts that copy a model's columns.giveninputs are allrows: []. The model's logic over its inputs is never exercised.Fixes found by the end-to-end run
catalog.json. It now retries without the schema.validate: Table '<x>' not foundwarnings. A manifest-only schema produced this engine warning for undocumented models; 50 new models meant 50 warnings and a blocked gate. It's still reported when a completecatalog.jsonschema is loaded.How did you verify your code works?
test/altimate/review-contracts-dialect.test.ts: engine table per dialect, string literals, exposures and contracts, the opt-ins, unit-test fixtures, and dialect packs.modelConsumersand the partial-schema filter, using the engine's real response shape.bun typecheckis clean, and the real-world corpus is unchanged (15/15 caught, 0/5 false positives).revenue_dashboardand the contract; 0.12.10 reports nothing for it.dbt parseonly, base from the GitHub event). Off-diff findings went from 219–427 per PR to 0 (e.g. 431 findings → 1 on a 2-file PR), and no earlier catch was lost. That's mostly from the earlier PRs in the stack; this PR's fixes keep the count at 0.Cost side effects
None. No model calls or warehouse queries. Column lineage may run a second time, without the schema, when the first call returns nothing (in-process, bounded by the number of changed models).
Screenshots / recordings
N/A (CLI).
Checklist
🤖 Generated with Claude Code
Summary by cubic
Adds deterministic review checks that run on any dbt project from the manifest and the diff, plus fixes from an end-to-end run on a real dbt project.
New checks
int / int(andcount(…) / count(…)) is a correctness finding on Postgres, Redshift, Trino/Athena and SQL Server; on Snowflake, BigQuery, Databricks and DuckDB it is portability advice for multi-adapter packages only, and text inside string literals is ignored. Ratios already made numeric (100.0 * count(…), a cast on the divisor) don't match.exposuresor an enforced contract now names those consumers (newmodelConsumersrunner method).checkContractColumns(opt-in, never blocks): a new column missing from an enforced contract is reported;dbt buildalready fails on it, so it's off by default. A column declared for that model in a changed yml counts as declared.consumerColumnLists(opt-in): named files outside dbt that copy a model's columns must follow added and removed columns; each regex match is its own list.giveninputs are allrows: []are reported, since they never exercise the model's logic.Fixes
catalog.json.Table '<x>' not foundvalidation warnings; these are kept when a completecatalog.jsonschema is loaded.Written for commit 9fa035e. Summary will update on new commits.