Repository navigation
feat(review): post fewer, better-placed comments: rule tiers, inline budget, pre-existing tags, coverage ledger - #1758
RaghwendraSingh wants to merge 28 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
There was a problem hiding this comment.
9 issues found across 31 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/src/altimate/review/diff-filter.ts">
<violation number="1" location="packages/opencode/src/altimate/review/diff-filter.ts:102">
P2: `posix.join` treats an absolute `*-paths` entry as relative here, so `/shared/macros` becomes `<project-root>/shared/macros` instead of being dropped. Reject absolute entries before joining to avoid classifying and scoping the wrong files.</violation>
</file>
<file name="packages/opencode/src/altimate/review/ai-review.ts">
<violation number="1" location="packages/opencode/src/altimate/review/ai-review.ts:106">
P2: `ALTIMATE_REVIEW_AI_BATCH_CHARS` does not prevent large files from being cut: batching only groups whole files, and an oversized file is sent once after `truncate` removes its content. Split oversized file sections across calls, or document and enforce a hard per-file limit instead of promising preservation.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/ai-review.ts:346">
P3: This comment's rationale looks wrong: the module already statically imports `@/session/llm` and `@/session/message-v2`, so the lazy `@/session` import does not keep the session module out of the graph. Either drop the claim or confirm the reason before keeping the comment.</violation>
</file>
<file name="packages/opencode/test/altimate/review-ci-honesty.test.ts">
<violation number="1" location="packages/opencode/test/altimate/review-ci-honesty.test.ts:59">
P2: This assertion can never fail: no finding uses the rule ID `altimate-core.equivalence:undecided`, so the test does not check that the undecided result stays out of the findings. Assert on the degraded flag or the semantic_change category instead.</violation>
</file>
<file name="packages/opencode/src/altimate/review/verdict.ts">
<violation number="1" location="packages/opencode/src/altimate/review/verdict.ts:49">
P2: Pre-existing findings are excluded from eligibility but still force `computeIdealVerdict` to return `COMMENT` because it checks the raw finding count first. Filter to verdict-eligible findings before the empty check so a run containing only pre-existing findings remains APPROVE and can be silent.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/verdict.ts:142">
P2: `notReviewed` uses `.default([])` on a signed field. Parsing a legacy envelope with a `scope` but no `notReviewed` adds `notReviewed: []` to the canonical body, so `verifyEnvelope` fails for that envelope. Make the field `.optional()` like `policy.posting`, and have readers already using `?? []` treat a missing ledger as empty.</violation>
</file>
<file name="packages/opencode/src/altimate/review/rule-tiers.ts">
<violation number="1" location="packages/opencode/src/altimate/review/rule-tiers.ts:130">
P2: `ENGINE_FLOOR` omits engine checks that can affect the verdict. Every `altimate-core.*` finding is verdict-eligible, but `altimate-core.grain:`, `altimate-core.source-filters:` and `altimate-core.structural-diff:` are not in the floor. Their default `off` entries drop findings before the verdict, and a `tool:*` glob can silence them. Add these prefixes to `ENGINE_FLOOR`, or move the affected entries out of `OFF`.</violation>
</file>
<file name="packages/opencode/test/altimate/review-noise.test.ts">
<violation number="1" location="packages/opencode/test/altimate/review-noise.test.ts:447">
P3: This guard test claims engine rules are never off, but its prefix list omits `altimate-core.column-lineage:`, `altimate-core.classify-pii:`, and `consumer-column-list:` from `ENGINE_FLOOR`. Import the floor list or extend the list, and assert `ruleTier()` output for a floor rule.</violation>
</file>
<file name="packages/opencode/src/altimate/review/config.ts">
<violation number="1" location="packages/opencode/src/altimate/review/config.ts:87">
P2: Rule keys are not validated, so a typo in `rules:` is accepted and silently has no effect. Constrain keys to the `tool:rule` or `tool:*` shape so a malformed key fails at config load.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| const out: Array<{ role: DbtRole; dir: string }> = [] | ||
| for (const [role, list] of Object.entries(layout.dirs) as Array<[DbtRole, string[]]>) { | ||
| for (const d of list) { | ||
| const dir = posix.normalize(posix.join(root || ".", norm(d))).replace(/\/+$/, "") |
There was a problem hiding this comment.
P2: posix.join treats an absolute *-paths entry as relative here, so /shared/macros becomes <project-root>/shared/macros instead of being dropped. Reject absolute entries before joining to avoid classifying and scoping the wrong files.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/diff-filter.ts, line 102:
<comment>`posix.join` treats an absolute `*-paths` entry as relative here, so `/shared/macros` becomes `<project-root>/shared/macros` instead of being dropped. Reject absolute entries before joining to avoid classifying and scoping the wrong files.</comment>
<file context>
@@ -86,17 +88,34 @@ export function relativeToRoot(p: string, root: string): string | undefined {
+ const out: Array<{ role: DbtRole; dir: string }> = []
+ for (const [role, list] of Object.entries(layout.dirs) as Array<[DbtRole, string[]]>) {
+ for (const d of list) {
+ const dir = posix.normalize(posix.join(root || ".", norm(d))).replace(/\/+$/, "")
+ if (!dir || dir === "." || dir === ".." || dir.startsWith("../") || dir.startsWith("/")) continue
+ out.push({ role, dir })
</file context>
| const dir = posix.normalize(posix.join(root || ".", norm(d))).replace(/\/+$/, "") | |
| if (norm(d).startsWith("/")) continue | |
| const dir = posix.normalize(posix.join(root || ".", norm(d))).replace(/\/+$/, "") |
There was a problem hiding this comment.
Follow-up: low priority, tracked. An absolute entry in a project-relative path list is unusual; we'll skip leading-/ entries in a follow-up.
| test("undecidable equivalence on a manifest-backed run goes to the ledger, not lint-only", async () => { | ||
| const env = await review() | ||
| expect(env.findings.some((f) => f.degraded)).toBe(true) | ||
| expect(env.findings.some((f) => f.rule === "altimate-core.equivalence:undecided")).toBe(false) |
There was a problem hiding this comment.
P2: This assertion can never fail: no finding uses the rule ID altimate-core.equivalence:undecided, so the test does not check that the undecided result stays out of the findings. Assert on the degraded flag or the semantic_change category instead.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/review-ci-honesty.test.ts, line 59:
<comment>This assertion can never fail: no finding uses the rule ID `altimate-core.equivalence:undecided`, so the test does not check that the undecided result stays out of the findings. Assert on the degraded flag or the semantic_change category instead.</comment>
<file context>
@@ -54,13 +54,12 @@ function review(extra: Partial<Parameters<typeof runReview>[0]> = {}) {
+ test("undecidable equivalence on a manifest-backed run goes to the ledger, not lint-only", async () => {
const env = await review()
- expect(env.findings.some((f) => f.degraded)).toBe(true)
+ expect(env.findings.some((f) => f.rule === "altimate-core.equivalence:undecided")).toBe(false)
expect(env.summary.degraded).toBe(false)
- const md = renderSummary(env)
</file context>
There was a problem hiding this comment.
Follow-up: low priority, tracked. Agreed the assertion can't fail (the undecided path returns []); we'll assert on degraded findings instead.
| /** Every changed file the review did not look at (excluded, outside the | ||
| * project, moved, not a dbt file, no check at this tier, no manifest), with | ||
| * the reason; also repeats the `unverified` entries. */ | ||
| notReviewed: z.array(z.object({ path: z.string(), reason: z.string() })).default([]), |
There was a problem hiding this comment.
P2: notReviewed uses .default([]) on a signed field. Parsing a legacy envelope with a scope but no notReviewed adds notReviewed: [] to the canonical body, so verifyEnvelope fails for that envelope. Make the field .optional() like policy.posting, and have readers already using ?? [] treat a missing ledger as empty.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/verdict.ts, line 142:
<comment>`notReviewed` uses `.default([])` on a signed field. Parsing a legacy envelope with a `scope` but no `notReviewed` adds `notReviewed: []` to the canonical body, so `verifyEnvelope` fails for that envelope. Make the field `.optional()` like `policy.posting`, and have readers already using `?? []` treat a missing ledger as empty.</comment>
<file context>
@@ -118,6 +136,10 @@ export const ReviewScope = z.object({
+ /** Every changed file the review did not look at (excluded, outside the
+ * project, moved, not a dbt file, no check at this tier, no manifest), with
+ * the reason; also repeats the `unverified` entries. */
+ notReviewed: z.array(z.object({ path: z.string(), reason: z.string() })).default([]),
})
export type ReviewScope = z.infer<typeof ReviewScope>
</file context>
| notReviewed: z.array(z.object({ path: z.string(), reason: z.string() })).default([]), | |
| notReviewed: z.array(z.object({ path: z.string(), reason: z.string() })).optional(), |
There was a problem hiding this comment.
Follow-up: low priority, tracked. Latent today (only tests call verifyEnvelope); switching to .optional() is queued.
|
|
||
| /** Engine rules that may never default below summary (verdict-relevant). */ | ||
| const ENGINE_FLOOR = [ | ||
| "altimate-core.equivalence:", |
There was a problem hiding this comment.
P2: ENGINE_FLOOR omits engine checks that can affect the verdict. Every altimate-core.* finding is verdict-eligible, but altimate-core.grain:, altimate-core.source-filters: and altimate-core.structural-diff: are not in the floor. Their default off entries drop findings before the verdict, and a tool:* glob can silence them. Add these prefixes to ENGINE_FLOOR, or move the affected entries out of OFF.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/rule-tiers.ts, line 130:
<comment>`ENGINE_FLOOR` omits engine checks that can affect the verdict. Every `altimate-core.*` finding is verdict-eligible, but `altimate-core.grain:`, `altimate-core.source-filters:` and `altimate-core.structural-diff:` are not in the floor. Their default `off` entries drop findings before the verdict, and a `tool:*` glob can silence them. Add these prefixes to `ENGINE_FLOOR`, or move the affected entries out of `OFF`.</comment>
<file context>
@@ -0,0 +1,161 @@
+
+/** Engine rules that may never default below summary (verdict-relevant). */
+const ENGINE_FLOOR = [
+ "altimate-core.equivalence:",
+ "altimate-core.column-lineage:",
+ "impact-analysis:",
</file context>
There was a problem hiding this comment.
Not changing in this PR: turning those prefixes off is deliberate after judging, and an off rule is meant to leave the verdict too. We'll fix the ENGINE_FLOOR comment so it no longer reads as a contradiction.
| * rule-tiers.ts. Read from the base branch, so a PR cannot relax it. | ||
| * Engine rules that decide the verdict can be turned off only by exact id. | ||
| */ | ||
| rules: z.record(z.string(), z.enum(["inline", "summary", "off"])).default({}), |
There was a problem hiding this comment.
P2: Rule keys are not validated, so a typo in rules: is accepted and silently has no effect. Constrain keys to the tool:rule or tool:* shape so a malformed key fails at config load.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/config.ts, line 87:
<comment>Rule keys are not validated, so a typo in `rules:` is accepted and silently has no effect. Constrain keys to the `tool:rule` or `tool:*` shape so a malformed key fails at config load.</comment>
<file context>
@@ -77,6 +77,27 @@ export const ReviewConfig = z.object({
+ * rule-tiers.ts. Read from the base branch, so a PR cannot relax it.
+ * Engine rules that decide the verdict can be turned off only by exact id.
+ */
+ rules: z.record(z.string(), z.enum(["inline", "summary", "off"])).default({}),
+ /**
+ * How findings reach the PR: `inlineBudget` caps new inline comments per run
</file context>
| rules: z.record(z.string(), z.enum(["inline", "summary", "off"])).default({}), | |
| rules: z.record(z.string().regex(/^[\w.-]+:(\*|[\w-]+)$/), z.enum(["inline", "summary", "off"])).default({}), |
There was a problem hiding this comment.
Not changing: nit. Unknown keys are harmless no-ops, so we left validation out of this PR.
| let cur: AiReviewFile[] = [] | ||
| let size = 0 | ||
| for (const f of files) { | ||
| const n = fileSection(f, budget.maxFileChars).length |
There was a problem hiding this comment.
P2: ALTIMATE_REVIEW_AI_BATCH_CHARS does not prevent large files from being cut: batching only groups whole files, and an oversized file is sent once after truncate removes its content. Split oversized file sections across calls, or document and enforce a hard per-file limit instead of promising preservation.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 106:
<comment>`ALTIMATE_REVIEW_AI_BATCH_CHARS` does not prevent large files from being cut: batching only groups whole files, and an oversized file is sent once after `truncate` removes its content. Split oversized file sections across calls, or document and enforce a hard per-file limit instead of promising preservation.</comment>
<file context>
@@ -48,18 +75,62 @@ function kindRank(kind: string | undefined): number {
+ let cur: AiReviewFile[] = []
+ let size = 0
+ for (const f of files) {
+ const n = fileSection(f, budget.maxFileChars).length
+ if (cur.length && size + n > budget.batchChars) {
+ batches.push(cur)
</file context>
There was a problem hiding this comment.
Not changing: nothing promises that. batchAiFiles groups whole files, and per-file truncation is the separate maxFileChars knob (ai-review.ts:41-42, 92-95).
| * rule in `blockingRules`; advisory AI findings never. A finding on a line the | ||
| * PR did not add (`preExisting`) never does: the PR did not introduce it. */ | ||
| export function verdictEligible(f: Finding, rubric: Rubric = DEFAULT_RUBRIC): boolean { | ||
| if (f.preExisting) return false |
There was a problem hiding this comment.
P2: Pre-existing findings are excluded from eligibility but still force computeIdealVerdict to return COMMENT because it checks the raw finding count first. Filter to verdict-eligible findings before the empty check so a run containing only pre-existing findings remains APPROVE and can be silent.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/verdict.ts, line 49:
<comment>Pre-existing findings are excluded from eligibility but still force `computeIdealVerdict` to return `COMMENT` because it checks the raw finding count first. Filter to verdict-eligible findings before the empty check so a run containing only pre-existing findings remains APPROVE and can be silent.</comment>
<file context>
@@ -43,8 +43,10 @@ export const VCS_EVENT: Record<Verdict, "COMMENT" | "REQUEST_CHANGES"> = {
+ * rule in `blockingRules`; advisory AI findings never. A finding on a line the
+ * PR did not add (`preExisting`) never does: the PR did not introduce it. */
export function verdictEligible(f: Finding, rubric: Rubric = DEFAULT_RUBRIC): boolean {
+ if (f.preExisting) return false
const origin = findingOrigin(f)
if (origin === "engine") return true
</file context>
There was a problem hiding this comment.
Not changing: COMMENT on any finding is the documented rubric (verdict.ts:62). silentWhenClean also requires !env.findings.length (format.ts:64), so a run with only pre-existing findings behaves the same either way.
| const usage = await stream.totalUsage | ||
| const metadata = await stream.providerMetadata | ||
| if (!usage) return undefined | ||
| // Loaded lazily, as in learn/usage.ts, to keep the session module out of |
There was a problem hiding this comment.
P3: This comment's rationale looks wrong: the module already statically imports @/session/llm and @/session/message-v2, so the lazy @/session import does not keep the session module out of the graph. Either drop the claim or confirm the reason before keeping the comment.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 346:
<comment>This comment's rationale looks wrong: the module already statically imports `@/session/llm` and `@/session/message-v2`, so the lazy `@/session` import does not keep the session module out of the graph. Either drop the claim or confirm the reason before keeping the comment.</comment>
<file context>
@@ -177,6 +274,110 @@ export async function runAiReview(input: AiReviewInput): Promise<AiReviewResult>
+ const usage = await stream.totalUsage
+ const metadata = await stream.providerMetadata
+ if (!usage) return undefined
+ // Loaded lazily, as in learn/usage.ts, to keep the session module out of
+ // this module's import graph.
+ const { Session } = await import("@/session")
</file context>
There was a problem hiding this comment.
Not changing: nit (ai-review.ts:346, lazy-import comment).
| "dbt-patterns:fanout-join", | ||
| ]) | ||
| expect(ruleTier(k)).toBe("summary") | ||
| for (const p of ["altimate-core.equivalence:", "impact-analysis:", "schema.detect-pii:", "contract-check:", "data.diff:"]) |
There was a problem hiding this comment.
P3: This guard test claims engine rules are never off, but its prefix list omits altimate-core.column-lineage:, altimate-core.classify-pii:, and consumer-column-list: from ENGINE_FLOOR. Import the floor list or extend the list, and assert ruleTier() output for a floor rule.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/review-noise.test.ts, line 447:
<comment>This guard test claims engine rules are never off, but its prefix list omits `altimate-core.column-lineage:`, `altimate-core.classify-pii:`, and `consumer-column-list:` from `ENGINE_FLOOR`. Import the floor list or extend the list, and assert `ruleTier()` output for a floor rule.</comment>
<file context>
@@ -319,3 +327,551 @@ describe("repeats of one rule within a file", () => {
+ "dbt-patterns:fanout-join",
+ ])
+ expect(ruleTier(k)).toBe("summary")
+ for (const p of ["altimate-core.equivalence:", "impact-analysis:", "schema.detect-pii:", "contract-check:", "data.diff:"])
+ expect(keys.filter((k) => k.startsWith(p) && DEFAULT_RULE_TIERS[k] === "off")).toEqual([])
+ })
</file context>
| for (const p of ["altimate-core.equivalence:", "impact-analysis:", "schema.detect-pii:", "contract-check:", "data.diff:"]) | |
| for (const p of ["altimate-core.equivalence:", "altimate-core.column-lineage:", "altimate-core.classify-pii:", "impact-analysis:", "schema.detect-pii:", "contract-check:", "consumer-column-list:", "data.diff:"]) |
There was a problem hiding this comment.
Not changing: nit. The test pins the prefixes that matter for this change.
| for (const uid of asArray<string>(node.depends_on?.nodes)) { | ||
| const parent = raw.nodes?.[uid] | ||
| if (parent?.original_file_path && ["model", "seed", "snapshot"].includes(parent.resource_type)) { | ||
| add(uid, { path: base + parent.original_file_path, kind: parent.resource_type, name: parent.name }, changed) |
There was a problem hiding this comment.
[WARNING]: Package parents resolve to the wrong file
original_file_path is relative to the package that defines the node, but the path here is built as base + parent.original_file_path for every model/seed/snapshot. A parent from an installed package (e.g. dbt_utils) therefore becomes a root-project path such as models/stg.sql instead of dbt_packages/dbt_utils/models/stg.sql. aiGraphContext then either silently drops the parent (file missing) or, if the root project happens to have a file at that relative path, sends unrelated content to the model as the upstream parent. The macro branch below already guards with macro.package_name === project; the node branches need the same check (or the package-root prefix).
There was a problem hiding this comment.
Fixed in 4733424: graphParents now skips installed-package nodes, parents and sources, because their original_file_path is relative to the package. Test added.
| return changed.some((l) => l.includes(head)) && !/partition\s+by/i.test(sql.slice(open, end)) | ||
| }) | ||
| const hit = unpartitioned ? changed[0] : undefined | ||
| const hit = unpartitioned ? c.added.find((l) => stripComments(l).trim() === changed[0]) : undefined |
There was a problem hiding this comment.
[SUGGESTION]: Anchors to the first OVER ( line, not the unpartitioned one
changed[0] is the first added line that contains any over (, while unpartitioned may be true for a later clause. When a diff adds a partitioned window before an unpartitioned one, the finding anchors to the correctly partitioned line with a window function has no PARTITION BY message, and once dbt-patterns:window-no-partition is promoted to inline the comment is posted there. Have the matchAll predicate record the offending head (use find rather than some) and anchor that line.
There was a problem hiding this comment.
Follow-up: low priority, tracked. The rule is off by default, so this is dormant. We'll anchor it to the offending OVER when we enable it.
| "The existing table keeps its old partitioning; incremental runs don't rebuild it. Run `--full-refresh` on deploy.", | ||
| // The `partition_by` key, else an added line inside its value. | ||
| addedLine(/\bpartition_by\b/) ?? | ||
| input.added.find((l) => /\w{2}/.test(l) && (inlineNew.partition_by ?? "").includes(l.trim()))), |
There was a problem hiding this comment.
[SUGGESTION]: The fallback anchor condition is inverted
inlineNew.partition_by is the config value (for example ['block_month']) and l is a full raw added line, so (inlineNew.partition_by ?? "").includes(l.trim()) asks whether the short value contains the whole line. That is essentially never true, so this fallback is dead: a partition_by change that edited only the value line (no added line holds the partition_by key) loses its anchor and stays in the summary. The comparison belongs the other way round (l.includes(...) on part of the value, guarded against an empty value).
There was a problem hiding this comment.
Not changing: inlineConfig stores the raw multi-line value text (incremental-semantics.ts:72-84), so an inner line of partition_by={...} is a substring of it and the fallback anchors correctly.
| let cur: AiReviewFile[] = [] | ||
| let size = 0 | ||
| for (const f of files) { | ||
| const n = fileSection(f, budget.maxFileChars).length |
There was a problem hiding this comment.
[SUGGESTION]: batchChars does not bound the request that is actually sent
batchAiFiles measures only fileSection(...), but reviewBatch calls buildUserMessage, which also appends relatedContext (up to CONTEXT_TOTAL_CHARS = 40k characters, re-sent in every batch when ALTIMATE_REVIEW_AI_GRAPH=1), the grounding findings and the PR title/body. With the env switches on, a batch sized to ALTIMATE_REVIEW_AI_BATCH_CHARS can exceed it by roughly 40k characters per call, and neither the number of calls nor the total wall time (up to 5 minutes each, two at a time) has a hard cap. Include the shared context/grounding in the sizing, or clamp the final message.
There was a problem hiding this comment.
Follow-up: low priority, tracked. This only applies with the opt-in env (the default is unbounded); we'll include the shared context in the sizing.
| return done([], { state: "timeout", reason: `no answer within ${Math.round(timeoutMs / 1000)} s`, model: modelLabel }) | ||
| if (!text) return done([], { state: "error", reason: "the model returned no text", model: modelLabel }) | ||
| if (controller.signal.aborted) return timedOut() | ||
| usage = await streamUsage(stream, model) |
There was a problem hiding this comment.
[SUGGESTION]: Timed-out and failed calls spend tokens that are never accounted
Usage is read only after readStreamText resolves and the abort check passes. timedOut() above and the catch path return no usage, while laneUsage still counts those results in calls. Since this PR exists to establish a cost baseline, ai_tokens_total/ai_cost omit exactly the slow, aborted and provider-error calls and can report ai_calls > 0 with zero tokens. Read usage before the abort/error returns, or count only the calls that reported it.
There was a problem hiding this comment.
Follow-up: low priority, tracked. We'll read usage before the abort/error returns so cost isn't undercounted.
| // GitHub applies a suggestion to the comment's line range, so the | ||
| // comment spans exactly the replaced lines. | ||
| if (!s) return { path: f.file, line: f.startLine!, side: "RIGHT", body } | ||
| return { |
There was a problem hiding this comment.
[SUGGESTION]: A suggestion range spanning two hunks takes down every inline comment
validateSuggestions only checks that each line of the range is an added line; two added lines in different hunks pass. GitHub rejects a multi-line comment whose start_line and line are not in the same hunk, and postGitHubReview classifies that 422 as a positioning error and re-posts the review with no inline comments at all (the whole batch is lost, not just the suggestion). Validate that the range lies inside one hunk, or drop the suggestion and fall back to the text block. Dormant until a lane emits suggestions.
There was a problem hiding this comment.
Not changing: validateSuggestions (orchestrate.ts) requires every line in startLine..endLine to be added. That makes the range one contiguous added block, which can't span two hunks.
| .filter((f) => shouldReview(f.path)) | ||
| .filter((f) => !excluders.some((re) => re.test(norm(f.path)))) | ||
| .map((f) => ({ ...f, kind: classifyDbtFile(f.path, layout) })) | ||
| return partitionChangedFiles(files, extraExcludeGlobs, layout).reviewable |
There was a problem hiding this comment.
[SUGGESTION]: filterChangedFiles is now dead code
Nothing calls this wrapper any more (only doc comments reference it) since runReview switched to partitionChangedFiles, and keeping both names invites a future caller to pick the one that silently discards the dropped-file reasons the coverage ledger needs. Remove the wrapper, or keep one entry point that returns the reviewable set plus the dropped reasons.
There was a problem hiding this comment.
Not changing in this PR: nit. We'll remove it in a cleanup.
|
|
||
| /** The first added line matching `re`, for anchoring a finding to the diff. */ | ||
| const addedLine = (file: ChangedFile, re: RegExp) => addedLines(file.diff).find((l) => re.test(l)) | ||
| const escapeRe = (s: string) => s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&") |
There was a problem hiding this comment.
[SUGGESTION]: Duplicate regex-escape helper and a second diff parser in one module
escapeRe is byte-for-byte the helper added at orchestrate.ts:382, and this file now keeps its own raw-string addedLines/addedLine parser next to the imported walkDiffLines (used only by signatureAnchor). The two diff readers treat a diff differently (raw + prefixes vs hunk-aware head lines), which is the class of mismatch this anchoring work is fixing. Move the escape helper to one place and read added lines through walkDiffLines.
There was a problem hiding this comment.
Not changing in this PR: nit. We'll consolidate it in a cleanup.
Code Review SummaryStatus: 12 Issues Found (1 critical, 3 warning, 8 suggestion) | Recommendation: Address before merge Overview
New in this round (2)
Previously reported, re-verified still present (10)
Files Reviewed (21 source files + 10 test files)
Fix these issues in Kilo Cloud Previous Review Summaries (2 snapshots, latest commit 4733424)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit 4733424)Status: 5 New Issues Found (1 critical) + 7 previously reported issues still unresolved | Recommendation: Address before merge Overview
New Issue Details (click to expand)CRITICAL
WARNING
SUGGESTION
Previously reported, verified still unresolved (click to expand)These keep their existing inline comments from the prior review round and were re-checked against HEAD.
Also unresolved from another reviewer's open inline comment: the inline-review failure path ( Files Reviewed (31 files)
Fix these issues in Kilo Cloud Previous review (commit bcc75a9)Status: 8 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (31 files)
Reviewed by deepseek-v4.1-flash-1017 · Input: 1.2M · Output: 122K · Cached: 14.9M Review guidance: REVIEW.md from base branch |
Review order for the dbt PR review stackEach PR is based on the one above it, so review and merge top to bottom.
Overall on 25 repos never used for tuning (101 PRs, 153 reviewer-found defects):
For comparison, on the same repos CodeRabbit's comments were 19% useful / 8% wrong, and human reviewers' 22% / 1%. The AI improvements beyond #1769 stay opt-in: their gain is within noise on fresh repos. 🤖 Generated with Claude Code Merge planBase first, one PR at a time:
Review feedback on a PR lands as a new commit on that PR (not a rebase), so "changes since your last review" shows only the fix. The PRs above it are then rebased. |
hkaltai
left a comment
There was a problem hiding this comment.
Verdict: request changes (1 MAJOR, 4 MINOR, plus NITs). Panel: Claude, Codex and lead. All 31 files were covered. bun test test/altimate/review passes (591).
Counts: CRITICAL 0, MAJOR 1, MINOR 4 (inline) + 4 more below, NIT 8.
The design is sound: shared diff walk, deletion-site anchoring, ledger of unreviewed files, signed posting policy. The MAJOR is in the posting fallback; the inline comments carry details.
More MINOR items (no inline anchor)
rule-tiers.ts:OFFcontains engine-origin rules not covered byENGINE_FLOOR(altimate-core.grain:conflicts-declared-uniqueness,altimate-core.source-filters:sibling-mismatch,altimate-core.structural-diff:*). Those are verdict-eligible, so the defaults change verdicts, which the floor comment and config doc say should not happen. Either add them to the floor or reword and call it out.rule-tiers.tsruleTier: override keys are matched raw while ids go throughnormRuleToken, so{"dbt_patterns:self_join":"off"}silently fails to matchdbt-patterns:self-join. Normalise keys on both sides.project-rules.tsTEMP_MARKER: new alternativestake (this|it) outandwhile debuggingcan match comments meant to stay (inline tier). Require removal intent or move them to summary.finding.ts/orchestrate.tsvalidateSuggestions: no production code setsFinding.suggestion, so suggestion blocks are untested end to end; the fallback also emits a no-op suggestion when replacement equals current, with a hard-coded sql fence.
Missing tests: postGitHubReview fallback summary content; AI finding with summary override; non-normalised override keys; AI file cap vs ledger; partial batch failure rendering; clampBody cutting inside a code fence.
Note: tsc --noEmit is not fully clean at the repo level (11 errors in packages/tui/src/component/dialog-move-session.tsx, untouched by this PR).
Nits
orchestrate.ts:2065manifestMissing && !anyManifest:manifestMissingalready implies!anyManifest; drop the second term.orchestrate.ts:~1686anchorDedupIssuerecovers column names by parsing[...]from the human-readable body; carry them as structured data.ledgerBlockwraps paths in a plain backtick span; usecodeSpan.normalizeConditionlowercases double-quoted identifiers, which are case-sensitive on Snowflake and Postgres.orchestrate.tsledger heading "Not reviewed" also lists model files that SQL/pattern/PII lanes did reach (no-manifest entry); use a "partially reviewed" group. Theno checks run for ${kind} files at the ${tier} tierwording is wrong where tier is irrelevant.rule-tiers.tsheader and held-out comment refer to a benchmark readers cannot see; add a pointer.diff-filter.ts:~669comment "Normalize to forward slashes..." sits above theexcludersline rather than thenorm(f.path)call it describes.- Naming:
evidenceLineholds text (considerevidenceText);touch/touchedcould bereachedByLane;scanDiffLines/walkDiffLines/walkDiffRowsare easy to confuse.
| // missing it would post a duplicate summary on every rerun. A silent clean | ||
| // run posts nothing, but still overwrites an earlier summary so it does | ||
| // not keep showing findings that are gone. | ||
| const summary = renderSummary(env, { alreadyPosted: posted }) |
There was a problem hiding this comment.
[MAJOR] Inline-eligible findings lose their body if the inline review is rejected
const summary = renderSummary(env, { alreadyPosted: posted })The summary is rendered and posted before the inline review is attempted. Inline-eligible findings appear in it only as a one-line pointer (file:line and title); their bodies live only in the inline comments. If createReview then fails (a positioning 422, "can not request changes on your own PR", missing permissions), the fallback posts only the verdict headline and the summary is never re-rendered. Failure scenario: a temporary-code-marker warning on an added line. The summary shows ### Inline (1) plus the title, GitHub rejects the batch, and the PR carries no explanation or fix for the finding. Before this PR the summary listed every finding in full, so the fallback lost nothing. Reproduced by two reviewers with a client that returns a 422 and then accepts the event-only review.
Suggested fix: when inline posting falls back or errors, re-render the summary with inline: false and update the summary comment (or include the bodies in the fallback review).
There was a problem hiding this comment.
Thanks, good catch: the fallback path really did lose the bodies. Fixed in "fix(review): address review feedback: full summary on inline fallback, …" (e7a80ad929). When createReview with comments fails and the event-only review is posted, postGitHubReview now rewrites the summary comment with renderSummary(env, { inline: false }), so every finding (including the ones that would have gone inline) carries its full body and fix. This applies to any failure (positioning 422, own-PR, permissions). A new test in review-failures-gate.test.ts uses a client that returns 422 and then accepts the event-only review. It checks that the first summary has only the ### Inline (1) pointer and the rewritten summary has the full body.
There was a problem hiding this comment.
✅ Fixed — post-github.ts:257-264: when the inline createReview fails and comments existed, the summary is re-rendered with inline: false, allowSilent: false and updateCommentd before the event-only review; a failed update is swallowed. Applies to the 422 and non-422 paths. review-failures-gate.test.ts (422 client) asserts the full body after the rewrite; neutralising the guard makes it fail. Optional extra tests: a non-positioning error and an updateComment failure.
| let level = 0 | ||
| let close = -1 | ||
| for (let i = open; i < text.length && close < 0; i++) { | ||
| if (text[i] === "(") level++ |
There was a problem hiding this comment.
[MINOR] CTE paren balancing counts parentheses inside string literals
if (text[i] === "(") level++text only has comments blanked, so quoted literals still contribute parentheses. with final as (select id, ')' as label, 1 as amount from t) select * from final makes finalSelectNames() return ["id"] (reproduced), so amount is reported as a removed final-layer column and the PR can get a false REQUEST_CHANGES. The trigger is narrow (a paren inside a string in a select * from <cte> model), which is why this is MINOR.
Suggested fix: scan the output of the existing literal-blanking step (or skip quoted strings and identifiers) while balancing, keeping offsets into the original SQL.
There was a problem hiding this comment.
Thanks for the precise repro. This is already handled at the branch head: finalSelectNames balances parentheses over a copy of the SQL with comments and string contents blanked ('...' becomes underscores of the same length), so offsets still line up and a ')' literal can no longer close the CTE body early. This landed in "fix(review): address review comments: UNION output names, package parents in graph context" (4ad40fce41). Your example now returns ["id", "label", "amount"].
There was a problem hiding this comment.
✅ Fixed — select-list.ts:120 blanks string-literal contents (incl. '' escapes) before paren balancing; the repro now returns ["id","label","amount"]. Regression in review-select-list.test.ts fails when the blanking is reverted. Residual NIT: parens inside double-quoted or backtick identifiers (")x") are still counted.
| /** Whether a finding may be posted as an inline comment: an inline-tier rule | ||
| * or the AI lane, on a line the PR added, at or above the severity floor. */ | ||
| export function inlineEligible(f: Finding, posting: PostingPolicy): boolean { | ||
| if (f.tier !== "inline" && findingOrigin(f) !== "ai") return false |
There was a problem hiding this comment.
[MINOR] AI findings bypass an explicit summary tier override
if (f.tier !== "inline" && findingOrigin(f) !== "ai") return falseWith rules: { "ai-review:*": summary } the orchestrator stamps tier: "summary", but this origin exemption still returns eligible, so the comment goes inline (confirmed end to end by two reviewers). off works because those findings are dropped earlier. This contradicts the config doc ("summary keeps it to the summary comment").
Suggested fix: resolve AI findings to inline as the default tier inside ruleTier and drop the origin bypass, so an explicit override wins.
There was a problem hiding this comment.
Agreed, the origin exemption contradicted the config doc. Fixed in "fix(review): address review feedback: …, AI tier overrides, …" (e7a80ad929) the way you suggested. ruleTier now resolves ai-review:* rules to inline as the default, after exact and glob overrides, and the findingOrigin(f) === "ai" bypass in inlineEligible is removed. rules: { "ai-review:*": summary } therefore keeps AI findings in the summary. An end-to-end test in review-ai-inputs.test.ts runs the orchestrator with and without that override, and review-noise.test.ts has a resolution-order test.
Heads-up for the next PR: the verifier PR (#1769) changes the same condition in inlineEligible, and I will carry this change through when restacking.
There was a problem hiding this comment.
✅ Fixed — the findingOrigin === "ai" bypass is gone from inlineEligible (format.ts:398); ruleTier defaults ai-review:* to inline after exact/glob overrides (rule-tiers.ts:181). Tests in review-noise.test.ts and the end-to-end review-ai-inputs.test.ts pass and fail when either change is reverted.
| supporting.forEach((f, i) => | ||
| aiFiles.push({ path: f.path, status: f.status, model: modelNameFromPath(f.path), kind: f.kind, diff: f.diff, sql: contents[i] }), | ||
| ) | ||
| touch(aiFiles) |
There was a problem hiding this comment.
[MINOR] Coverage ledger marks AI candidates reviewed before the file cap applies
touch(aiFiles)Every candidate is recorded as reached before runAiReview applies its file limit. With 21 changed seed files and a default limit of 20, one file gets no deterministic check and no ledger entry, so the ledger is empty and the verdict is APPROVE (reported by Codex). Suggested fix: have the AI lane return the paths it actually sent and build coverage from that.
There was a problem hiding this comment.
Thanks, confirmed: with 21 seed files one fell out of both the AI call and the ledger. Fixed in "fix(review): address review feedback: …, AI file-cap ledger" (e7a80ad929). The orchestrator now marks reached only selectAiFiles(aiFiles, aiInputBudget().maxFiles), the same selection and cap runAiReview applies, so the 21st file stays under "Not reviewed". New test: 21 changed seeds give a ledger of exactly seeds/s20.csv, and the summary shows it. One note: a "Not reviewed" entry alone does not stop an APPROVE verdict. That matches how other unchecked files are treated today, so the verdict policy is unchanged here. Only the ledger is now accurate.
There was a problem hiding this comment.
✅ Fixed (for the cap) — orchestrate.ts:2477 now touches selectAiFiles(aiFiles, aiInputBudget().maxFiles), the same selection the lane applies; the 21-seed test keeps seeds/s20.csv in the ledger. Related remaining item (new MINOR below): the touch still happens before the lane result is known.
| if (failed.length === results.length) | ||
| return done([], { state: failed[0].state, reason: failed[0].reason, model: modelLabel, ...spent }) | ||
| const reason = failed.length ? `${failed.length} of ${results.length} calls failed: ${failed[0].reason}` : undefined | ||
| return done(findings, { state: "ok", model: modelLabel, ...(reason ? { reason } : {}), ...spent }) |
There was a problem hiding this comment.
[MINOR] Partial batch failure is reported as state: "ok" and its reason is not rendered
return done(findings, { state: "ok", model: modelLabel, ...(reason ? { reason } : {}), ...spent })coverageNotes() shows reasons only for non-ok states, so when one batch is clean and another times out, the run can look clean and, with silentWhenClean, post nothing. Suggested fix: use an explicit partial state, or render the reason whenever one is present, including the affected paths.
There was a problem hiding this comment.
Good point, the reason was recorded but never shown. Fixed in "fix(review): address review feedback: …" (e7a80ad929) by taking the second option. coverageNotes now renders Advisory AI review partly failed: <reason> whenever the lane is ok but carries a reason (for example "1 of 2 calls failed: timed out"). That makes the summary non-silent even under silentWhenClean. The lane state stays ok so the telemetry and envelope schema don't change. Listing the affected paths would mean threading per-batch file lists through BatchResult, and I left that as a follow-up. A test covers the rendered note.
There was a problem hiding this comment.
format.ts:281-284 now renders Advisory AI review partly failed: <reason> for an ok lane with a reason, which also stops silentWhenClean from going silent, and a test covers it. Still open (non-blocking, acknowledged as follow-up): the note does not name the files of the failed batches, and those files remain marked as reached in the ledger.
3d72154 to
bb6a1fe
Compare
bcc75a9 to
4733424
Compare
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
| "dbt-patterns:float-equality", | ||
| "altimate-core.structural-diff:group-by-change", | ||
| "rule-catalog:bq-array-no-offset", | ||
| "altimate-core.structural-diff:join-key-regression", |
There was a problem hiding this comment.
CRITICAL: Default off silently disables the engine's blocking join-key check
altimate-core.structural-diff:join-key-regression is emitted by structuralChangeLane as category: "join_risk", escalated to critical when core returns severity: "error" (orchestrate.ts:1637), and join_risk is in the default blockOn (rubric.ts:49). But cleanFindings drops every off rule before buildEnvelope (orchestrate.ts:2417-2420), and ENGINE_FLOOR has no altimate-core.structural-diff: prefix — so with the shipped defaults a PR that changes a join key produces no finding at all, and computeIdealVerdict can return APPROVE/COMMENT instead of REQUEST_CHANGES. The PR's own blocking test only passes because it forces rules: { "*": "summary" } (review.test.ts:1268), which confirms the default suppresses the blocker. Move this rule (and the other verdict-capable altimate-core.structural-diff rules) to summary, or add that prefix to ENGINE_FLOOR.
| const parsedRaw = ((parseRes.data ?? {}) as Record<string, unknown>).findings | ||
| if (parseRes.success === false || !Array.isArray(parsedRaw)) | ||
| return done([], { | ||
| return { |
There was a problem hiding this comment.
WARNING: The parse-failure return drops the usage it just read
usage is assigned at line 422, but this return (rewritten by this PR) omits it, unlike the no-text path at 423 and the success path at 475. A billed call whose answer core cannot parse is therefore counted in ai_calls with zero tokens and zero cost — exactly the accounting gap this PR adds telemetry to close. Add usage here too.
| // both SQL sides. This avoids duplicate engine calls and lets tiering see PII. | ||
| const modelFiles = reviewable.filter((f) => f.kind === "model_sql" || f.kind === "python_model") | ||
| // Every model gets at least the PII check; one that fails is recorded as unverified. | ||
| touch(modelFiles) |
There was a problem hiding this comment.
WARNING: A model whose head content cannot be read is still marked reviewed
touch(modelFiles) marks every model file as looked at before any lane runs, so the coverage ledger (reviewable.filter((f) => !touched.has(f.path)), line 2501) never lists it. getContent resolves to undefined on any git read failure instead of throwing (git.ts:148-150), and the only not_compiled record requires newSql to be truthy (lines 1912-1913). A shallow checkout, a rename the base cannot resolve, or a containment failure therefore skips every SQL lane for that model with no unverified entry and no ledger row — the run can still APPROVE while that model's SQL was never examined. Record unverified/notReviewed when a non-deleted model's head content resolves to undefined, or touch only after content was actually delivered to a lane.
| unverified_models: env.scope?.unverified?.length ?? 0, | ||
| ai_state: env.ai?.state, | ||
| ai_duration_ms: env.ai?.durationMs, | ||
| ai_tokens_total: env.ai?.tokens ? env.ai.tokens.inputTotal + env.ai.tokens.output : undefined, |
There was a problem hiding this comment.
SUGGESTION: ai_tokens_total omits reasoning tokens
Session.getUsage returns tokens.output without reasoning and bills tokens.reasoning separately at the output rate (session/index.ts:917-944), and this PR stores reasoning on the lane's tokens. Summing only inputTotal + output therefore under-reports total tokens on reasoning models while ai_cost includes them, so the two metrics disagree; the doc comment added in telemetry/index.ts:1608 explicitly says output includes reasoning. Add reasoning (and set ai_tokens_output to match), or correct the doc.
| "dbt-project-rules:ref-target-missing", | ||
| "rule-catalog:qualify-unsupported", | ||
| "dbt-project-rules:macro-param-shifted", | ||
| "rule-catalog:case-duplicate-when", |
There was a problem hiding this comment.
SUGGESTION: inline tier is inert for these suggestion-severity rules
rule-catalog:case-duplicate-when (rule-catalog.ts:594-596) and dbt-patterns:macro-var-no-default (dbt-patterns.ts:1006-1014) are emitted at suggestion severity, while inlineEligible requires severity >= inlineSeverityFloor and the default floor is warning. Under the shipped posting defaults these two of the 14 inline rules can never post inline (they fall through to the summary), so the tier entry is a no-op unless a repo lowers the floor. Raise their severity, or move them out of INLINE so the table reflects what actually posts.
bb6a1fe to
416ac62
Compare
4733424 to
e7a80ad
Compare
|
Thanks for the thorough pass, Harish. Addressed in "fix(review): address review feedback: full summary on inline fallback, AI tier overrides, override key spelling, AI file-cap ledger" (
|
hkaltai
left a comment
There was a problem hiding this comment.
Re-review at e7a80ad (previously reviewed bcc75a9). Panel: Claude, Codex and lead. The base branch moved since the last round, so only this PR's own diff against its current base was reviewed.
Verdict: no CRITICAL/MAJOR remains. The prior MAJOR is fixed, with a regression test that fails without the fix. 0 CRITICAL, 0 MAJOR, 1 MINOR (new, inline), plus nits. I ran the review-failures-gate, select-list, noise and ai-inputs suites (105 tests) and they pass.
The author's claim holds: when the inline review is rejected, the summary is rewritten with every finding in full. Replies are posted in each prior thread.
Prior findings
| Prior finding | Status |
|---|---|
| MAJOR post-github.ts fallback loses bodies | Fixed |
| MINOR select-list CTE paren scan in strings | Fixed (double-quoted/backtick identifiers remain, NIT) |
| MINOR format.ts AI findings ignore summary override | Fixed |
| MINOR orchestrate touch(aiFiles) before cap | Fixed for the cap (see new MINOR) |
| MINOR ai-review partial batch failure state ok | Partially fixed: now visible and non-silent, but failed files are not named and still count as reached |
| Body: engine rules default off but are not on ENGINE_FLOOR | Partially fixed: the ENGINE_FLOOR comment is updated, the config.ts:85 doc still overstates exact-id protection |
| Body: override keys not normalised | Fixed |
| Body: temporary-marker phrases too broad | Fixed |
| Body: suggestion blocks have no production producer and a no-op replacement is still rendered as a fix | Not fixed (non-blocking) |
Nits still open or new
orchestrate.ts:2523manifestMissing && !anyManifest: drop the redundant second term.orchestrate.ts:627anchorDedupIssuestill parses[...]from the body: carry the column names as structured data.orchestrate.ts:2522ledger textno checks run for ${kind} files at the ${tier} tieris also used for files dropped by the AI file cap; use a dedicated reason such as "past the AI file cap (ALTIMATE_REVIEW_AI_MAX_FILES)".- "Not reviewed" also lists models the SQL/pattern lanes did reach: use a "partially reviewed" group.
rule-tiers.ts:11still refers to a benchmark readers cannot see: add a pointer.diff-filter.ts:238comment about normalising slashes sits aboveexcludersrather than thenorm(f.path)call.rule-tiers.ts:150-169normRuleKeyturnsrule_catalog:join_*intorule-catalog:join*, which also matchesjoinfoo. Two raw keys that normalise to the same id resolve by insertion order. Keep the trailing-before*and warn on collisions.select-list.ts:120blank"..."and backtick identifiers like string literals before balancing parens.format.ts:282says failed calls' files "were not reviewed by the AI", but the ledger counts them as reached.orchestrate.ts:2476comment "Only the files within the AI lane's file cap are reached" overstates the same.config.ts:85doc: say that some engine rules are off by default and are not protected by the floor.ai-review.ts:435(base code): the in-band parse-failure return drops the already-readusage, so tokens and cost are undercounted. Already tracked as a follow-up by the author.- Naming:
evidenceLineholds text (evidenceText),touch/touchedcould bereachedByLane, andscanDiffLines/walkDiffLines/walkDiffRowsare easy to confuse.
Missing tests (optional): fallback on a non-positioning error and on an updateComment failure; an AI lane that is skipped or timed out with AI-only files; a paren inside a double-quoted identifier.
| aiFiles.push({ path: f.path, status: f.status, model: modelNameFromPath(f.path), kind: f.kind, diff: f.diff, sql: contents[i] }), | ||
| ) | ||
| // Only the files within the AI lane's file cap are reached; the rest stay in the ledger. | ||
| touch(selectAiFiles(aiFiles, aiInputBudget().maxFiles)) |
There was a problem hiding this comment.
[MINOR] AI files are marked "reached" before the lane result is known
touch(selectAiFiles(aiFiles, aiInputBudget().maxFiles))touch runs before input.aiReview is called, so the ledger treats these files as reviewed even when the lane returns skipped (no usable model), timeout or error. Scenario: a PR that changes only seeds/a.csv with no default model configured. scope.notReviewed stays empty, the verdict is APPROVE and the summary says "No issues found", with only a one-line "Advisory AI review skipped" note. With --no-ai the same seed is listed under "Not reviewed", so the two cases are inconsistent. A partial batch failure also leaves the failed batch's files counted as covered.
Suggested fix: touch only after the lane returns, and only for files it actually reviewed. For example, have runAiReview return the reviewed paths, which also removes the duplicated selectAiFiles/aiInputBudget selection here.
…at changed (opt-in) Of the defects the AI-on review missed on the public benchmark, 55% were in files the lane never fully saw: cut at 6,000 characters, or dropped by the 20-file cap. Most of the rest needed another file (an upstream model's columns, a seed's rows). Both are now available behind environment switches; defaults are unchanged. - `ALTIMATE_REVIEW_AI_MAX_FILES`, `ALTIMATE_REVIEW_AI_MAX_FILE_CHARS`: widen the file cap and per-file limit - `ALTIMATE_REVIEW_AI_BATCH_CHARS`: split the files across several calls of about this size instead of cutting them; findings merge, and a partly failed run says how many calls failed - `ALTIMATE_REVIEW_AI_GRAPH=1`: send the changed files' unchanged dbt parents as read-only context: models, seeds and snapshots by file, sources as their declared columns, and the project's own macros (`graphParents` on the runner, from the manifest; a changed yml maps through its tests) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…root A monorepo of dbt sub-projects often shares one macro directory, e.g. `dbt_subprojects/<name>/dbt_project.yml` with `macro-paths: ["macros", "../../dbt_macros"]`. The review stripped the leading `.` from such entries so they never matched, and it dropped every file outside the project root as out of scope, so shared macros were never reviewed. - Resolve each layout dir to a repo-relative path (`resolvedLayoutDirs`), dropping any that escapes the repository. - `classifyDbtFile` matches the full repo path against those dirs, so `dbt_macros/shared/x.sql` is a `macro`. - `scopeToProject` takes the layout and keeps files under any resolved layout dir as well as under the project root; `run.ts` passes it. - Tests cover the shared-macro case end to end (classification, scope, and the file reaching the AI lane) and a dir that escapes the repo. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The per-rule posting policy, the coverage ledger and per-rule measurement all need to say which rule produced a finding. Until now that lived in a mix of `evidence.result.rule`, `evidence.result.code` and free-text titles, so there was no single key to configure or count by. `makeFinding` now stamps `rule` as `tool:rule`, normalised by `ruleId` (lowercase, `_` to `-`), derived from the evidence when the caller does not pass one. Engine and manifest findings whose evidence carries no rule name get an explicit id at the call site, and the AI lane uses `ai-review:<category>`. The fingerprint is unchanged, so finding ids stay stable across re-reviews. The schema also gains the optional `preExisting`, `tier`, `evidenceLine` and `suggestion` fields that the following posting-policy changes fill in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Most review noise comes from a handful of rules that blind judging found wrong at least half the time and never valuable, while a small set of rules was right nearly every time but got lost among them. One severity knob cannot separate the two, so each rule now has a posting tier. `rule-tiers.ts` holds the default table: 15 rules go inline, the rules judged mostly wrong with no real benchmark catch are off, and everything else stays in the summary. Rules that were mostly wrong but still caught a real defect in the benchmark are kept at summary. Teams override any rule, or a whole tool with `tool:*`, under `rules:` in `.altimate/review.yml`; since the policy is read from the base branch, a PR cannot relax it. The engine rules that decide the verdict (equivalence, lineage, PII, contracts, data-diff) can be turned off only by exact id, never by a glob. `cleanFindings` drops `off` rules and stamps `tier` on the rest. The AI lane is grounded in the same cleaned list, so an off rule no longer steers it either. Detection tests in `review.test.ts` that exercise now-off rules opt into reporting every rule, since they test the lane, not the posting policy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A finding anchored on a diff context line points at code the PR left alone. Reporting it as if the PR introduced it is the most common reason reviewers dismiss a comment, and letting it count toward the verdict blames the author for code they did not write. - `scanDiffLines` returns both the commentable lines and the added (`+`) lines of a diff; `commentableLines` is now a thin wrapper with unchanged behaviour. - `anchorToDiff` keeps a context-line anchor but sets `preExisting: true`. Every line of a newly added file counts as added. - `addedLinesByFile` exposes the added-line sets for later tasks (suggestion validation, the verifier). - `mergeRepeats` no longer merges pre-existing and new instances of a rule. - `verdictEligible` ignores pre-existing findings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ndings go
Every finding with a line used to become an inline comment, so a busy PR got
dozens of diff comments, most from rules a reviewer would not act on, and a
clean run still posted a "no issues" comment and review on every push.
- New `posting` policy (config and signed envelope): `inlineBudget` (10),
`inlineSeverityFloor` (warning), `silentWhenClean` (true). Read from the
base branch with the rest of the policy, so a PR cannot loosen it.
- `inlineComments` returns `{ comments, deferred }`. Only inline-tier rules
and AI-lane findings on added lines at or above the floor are eligible,
ordered by severity, confidence, file and line; findings already on the
PR do not count toward the budget. Over-budget findings stay in the
summary.
- `renderSummary` lists inline findings as one-line pointers, renders the
rest by severity, and collapses pre-existing findings into a `<details>`
block; the size budget logic is unchanged. A clean run with nothing to
say about coverage renders `""` under `silentWhenClean`.
- `postGitHubReview` reads existing comments before rendering, skips the
summary and the review on a silent clean run (an earlier summary is still
overwritten so it does not show stale findings), and dismissal of a stale
blocking review still happens.
- Local output (CLI stdout, the agent tool) renders every finding in full.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…h the reason
A review that silently skips files reads as a clean bill of health for them.
The summary only counted out-of-scope and moved files, and said nothing about
files dropped by extension or exclude glob, non-dbt files, kinds no lane ran
on at the chosen tier, or models checked without a manifest.
- `ReviewScope.notReviewed` (signed, defaults to `[]` so older envelopes
still parse) holds `{ path, reason }` for each such file.
- `partitionChangedFiles` returns the reviewable files and the dropped ones
with their reason; `filterChangedFiles` wraps it unchanged.
- `scopeToProject` also returns `outOfScopePaths` and
`movedUnchangedPaths`; `run.ts` passes them into the scope.
- `runReview` adds dropped files, `other`/`analysis` files, files no lane
received at this tier, and a no-manifest note per model (only when the
runner reports no manifest), de-duplicated by path, plus the existing
`unverified` entries.
- The summary renders a collapsed "Not reviewed: n file(s)" block grouped
by reason (30 paths, then a count), replacing the out-of-scope and moved
count lines when the paths are known.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Moved files now go to the "Not reviewed" ledger by path, which replaced the count line that told readers `dbt_project.yml` folder config (`+schema`, `+materialized`, `+tags`) can still change how a moved model builds. The ledger's "moved without edits" group now carries that hint under its paths, and the count line reuses the same text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stion blocks Long finding bodies made inline comments and summary bullets hard to scan, the reader had to open the file to see which line a pattern matched, and a fix could only be described in prose. - `clampBody` cuts a body to 600 characters at a sentence (or word) boundary and appends "…", closing a code fence the cut left open. It applies only when rendering inline comments and summary bullets; the envelope keeps the full body. - `cleanFindings` copies the matched source text a lane records in `evidence.result.line` (today the dbt-patterns detectors) into `evidenceLine`, rendered as `` `file:line` — `<text>` `` in inline comments and single-finding summary bullets. - `validateSuggestions` runs before the final `cleanFindings`: a `suggestion` survives only when every line of its range was added by the PR, the head file has those lines, and the replacement changes them. Otherwise the field is dropped and the replacement is appended to the body as a "Suggested fix" SQL block. - `inlineComments` posts a surviving suggestion as a GitHub `suggestion` block anchored on its range: `line` is the range end, and a multi-line range adds `start_line`/`start_side`. `InlineComment` already is the `pulls.createReview` payload, so `post-github.ts` passes it through as is. No lane emits suggestions yet; the tests build findings with one directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Any decision about a verifier pass (the 2x token rule) needs a baseline of
what the advisory lane spends today, and nothing recorded it.
- After a batch's stream is drained, `streamUsage` reads `totalUsage` and
`providerMetadata` and runs them through `Session.getUsage`, the same
accounting the chat session uses (cached-token handling, model price
table). `Session` is imported lazily, as in `learn/usage.ts`. Any failure
returns no usage; accounting never fails the lane.
- `AiLaneStatus` gains `tokens { input, inputTotal, output, reasoning,
cacheRead }`, `cost` and `calls`. `laneUsage` sums the batches that
reported usage and counts every batch as a call. Usage survives a batch
whose response later fails to parse.
- The envelope's `ai` status carries the same optional fields.
- The `review_run` telemetry event gains `ai_tokens_total` (input including
cached, plus output), `ai_tokens_input`, `ai_tokens_output`, `ai_cost` and
`ai_calls`.
- The summary footer shows `· tokens N` when the lane reported usage.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rule findings had no line, so inline-tier rules never posted inline and were never tagged pre-existing. Each rule now records the raw added line it matched in `evidence.result.line`, and a new `anchorByEvidence` step in `cleanFindings` finds that line in the file's diff before `anchorToDiff` runs. - Rule catalog: `matchRule` returns the first added line that satisfied the `added` condition (the raw line, not the comment-stripped one), else the added line inside the matching ON clause, else the first added line where the `newSql` pattern shows. `ruleMatches` wraps it. - dbt-patterns: detectors that matched added lines without recording one now do (clock, fan-out join, COUNT distinctness, CASE without ELSE, self-join, window without PARTITION BY, multiple COUNT DISTINCT, macro var without a default). The dedup detectors record the added line that opens the window and keep the window text in `window`. - Project rules and incremental semantics record the matched line where a rule matches one (temporary marker, utcnow, ref target, macro signature, config keys, tests on NULL columns, `partition_by` change). - `anchorByEvidence` picks the first added line with the same trimmed text, else the first context line with it (marked pre-existing downstream), else the first added line containing it when it is at least 12 characters. It never anchors outside the diff. - `walkDiffLines` yields each hunk line with its head-side number and kind; `scanDiffLines` now uses it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ary tier Both rules were switched off on per-rule judging alone, but each one produced a real catch in the replay benchmark. Rules with a benchmark catch stay at summary tier, like the other exceptions in that list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the engine cannot decide whether a refactor is equivalent (no schema or unsupported SQL), the review no longer emits a finding. The model is listed under "Not reviewed" with a data-diff hint instead. Only 1 of 426 such findings matched a real defect on replay, and they were the largest source of volume (about one per PR). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Blind judging of rule findings on repos held out from tuning, with the decision rule fixed before judging (off rule with at least 2 of 6 valuable -> summary; inline rule with at least 3 of 6 wrong -> summary): - `rule-catalog:distinct-and-group-by`: off -> summary (2 of 3 valuable) - `dbt-project-rules:test-on-null-column`: inline -> summary (3 of 6 wrong) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The detector looked for the raw added line inside the tokenised `is_incremental()` filter, which never matches, so the finding never got a line and could not post inline. Match the added line on the creation column and a comparison instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two WHEN conditions that differ only in spacing around operators (`>= 1000` vs `>=1000`) were compared as different text, so the unreachable branch went unreported. Compare conditions with case and spacing folded outside string literals; literal text is compared verbatim, so `'A'` and `'a'` stay distinct. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s window form `ignore-nulls-inside-call` required `IGNORE NULLS` right before the closing parenthesis, so `ARRAY_AGG(status IGNORE NULLS ORDER BY d)` was missed. Allow a trailing `ORDER BY` / `LIMIT` inside the call. DuckDB accepts the in-call form on navigation window functions (`first_value`, `last_value`, `nth_value`, `lead`, `lag` with `OVER`) but rejects it on aggregates. A catalog rule's portability can now name engines that accept the construct only when a check holds on the file, so the advice is dropped for a DuckDB project's window functions and kept for its aggregates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e-marker` A comment such as `-- sample while debugging locally, take this out before release` was not recognised: the marker needed `temp`, `revert`, `do not merge` or `remove before`. Also match `take this/it out`, `while debugging` and `remove/delete/drop this/it before merge/release/deploy/prod`. The match still runs on comment text only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rule fired whenever `partition_by` carried a `granularity` key, so it reported a dbt-bigquery failure on DuckDB, Postgres and Snowflake projects, where the mismatch is not an error. Record the requirement as a BigQuery capability (`microbatchBatchSizeMatchesPartition`, from dbt-bigquery's `bq_validate_microbatch_config`) and fire only where it holds. When the adapter is a default guess, the `granularity` dict still identifies BigQuery, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A final-layer model renaming an alias (`customer_lifetime_value` -> `lifetime_value`) got no finding. Its output columns come from column lineage plus the final SELECT's own list; lineage without a schema drops columns built from aggregates over `select *` sources, and the final `select * from final` gave no list, so both sides looked the same. Follow a final `select * from <cte>` to that CTE's select list (up to five hops; a star over a table, a join or another star stays unknown). Used for compiled SQL and for the raw-model fallback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e text Core reports line numbers in the compiled SQL, which don't exist in a Jinja model's file, so its findings there (e.g. `non-deterministic-dedup`) never got a line and could not post inline. Record the compiled line's text as the evidence line (the number moves to `compiledLine`) so `anchorByEvidence` finds it among the added lines. Only text of at least 12 characters that is unique in the compiled SQL is used; anything else keeps no line, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`dbt.manifest:removed-final-layer-columns` and `dbt.manifest:removed-columns-declared-consumers` are whole-model findings with no line, so they could never post inline. They now anchor on the head side of the removal: - find the base line that defined the first removed column the diff shows (the last removed select item ending in the column, else the first removed line naming it); - take the `+` line at the same position in its change block (the rename or replacement), else the nearest `+` line in the hunk, else, for a pure deletion, the next (or previous) context line in the hunk; - a column no removed line names stays unanchored, and the body is unchanged. A new `deletionSite` field marks such a line as standing in for removed code, so `anchorToDiff` does not mark a context-line anchor `preExisting`; inline posting and the verdict treat it as the PR's change. Comments stay RIGHT-side. `walkDiffLines` moves to `diff-filter.ts` beside the new `walkDiffRows` (which also yields `-` rows) and is re-exported from `orchestrate.ts`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rule anchored only through the `{% macro name(` line's text, so a
multi-line signature whose parameter list changed on a later line (the
usual shape: one parameter per line) got no line. The finding now anchors
on the first added head line inside that macro's parameter list in the head
file, which is also the `{% macro` line when a single-line signature
changed. With no added line in the signature it falls back to the previous
evidence-text anchor.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… by the time column
The compiled-line text anchor misses when the window comes from a macro
(e.g. `{{ deduplicate_raw_table(order_by='_airbyte_extracted_at', ...) }}`)
or a Jinja-built line, so the compiled text is nowhere in the raw diff.
When that anchor fails on compiled SQL, the finding now takes the first
added line that opens a `row_number()` naming the time column the issue
reports, else the first added line ordering by it (`order by col`,
`order_by='col'`). No added line names the column → no line, as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rule is inline-tier and now anchors to the changed line, but at suggestion severity it stayed below the default `inlineSeverityFloor` (warning) and never posted inline. Dev-split precision draw: 5 of 7 inline findings judged valuable, none wrong. Effect on the no-AI bench (anchor mode): dev 18 newly inline-eligible findings on 11 PRs, 3 PRs reach the warning-pattern threshold; heldout_a 11 findings on 9 PRs, 3 PRs reach the threshold. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ummary A leave-one-repo-out re-derivation of the tier table moved 19 `off` rules to another tier when a single repo was dropped from the judging sample, so their `off` verdict is not robust. They now default to `summary` until they are re-judged on held-out repositories. The 7 inline rules that moved in the same check stay inline pending that re-judging. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ents in graph context - `select-list.ts`: the select list stops at a set operator, so a first branch without `FROM` (`select 1 as one union select 2`) keeps its names, and `finalSelectNames` follows `select * from <cte>` only from the branch that names the output (shared `finalSelectStart`), never from a later UNION branch. - `select-list.ts`: the CTE close-paren scan in `finalSelectNames` masks string literals, so a `')'` literal can't end the CTE body early. - `runner.ts`: `graphParents` skips nodes, model/seed/snapshot parents and sources from installed packages, whose `original_file_path` is relative to the package rather than the project (only with `ALTIMATE_REVIEW_AI_GRAPH=1`). - Tests for each. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, AI tier overrides, override key spelling, AI file-cap ledger - `post-github.ts`: when the inline review is rejected and the event-only fallback runs, rewrite the summary comment with every finding in full (`inline: false`), so inline-eligible findings keep their body and fix. - `rule-tiers.ts` / `format.ts`: AI findings default to `inline` inside `ruleTier`; the origin bypass in `inlineEligible` is gone, so an explicit `ai-review:*: summary` (or `off`) override wins. - `rule-tiers.ts`: override keys and rule ids are compared in the normalised `ruleId` spelling, so `dbt_patterns:self_join` matches `dbt-patterns:self-join`; the `ENGINE_FLOOR` comment now names the engine rules that default to `off`. - `orchestrate.ts`: the coverage ledger marks only the files within the AI lane's file cap as reached; files past the cap stay under "Not reviewed". - `format.ts`: a partial AI failure (state `ok` with a reason) is rendered as "Advisory AI review partly failed"; ledger paths use `codeSpan`. - `project-rules.ts`: `temporary-code-marker` requires removal intent for "take this out" and "while debugging". - Tests for each change. - Restack fixups: one `normalizeCondition` (literals and double-quoted identifiers kept verbatim), and the DuckDB `IGNORE NULLS` test checks the `alsoWorksOn` window-form exception. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
416ac62 to
3be068e
Compare
e7a80ad to
2dcef4b
Compare
| * rule in `blockingRules`; advisory AI findings never. A finding on a line the | ||
| * PR did not add (`preExisting`) never does: the PR did not introduce it. */ | ||
| export function verdictEligible(f: Finding, rubric: Rubric = DEFAULT_RUBRIC): boolean { | ||
| if (f.preExisting) return false |
There was a problem hiding this comment.
WARNING: preExisting findings are excluded from the verdict but still counted in the headline
computeIdealVerdict filters through verdictEligible, so this early return means a finding anchored on a context line cannot block (or even downgrade) the run. But summarize() - which feeds env.summary and therefore verdictHeadline (format.ts:25-36) - still tallies every finding. A PR whose only finding is pre-existing therefore renders as Approved — 1 critical (full tier) (or Approved — 3 warning) while the body files the same finding under "Pre-existing". That is the most visible way the new trust rule can look self-contradictory. Count verdict-eligible findings for the headline, or subtract preExisting from the tally and report it separately.
| * the replacement changes them. Otherwise the replacement moves into the body | ||
| * as a plain "Suggested fix" block. | ||
| */ | ||
| export async function validateSuggestions( |
There was a problem hiding this comment.
SUGGESTION: The suggestion pipeline has no production producer
Nothing sets Finding.suggestion: every makeFinding caller (ai-review.ts, dbt-patterns.ts, project-rules.ts, incremental-semantics.ts, rule-catalog.ts) omits it, including the AI transport, which maps only explicit fields from the core parse result. So validateSuggestions here, suggestionBlock in format.ts, and the start_line/start_side branch of inlineComments are exercised only by tests. Either wire it to the core review_ai_parse contract (the parser would have to return a replacement) or drop the field and the code that renders it.
Issue for this PR
Part of #1792
Type of change
What does this PR do?
Stacked on #1739. This PR changes what the review posts and where, so each PR gets fewer, better-placed comments. It adds no model calls.
Results (replay of public dbt PRs, rules only)
What changed
Posting policy
ruleid.rule-tiers.ts) puts each rule inline, in the summary, or off. Repos can override it withrules:in the review config, which is read from the base branch. Engine checks that decide the verdict can only be turned off by an exact key.postingconfig: an inline budget (default 10), a severity floor (defaultwarning), and a silent clean run (default on).Placement and form
Coverage
macro-pathsthat point outside the dbt project root are honoured.Measurement
ALTIMATE_REVIEW_AI_MAX_FILES,ALTIMATE_REVIEW_AI_MAX_FILE_CHARS,ALTIMATE_REVIEW_AI_BATCH_CHARS,ALTIMATE_REVIEW_AI_GRAPH). All are off by default.Rule fixes from the planted-bug suite
case-duplicate-when,IGNORE NULLSplacement,temporary-code-markerphrasing,microbatch-granularity-mismatchadapter gating,select *through CTEs.removed-final-layer-columnsis raised to warning so it can post inline.How did you verify your code works?
bun test test/altimate/review: 591 pass.bunx tsc --noEmit: clean.Cost side effects
None by default. No new model calls, warehouse queries or cloud resources; posting policy and placement are local. The opt-in switches (
ALTIMATE_REVIEW_AI_MAX_FILES,ALTIMATE_REVIEW_AI_MAX_FILE_CHARS,ALTIMATE_REVIEW_AI_BATCH_CHARS,ALTIMATE_REVIEW_AI_GRAPH) can enlarge AI-lane prompts when set; all are off by default. The AI lane's tokens, cost and calls are now recorded in the envelope and telemetry, so the cost is visible.Screenshots / recordings
Not a UI change.
Checklist
Changes since review:
e7a80ad929addresses the second-round review feedback: full summary on the event-only inline fallback, explicitai-review:*tier overrides win, override keys matched in normalised spelling, AI file-cap respected in the coverage ledger.🤖 Generated with Claude Code
Summary by cubic
The baseline description already covers most of the changes well. The new commits since the last summary haven't materially changed the PR's scope; they add a few more rule fixes and edge cases, but the description's key facts (rule tiers, inline budget, pre-existing tags, coverage ledger, measurement, planted-bug fixes) remain accurate.
However, the checklist/bullet count is long. Let me condense the "Posting policy" section and merge the rule-fix items. Key new details worth keeping:
select *lineage through CTEs (already in baseline)deletionSite(add this detail)non-deterministic-dedupordering-line anchoring (minor)The baseline says "591 tests pass" — that's still the count in the latest code.
I'll keep the structure but trim wording, merge the rule-fix bullet points, and add the
deletionSitenotion briefly. Target length similar to baseline (which is ~400 words).Draft:
Changes what the dbt review posts and where, so each PR gets fewer, better-placed comments without adding model calls. Replayed on public dbt PRs, findings per PR drop from 15.6 to 10.0 (8.0 to 5.1 on repos never used for tuning) while keeping the same reviewer-detected defects; blind judging rates 48% of inline comments useful (up from 7%) and 5% wrong (down from 29%).
Posting policy
ruleid; a default tier table (rule-tiers.ts) puts each rule inline, in the summary, or off. Repos override it withrules:in the review config, read from the base branch so a PR cannot relax its own review; verdict-deciding engine checks are only disabled by exact key.deletionSite, counts as the PR's own change): they never post inline and never count toward the verdict.postingconfig adds an inline budget (10), a severity floor (warning), and silent clean runs that post nothing.distinct-and-group-byoff→summary andtest-on-null-columninline→summary.Coverage and measurement
macro-pathsoutside the project root are honored.case-duplicate-whenspacing,IGNORE NULLSplacement,temporary-code-markerphrasing,microbatch-granularity-mismatchadapter gating, andselect *lineage through CTEs.removed-final-layer-columnsis raised to warning so it can post inline. 591 tests pass andtsc --noEmitis clean.Changes made vs baseline:
deletionSitedetail to the pre-existing bullet.That's about the same length as the baseline, covers the new commits, and applies the writing rules.Changes what the dbt review posts and where, so each PR gets fewer, better-placed comments without adding model calls. Replayed on public dbt PRs, findings per PR drop from 15.6 to 10.0 (8.0 to 5.1 on repos never used for tuning) while keeping the same reviewer-detected defects; blind judging rates 48% of inline comments useful (up from 7%) and 5% wrong (down from 29%).
Posting policy
ruleid; a default tier table (rule-tiers.ts) puts each rule inline, in the summary, or off. Repos override it withrules:in the review config, read from the base branch so a PR cannot relax its own review; verdict-deciding engine checks are only disabled by exact key.deletionSite, counts as the PR's own change): they never post inline and never count toward the verdict.postingconfig adds an inline budget (10), a severity floor (warning), and silent clean runs that post nothing.distinct-and-group-byoff→summary andtest-on-null-columninline→summary.Coverage and measurement
macro-pathsoutside the project root are honored.case-duplicate-whenspacing,IGNORE NULLSplacement,temporary-code-markerphrasing,microbatch-granularity-mismatchadapter gating, andselect *lineage through CTEs.removed-final-layer-columnsis raised to warning so it can post inline. 591 tests pass andtsc --noEmitis clean.Written for commit 2dcef4b. Summary will update on new commits.