Skip to content

feat(review): post fewer, better-placed comments: rule tiers, inline budget, pre-existing tags, coverage ledger - #1758

Open
RaghwendraSingh wants to merge 28 commits into
fix/review-false-positivesfrom
feat/review-posting-policy
Open

RaghwendraSingh wants to merge 28 commits into
fix/review-false-positivesfrom
feat/review-posting-policy

Conversation

@RaghwendraSingh

@RaghwendraSingh RaghwendraSingh commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Issue for this PR

Part of #1792

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Stacked on #1739. This PR changes what the review posts and where, so each PR gets fewer, better-placed comments. It adds no model calls.

Results (replay of public dbt PRs, rules only)

Split Findings per PR, before → after Reviewer-found defects caught, before → after
Tuning repos, 342 PRs 15.6 → 10.0 52 → 51
Repos never used for tuning, 71 PRs 8.0 → 5.1 10 → 10
  • At most 10 inline comments per PR. 99–100% of them sit on a line the PR changed (before: 35–75%).
  • Inline rule comments judged blind: 48% useful and 5% wrong (before: 7% useful, 29% wrong).
  • Planted-bug suite (104 injected defects, each with a no-defect twin): 100 caught, 0 false alarms on the twins.

What changed

Posting policy

  • Every finding carries a stable rule id.
  • A per-rule tier table (rule-tiers.ts) puts each rule inline, in the summary, or off. Repos can override it with rules: in the review config, which is read from the base branch. Engine checks that decide the verdict can only be turned off by an exact key.
  • Rules whose tier changed when one repo was left out of the judging sample stay at summary until they are re-judged.
  • Findings on lines the PR did not add are tagged pre-existing. They never post inline and never affect the verdict.
  • posting config: an inline budget (default 10), a severity floor (default warning), and a silent clean run (default on).

Placement and form

  • Rule findings are anchored to the diff line they matched, including Jinja models (by compiled line text) and removed columns.
  • Bodies are capped at 600 characters when rendered (the JSON envelope keeps the full text). The matched line is quoted, and valid fixes post as GitHub suggestion blocks.
  • The summary lists inline comments, then the remaining findings by severity, then pre-existing ones.

Coverage

  • A ledger lists every changed file the review did not look at, with the reason (excluded, out of scope, no checks for that kind, no manifest, undecidable equivalence).
  • macro-paths that point outside the dbt project root are honoured.

Measurement

  • The AI lane records tokens, cost and calls in the envelope and telemetry.
  • Opt-in switches let the AI lane see whole files and dbt parents (ALTIMATE_REVIEW_AI_MAX_FILES, ALTIMATE_REVIEW_AI_MAX_FILE_CHARS, ALTIMATE_REVIEW_AI_BATCH_CHARS, ALTIMATE_REVIEW_AI_GRAPH). All are off by default.

Rule fixes from the planted-bug suite

  • case-duplicate-when, IGNORE NULLS placement, temporary-code-marker phrasing, microbatch-granularity-mismatch adapter gating, select * through CTEs.
  • removed-final-layer-columns is raised to warning so it can post inline.

How did you verify your code works?

  • bun test test/altimate/review: 591 pass.
  • bunx tsc --noEmit: clean.

Cost side effects

None by default. No new model calls, warehouse queries or cloud resources; posting policy and placement are local. The opt-in switches (ALTIMATE_REVIEW_AI_MAX_FILES, ALTIMATE_REVIEW_AI_MAX_FILE_CHARS, ALTIMATE_REVIEW_AI_BATCH_CHARS, ALTIMATE_REVIEW_AI_GRAPH) can enlarge AI-lane prompts when set; all are off by default. The AI lane's tokens, cost and calls are now recorded in the envelope and telemetry, so the cost is visible.

Screenshots / recordings

Not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR
  • I checked for cost side effects (model calls, queries or cloud resources that grow with user data) and described them, or wrote "none"

Changes since review: e7a80ad929 addresses the second-round review feedback: full summary on the event-only inline fallback, explicit ai-review:* tier overrides win, override keys matched in normalised spelling, AI file-cap respected in the coverage ledger.

🤖 Generated with Claude Code


Summary by cubic

The baseline description already covers most of the changes well. The new commits since the last summary haven't materially changed the PR's scope; they add a few more rule fixes and edge cases, but the description's key facts (rule tiers, inline budget, pre-existing tags, coverage ledger, measurement, planted-bug fixes) remain accurate.

However, the checklist/bullet count is long. Let me condense the "Posting policy" section and merge the rule-fix items. Key new details worth keeping:

  • select * lineage through CTEs (already in baseline)
  • Compiled-line text anchoring for Jinja models (already in baseline)
  • Removed-column anchoring with deletionSite (add this detail)
  • Macro signature anchoring, non-deterministic-dedup ordering-line anchoring (minor)

The baseline says "591 tests pass" — that's still the count in the latest code.

I'll keep the structure but trim wording, merge the rule-fix bullet points, and add the deletionSite notion briefly. Target length similar to baseline (which is ~400 words).

Draft:


Changes what the dbt review posts and where, so each PR gets fewer, better-placed comments without adding model calls. Replayed on public dbt PRs, findings per PR drop from 15.6 to 10.0 (8.0 to 5.1 on repos never used for tuning) while keeping the same reviewer-detected defects; blind judging rates 48% of inline comments useful (up from 7%) and 5% wrong (down from 29%).

Posting policy

  • Every finding carries a stable rule id; a default tier table (rule-tiers.ts) puts each rule inline, in the summary, or off. Repos override it with rules: in the review config, read from the base branch so a PR cannot relax its own review; verdict-deciding engine checks are only disabled by exact key.
  • Findings on lines the PR did not add are tagged pre-existing (a removal-site anchor, deletionSite, counts as the PR's own change): they never post inline and never count toward the verdict.
  • The posting config adds an inline budget (10), a severity floor (warning), and silent clean runs that post nothing.
  • Findings anchor to the matched diff line — including Jinja models (by compiled line text), core-check issues, macro signatures and removed columns; bodies cap at 600 characters when rendered, the matched line is quoted, and valid fixes post as GitHub suggestion blocks.
  • The summary lists inline comments, then remaining findings by severity, then pre-existing ones. Rules whose tier depended on the judging sample stay at summary until re-judged; held-out judging so far moved distinct-and-group-by off→summary and test-on-null-column inline→summary.

Coverage and measurement

  • A ledger lists every changed file the review did not look at, with the reason (excluded, out of scope, no matching checks, no manifest, undecidable equivalence); macro-paths outside the project root are honored.
  • The AI lane records tokens, cost, and calls in the envelope and telemetry; opt-in switches let it see whole files and dbt parents, all off by default.
  • Planted-bug fixes: case-duplicate-when spacing, IGNORE NULLS placement, temporary-code-marker phrasing, microbatch-granularity-mismatch adapter gating, and select * lineage through CTEs. removed-final-layer-columns is raised to warning so it can post inline. 591 tests pass and tsc --noEmit is clean.

Changes made vs baseline:

  1. Added the deletionSite detail to the pre-existing bullet.
  2. Expanded the anchoring bullet to mention core-check issues and macro signatures.
  3. Trimmed some redundant phrasing elsewhere.

That's about the same length as the baseline, covers the new commits, and applies the writing rules.Changes what the dbt review posts and where, so each PR gets fewer, better-placed comments without adding model calls. Replayed on public dbt PRs, findings per PR drop from 15.6 to 10.0 (8.0 to 5.1 on repos never used for tuning) while keeping the same reviewer-detected defects; blind judging rates 48% of inline comments useful (up from 7%) and 5% wrong (down from 29%).

Posting policy

  • Every finding carries a stable rule id; a default tier table (rule-tiers.ts) puts each rule inline, in the summary, or off. Repos override it with rules: in the review config, read from the base branch so a PR cannot relax its own review; verdict-deciding engine checks are only disabled by exact key.
  • Findings on lines the PR did not add are tagged pre-existing (a removal-site anchor, deletionSite, counts as the PR's own change): they never post inline and never count toward the verdict.
  • The posting config adds an inline budget (10), a severity floor (warning), and silent clean runs that post nothing.
  • Findings anchor to the matched diff line — including Jinja models (by compiled line text), core-check issues, macro signatures, and removed columns; bodies cap at 600 characters when rendered, the matched line is quoted, and valid fixes post as GitHub suggestion blocks.
  • The summary lists inline comments, then remaining findings by severity, then pre-existing ones. Rules whose tier depended on the judging sample stay at summary until re-judged; held-out judging so far moved distinct-and-group-by off→summary and test-on-null-column inline→summary.

Coverage and measurement

  • A ledger lists every changed file the review did not look at, with the reason (excluded, out of scope, no matching checks, no manifest, undecidable equivalence); macro-paths outside the project root are honored.
  • The AI lane records tokens, cost, and calls in the envelope and telemetry; opt-in switches let it see whole files and dbt parents, all off by default.
  • Planted-bug fixes: case-duplicate-when spacing, IGNORE NULLS placement, temporary-code-marker phrasing, microbatch-granularity-mismatch adapter gating, and select * lineage through CTEs. removed-final-layer-columns is raised to warning so it can post inline. 591 tests pass and tsc --noEmit is clean.

Written for commit 2dcef4b. Summary will update on new commits.

View guided diff Turn on auto-fix

@coderabbitai

coderabbitai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: f2b60ba9-60b8-4a13-b4cf-32421a262d30

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

9 issues found across 31 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/opencode/src/altimate/review/diff-filter.ts">

<violation number="1" location="packages/opencode/src/altimate/review/diff-filter.ts:102">
P2: `posix.join` treats an absolute `*-paths` entry as relative here, so `/shared/macros` becomes `<project-root>/shared/macros` instead of being dropped. Reject absolute entries before joining to avoid classifying and scoping the wrong files.</violation>
</file>

<file name="packages/opencode/src/altimate/review/ai-review.ts">

<violation number="1" location="packages/opencode/src/altimate/review/ai-review.ts:106">
P2: `ALTIMATE_REVIEW_AI_BATCH_CHARS` does not prevent large files from being cut: batching only groups whole files, and an oversized file is sent once after `truncate` removes its content. Split oversized file sections across calls, or document and enforce a hard per-file limit instead of promising preservation.</violation>

<violation number="2" location="packages/opencode/src/altimate/review/ai-review.ts:346">
P3: This comment's rationale looks wrong: the module already statically imports `@/session/llm` and `@/session/message-v2`, so the lazy `@/session` import does not keep the session module out of the graph. Either drop the claim or confirm the reason before keeping the comment.</violation>
</file>

<file name="packages/opencode/test/altimate/review-ci-honesty.test.ts">

<violation number="1" location="packages/opencode/test/altimate/review-ci-honesty.test.ts:59">
P2: This assertion can never fail: no finding uses the rule ID `altimate-core.equivalence:undecided`, so the test does not check that the undecided result stays out of the findings. Assert on the degraded flag or the semantic_change category instead.</violation>
</file>

<file name="packages/opencode/src/altimate/review/verdict.ts">

<violation number="1" location="packages/opencode/src/altimate/review/verdict.ts:49">
P2: Pre-existing findings are excluded from eligibility but still force `computeIdealVerdict` to return `COMMENT` because it checks the raw finding count first. Filter to verdict-eligible findings before the empty check so a run containing only pre-existing findings remains APPROVE and can be silent.</violation>

<violation number="2" location="packages/opencode/src/altimate/review/verdict.ts:142">
P2: `notReviewed` uses `.default([])` on a signed field. Parsing a legacy envelope with a `scope` but no `notReviewed` adds `notReviewed: []` to the canonical body, so `verifyEnvelope` fails for that envelope. Make the field `.optional()` like `policy.posting`, and have readers already using `?? []` treat a missing ledger as empty.</violation>
</file>

<file name="packages/opencode/src/altimate/review/rule-tiers.ts">

<violation number="1" location="packages/opencode/src/altimate/review/rule-tiers.ts:130">
P2: `ENGINE_FLOOR` omits engine checks that can affect the verdict. Every `altimate-core.*` finding is verdict-eligible, but `altimate-core.grain:`, `altimate-core.source-filters:` and `altimate-core.structural-diff:` are not in the floor. Their default `off` entries drop findings before the verdict, and a `tool:*` glob can silence them. Add these prefixes to `ENGINE_FLOOR`, or move the affected entries out of `OFF`.</violation>
</file>

<file name="packages/opencode/test/altimate/review-noise.test.ts">

<violation number="1" location="packages/opencode/test/altimate/review-noise.test.ts:447">
P3: This guard test claims engine rules are never off, but its prefix list omits `altimate-core.column-lineage:`, `altimate-core.classify-pii:`, and `consumer-column-list:` from `ENGINE_FLOOR`. Import the floor list or extend the list, and assert `ruleTier()` output for a floor rule.</violation>
</file>

<file name="packages/opencode/src/altimate/review/config.ts">

<violation number="1" location="packages/opencode/src/altimate/review/config.ts:87">
P2: Rule keys are not validated, so a typo in `rules:` is accepted and silently has no effect. Constrain keys to the `tool:rule` or `tool:*` shape so a malformed key fails at config load.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

const out: Array<{ role: DbtRole; dir: string }> = []
for (const [role, list] of Object.entries(layout.dirs) as Array<[DbtRole, string[]]>) {
for (const d of list) {
const dir = posix.normalize(posix.join(root || ".", norm(d))).replace(/\/+$/, "")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: posix.join treats an absolute *-paths entry as relative here, so /shared/macros becomes <project-root>/shared/macros instead of being dropped. Reject absolute entries before joining to avoid classifying and scoping the wrong files.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/diff-filter.ts, line 102:

<comment>`posix.join` treats an absolute `*-paths` entry as relative here, so `/shared/macros` becomes `<project-root>/shared/macros` instead of being dropped. Reject absolute entries before joining to avoid classifying and scoping the wrong files.</comment>

<file context>
@@ -86,17 +88,34 @@ export function relativeToRoot(p: string, root: string): string | undefined {
+  const out: Array<{ role: DbtRole; dir: string }> = []
+  for (const [role, list] of Object.entries(layout.dirs) as Array<[DbtRole, string[]]>) {
+    for (const d of list) {
+      const dir = posix.normalize(posix.join(root || ".", norm(d))).replace(/\/+$/, "")
+      if (!dir || dir === "." || dir === ".." || dir.startsWith("../") || dir.startsWith("/")) continue
+      out.push({ role, dir })
</file context>
Suggested change
const dir = posix.normalize(posix.join(root || ".", norm(d))).replace(/\/+$/, "")
if (norm(d).startsWith("/")) continue
const dir = posix.normalize(posix.join(root || ".", norm(d))).replace(/\/+$/, "")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. An absolute entry in a project-relative path list is unusual; we'll skip leading-/ entries in a follow-up.

test("undecidable equivalence on a manifest-backed run goes to the ledger, not lint-only", async () => {
const env = await review()
expect(env.findings.some((f) => f.degraded)).toBe(true)
expect(env.findings.some((f) => f.rule === "altimate-core.equivalence:undecided")).toBe(false)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This assertion can never fail: no finding uses the rule ID altimate-core.equivalence:undecided, so the test does not check that the undecided result stays out of the findings. Assert on the degraded flag or the semantic_change category instead.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/review-ci-honesty.test.ts, line 59:

<comment>This assertion can never fail: no finding uses the rule ID `altimate-core.equivalence:undecided`, so the test does not check that the undecided result stays out of the findings. Assert on the degraded flag or the semantic_change category instead.</comment>

<file context>
@@ -54,13 +54,12 @@ function review(extra: Partial<Parameters<typeof runReview>[0]> = {}) {
+  test("undecidable equivalence on a manifest-backed run goes to the ledger, not lint-only", async () => {
     const env = await review()
-    expect(env.findings.some((f) => f.degraded)).toBe(true)
+    expect(env.findings.some((f) => f.rule === "altimate-core.equivalence:undecided")).toBe(false)
     expect(env.summary.degraded).toBe(false)
-    const md = renderSummary(env)
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. Agreed the assertion can't fail (the undecided path returns []); we'll assert on degraded findings instead.

/** Every changed file the review did not look at (excluded, outside the
* project, moved, not a dbt file, no check at this tier, no manifest), with
* the reason; also repeats the `unverified` entries. */
notReviewed: z.array(z.object({ path: z.string(), reason: z.string() })).default([]),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: notReviewed uses .default([]) on a signed field. Parsing a legacy envelope with a scope but no notReviewed adds notReviewed: [] to the canonical body, so verifyEnvelope fails for that envelope. Make the field .optional() like policy.posting, and have readers already using ?? [] treat a missing ledger as empty.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/verdict.ts, line 142:

<comment>`notReviewed` uses `.default([])` on a signed field. Parsing a legacy envelope with a `scope` but no `notReviewed` adds `notReviewed: []` to the canonical body, so `verifyEnvelope` fails for that envelope. Make the field `.optional()` like `policy.posting`, and have readers already using `?? []` treat a missing ledger as empty.</comment>

<file context>
@@ -118,6 +136,10 @@ export const ReviewScope = z.object({
+  /** Every changed file the review did not look at (excluded, outside the
+   *  project, moved, not a dbt file, no check at this tier, no manifest), with
+   *  the reason; also repeats the `unverified` entries. */
+  notReviewed: z.array(z.object({ path: z.string(), reason: z.string() })).default([]),
 })
 export type ReviewScope = z.infer<typeof ReviewScope>
</file context>
Suggested change
notReviewed: z.array(z.object({ path: z.string(), reason: z.string() })).default([]),
notReviewed: z.array(z.object({ path: z.string(), reason: z.string() })).optional(),

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. Latent today (only tests call verifyEnvelope); switching to .optional() is queued.


/** Engine rules that may never default below summary (verdict-relevant). */
const ENGINE_FLOOR = [
"altimate-core.equivalence:",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: ENGINE_FLOOR omits engine checks that can affect the verdict. Every altimate-core.* finding is verdict-eligible, but altimate-core.grain:, altimate-core.source-filters: and altimate-core.structural-diff: are not in the floor. Their default off entries drop findings before the verdict, and a tool:* glob can silence them. Add these prefixes to ENGINE_FLOOR, or move the affected entries out of OFF.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/rule-tiers.ts, line 130:

<comment>`ENGINE_FLOOR` omits engine checks that can affect the verdict. Every `altimate-core.*` finding is verdict-eligible, but `altimate-core.grain:`, `altimate-core.source-filters:` and `altimate-core.structural-diff:` are not in the floor. Their default `off` entries drop findings before the verdict, and a `tool:*` glob can silence them. Add these prefixes to `ENGINE_FLOOR`, or move the affected entries out of `OFF`.</comment>

<file context>
@@ -0,0 +1,161 @@
+
+/** Engine rules that may never default below summary (verdict-relevant). */
+const ENGINE_FLOOR = [
+  "altimate-core.equivalence:",
+  "altimate-core.column-lineage:",
+  "impact-analysis:",
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing in this PR: turning those prefixes off is deliberate after judging, and an off rule is meant to leave the verdict too. We'll fix the ENGINE_FLOOR comment so it no longer reads as a contradiction.

* rule-tiers.ts. Read from the base branch, so a PR cannot relax it.
* Engine rules that decide the verdict can be turned off only by exact id.
*/
rules: z.record(z.string(), z.enum(["inline", "summary", "off"])).default({}),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Rule keys are not validated, so a typo in rules: is accepted and silently has no effect. Constrain keys to the tool:rule or tool:* shape so a malformed key fails at config load.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/config.ts, line 87:

<comment>Rule keys are not validated, so a typo in `rules:` is accepted and silently has no effect. Constrain keys to the `tool:rule` or `tool:*` shape so a malformed key fails at config load.</comment>

<file context>
@@ -77,6 +77,27 @@ export const ReviewConfig = z.object({
+   * rule-tiers.ts. Read from the base branch, so a PR cannot relax it.
+   * Engine rules that decide the verdict can be turned off only by exact id.
+   */
+  rules: z.record(z.string(), z.enum(["inline", "summary", "off"])).default({}),
+  /**
+   * How findings reach the PR: `inlineBudget` caps new inline comments per run
</file context>
Suggested change
rules: z.record(z.string(), z.enum(["inline", "summary", "off"])).default({}),
rules: z.record(z.string().regex(/^[\w.-]+:(\*|[\w-]+)$/), z.enum(["inline", "summary", "off"])).default({}),

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing: nit. Unknown keys are harmless no-ops, so we left validation out of this PR.

let cur: AiReviewFile[] = []
let size = 0
for (const f of files) {
const n = fileSection(f, budget.maxFileChars).length

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: ALTIMATE_REVIEW_AI_BATCH_CHARS does not prevent large files from being cut: batching only groups whole files, and an oversized file is sent once after truncate removes its content. Split oversized file sections across calls, or document and enforce a hard per-file limit instead of promising preservation.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 106:

<comment>`ALTIMATE_REVIEW_AI_BATCH_CHARS` does not prevent large files from being cut: batching only groups whole files, and an oversized file is sent once after `truncate` removes its content. Split oversized file sections across calls, or document and enforce a hard per-file limit instead of promising preservation.</comment>

<file context>
@@ -48,18 +75,62 @@ function kindRank(kind: string | undefined): number {
+  let cur: AiReviewFile[] = []
+  let size = 0
+  for (const f of files) {
+    const n = fileSection(f, budget.maxFileChars).length
+    if (cur.length && size + n > budget.batchChars) {
+      batches.push(cur)
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing: nothing promises that. batchAiFiles groups whole files, and per-file truncation is the separate maxFileChars knob (ai-review.ts:41-42, 92-95).

* rule in `blockingRules`; advisory AI findings never. A finding on a line the
* PR did not add (`preExisting`) never does: the PR did not introduce it. */
export function verdictEligible(f: Finding, rubric: Rubric = DEFAULT_RUBRIC): boolean {
if (f.preExisting) return false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Pre-existing findings are excluded from eligibility but still force computeIdealVerdict to return COMMENT because it checks the raw finding count first. Filter to verdict-eligible findings before the empty check so a run containing only pre-existing findings remains APPROVE and can be silent.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/verdict.ts, line 49:

<comment>Pre-existing findings are excluded from eligibility but still force `computeIdealVerdict` to return `COMMENT` because it checks the raw finding count first. Filter to verdict-eligible findings before the empty check so a run containing only pre-existing findings remains APPROVE and can be silent.</comment>

<file context>
@@ -43,8 +43,10 @@ export const VCS_EVENT: Record<Verdict, "COMMENT" | "REQUEST_CHANGES"> = {
+ *  rule in `blockingRules`; advisory AI findings never. A finding on a line the
+ *  PR did not add (`preExisting`) never does: the PR did not introduce it. */
 export function verdictEligible(f: Finding, rubric: Rubric = DEFAULT_RUBRIC): boolean {
+  if (f.preExisting) return false
   const origin = findingOrigin(f)
   if (origin === "engine") return true
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing: COMMENT on any finding is the documented rubric (verdict.ts:62). silentWhenClean also requires !env.findings.length (format.ts:64), so a run with only pre-existing findings behaves the same either way.

const usage = await stream.totalUsage
const metadata = await stream.providerMetadata
if (!usage) return undefined
// Loaded lazily, as in learn/usage.ts, to keep the session module out of

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This comment's rationale looks wrong: the module already statically imports @/session/llm and @/session/message-v2, so the lazy @/session import does not keep the session module out of the graph. Either drop the claim or confirm the reason before keeping the comment.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 346:

<comment>This comment's rationale looks wrong: the module already statically imports `@/session/llm` and `@/session/message-v2`, so the lazy `@/session` import does not keep the session module out of the graph. Either drop the claim or confirm the reason before keeping the comment.</comment>

<file context>
@@ -177,6 +274,110 @@ export async function runAiReview(input: AiReviewInput): Promise<AiReviewResult>
+    const usage = await stream.totalUsage
+    const metadata = await stream.providerMetadata
+    if (!usage) return undefined
+    // Loaded lazily, as in learn/usage.ts, to keep the session module out of
+    // this module's import graph.
+    const { Session } = await import("@/session")
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing: nit (ai-review.ts:346, lazy-import comment).

"dbt-patterns:fanout-join",
])
expect(ruleTier(k)).toBe("summary")
for (const p of ["altimate-core.equivalence:", "impact-analysis:", "schema.detect-pii:", "contract-check:", "data.diff:"])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This guard test claims engine rules are never off, but its prefix list omits altimate-core.column-lineage:, altimate-core.classify-pii:, and consumer-column-list: from ENGINE_FLOOR. Import the floor list or extend the list, and assert ruleTier() output for a floor rule.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/review-noise.test.ts, line 447:

<comment>This guard test claims engine rules are never off, but its prefix list omits `altimate-core.column-lineage:`, `altimate-core.classify-pii:`, and `consumer-column-list:` from `ENGINE_FLOOR`. Import the floor list or extend the list, and assert `ruleTier()` output for a floor rule.</comment>

<file context>
@@ -319,3 +327,551 @@ describe("repeats of one rule within a file", () => {
+      "dbt-patterns:fanout-join",
+    ])
+      expect(ruleTier(k)).toBe("summary")
+    for (const p of ["altimate-core.equivalence:", "impact-analysis:", "schema.detect-pii:", "contract-check:", "data.diff:"])
+      expect(keys.filter((k) => k.startsWith(p) && DEFAULT_RULE_TIERS[k] === "off")).toEqual([])
+  })
</file context>
Suggested change
for (const p of ["altimate-core.equivalence:", "impact-analysis:", "schema.detect-pii:", "contract-check:", "data.diff:"])
for (const p of ["altimate-core.equivalence:", "altimate-core.column-lineage:", "altimate-core.classify-pii:", "impact-analysis:", "schema.detect-pii:", "contract-check:", "consumer-column-list:", "data.diff:"])

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing: nit. The test pins the prefixes that matter for this change.

for (const uid of asArray<string>(node.depends_on?.nodes)) {
const parent = raw.nodes?.[uid]
if (parent?.original_file_path && ["model", "seed", "snapshot"].includes(parent.resource_type)) {
add(uid, { path: base + parent.original_file_path, kind: parent.resource_type, name: parent.name }, changed)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[WARNING]: Package parents resolve to the wrong file

original_file_path is relative to the package that defines the node, but the path here is built as base + parent.original_file_path for every model/seed/snapshot. A parent from an installed package (e.g. dbt_utils) therefore becomes a root-project path such as models/stg.sql instead of dbt_packages/dbt_utils/models/stg.sql. aiGraphContext then either silently drops the parent (file missing) or, if the root project happens to have a file at that relative path, sends unrelated content to the model as the upstream parent. The macro branch below already guards with macro.package_name === project; the node branches need the same check (or the package-root prefix).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 4733424: graphParents now skips installed-package nodes, parents and sources, because their original_file_path is relative to the package. Test added.

return changed.some((l) => l.includes(head)) && !/partition\s+by/i.test(sql.slice(open, end))
})
const hit = unpartitioned ? changed[0] : undefined
const hit = unpartitioned ? c.added.find((l) => stripComments(l).trim() === changed[0]) : undefined

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION]: Anchors to the first OVER ( line, not the unpartitioned one

changed[0] is the first added line that contains any over (, while unpartitioned may be true for a later clause. When a diff adds a partitioned window before an unpartitioned one, the finding anchors to the correctly partitioned line with a window function has no PARTITION BY message, and once dbt-patterns:window-no-partition is promoted to inline the comment is posted there. Have the matchAll predicate record the offending head (use find rather than some) and anchor that line.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. The rule is off by default, so this is dormant. We'll anchor it to the offending OVER when we enable it.

"The existing table keeps its old partitioning; incremental runs don't rebuild it. Run `--full-refresh` on deploy.",
// The `partition_by` key, else an added line inside its value.
addedLine(/\bpartition_by\b/) ??
input.added.find((l) => /\w{2}/.test(l) && (inlineNew.partition_by ?? "").includes(l.trim()))),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION]: The fallback anchor condition is inverted

inlineNew.partition_by is the config value (for example ['block_month']) and l is a full raw added line, so (inlineNew.partition_by ?? "").includes(l.trim()) asks whether the short value contains the whole line. That is essentially never true, so this fallback is dead: a partition_by change that edited only the value line (no added line holds the partition_by key) loses its anchor and stays in the summary. The comparison belongs the other way round (l.includes(...) on part of the value, guarded against an empty value).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing: inlineConfig stores the raw multi-line value text (incremental-semantics.ts:72-84), so an inner line of partition_by={...} is a substring of it and the fallback anchors correctly.

let cur: AiReviewFile[] = []
let size = 0
for (const f of files) {
const n = fileSection(f, budget.maxFileChars).length

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION]: batchChars does not bound the request that is actually sent

batchAiFiles measures only fileSection(...), but reviewBatch calls buildUserMessage, which also appends relatedContext (up to CONTEXT_TOTAL_CHARS = 40k characters, re-sent in every batch when ALTIMATE_REVIEW_AI_GRAPH=1), the grounding findings and the PR title/body. With the env switches on, a batch sized to ALTIMATE_REVIEW_AI_BATCH_CHARS can exceed it by roughly 40k characters per call, and neither the number of calls nor the total wall time (up to 5 minutes each, two at a time) has a hard cap. Include the shared context/grounding in the sizing, or clamp the final message.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. This only applies with the opt-in env (the default is unbounded); we'll include the shared context in the sizing.

return done([], { state: "timeout", reason: `no answer within ${Math.round(timeoutMs / 1000)} s`, model: modelLabel })
if (!text) return done([], { state: "error", reason: "the model returned no text", model: modelLabel })
if (controller.signal.aborted) return timedOut()
usage = await streamUsage(stream, model)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION]: Timed-out and failed calls spend tokens that are never accounted

Usage is read only after readStreamText resolves and the abort check passes. timedOut() above and the catch path return no usage, while laneUsage still counts those results in calls. Since this PR exists to establish a cost baseline, ai_tokens_total/ai_cost omit exactly the slow, aborted and provider-error calls and can report ai_calls > 0 with zero tokens. Read usage before the abort/error returns, or count only the calls that reported it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. We'll read usage before the abort/error returns so cost isn't undercounted.

// GitHub applies a suggestion to the comment's line range, so the
// comment spans exactly the replaced lines.
if (!s) return { path: f.file, line: f.startLine!, side: "RIGHT", body }
return {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION]: A suggestion range spanning two hunks takes down every inline comment

validateSuggestions only checks that each line of the range is an added line; two added lines in different hunks pass. GitHub rejects a multi-line comment whose start_line and line are not in the same hunk, and postGitHubReview classifies that 422 as a positioning error and re-posts the review with no inline comments at all (the whole batch is lost, not just the suggestion). Validate that the range lies inside one hunk, or drop the suggestion and fall back to the text block. Dormant until a lane emits suggestions.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing: validateSuggestions (orchestrate.ts) requires every line in startLine..endLine to be added. That makes the range one contiguous added block, which can't span two hunks.

.filter((f) => shouldReview(f.path))
.filter((f) => !excluders.some((re) => re.test(norm(f.path))))
.map((f) => ({ ...f, kind: classifyDbtFile(f.path, layout) }))
return partitionChangedFiles(files, extraExcludeGlobs, layout).reviewable

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION]: filterChangedFiles is now dead code

Nothing calls this wrapper any more (only doc comments reference it) since runReview switched to partitionChangedFiles, and keeping both names invites a future caller to pick the one that silently discards the dropped-file reasons the coverage ledger needs. Remove the wrapper, or keep one entry point that returns the reviewable set plus the dropped reasons.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing in this PR: nit. We'll remove it in a cleanup.


/** The first added line matching `re`, for anchoring a finding to the diff. */
const addedLine = (file: ChangedFile, re: RegExp) => addedLines(file.diff).find((l) => re.test(l))
const escapeRe = (s: string) => s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION]: Duplicate regex-escape helper and a second diff parser in one module

escapeRe is byte-for-byte the helper added at orchestrate.ts:382, and this file now keeps its own raw-string addedLines/addedLine parser next to the imported walkDiffLines (used only by signatureAnchor). The two diff readers treat a diff differently (raw + prefixes vs hunk-aware head lines), which is the class of mismatch this anchoring work is fixing. Move the escape helper to one place and read added lines through walkDiffLines.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing in this PR: nit. We'll consolidate it in a cleanup.

@kilo-code-bot

kilo-code-bot Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Code Review Summary

Status: 12 Issues Found (1 critical, 3 warning, 8 suggestion) | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 1
WARNING 3
SUGGESTION 8
New in this round (2)
File Line Issue
packages/opencode/src/altimate/review/verdict.ts 49 preExisting findings are excluded from computeIdealVerdict but summarize() still tallies them, so a PR whose only finding is on a context line renders as Approved — 1 critical / Approved — 3 warning while the body files it under "Pre-existing".
packages/opencode/src/altimate/review/orchestrate.ts 336 No producer sets Finding.suggestion (AI transport maps explicit fields only), so validateSuggestions, suggestionBlock and the start_line/start_side comment branch run only in tests. Wire it to the core parse contract or drop ~80 lines.
Previously reported, re-verified still present (10)
File Line Issue
packages/opencode/src/altimate/review/rule-tiers.ts 64 CRITICAL: altimate-core.structural-diff:join-key-regression defaults to off and ENGINE_FLOOR omits the structural-diff prefix, so cleanFindings drops the only structural rule allowed to block before the envelope; a changed join key can produce no finding and APPROVE.
packages/opencode/src/altimate/review/ai-review.ts 435 The parse-failure return omits usage, so a billed call whose answer core cannot parse is counted in ai_calls with 0 tokens and $0 (the other returns include it).
packages/opencode/src/altimate/review/orchestrate.ts 1884 touch(modelFiles) marks every model reviewed before any lane runs; a model whose content is unavailable skips its lanes with no unverified entry and no ledger row.
packages/opencode/src/altimate/review/ai-review.ts 107 batchAiFiles sizes only file sections; related context (up to 40k), grounding and PR text are unaccounted, and calls/wall time have no hard cap.
packages/opencode/src/altimate/review/ai-review.ts 422 Usage is read only after a successful stream, so timed-out and failed calls count in ai_calls with no tokens or cost.
packages/opencode/src/altimate/review/telemetry.ts 118 ai_tokens_total = inputTotal + output excludes reasoning, while Session.getUsage bills it separately and the new doc says output includes it.
packages/opencode/src/altimate/review/rule-tiers.ts 28 rule-catalog:case-duplicate-when and dbt-patterns:macro-var-no-default are suggestion severity, so with the default inlineSeverityFloor: warning their inline tier can never post inline.
packages/opencode/src/altimate/review/dbt-patterns.ts 822 Window-without-PARTITION anchors to changed[0], the first added OVER ( line, not the unpartitioned window that actually matched.
packages/opencode/src/altimate/review/diff-filter.ts 263 filterChangedFiles is an unused wrapper over partitionChangedFiles that discards the dropped-file reasons the ledger needs.
packages/opencode/src/altimate/review/project-rules.ts 44 Duplicate escapeRe helper: this PR also adds the same expression in orchestrate.ts; both are one-liners that belong in one shared place.
Files Reviewed (21 source files + 10 test files)
  • packages/opencode/src/altimate/review/adapter-capabilities.ts
  • packages/opencode/src/altimate/review/ai-review.ts - 3 issues
  • packages/opencode/src/altimate/review/config.ts
  • packages/opencode/src/altimate/review/dbt-patterns.ts - 1 issue
  • packages/opencode/src/altimate/review/diff-filter.ts - 1 issue
  • packages/opencode/src/altimate/review/finding.ts
  • packages/opencode/src/altimate/review/format.ts
  • packages/opencode/src/altimate/review/incremental-semantics.ts
  • packages/opencode/src/altimate/review/orchestrate.ts - 2 issues
  • packages/opencode/src/altimate/review/post-github.ts
  • packages/opencode/src/altimate/review/project-rules.ts - 1 issue
  • packages/opencode/src/altimate/review/rule-catalog.ts
  • packages/opencode/src/altimate/review/rule-tiers.ts - 2 issues
  • packages/opencode/src/altimate/review/run.ts
  • packages/opencode/src/altimate/review/runner.ts
  • packages/opencode/src/altimate/review/select-list.ts
  • packages/opencode/src/altimate/review/telemetry.ts - 1 issue
  • packages/opencode/src/altimate/review/verdict.ts - 1 issue
  • packages/opencode/src/altimate/telemetry/index.ts
  • packages/opencode/src/altimate/tools/dbt-pr-review.ts
  • packages/opencode/src/cli/cmd/review.ts
  • packages/opencode/test/altimate/* (10 review test files)

Fix these issues in Kilo Cloud

Previous Review Summaries (2 snapshots, latest commit 4733424)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 4733424)

Status: 5 New Issues Found (1 critical) + 7 previously reported issues still unresolved | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 1
WARNING 2
SUGGESTION 2
New Issue Details (click to expand)

CRITICAL

File Line Issue
packages/opencode/src/altimate/review/rule-tiers.ts 62 altimate-core.structural-diff:join-key-regression defaults to off, and ENGINE_FLOOR omits that prefix. cleanFindings drops off rules before the envelope, so the only structural rule allowed to block (join_risk + error → critical, rubric blockOn includes join_risk) never reaches the verdict; a changed join key can now produce no finding and APPROVE. The PR's own SC010 blocking test needs rules: {"*": "summary"} to keep passing.

WARNING

File Line Issue
packages/opencode/src/altimate/review/ai-review.ts 435 The parse-failure return omits the usage read at line 422 (the no-text and success paths pass it). A billed call whose answer core cannot parse is counted in ai_calls with 0 tokens and $0, which is the accounting gap the new telemetry is meant to close.
packages/opencode/src/altimate/review/orchestrate.ts 1869 touch(modelFiles) marks every model reviewed before any lane runs, while getContent resolves undefined on git read errors (no throw) and not_compiled only records when newSql is truthy. Such a model skips all SQL lanes with no unverified entry and no ledger row, so the run can APPROVE while its SQL was never examined.

SUGGESTION

File Line Issue
packages/opencode/src/altimate/review/telemetry.ts 118 ai_tokens_total = inputTotal + output excludes reasoning, but Session.getUsage bills reasoning separately at the output rate and the new doc comment says output includes reasoning, so token and cost telemetry disagree.
packages/opencode/src/altimate/review/rule-tiers.ts 26 rule-catalog:case-duplicate-when and dbt-patterns:macro-var-no-default are suggestion severity, so with the default inlineSeverityFloor: warning their inline tier can never post inline; 2 of the 14 inline rules are effectively no-ops.
Previously reported, verified still unresolved (click to expand)

These keep their existing inline comments from the prior review round and were re-checked against HEAD.

File Line Issue
packages/opencode/src/altimate/review/dbt-patterns.ts 816 Window-without-PARTITION anchors to changed[0], the first added OVER ( line, not the unpartitioned window that matched.
packages/opencode/src/altimate/review/incremental-semantics.ts 329 partition_by fallback anchor is inverted ((inlineNew.partition_by ?? "").includes(l.trim())), so it never matches and the finding loses its line.
packages/opencode/src/altimate/review/ai-review.ts 108 batchChars sizes only file sections; related context (up to 40k chars), grounding and PR text are unaccounted, and calls/wall time have no hard cap.
packages/opencode/src/altimate/review/ai-review.ts 422 Usage is read only after a successful stream, so timed-out and failed calls count in ai_calls with no tokens or cost.
packages/opencode/src/altimate/review/format.ts 430 validateSuggestions accepts a range spanning two hunks; GitHub rejects it and postGitHubReview drops the whole inline batch.
packages/opencode/src/altimate/review/diff-filter.ts 263 filterChangedFiles is an unused wrapper over partitionChangedFiles that discards the dropped-file reasons the ledger needs.
packages/opencode/src/altimate/review/project-rules.ts 42 Duplicate escapeRe helper and a second raw-string diff parser beside the imported walkDiffLines.

Also unresolved from another reviewer's open inline comment: the inline-review failure path (post-github.ts:145) still leaves inline-tier bodies out of the already-posted summary, and the CLI still reports "fell back to summary-only".

Files Reviewed (31 files)
  • packages/opencode/src/altimate/review/adapter-capabilities.ts
  • packages/opencode/src/altimate/review/ai-review.ts - 2 issues
  • packages/opencode/src/altimate/review/config.ts
  • packages/opencode/src/altimate/review/dbt-patterns.ts - 1 issue (carried)
  • packages/opencode/src/altimate/review/diff-filter.ts - 1 issue (carried)
  • packages/opencode/src/altimate/review/finding.ts
  • packages/opencode/src/altimate/review/format.ts - 1 issue (carried)
  • packages/opencode/src/altimate/review/incremental-semantics.ts - 1 issue (carried)
  • packages/opencode/src/altimate/review/orchestrate.ts - 1 issue
  • packages/opencode/src/altimate/review/post-github.ts
  • packages/opencode/src/altimate/review/project-rules.ts - 1 issue (carried)
  • packages/opencode/src/altimate/review/rule-catalog.ts
  • packages/opencode/src/altimate/review/rule-tiers.ts - 2 issues
  • packages/opencode/src/altimate/review/run.ts
  • packages/opencode/src/altimate/review/runner.ts
  • packages/opencode/src/altimate/review/select-list.ts
  • packages/opencode/src/altimate/review/telemetry.ts - 1 issue
  • packages/opencode/src/altimate/review/verdict.ts
  • packages/opencode/src/altimate/telemetry/index.ts
  • packages/opencode/src/altimate/tools/dbt-pr-review.ts
  • packages/opencode/src/cli/cmd/review.ts
  • packages/opencode/test/altimate/review-ai-inputs.test.ts
  • packages/opencode/test/altimate/review-ci-honesty.test.ts
  • packages/opencode/test/altimate/review-contracts-dialect.test.ts
  • packages/opencode/test/altimate/review-failures-gate.test.ts
  • packages/opencode/test/altimate/review-noise.test.ts
  • packages/opencode/test/altimate/review-project-rules.test.ts
  • packages/opencode/test/altimate/review-scope-diff.test.ts
  • packages/opencode/test/altimate/review-select-list.test.ts
  • packages/opencode/test/altimate/review-verdict-trust.test.ts
  • packages/opencode/test/altimate/review.test.ts

Fix these issues in Kilo Cloud

Previous review (commit bcc75a9)

Status: 8 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 7
Issue Details (click to expand)

WARNING

File Line Issue
packages/opencode/src/altimate/review/runner.ts 436 Graph parents from installed packages keep a package-relative original_file_path, so the joined path is wrong: the parent is silently dropped or an unrelated root file is sent as context.

SUGGESTION

File Line Issue
packages/opencode/src/altimate/review/dbt-patterns.ts 781 Anchors to the first added OVER ( line instead of the unpartitioned window that matched, so the comment can land on a correctly partitioned line.
packages/opencode/src/altimate/review/incremental-semantics.ts 279 Fallback anchor condition is inverted (value.includes(fullLine)), so it never matches and the partition-change finding loses its line.
packages/opencode/src/altimate/review/ai-review.ts 106 batchChars sizes only file sections; related context (up to 40k chars), grounding and PR text are unaccounted, and calls/wall time have no hard cap.
packages/opencode/src/altimate/review/ai-review.ts 412 Usage is read only after a successful stream, so timed-out/failed calls are counted as calls but contribute no tokens or cost.
packages/opencode/src/altimate/review/format.ts 424 A suggestion range spanning two hunks passes validation but GitHub rejects it, dropping every inline comment in the batch.
packages/opencode/src/altimate/review/diff-filter.ts 256 filterChangedFiles is now an unused wrapper; keeping both entry points invites dropping the ledger reasons.
packages/opencode/src/altimate/review/project-rules.ts 42 Duplicate escapeRe helper (also in orchestrate.ts:382) plus a second, non-hunk-aware diff parser beside the imported walkDiffLines.
Files Reviewed (31 files)
  • packages/opencode/src/altimate/review/adapter-capabilities.ts
  • packages/opencode/src/altimate/review/ai-review.ts - 2 issues
  • packages/opencode/src/altimate/review/config.ts
  • packages/opencode/src/altimate/review/dbt-patterns.ts - 1 issue
  • packages/opencode/src/altimate/review/diff-filter.ts - 1 issue
  • packages/opencode/src/altimate/review/finding.ts
  • packages/opencode/src/altimate/review/format.ts - 1 issue
  • packages/opencode/src/altimate/review/incremental-semantics.ts - 1 issue
  • packages/opencode/src/altimate/review/orchestrate.ts
  • packages/opencode/src/altimate/review/post-github.ts
  • packages/opencode/src/altimate/review/project-rules.ts - 1 issue
  • packages/opencode/src/altimate/review/rule-catalog.ts
  • packages/opencode/src/altimate/review/rule-tiers.ts
  • packages/opencode/src/altimate/review/run.ts
  • packages/opencode/src/altimate/review/runner.ts - 1 issue
  • packages/opencode/src/altimate/review/select-list.ts
  • packages/opencode/src/altimate/review/telemetry.ts
  • packages/opencode/src/altimate/review/verdict.ts
  • packages/opencode/src/altimate/telemetry/index.ts
  • packages/opencode/src/altimate/tools/dbt-pr-review.ts
  • packages/opencode/src/cli/cmd/review.ts
  • packages/opencode/test/altimate/review-ai-inputs.test.ts
  • packages/opencode/test/altimate/review-ci-honesty.test.ts
  • packages/opencode/test/altimate/review-contracts-dialect.test.ts
  • packages/opencode/test/altimate/review-failures-gate.test.ts
  • packages/opencode/test/altimate/review-noise.test.ts
  • packages/opencode/test/altimate/review-project-rules.test.ts
  • packages/opencode/test/altimate/review-scope-diff.test.ts
  • packages/opencode/test/altimate/review-select-list.test.ts
  • packages/opencode/test/altimate/review-verdict-trust.test.ts
  • packages/opencode/test/altimate/review.test.ts

Fix these issues in Kilo Cloud


Reviewed by deepseek-v4.1-flash-1017 · Input: 1.2M · Output: 122K · Cached: 14.9M

Review guidance: REVIEW.md from base branch fix/review-false-positives

@RaghwendraSingh

RaghwendraSingh commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor Author

Review order for the dbt PR review stack

Each PR is based on the one above it, so review and merge top to bottom.

# PR What it does Proof (blind Claude judge on public dbt PR replays)
1 #1719 Review the PR's own changes (real base, merge-base diff, project scope)
2 #1720 Honest PR comment (lint-only banner, AI lane status)
3 #1721 Readable comment (diff-anchored inline, no reposts, size budget)
4 #1722 Verdict trust (only the engine blocks; policy from base branch)
5 #1723 Failures are never clean
6 #1724 Consumers outside dbt, engine-aware integer division, contract checks
7 #1725 Incremental and snapshot semantics
8 #1726 Action dbt_prepare, data-test adequacy, parse-only column changes
9 #1739 Precision and new rules from a replay of human-caught defects
10 #1758 Posting policy: rule tiers, inline budget, pre-existing tags, coverage ledger Findings/PR 15.6 → 10.0, catches 52 → 51; inline rule comments 7% → 48% useful, 29% → 5% wrong
11 #1769 Opt-in AI verifier Verifier on vs off: useful 45% → 53%, wrong 1% → 0%. Fresh repos: 35 caught vs released v0.12.11's 24 (+11, 95% CI +1 to +22)
12 #1770 11 rules for macro, test and yml defects Fresh repos 11 → 12 (CI 0 to +3), findings/PR +0.6%; planted bugs 236/238, 1/237 false alarms
13 #1771 Opt-in AI input coverage + manifest context card 21 dev PRs: 6 → 10 caught at 1.32× tokens
14 #1772 Opt-in test-gap check, focused batches, planted-bug fixes Missing behavioural test raised 10/10 vs 2/21; useful 51% → 57%, wrong 3% → 0%
15 #1773 --all-projects: one run for every dbt project a PR touches 5-project monorepo PR: 5 runs → 1
side AltimateAI/altimate-core-internal#786 → #1774 (draft) AI output fields (evidence line, pre-existing, suggestion, cites) + base-branch repo instructions Needs core 0.10.0; the remit widening was dropped after it failed on held-out repos

Overall on 25 repos never used for tuning (101 PRs, 153 reviewer-found defects):

Configuration Caught AI comments useful / wrong
Released v0.12.11 24 34% / 3.6%
#1769–#1770 (AI + verifier + rules) 35 43% / 1.3%
#1771–#1772 opt-ins on 40 48% / 1.3%

For comparison, on the same repos CodeRabbit's comments were 19% useful / 8% wrong, and human reviewers' 22% / 1%. The AI improvements beyond #1769 stay opt-in: their gain is within noise on fresh repos.

🤖 Generated with Claude Code

Merge plan

Base first, one PR at a time:

  1. Squash-merge fix(review): review the PR's own changes: real base, merge-base diff, project scope, renames, model-paths, no raw Jinja #1719 into main once it is approved.
  2. Rebase the next PR onto main, dropping the commits that are already merged (git rebase --onto main <old base> <branch>), force-push with lease, and change its base to main. Its own diff stays the same, and approvals are kept: this repo does not dismiss reviews on push.
  3. Squash-merge it, then repeat up the stack.

Review feedback on a PR lands as a new commit on that PR (not a rebase), so "changes since your last review" shows only the fix. The PRs above it are then rebased.

@hkaltai hkaltai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes (1 MAJOR, 4 MINOR, plus NITs). Panel: Claude, Codex and lead. All 31 files were covered. bun test test/altimate/review passes (591).

Counts: CRITICAL 0, MAJOR 1, MINOR 4 (inline) + 4 more below, NIT 8.

The design is sound: shared diff walk, deletion-site anchoring, ledger of unreviewed files, signed posting policy. The MAJOR is in the posting fallback; the inline comments carry details.

More MINOR items (no inline anchor)

  • rule-tiers.ts: OFF contains engine-origin rules not covered by ENGINE_FLOOR (altimate-core.grain:conflicts-declared-uniqueness, altimate-core.source-filters:sibling-mismatch, altimate-core.structural-diff:*). Those are verdict-eligible, so the defaults change verdicts, which the floor comment and config doc say should not happen. Either add them to the floor or reword and call it out.
  • rule-tiers.ts ruleTier: override keys are matched raw while ids go through normRuleToken, so {"dbt_patterns:self_join":"off"} silently fails to match dbt-patterns:self-join. Normalise keys on both sides.
  • project-rules.ts TEMP_MARKER: new alternatives take (this|it) out and while debugging can match comments meant to stay (inline tier). Require removal intent or move them to summary.
  • finding.ts / orchestrate.ts validateSuggestions: no production code sets Finding.suggestion, so suggestion blocks are untested end to end; the fallback also emits a no-op suggestion when replacement equals current, with a hard-coded sql fence.

Missing tests: postGitHubReview fallback summary content; AI finding with summary override; non-normalised override keys; AI file cap vs ledger; partial batch failure rendering; clampBody cutting inside a code fence.

Note: tsc --noEmit is not fully clean at the repo level (11 errors in packages/tui/src/component/dialog-move-session.tsx, untouched by this PR).

Nits

  • orchestrate.ts:2065 manifestMissing && !anyManifest: manifestMissing already implies !anyManifest; drop the second term.
  • orchestrate.ts:~1686 anchorDedupIssue recovers column names by parsing [...] from the human-readable body; carry them as structured data.
  • ledgerBlock wraps paths in a plain backtick span; use codeSpan.
  • normalizeCondition lowercases double-quoted identifiers, which are case-sensitive on Snowflake and Postgres.
  • orchestrate.ts ledger heading "Not reviewed" also lists model files that SQL/pattern/PII lanes did reach (no-manifest entry); use a "partially reviewed" group. The no checks run for ${kind} files at the ${tier} tier wording is wrong where tier is irrelevant.
  • rule-tiers.ts header and held-out comment refer to a benchmark readers cannot see; add a pointer.
  • diff-filter.ts:~669 comment "Normalize to forward slashes..." sits above the excluders line rather than the norm(f.path) call it describes.
  • Naming: evidenceLine holds text (consider evidenceText); touch/touched could be reachedByLane; scanDiffLines/walkDiffLines/walkDiffRows are easy to confuse.

// missing it would post a duplicate summary on every rerun. A silent clean
// run posts nothing, but still overwrites an earlier summary so it does
// not keep showing findings that are gone.
const summary = renderSummary(env, { alreadyPosted: posted })

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] Inline-eligible findings lose their body if the inline review is rejected

const summary = renderSummary(env, { alreadyPosted: posted })

The summary is rendered and posted before the inline review is attempted. Inline-eligible findings appear in it only as a one-line pointer (file:line and title); their bodies live only in the inline comments. If createReview then fails (a positioning 422, "can not request changes on your own PR", missing permissions), the fallback posts only the verdict headline and the summary is never re-rendered. Failure scenario: a temporary-code-marker warning on an added line. The summary shows ### Inline (1) plus the title, GitHub rejects the batch, and the PR carries no explanation or fix for the finding. Before this PR the summary listed every finding in full, so the fallback lost nothing. Reproduced by two reviewers with a client that returns a 422 and then accepts the event-only review.

Suggested fix: when inline posting falls back or errors, re-render the summary with inline: false and update the summary comment (or include the bodies in the fallback review).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, good catch: the fallback path really did lose the bodies. Fixed in "fix(review): address review feedback: full summary on inline fallback, …" (e7a80ad929). When createReview with comments fails and the event-only review is posted, postGitHubReview now rewrites the summary comment with renderSummary(env, { inline: false }), so every finding (including the ones that would have gone inline) carries its full body and fix. This applies to any failure (positioning 422, own-PR, permissions). A new test in review-failures-gate.test.ts uses a client that returns 422 and then accepts the event-only review. It checks that the first summary has only the ### Inline (1) pointer and the rewritten summary has the full body.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Fixed — post-github.ts:257-264: when the inline createReview fails and comments existed, the summary is re-rendered with inline: false, allowSilent: false and updateCommentd before the event-only review; a failed update is swallowed. Applies to the 422 and non-422 paths. review-failures-gate.test.ts (422 client) asserts the full body after the rewrite; neutralising the guard makes it fail. Optional extra tests: a non-positioning error and an updateComment failure.

let level = 0
let close = -1
for (let i = open; i < text.length && close < 0; i++) {
if (text[i] === "(") level++

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] CTE paren balancing counts parentheses inside string literals

if (text[i] === "(") level++

text only has comments blanked, so quoted literals still contribute parentheses. with final as (select id, ')' as label, 1 as amount from t) select * from final makes finalSelectNames() return ["id"] (reproduced), so amount is reported as a removed final-layer column and the PR can get a false REQUEST_CHANGES. The trigger is narrow (a paren inside a string in a select * from <cte> model), which is why this is MINOR.

Suggested fix: scan the output of the existing literal-blanking step (or skip quoted strings and identifiers) while balancing, keeping offsets into the original SQL.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the precise repro. This is already handled at the branch head: finalSelectNames balances parentheses over a copy of the SQL with comments and string contents blanked ('...' becomes underscores of the same length), so offsets still line up and a ')' literal can no longer close the CTE body early. This landed in "fix(review): address review comments: UNION output names, package parents in graph context" (4ad40fce41). Your example now returns ["id", "label", "amount"].

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Fixed — select-list.ts:120 blanks string-literal contents (incl. '' escapes) before paren balancing; the repro now returns ["id","label","amount"]. Regression in review-select-list.test.ts fails when the blanking is reverted. Residual NIT: parens inside double-quoted or backtick identifiers (")x") are still counted.

/** Whether a finding may be posted as an inline comment: an inline-tier rule
* or the AI lane, on a line the PR added, at or above the severity floor. */
export function inlineEligible(f: Finding, posting: PostingPolicy): boolean {
if (f.tier !== "inline" && findingOrigin(f) !== "ai") return false

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] AI findings bypass an explicit summary tier override

if (f.tier !== "inline" && findingOrigin(f) !== "ai") return false

With rules: { "ai-review:*": summary } the orchestrator stamps tier: "summary", but this origin exemption still returns eligible, so the comment goes inline (confirmed end to end by two reviewers). off works because those findings are dropped earlier. This contradicts the config doc ("summary keeps it to the summary comment").

Suggested fix: resolve AI findings to inline as the default tier inside ruleTier and drop the origin bypass, so an explicit override wins.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, the origin exemption contradicted the config doc. Fixed in "fix(review): address review feedback: …, AI tier overrides, …" (e7a80ad929) the way you suggested. ruleTier now resolves ai-review:* rules to inline as the default, after exact and glob overrides, and the findingOrigin(f) === "ai" bypass in inlineEligible is removed. rules: { "ai-review:*": summary } therefore keeps AI findings in the summary. An end-to-end test in review-ai-inputs.test.ts runs the orchestrator with and without that override, and review-noise.test.ts has a resolution-order test.

Heads-up for the next PR: the verifier PR (#1769) changes the same condition in inlineEligible, and I will carry this change through when restacking.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Fixed — the findingOrigin === "ai" bypass is gone from inlineEligible (format.ts:398); ruleTier defaults ai-review:* to inline after exact/glob overrides (rule-tiers.ts:181). Tests in review-noise.test.ts and the end-to-end review-ai-inputs.test.ts pass and fail when either change is reverted.

supporting.forEach((f, i) =>
aiFiles.push({ path: f.path, status: f.status, model: modelNameFromPath(f.path), kind: f.kind, diff: f.diff, sql: contents[i] }),
)
touch(aiFiles)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Coverage ledger marks AI candidates reviewed before the file cap applies

touch(aiFiles)

Every candidate is recorded as reached before runAiReview applies its file limit. With 21 changed seed files and a default limit of 20, one file gets no deterministic check and no ledger entry, so the ledger is empty and the verdict is APPROVE (reported by Codex). Suggested fix: have the AI lane return the paths it actually sent and build coverage from that.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, confirmed: with 21 seed files one fell out of both the AI call and the ledger. Fixed in "fix(review): address review feedback: …, AI file-cap ledger" (e7a80ad929). The orchestrator now marks reached only selectAiFiles(aiFiles, aiInputBudget().maxFiles), the same selection and cap runAiReview applies, so the 21st file stays under "Not reviewed". New test: 21 changed seeds give a ledger of exactly seeds/s20.csv, and the summary shows it. One note: a "Not reviewed" entry alone does not stop an APPROVE verdict. That matches how other unchecked files are treated today, so the verdict policy is unchanged here. Only the ledger is now accurate.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Fixed (for the cap) — orchestrate.ts:2477 now touches selectAiFiles(aiFiles, aiInputBudget().maxFiles), the same selection the lane applies; the 21-seed test keeps seeds/s20.csv in the ledger. Related remaining item (new MINOR below): the touch still happens before the lane result is known.

if (failed.length === results.length)
return done([], { state: failed[0].state, reason: failed[0].reason, model: modelLabel, ...spent })
const reason = failed.length ? `${failed.length} of ${results.length} calls failed: ${failed[0].reason}` : undefined
return done(findings, { state: "ok", model: modelLabel, ...(reason ? { reason } : {}), ...spent })

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Partial batch failure is reported as state: "ok" and its reason is not rendered

return done(findings, { state: "ok", model: modelLabel, ...(reason ? { reason } : {}), ...spent })

coverageNotes() shows reasons only for non-ok states, so when one batch is clean and another times out, the run can look clean and, with silentWhenClean, post nothing. Suggested fix: use an explicit partial state, or render the reason whenever one is present, including the affected paths.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, the reason was recorded but never shown. Fixed in "fix(review): address review feedback: …" (e7a80ad929) by taking the second option. coverageNotes now renders Advisory AI review partly failed: <reason> whenever the lane is ok but carries a reason (for example "1 of 2 calls failed: timed out"). That makes the summary non-silent even under silentWhenClean. The lane state stays ok so the telemetry and envelope schema don't change. Listing the affected paths would mean threading per-batch file lists through BatchResult, and I left that as a follow-up. A test covers the rendered note.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Partially fixed — format.ts:281-284 now renders Advisory AI review partly failed: <reason> for an ok lane with a reason, which also stops silentWhenClean from going silent, and a test covers it. Still open (non-blocking, acknowledged as follow-up): the note does not name the files of the failed batches, and those files remain marked as reached in the ledger.

@github-actions

Copy link
Copy Markdown

Thanks for updating your PR! It now meets our contributing guidelines. 👍

"dbt-patterns:float-equality",
"altimate-core.structural-diff:group-by-change",
"rule-catalog:bq-array-no-offset",
"altimate-core.structural-diff:join-key-regression",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CRITICAL: Default off silently disables the engine's blocking join-key check

altimate-core.structural-diff:join-key-regression is emitted by structuralChangeLane as category: "join_risk", escalated to critical when core returns severity: "error" (orchestrate.ts:1637), and join_risk is in the default blockOn (rubric.ts:49). But cleanFindings drops every off rule before buildEnvelope (orchestrate.ts:2417-2420), and ENGINE_FLOOR has no altimate-core.structural-diff: prefix — so with the shipped defaults a PR that changes a join key produces no finding at all, and computeIdealVerdict can return APPROVE/COMMENT instead of REQUEST_CHANGES. The PR's own blocking test only passes because it forces rules: { "*": "summary" } (review.test.ts:1268), which confirms the default suppresses the blocker. Move this rule (and the other verdict-capable altimate-core.structural-diff rules) to summary, or add that prefix to ENGINE_FLOOR.

const parsedRaw = ((parseRes.data ?? {}) as Record<string, unknown>).findings
if (parseRes.success === false || !Array.isArray(parsedRaw))
return done([], {
return {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: The parse-failure return drops the usage it just read

usage is assigned at line 422, but this return (rewritten by this PR) omits it, unlike the no-text path at 423 and the success path at 475. A billed call whose answer core cannot parse is therefore counted in ai_calls with zero tokens and zero cost — exactly the accounting gap this PR adds telemetry to close. Add usage here too.

// both SQL sides. This avoids duplicate engine calls and lets tiering see PII.
const modelFiles = reviewable.filter((f) => f.kind === "model_sql" || f.kind === "python_model")
// Every model gets at least the PII check; one that fails is recorded as unverified.
touch(modelFiles)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: A model whose head content cannot be read is still marked reviewed

touch(modelFiles) marks every model file as looked at before any lane runs, so the coverage ledger (reviewable.filter((f) => !touched.has(f.path)), line 2501) never lists it. getContent resolves to undefined on any git read failure instead of throwing (git.ts:148-150), and the only not_compiled record requires newSql to be truthy (lines 1912-1913). A shallow checkout, a rename the base cannot resolve, or a containment failure therefore skips every SQL lane for that model with no unverified entry and no ledger row — the run can still APPROVE while that model's SQL was never examined. Record unverified/notReviewed when a non-deleted model's head content resolves to undefined, or touch only after content was actually delivered to a lane.

unverified_models: env.scope?.unverified?.length ?? 0,
ai_state: env.ai?.state,
ai_duration_ms: env.ai?.durationMs,
ai_tokens_total: env.ai?.tokens ? env.ai.tokens.inputTotal + env.ai.tokens.output : undefined,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: ai_tokens_total omits reasoning tokens

Session.getUsage returns tokens.output without reasoning and bills tokens.reasoning separately at the output rate (session/index.ts:917-944), and this PR stores reasoning on the lane's tokens. Summing only inputTotal + output therefore under-reports total tokens on reasoning models while ai_cost includes them, so the two metrics disagree; the doc comment added in telemetry/index.ts:1608 explicitly says output includes reasoning. Add reasoning (and set ai_tokens_output to match), or correct the doc.

"dbt-project-rules:ref-target-missing",
"rule-catalog:qualify-unsupported",
"dbt-project-rules:macro-param-shifted",
"rule-catalog:case-duplicate-when",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: inline tier is inert for these suggestion-severity rules

rule-catalog:case-duplicate-when (rule-catalog.ts:594-596) and dbt-patterns:macro-var-no-default (dbt-patterns.ts:1006-1014) are emitted at suggestion severity, while inlineEligible requires severity >= inlineSeverityFloor and the default floor is warning. Under the shipped posting defaults these two of the 14 inline rules can never post inline (they fall through to the summary), so the tier entry is a no-op unless a repo lowers the floor. Raise their severity, or move them out of INLINE so the table reflects what actually posts.

@RaghwendraSingh
RaghwendraSingh force-pushed the fix/review-false-positives branch from bb6a1fe to 416ac62 Compare November 20, 2026 08:22
@RaghwendraSingh
RaghwendraSingh force-pushed the feat/review-posting-policy branch from 4733424 to e7a80ad Compare November 20, 2026 08:22
@RaghwendraSingh

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough pass, Harish. Addressed in "fix(review): address review feedback: full summary on inline fallback, AI tier overrides, override key spelling, AI file-cap ledger" (e7a80ad929):

  • MAJOR, fallback summary: fixed (summary re-rendered in full when inline posting falls back).
  • AI summary override, ledger vs AI file cap, partial batch failure: fixed, see the inline replies. The CTE literal parens issue was already fixed in 4ad40fce41.
  • ENGINE_FLOOR vs OFF engine rules: I kept the defaults, because the benchmark judged those detectors mostly wrong. I reworded the floor comment to state that structural-diff, grain and source-filters rules are not on the floor and default to off, so they no longer count toward the verdict unless a team turns them back on.
  • Non-normalised override keys: fixed. Keys and ids are compared in the ruleId spelling, globs included, with tests.
  • TEMP_MARKER: "take this out" now needs a removal cue (before/once/after/when), and "while debugging" needs "added/enabled/only". New negative tests cover comments meant to stay.
  • Nits: the ledger now uses codeSpan. I kept manifestMissing && !anyManifest, because anyManifest can flip to true later when an impact call finds a manifest (orchestrate.ts ~1921) while manifestMissing stays set, so the second term is not redundant.
  • Not changed in this round (follow-ups): an end-to-end test for Finding.suggestion from production code (and the no-op suggestion when the replacement equals the current text), structured columns for anchorDedupIssue, case of quoted identifiers in normalizeCondition, a "partially reviewed" ledger group, comment placement/naming nits, and the pre-existing packages/tui tsc errors.
  • bun test test/altimate/review: 649 pass; tsc --noEmit clean outside /tui/.

@hkaltai hkaltai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at e7a80ad (previously reviewed bcc75a9). Panel: Claude, Codex and lead. The base branch moved since the last round, so only this PR's own diff against its current base was reviewed.

Verdict: no CRITICAL/MAJOR remains. The prior MAJOR is fixed, with a regression test that fails without the fix. 0 CRITICAL, 0 MAJOR, 1 MINOR (new, inline), plus nits. I ran the review-failures-gate, select-list, noise and ai-inputs suites (105 tests) and they pass.

The author's claim holds: when the inline review is rejected, the summary is rewritten with every finding in full. Replies are posted in each prior thread.

Prior findings

Prior finding Status
MAJOR post-github.ts fallback loses bodies Fixed
MINOR select-list CTE paren scan in strings Fixed (double-quoted/backtick identifiers remain, NIT)
MINOR format.ts AI findings ignore summary override Fixed
MINOR orchestrate touch(aiFiles) before cap Fixed for the cap (see new MINOR)
MINOR ai-review partial batch failure state ok Partially fixed: now visible and non-silent, but failed files are not named and still count as reached
Body: engine rules default off but are not on ENGINE_FLOOR Partially fixed: the ENGINE_FLOOR comment is updated, the config.ts:85 doc still overstates exact-id protection
Body: override keys not normalised Fixed
Body: temporary-marker phrases too broad Fixed
Body: suggestion blocks have no production producer and a no-op replacement is still rendered as a fix Not fixed (non-blocking)

Nits still open or new

  • orchestrate.ts:2523 manifestMissing && !anyManifest: drop the redundant second term.
  • orchestrate.ts:627 anchorDedupIssue still parses [...] from the body: carry the column names as structured data.
  • orchestrate.ts:2522 ledger text no checks run for ${kind} files at the ${tier} tier is also used for files dropped by the AI file cap; use a dedicated reason such as "past the AI file cap (ALTIMATE_REVIEW_AI_MAX_FILES)".
  • "Not reviewed" also lists models the SQL/pattern lanes did reach: use a "partially reviewed" group.
  • rule-tiers.ts:11 still refers to a benchmark readers cannot see: add a pointer.
  • diff-filter.ts:238 comment about normalising slashes sits above excluders rather than the norm(f.path) call.
  • rule-tiers.ts:150-169 normRuleKey turns rule_catalog:join_* into rule-catalog:join*, which also matches joinfoo. Two raw keys that normalise to the same id resolve by insertion order. Keep the trailing - before * and warn on collisions.
  • select-list.ts:120 blank "..." and backtick identifiers like string literals before balancing parens.
  • format.ts:282 says failed calls' files "were not reviewed by the AI", but the ledger counts them as reached. orchestrate.ts:2476 comment "Only the files within the AI lane's file cap are reached" overstates the same.
  • config.ts:85 doc: say that some engine rules are off by default and are not protected by the floor.
  • ai-review.ts:435 (base code): the in-band parse-failure return drops the already-read usage, so tokens and cost are undercounted. Already tracked as a follow-up by the author.
  • Naming: evidenceLine holds text (evidenceText), touch/touched could be reachedByLane, and scanDiffLines/walkDiffLines/walkDiffRows are easy to confuse.

Missing tests (optional): fallback on a non-positioning error and on an updateComment failure; an AI lane that is skipped or timed out with AI-only files; a paren inside a double-quoted identifier.

aiFiles.push({ path: f.path, status: f.status, model: modelNameFromPath(f.path), kind: f.kind, diff: f.diff, sql: contents[i] }),
)
// Only the files within the AI lane's file cap are reached; the rest stay in the ledger.
touch(selectAiFiles(aiFiles, aiInputBudget().maxFiles))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] AI files are marked "reached" before the lane result is known

touch(selectAiFiles(aiFiles, aiInputBudget().maxFiles))

touch runs before input.aiReview is called, so the ledger treats these files as reviewed even when the lane returns skipped (no usable model), timeout or error. Scenario: a PR that changes only seeds/a.csv with no default model configured. scope.notReviewed stays empty, the verdict is APPROVE and the summary says "No issues found", with only a one-line "Advisory AI review skipped" note. With --no-ai the same seed is listed under "Not reviewed", so the two cases are inconsistent. A partial batch failure also leaves the failed batch's files counted as covered.

Suggested fix: touch only after the lane returns, and only for files it actually reviewed. For example, have runAiReview return the reviewed paths, which also removes the duplicated selectAiFiles/aiInputBudget selection here.

RaghwendraSingh and others added 10 commits November 20, 2026 16:12
…at changed (opt-in)

Of the defects the AI-on review missed on the public benchmark, 55% were in files
the lane never fully saw: cut at 6,000 characters, or dropped by the 20-file cap.
Most of the rest needed another file (an upstream model's columns, a seed's rows).
Both are now available behind environment switches; defaults are unchanged.

- `ALTIMATE_REVIEW_AI_MAX_FILES`, `ALTIMATE_REVIEW_AI_MAX_FILE_CHARS`: widen the file cap and per-file limit
- `ALTIMATE_REVIEW_AI_BATCH_CHARS`: split the files across several calls of about this size instead of cutting them; findings merge, and a partly failed run says how many calls failed
- `ALTIMATE_REVIEW_AI_GRAPH=1`: send the changed files' unchanged dbt parents as read-only context: models, seeds and snapshots by file, sources as their declared columns, and the project's own macros (`graphParents` on the runner, from the manifest; a changed yml maps through its tests)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…root

A monorepo of dbt sub-projects often shares one macro directory, e.g.
`dbt_subprojects/<name>/dbt_project.yml` with
`macro-paths: ["macros", "../../dbt_macros"]`. The review stripped the
leading `.` from such entries so they never matched, and it dropped every
file outside the project root as out of scope, so shared macros were never
reviewed.

- Resolve each layout dir to a repo-relative path (`resolvedLayoutDirs`),
  dropping any that escapes the repository.
- `classifyDbtFile` matches the full repo path against those dirs, so
  `dbt_macros/shared/x.sql` is a `macro`.
- `scopeToProject` takes the layout and keeps files under any resolved
  layout dir as well as under the project root; `run.ts` passes it.
- Tests cover the shared-macro case end to end (classification, scope, and
  the file reaching the AI lane) and a dir that escapes the repo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The per-rule posting policy, the coverage ledger and per-rule measurement all
need to say which rule produced a finding. Until now that lived in a mix of
`evidence.result.rule`, `evidence.result.code` and free-text titles, so there
was no single key to configure or count by.

`makeFinding` now stamps `rule` as `tool:rule`, normalised by `ruleId`
(lowercase, `_` to `-`), derived from the evidence when the caller does not
pass one. Engine and manifest findings whose evidence carries no rule name
get an explicit id at the call site, and the AI lane uses
`ai-review:<category>`. The fingerprint is unchanged, so finding ids stay
stable across re-reviews.

The schema also gains the optional `preExisting`, `tier`, `evidenceLine` and
`suggestion` fields that the following posting-policy changes fill in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Most review noise comes from a handful of rules that blind judging found
wrong at least half the time and never valuable, while a small set of rules
was right nearly every time but got lost among them. One severity knob
cannot separate the two, so each rule now has a posting tier.

`rule-tiers.ts` holds the default table: 15 rules go inline, the rules
judged mostly wrong with no real benchmark catch are off, and everything
else stays in the summary. Rules that were mostly wrong but still caught a
real defect in the benchmark are kept at summary. Teams override any rule,
or a whole tool with `tool:*`, under `rules:` in `.altimate/review.yml`;
since the policy is read from the base branch, a PR cannot relax it. The
engine rules that decide the verdict (equivalence, lineage, PII, contracts,
data-diff) can be turned off only by exact id, never by a glob.

`cleanFindings` drops `off` rules and stamps `tier` on the rest. The AI
lane is grounded in the same cleaned list, so an off rule no longer steers
it either.

Detection tests in `review.test.ts` that exercise now-off rules opt into
reporting every rule, since they test the lane, not the posting policy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A finding anchored on a diff context line points at code the PR left alone.
Reporting it as if the PR introduced it is the most common reason reviewers
dismiss a comment, and letting it count toward the verdict blames the author
for code they did not write.

- `scanDiffLines` returns both the commentable lines and the added (`+`)
  lines of a diff; `commentableLines` is now a thin wrapper with unchanged
  behaviour.
- `anchorToDiff` keeps a context-line anchor but sets `preExisting: true`.
  Every line of a newly added file counts as added.
- `addedLinesByFile` exposes the added-line sets for later tasks
  (suggestion validation, the verifier).
- `mergeRepeats` no longer merges pre-existing and new instances of a rule.
- `verdictEligible` ignores pre-existing findings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ndings go

Every finding with a line used to become an inline comment, so a busy PR got
dozens of diff comments, most from rules a reviewer would not act on, and a
clean run still posted a "no issues" comment and review on every push.

- New `posting` policy (config and signed envelope): `inlineBudget` (10),
  `inlineSeverityFloor` (warning), `silentWhenClean` (true). Read from the
  base branch with the rest of the policy, so a PR cannot loosen it.
- `inlineComments` returns `{ comments, deferred }`. Only inline-tier rules
  and AI-lane findings on added lines at or above the floor are eligible,
  ordered by severity, confidence, file and line; findings already on the
  PR do not count toward the budget. Over-budget findings stay in the
  summary.
- `renderSummary` lists inline findings as one-line pointers, renders the
  rest by severity, and collapses pre-existing findings into a `<details>`
  block; the size budget logic is unchanged. A clean run with nothing to
  say about coverage renders `""` under `silentWhenClean`.
- `postGitHubReview` reads existing comments before rendering, skips the
  summary and the review on a silent clean run (an earlier summary is still
  overwritten so it does not show stale findings), and dismissal of a stale
  blocking review still happens.
- Local output (CLI stdout, the agent tool) renders every finding in full.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…h the reason

A review that silently skips files reads as a clean bill of health for them.
The summary only counted out-of-scope and moved files, and said nothing about
files dropped by extension or exclude glob, non-dbt files, kinds no lane ran
on at the chosen tier, or models checked without a manifest.

- `ReviewScope.notReviewed` (signed, defaults to `[]` so older envelopes
  still parse) holds `{ path, reason }` for each such file.
- `partitionChangedFiles` returns the reviewable files and the dropped ones
  with their reason; `filterChangedFiles` wraps it unchanged.
- `scopeToProject` also returns `outOfScopePaths` and
  `movedUnchangedPaths`; `run.ts` passes them into the scope.
- `runReview` adds dropped files, `other`/`analysis` files, files no lane
  received at this tier, and a no-manifest note per model (only when the
  runner reports no manifest), de-duplicated by path, plus the existing
  `unverified` entries.
- The summary renders a collapsed "Not reviewed: n file(s)" block grouped
  by reason (30 paths, then a count), replacing the out-of-scope and moved
  count lines when the paths are known.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Moved files now go to the "Not reviewed" ledger by path, which replaced the
count line that told readers `dbt_project.yml` folder config (`+schema`,
`+materialized`, `+tags`) can still change how a moved model builds. The
ledger's "moved without edits" group now carries that hint under its paths,
and the count line reuses the same text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stion blocks

Long finding bodies made inline comments and summary bullets hard to scan,
the reader had to open the file to see which line a pattern matched, and a
fix could only be described in prose.

- `clampBody` cuts a body to 600 characters at a sentence (or word) boundary
  and appends "…", closing a code fence the cut left open. It applies only
  when rendering inline comments and summary bullets; the envelope keeps the
  full body.
- `cleanFindings` copies the matched source text a lane records in
  `evidence.result.line` (today the dbt-patterns detectors) into
  `evidenceLine`, rendered as `` `file:line` — `<text>` `` in inline comments
  and single-finding summary bullets.
- `validateSuggestions` runs before the final `cleanFindings`: a
  `suggestion` survives only when every line of its range was added by the
  PR, the head file has those lines, and the replacement changes them.
  Otherwise the field is dropped and the replacement is appended to the body
  as a "Suggested fix" SQL block.
- `inlineComments` posts a surviving suggestion as a GitHub `suggestion`
  block anchored on its range: `line` is the range end, and a multi-line
  range adds `start_line`/`start_side`. `InlineComment` already is the
  `pulls.createReview` payload, so `post-github.ts` passes it through as is.

No lane emits suggestions yet; the tests build findings with one directly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Any decision about a verifier pass (the 2x token rule) needs a baseline of
what the advisory lane spends today, and nothing recorded it.

- After a batch's stream is drained, `streamUsage` reads `totalUsage` and
  `providerMetadata` and runs them through `Session.getUsage`, the same
  accounting the chat session uses (cached-token handling, model price
  table). `Session` is imported lazily, as in `learn/usage.ts`. Any failure
  returns no usage; accounting never fails the lane.
- `AiLaneStatus` gains `tokens { input, inputTotal, output, reasoning,
  cacheRead }`, `cost` and `calls`. `laneUsage` sums the batches that
  reported usage and counts every batch as a call. Usage survives a batch
  whose response later fails to parse.
- The envelope's `ai` status carries the same optional fields.
- The `review_run` telemetry event gains `ai_tokens_total` (input including
  cached, plus output), `ai_tokens_input`, `ai_tokens_output`, `ai_cost` and
  `ai_calls`.
- The summary footer shows `· tokens N` when the lane reported usage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RaghwendraSingh and others added 18 commits November 20, 2026 16:12
Rule findings had no line, so inline-tier rules never posted inline and
were never tagged pre-existing. Each rule now records the raw added line it
matched in `evidence.result.line`, and a new `anchorByEvidence` step in
`cleanFindings` finds that line in the file's diff before `anchorToDiff`
runs.

- Rule catalog: `matchRule` returns the first added line that satisfied the
  `added` condition (the raw line, not the comment-stripped one), else the
  added line inside the matching ON clause, else the first added line where
  the `newSql` pattern shows. `ruleMatches` wraps it.
- dbt-patterns: detectors that matched added lines without recording one now
  do (clock, fan-out join, COUNT distinctness, CASE without ELSE, self-join,
  window without PARTITION BY, multiple COUNT DISTINCT, macro var without a
  default). The dedup detectors record the added line that opens the window
  and keep the window text in `window`.
- Project rules and incremental semantics record the matched line where a
  rule matches one (temporary marker, utcnow, ref target, macro signature,
  config keys, tests on NULL columns, `partition_by` change).
- `anchorByEvidence` picks the first added line with the same trimmed text,
  else the first context line with it (marked pre-existing downstream), else
  the first added line containing it when it is at least 12 characters.
  It never anchors outside the diff.
- `walkDiffLines` yields each hunk line with its head-side number and kind;
  `scanDiffLines` now uses it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ary tier

Both rules were switched off on per-rule judging alone, but each one
produced a real catch in the replay benchmark. Rules with a benchmark
catch stay at summary tier, like the other exceptions in that list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the engine cannot decide whether a refactor is equivalent (no
schema or unsupported SQL), the review no longer emits a finding. The
model is listed under "Not reviewed" with a data-diff hint instead.
Only 1 of 426 such findings matched a real defect on replay, and they
were the largest source of volume (about one per PR).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Blind judging of rule findings on repos held out from tuning, with the
decision rule fixed before judging (off rule with at least 2 of 6
valuable -> summary; inline rule with at least 3 of 6 wrong -> summary):

- `rule-catalog:distinct-and-group-by`: off -> summary (2 of 3 valuable)
- `dbt-project-rules:test-on-null-column`: inline -> summary (3 of 6 wrong)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The detector looked for the raw added line inside the tokenised
`is_incremental()` filter, which never matches, so the finding never got
a line and could not post inline. Match the added line on the creation
column and a comparison instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two WHEN conditions that differ only in spacing around operators
(`>= 1000` vs `>=1000`) were compared as different text, so the
unreachable branch went unreported. Compare conditions with case and
spacing folded outside string literals; literal text is compared
verbatim, so `'A'` and `'a'` stay distinct.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s window form

`ignore-nulls-inside-call` required `IGNORE NULLS` right before the
closing parenthesis, so `ARRAY_AGG(status IGNORE NULLS ORDER BY d)` was
missed. Allow a trailing `ORDER BY` / `LIMIT` inside the call.

DuckDB accepts the in-call form on navigation window functions
(`first_value`, `last_value`, `nth_value`, `lead`, `lag` with `OVER`)
but rejects it on aggregates. A catalog rule's portability can now name
engines that accept the construct only when a check holds on the file,
so the advice is dropped for a DuckDB project's window functions and
kept for its aggregates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e-marker`

A comment such as `-- sample while debugging locally, take this out
before release` was not recognised: the marker needed `temp`, `revert`,
`do not merge` or `remove before`. Also match `take this/it out`,
`while debugging` and `remove/delete/drop this/it before
merge/release/deploy/prod`. The match still runs on comment text only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rule fired whenever `partition_by` carried a `granularity` key, so
it reported a dbt-bigquery failure on DuckDB, Postgres and Snowflake
projects, where the mismatch is not an error. Record the requirement as
a BigQuery capability (`microbatchBatchSizeMatchesPartition`, from
dbt-bigquery's `bq_validate_microbatch_config`) and fire only where it
holds. When the adapter is a default guess, the `granularity` dict still
identifies BigQuery, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A final-layer model renaming an alias (`customer_lifetime_value` ->
`lifetime_value`) got no finding. Its output columns come from column
lineage plus the final SELECT's own list; lineage without a schema
drops columns built from aggregates over `select *` sources, and the
final `select * from final` gave no list, so both sides looked the same.
Follow a final `select * from <cte>` to that CTE's select list (up to
five hops; a star over a table, a join or another star stays unknown).
Used for compiled SQL and for the raw-model fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e text

Core reports line numbers in the compiled SQL, which don't exist in a
Jinja model's file, so its findings there (e.g.
`non-deterministic-dedup`) never got a line and could not post inline.
Record the compiled line's text as the evidence line (the number moves
to `compiledLine`) so `anchorByEvidence` finds it among the added lines.
Only text of at least 12 characters that is unique in the compiled SQL
is used; anything else keeps no line, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`dbt.manifest:removed-final-layer-columns` and
`dbt.manifest:removed-columns-declared-consumers` are whole-model findings
with no line, so they could never post inline. They now anchor on the head
side of the removal:

- find the base line that defined the first removed column the diff shows
  (the last removed select item ending in the column, else the first removed
  line naming it);
- take the `+` line at the same position in its change block (the rename or
  replacement), else the nearest `+` line in the hunk, else, for a pure
  deletion, the next (or previous) context line in the hunk;
- a column no removed line names stays unanchored, and the body is unchanged.

A new `deletionSite` field marks such a line as standing in for removed
code, so `anchorToDiff` does not mark a context-line anchor `preExisting`;
inline posting and the verdict treat it as the PR's change. Comments stay
RIGHT-side. `walkDiffLines` moves to `diff-filter.ts` beside the new
`walkDiffRows` (which also yields `-` rows) and is re-exported from
`orchestrate.ts`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rule anchored only through the `{% macro name(` line's text, so a
multi-line signature whose parameter list changed on a later line (the
usual shape: one parameter per line) got no line. The finding now anchors
on the first added head line inside that macro's parameter list in the head
file, which is also the `{% macro` line when a single-line signature
changed. With no added line in the signature it falls back to the previous
evidence-text anchor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… by the time column

The compiled-line text anchor misses when the window comes from a macro
(e.g. `{{ deduplicate_raw_table(order_by='_airbyte_extracted_at', ...) }}`)
or a Jinja-built line, so the compiled text is nowhere in the raw diff.
When that anchor fails on compiled SQL, the finding now takes the first
added line that opens a `row_number()` naming the time column the issue
reports, else the first added line ordering by it (`order by col`,
`order_by='col'`). No added line names the column → no line, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rule is inline-tier and now anchors to the changed line, but at
suggestion severity it stayed below the default `inlineSeverityFloor`
(warning) and never posted inline. Dev-split precision draw: 5 of 7
inline findings judged valuable, none wrong.

Effect on the no-AI bench (anchor mode): dev 18 newly inline-eligible
findings on 11 PRs, 3 PRs reach the warning-pattern threshold;
heldout_a 11 findings on 9 PRs, 3 PRs reach the threshold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ummary

A leave-one-repo-out re-derivation of the tier table moved 19 `off`
rules to another tier when a single repo was dropped from the judging
sample, so their `off` verdict is not robust. They now default to
`summary` until they are re-judged on held-out repositories.

The 7 inline rules that moved in the same check stay inline pending
that re-judging.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ents in graph context

- `select-list.ts`: the select list stops at a set operator, so a first branch
  without `FROM` (`select 1 as one union select 2`) keeps its names, and
  `finalSelectNames` follows `select * from <cte>` only from the branch that
  names the output (shared `finalSelectStart`), never from a later UNION branch.
- `select-list.ts`: the CTE close-paren scan in `finalSelectNames` masks string
  literals, so a `')'` literal can't end the CTE body early.
- `runner.ts`: `graphParents` skips nodes, model/seed/snapshot parents and
  sources from installed packages, whose `original_file_path` is relative to
  the package rather than the project (only with `ALTIMATE_REVIEW_AI_GRAPH=1`).
- Tests for each.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, AI tier overrides, override key spelling, AI file-cap ledger

- `post-github.ts`: when the inline review is rejected and the event-only
  fallback runs, rewrite the summary comment with every finding in full
  (`inline: false`), so inline-eligible findings keep their body and fix.
- `rule-tiers.ts` / `format.ts`: AI findings default to `inline` inside
  `ruleTier`; the origin bypass in `inlineEligible` is gone, so an explicit
  `ai-review:*: summary` (or `off`) override wins.
- `rule-tiers.ts`: override keys and rule ids are compared in the normalised
  `ruleId` spelling, so `dbt_patterns:self_join` matches `dbt-patterns:self-join`;
  the `ENGINE_FLOOR` comment now names the engine rules that default to `off`.
- `orchestrate.ts`: the coverage ledger marks only the files within the AI
  lane's file cap as reached; files past the cap stay under "Not reviewed".
- `format.ts`: a partial AI failure (state `ok` with a reason) is rendered as
  "Advisory AI review partly failed"; ledger paths use `codeSpan`.
- `project-rules.ts`: `temporary-code-marker` requires removal intent for
  "take this out" and "while debugging".
- Tests for each change.
- Restack fixups: one `normalizeCondition` (literals and double-quoted identifiers kept verbatim),
  and the DuckDB `IGNORE NULLS` test checks the `alsoWorksOn` window-form exception.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@RaghwendraSingh
RaghwendraSingh force-pushed the fix/review-false-positives branch from 416ac62 to 3be068e Compare November 20, 2026 11:37
@RaghwendraSingh
RaghwendraSingh force-pushed the feat/review-posting-policy branch from e7a80ad to 2dcef4b Compare November 20, 2026 11:37
* rule in `blockingRules`; advisory AI findings never. A finding on a line the
* PR did not add (`preExisting`) never does: the PR did not introduce it. */
export function verdictEligible(f: Finding, rubric: Rubric = DEFAULT_RUBRIC): boolean {
if (f.preExisting) return false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: preExisting findings are excluded from the verdict but still counted in the headline

computeIdealVerdict filters through verdictEligible, so this early return means a finding anchored on a context line cannot block (or even downgrade) the run. But summarize() - which feeds env.summary and therefore verdictHeadline (format.ts:25-36) - still tallies every finding. A PR whose only finding is pre-existing therefore renders as Approved — 1 critical (full tier) (or Approved — 3 warning) while the body files the same finding under "Pre-existing". That is the most visible way the new trust rule can look self-contradictory. Count verdict-eligible findings for the headline, or subtract preExisting from the tally and report it separately.

* the replacement changes them. Otherwise the replacement moves into the body
* as a plain "Suggested fix" block.
*/
export async function validateSuggestions(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: The suggestion pipeline has no production producer

Nothing sets Finding.suggestion: every makeFinding caller (ai-review.ts, dbt-patterns.ts, project-rules.ts, incremental-semantics.ts, rule-catalog.ts) omits it, including the AI transport, which maps only explicit fields from the core parse result. So validateSuggestions here, suggestionBlock in format.ts, and the start_line/start_side branch of inlineComments are exercised only by tests. Either wire it to the core review_ai_parse contract (the parser would have to return a replacement) or drop the field and the code that renders it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants