Skip to content

feat(review): opt-in AI input coverage, hunk excerpts and a dbt manifest context card - #1771

Open
RaghwendraSingh wants to merge 4 commits into
feat/review-recall-rulesfrom
feat/review-ai-input-coverage
Open

RaghwendraSingh wants to merge 4 commits into
feat/review-recall-rulesfrom
feat/review-ai-input-coverage

Conversation

@RaghwendraSingh

@RaghwendraSingh RaghwendraSingh commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Issue for this PR

Part of #1792

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Stacked on #1770. Gives the AI lane better input on large PRs. All of it is opt-in: with the flags unset, prompts are byte-identical to before (snapshot test).

  • ALTIMATE_REVIEW_AI_COVERAGE=1:
    • Per-kind file quotas (models 12, macros 8, tests 4, yml/config 6), so models can't crowd out supporting files.
    • Hunk-centred excerpts instead of cutting at the first 6,000 characters.
    • Files not sent are listed in the coverage ledger.
  • ALTIMATE_REVIEW_AI_CONTEXT_CARD=1: a short dbt context card per file from the manifest. For models: materialization, unique key, parents, children, exposures, declared columns with their tests, siblings. For macros: callers and dispatch variants.
  • ALTIMATE_REVIEW_AI_DRYRUN=1: builds the AI input and records its size without calling the model, for cost estimates.

Results (blind Claude judge)

Set Without With coverage + card Tokens
21 tuning-repo PRs where the input changes (39 defects) 6 caught 10 caught 1.32×
25 repos never used for tuning (129 snapshots, 188 defects), with the test-gap check from the next PR (config C vs A) 43 caught, 44% useful, 1.2% wrong 47 caught, 48% useful, 0.9% wrong 1.25×
  • The fresh-repo gain is +4 catches (95% CI −4 to +12). That leans better but is within noise, so these stay opt-in.

How did you verify your code works?

  • bun test test/altimate/review: 725 pass.
  • bunx tsc --noEmit: clean.
  • New cases in test/altimate/review-ai-inputs.test.ts: quotas, redistribution, hunk windows, ledger, card rendering, default prompt snapshot.

Cost side effects

None by default; every option is opt-in and the default prompts are byte-identical. With ALTIMATE_REVIEW_AI_COVERAGE=1 and ALTIMATE_REVIEW_AI_CONTEXT_CARD=1 the AI lane's input grows with the PR's changed files (bounded by the per-kind quotas), measured at 1.25-1.32x AI-lane tokens; the number of AI calls is unchanged. The context card is built from the local manifest (no warehouse queries). ALTIMATE_REVIEW_AI_DRYRUN=1 makes no model call and records the input size for cost estimates.

Screenshots / recordings

Not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR
  • I checked for cost side effects (model calls, queries or cloud resources that grow with user data) and described them, or wrote "none"

🤖 Generated with Claude Code


Summary by cubic

Adds opt-in flags that give the AI reviewer better input on large PRs while leaving the default prompt byte-identical (pinned by a snapshot test).

What changed

  • ALTIMATE_REVIEW_AI_COVERAGE=1 sends per-kind file quotas (models 12, macros 8, tests 4, yml/config 6), most-changed files first, with unused quota redistributed; quotas are overridable via ALTIMATE_REVIEW_AI_QUOTA_*.
  • The diff section gets its own 16,000-char budget (ALTIMATE_REVIEW_AI_COVERAGE_DIFF_CHARS) so hunks stay whole; over-budget files send ±20-line windows around changed lines with omission markers, and batches are capped at 120,000 chars per call.
  • For compiled SQL whose line numbers drift after Jinja expansion, windows anchor on the added lines' text instead of line numbers.
  • Files left out are listed in the coverage ledger and in scope.notReviewed as "not sent to the AI (file quota)".
  • ALTIMATE_REVIEW_AI_CONTEXT_CARD=1 adds a short dbt context card per file from the manifest: materialization and keys, parents/children/exposures, declared columns with tests, and macro callers and variants; yml and singular tests get the card of the model they test.
  • ALTIMATE_REVIEW_AI_DRYRUN=1 plans the AI input and records its size without calling the model, for cost estimates.
  • Per-file ai.input metadata (chars, truncated, hunks sent/total, card) is recorded on the verdict envelope under these flags.

In a blind evaluation across 25 repos never used for tuning, coverage plus the context card caught 47 vs 43 defects with fewer wrong findings at about 1.25× the tokens; the gain is within noise, so the flags stay opt-in.

Migration

  • No changes required; the flags default to off.

Written for commit bf6e861. Summary will update on new commits.

View guided diff Turn on auto-fix

@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 87e89335-68b0-4d5b-91fc-f53a18466c2e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

13 issues found across 9 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/opencode/test/altimate/fixtures/review-ai-prompt-input.ts">

<violation number="1" location="packages/opencode/test/altimate/fixtures/review-ai-prompt-input.ts:12">
P3: The hunk header counts in `bigDiff` don't match the hunk body: the old side has 3 lines and the new side has `size + 2`, but the header says `size + 2` and `size + 4`. Use the real counts so this fixture is a valid unified diff for any future hunk-window test.</violation>
</file>

<file name="packages/opencode/src/altimate/review/ai-review.ts">

<violation number="1" location="packages/opencode/src/altimate/review/ai-review.ts:391">
P2: A diff between `coverage.diffChars` and `maxChars` bypasses `excerptDiff`, so a lower `ALTIMATE_REVIEW_AI_COVERAGE_DIFF_CHARS` override is ignored. Compare against the effective limit, such as `Math.min(maxChars, coverage.diffChars)`.</violation>

<violation number="2" location="packages/opencode/src/altimate/review/ai-review.ts:421">
P3: Batch sizing counts every card, but `buildBatchMessage` omits cards after the 6,000-character per-call cap. Card-heavy inputs can therefore split into unnecessary model calls; apply the same cumulative card cap when estimating batches.</violation>

<violation number="3" location="packages/opencode/src/altimate/review/ai-review.ts:649">
P2: `planAiInput` now runs outside the `try` in `runAiReview`, so an exception while rendering the prompt escapes the lane. The doc comment promises this function never throws. Move `const plan = planAiInput(input)` inside the `try`, or wrap the planning in its own guard that returns an `error` status.</violation>
</file>

<file name="packages/opencode/src/altimate/review/runner.ts">

<violation number="1" location="packages/opencode/src/altimate/review/runner.ts:570">
P2: Column tests disappear from the card for manifests without `attached_node`; fall back to `depends_on.nodes` and `test_metadata.kwargs.column_name` when those fields are absent.</violation>

<violation number="2" location="packages/opencode/src/altimate/review/runner.ts:603">
P3: `text()` keeps only `partition_by.field` and drops `granularity`. Include the granularity in the rendered value, since the card is meant to show partitioning facts the diff does not.</violation>

<violation number="3" location="packages/opencode/src/altimate/review/runner.ts:627">
P2: `macroCard` tries the name before the changed file's path. For a file like macros/sf/hash.sql, the card then describes the dispatcher `hash` from another file, not the macro this PR changed. Try an exact `original_file_path` match first, then fall back to name.</violation>

<violation number="4" location="packages/opencode/src/altimate/review/runner.ts:639">
P2: These `.slice()` caps run before rendering, so the card's `+N more` marker can never show for callers, columns or siblings. A macro with 40 callers reads as having 10. Return the full lists and let `renderContextCard`'s `list()` cap them, or keep the true totals.</violation>
</file>

<file name="packages/opencode/src/altimate/review/ai-context-card.ts">

<violation number="1" location="packages/opencode/src/altimate/review/ai-context-card.ts:95">
P2: Seed files never get a context card. `MODEL_KINDS` leaves out `seed`, although `ModelCard` and `modelCard()` are documented to cover seeds. Add `seed` to the set, and drop the `kind === "model"` filter in the yml/test branch if seed parents should be carded there too.</violation>

<violation number="2" location="packages/opencode/src/altimate/review/ai-context-card.ts:122">
P3: The comment says a yml card covers 'the models it declares or tests', but `graphParents([f.path])` only returns models that generic tests in that yml depend on. A yml that declares a model without tests gets no card. Either fix the comment to say 'tests', or resolve declared models separately.</violation>

<violation number="3" location="packages/opencode/src/altimate/review/ai-context-card.ts:123">
P2: Schema and test changes attached to seed or snapshot nodes lose their related context card because this filter accepts only models. Include the resource kinds supported by `modelCard`.</violation>
</file>

<file name="packages/opencode/src/altimate/review/orchestrate.ts">

<violation number="1" location="packages/opencode/src/altimate/review/orchestrate.ts:2477">
P3: Seed files never receive the context card this new interface promises, because `attachContextCards` excludes `seed` from its cardable kinds. Add `seed` to the helper's model-card kinds.</violation>

<violation number="2" location="packages/opencode/src/altimate/review/orchestrate.ts:2549">
P2: These quota-skipped files were still checked by the deterministic lanes, but the ledger shows them under "Not reviewed". Consider a separate section or label (for example "Not sent to AI") so the header does not overstate coverage.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

const replay = aiReplayPath()
if (replay) return runAiReplay(input, replay)
const startedAt = Date.now()
const plan = planAiInput(input)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: planAiInput now runs outside the try in runAiReview, so an exception while rendering the prompt escapes the lane. The doc comment promises this function never throws. Move const plan = planAiInput(input) inside the try, or wrap the planning in its own guard that returns an error status.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 649:

<comment>`planAiInput` now runs outside the `try` in `runAiReview`, so an exception while rendering the prompt escapes the lane. The doc comment promises this function never throws. Move `const plan = planAiInput(input)` inside the `try`, or wrap the planning in its own guard that returns an `error` status.</comment>

<file context>
@@ -263,13 +646,20 @@ export async function runAiReview(input: AiReviewInput): Promise<AiReviewResult>
   const replay = aiReplayPath()
   if (replay) return runAiReplay(input, replay)
   const startedAt = Date.now()
+  const plan = planAiInput(input)
+  const { files, budget } = plan
+  const dryRun = aiDryRunEnabled()
</file context>

...(manifestMissing && !anyManifest
? modelFiles.map((f) => ({ path: f.path, reason: "no dbt manifest: lineage, config and graph checks skipped" }))
: []),
...aiUnsent.map((p) => ({ path: p, reason: "not sent to the AI (file quota)" })),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: These quota-skipped files were still checked by the deterministic lanes, but the ledger shows them under "Not reviewed". Consider a separate section or label (for example "Not sent to AI") so the header does not overstate coverage.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 2549:

<comment>These quota-skipped files were still checked by the deterministic lanes, but the ledger shows them under "Not reviewed". Consider a separate section or label (for example "Not sent to AI") so the header does not overstate coverage.</comment>

<file context>
@@ -2532,6 +2546,7 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
     ...(manifestMissing && !anyManifest
       ? modelFiles.map((f) => ({ path: f.path, reason: "no dbt manifest: lineage, config and graph checks skipped" }))
       : []),
+    ...aiUnsent.map((p) => ({ path: p, reason: "not sent to the AI (file quota)" })),
   ]
   const seen = new Set<string>()
</file context>

const byName = macros.filter(([, m]) => m?.name === name)
const path = cardOpts?.path
const hit =
byName.find(([, m]) => m.package_name === project) ??

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: macroCard tries the name before the changed file's path. For a file like macros/sf/hash.sql, the card then describes the dispatcher hash from another file, not the macro this PR changed. Try an exact original_file_path match first, then fall back to name.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/runner.ts, line 627:

<comment>`macroCard` tries the name before the changed file's path. For a file like macros/sf/hash.sql, the card then describes the dispatcher `hash` from another file, not the macro this PR changed. Try an exact `original_file_path` match first, then fall back to name.</comment>

<file context>
@@ -531,6 +532,129 @@ export function createDispatcherRunner(opts: DispatcherRunnerOptions): ReviewRun
+        const byName = macros.filter(([, m]) => m?.name === name)
+        const path = cardOpts?.path
+        const hit =
+          byName.find(([, m]) => m.package_name === project) ??
+          byName[0] ??
+          (path
</file context>

.filter(([id, n]) => id.startsWith("model.") && asArray<string>(n?.depends_on?.macros).includes(uid))
.map(([, n]) => String(n.name))
.sort((a, b) => Number(changed.has(b)) - Number(changed.has(a)))
.slice(0, 10)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: These .slice() caps run before rendering, so the card's +N more marker can never show for callers, columns or siblings. A macro with 40 callers reads as having 10. Return the full lists and let renderContextCard's list() cap them, or keep the true totals.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/runner.ts, line 639:

<comment>These `.slice()` caps run before rendering, so the card's `+N more` marker can never show for callers, columns or siblings. A macro with 40 callers reads as having 10. Return the full lists and let `renderContextCard`'s `list()` cap them, or keep the true totals.</comment>

<file context>
@@ -531,6 +532,129 @@ export function createDispatcherRunner(opts: DispatcherRunnerOptions): ReviewRun
+          .filter(([id, n]) => id.startsWith("model.") && asArray<string>(n?.depends_on?.macros).includes(uid))
+          .map(([, n]) => String(n.name))
+          .sort((a, b) => Number(changed.has(b)) - Number(changed.has(a)))
+          .slice(0, 10)
+        const macroName = String(macro.name)
+        const sep = macroName.indexOf("__")
</file context>

return text.slice(0, maxChars - 1) + "…"
}

const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot"])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Seed files never get a context card. MODEL_KINDS leaves out seed, although ModelCard and modelCard() are documented to cover seeds. Add seed to the set, and drop the kind === "model" filter in the yml/test branch if seed parents should be carded there too.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-context-card.ts, line 95:

<comment>Seed files never get a context card. `MODEL_KINDS` leaves out `seed`, although `ModelCard` and `modelCard()` are documented to cover seeds. Add `seed` to the set, and drop the `kind === "model"` filter in the yml/test branch if seed parents should be carded there too.</comment>

<file context>
@@ -0,0 +1,134 @@
+  return text.slice(0, maxChars - 1) + "…"
+}
+
+const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot"])
+const DECLARING_KINDS = new Set(["schema_yml", "source_yml", "test"])
+
</file context>
Suggested change
const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot"])
const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot", "seed"])

const out: string[] = []
for (let h = 0; h < hunks; h++) {
const start = 10 + h * 100
out.push(`@@ -${start},${size + 2} +${start},${size + 4} @@`)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The hunk header counts in bigDiff don't match the hunk body: the old side has 3 lines and the new side has size + 2, but the header says size + 2 and size + 4. Use the real counts so this fixture is a valid unified diff for any future hunk-window test.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/fixtures/review-ai-prompt-input.ts, line 12:

<comment>The hunk header counts in `bigDiff` don't match the hunk body: the old side has 3 lines and the new side has `size + 2`, but the header says `size + 2` and `size + 4`. Use the real counts so this fixture is a valid unified diff for any future hunk-window test.</comment>

<file context>
@@ -0,0 +1,64 @@
+  const out: string[] = []
+  for (let h = 0; h < hunks; h++) {
+    const start = 10 + h * 100
+    out.push(`@@ -${start},${size + 2} +${start},${size + 4} @@`)
+    out.push(` select ${h} as ctx_before`)
+    for (let i = 0; i < size; i++) out.push(`+  , coalesce(col_${h}_${i}, 'unknown_value_${i}') as renamed_column_${h}_${i}`)
</file context>
Suggested change
out.push(`@@ -${start},${size + 2} +${start},${size + 4} @@`)
out.push(`@@ -${start},3 +${start},${size + 2} @@`)

materialized: text(config.materialized),
uniqueKey: text(config.unique_key),
incrementalStrategy: text(config.incremental_strategy),
partitionBy: text(config.partition_by),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: text() keeps only partition_by.field and drops granularity. Include the granularity in the rendered value, since the card is meant to show partitioning facts the diff does not.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/runner.ts, line 603:

<comment>`text()` keeps only `partition_by.field` and drops `granularity`. Include the granularity in the rendered value, since the card is meant to show partitioning facts the diff does not.</comment>

<file context>
@@ -531,6 +532,129 @@ export function createDispatcherRunner(opts: DispatcherRunnerOptions): ReviewRun
+          materialized: text(config.materialized),
+          uniqueKey: text(config.unique_key),
+          incrementalStrategy: text(config.incremental_strategy),
+          partitionBy: text(config.partition_by),
+          clusterBy: text(config.cluster_by),
+          contractEnforced: config.contract?.enforced === true || node.contract?.enforced === true,
</file context>

return macro ? [{ name: macro.name, macro }] : []
}
if (!DECLARING_KINDS.has(kind)) return []
const parents = (await runner.graphParents?.([f.path])) ?? []

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The comment says a yml card covers 'the models it declares or tests', but graphParents([f.path]) only returns models that generic tests in that yml depend on. A yml that declares a model without tests gets no card. Either fix the comment to say 'tests', or resolve declared models separately.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-context-card.ts, line 122:

<comment>The comment says a yml card covers 'the models it declares or tests', but `graphParents([f.path])` only returns models that generic tests in that yml depend on. A yml that declares a model without tests gets no card. Either fix the comment to say 'tests', or resolve declared models separately.</comment>

<file context>
@@ -0,0 +1,134 @@
+      return macro ? [{ name: macro.name, macro }] : []
+    }
+    if (!DECLARING_KINDS.has(kind)) return []
+    const parents = (await runner.graphParents?.([f.path])) ?? []
+    const names = [...new Set(parents.filter((p) => p.kind === "model").map((p) => p.name))].slice(0, 3)
+    const cards = await Promise.all(names.map(async (name) => ({ name, model: await runner.modelCard?.(name) })))
</file context>

aiFiles.push({ path: f.path, status: f.status, model: modelNameFromPath(f.path), kind: f.kind, diff: f.diff, sql: contents[i] }),
)
touch(aiFiles)
if (input.aiContextCard && anyManifest) await attachContextCards(aiFiles, input.runner)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: Seed files never receive the context card this new interface promises, because attachContextCards excludes seed from its cardable kinds. Add seed to the helper's model-card kinds.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 2477:

<comment>Seed files never receive the context card this new interface promises, because `attachContextCards` excludes `seed` from its cardable kinds. Add `seed` to the helper's model-card kinds.</comment>

<file context>
@@ -2466,6 +2474,7 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
       aiFiles.push({ path: f.path, status: f.status, model: modelNameFromPath(f.path), kind: f.kind, diff: f.diff, sql: contents[i] }),
     )
     touch(aiFiles)
+    if (input.aiContextCard && anyManifest) await attachContextCards(aiFiles, input.runner)
     const context = input.aiGraphContext ? await aiGraphContext(aiFiles.map((f) => f.path), input.runner, getContent) : undefined
     // The verifier reads engine SQL first (the same SQL the engine lanes saw), then compiled output.
</file context>

let size = 0
for (const f of files) {
const n = fileSection(f, budget.maxFileChars).length
const n = fileSection(f, budget.maxFileChars, budget.coverage).length

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: Batch sizing counts every card, but buildBatchMessage omits cards after the 6,000-character per-call cap. Card-heavy inputs can therefore split into unnecessary model calls; apply the same cumulative card cap when estimating batches.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 421:

<comment>Batch sizing counts every card, but `buildBatchMessage` omits cards after the 6,000-character per-call cap. Card-heavy inputs can therefore split into unnecessary model calls; apply the same cumulative card cap when estimating batches.</comment>

<file context>
@@ -105,7 +418,7 @@ export function batchAiFiles(files: AiReviewFile[], budget: AiInputBudget): AiRe
   let size = 0
   for (const f of files) {
-    const n = fileSection(f, budget.maxFileChars).length
+    const n = fileSection(f, budget.maxFileChars, budget.coverage).length
     if (cur.length && size + n > budget.batchChars) {
       batches.push(cur)
</file context>

/** Head-side lines of context kept around each changed line in a file excerpt. */
const EXCERPT_CONTEXT_LINES = 20
/** Characters of a file's diff section under coverage (hunks beyond it are listed as omitted). */
const COVERAGE_DIFF_CHARS = 16_000

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Coverage mode can build a single ~660k-char call with no per-call cap

With ALTIMATE_REVIEW_AI_COVERAGE=1 the diff section can be COVERAGE_DIFF_CHARS (16,000) per file plus up to 6,000 of current file, while the new quotas allow 30 files (12 models + 8 macros + 4 tests + 6 yml) and batchChars still defaults to Infinity (line 57), so batchAiFiles keeps them all in one batch. One call can reach ~30 x 22k = 660k chars (~165k tokens) versus <= 20 x 12k = 240k before. That overruns a 128k-token model's context, reviewBatch throws, every batch fails and the lane reports error with no findings - on exactly the large PRs this mode targets. Consider defaulting batchChars to a bounded value when coverage is on, or scaling the per-file budgets against a per-call total.

const BUCKET_ORDER = [...new Set(KIND_ORDER.map((k) => BUCKET_OF[k]))]

/** Added and removed lines of a unified diff (file headers before the first hunk excluded). */
export function changedLineCount(diff: string | undefined): number {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Duplicates the existing countChangedLines

diff-filter.ts:329 already exports countChangedLines (imported from the same module for walkDiffLines) for this exact job, and it correctly ignores +++/--- file headers. This copy counts +++ b/... and --- a/... as changed lines whenever the diff has no @@ hunk (first === -1, so Math.max(first, 0) === 0), which inflates the most-changed ordering used by selectAiFilesByQuota for header-less diffs. Reuse countChangedLines(f.diff) here instead.

if (!kept.length) return { text: truncate(content, maxChars), truncated: true }
// Fill what is left with the file head, before the first window.
let head = 0
while (head + 1 < kept[0][0] && used + size(1, head + 1) + MARKER <= maxChars) head++

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Quadratic head fill

size(1, head + 1) re-slices and re-sums lines 1..head+1 on every iteration, so extending the head over H lines costs O(H^2) with a fresh array each step (H can approach maxChars for files with many short or blank lines, and this runs once per file under coverage). Accumulate the running length instead, e.g. keep headChars and add lines[head].length + 1 per step, then test used + headChars + MARKER <= maxChars.

return text.slice(0, maxChars - 1) + "…"
}

const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot"])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Changed seeds never get a context card

MODEL_KINDS omits "seed" and seeds are not in DECLARING_KINDS, so cardsFor returns [] for every seed file. But runner.modelCard explicitly resolves seed. nodes (runner.ts:543) and ModelCard is documented as facts about "one model, snapshot or seed", so that branch is unreachable through this feature. Either add "seed" to MODEL_KINDS (a changed seed is a legitimate review target) or drop the seed lookup from modelCard.

@kilo-code-bot

kilo-code-bot Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Code Review Summary

Status: 3 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 3
Issue Details (click to expand)

SUGGESTION

File Line Issue
packages/opencode/src/altimate/review/ai-review.ts 149 changedLineCount duplicates the existing countChangedLines in diff-filter.ts:378 and counts +++/--- headers for header-less diffs, inflating the most-changed ordering used by selectAiFilesByQuota.
packages/opencode/src/altimate/review/ai-review.ts 343 Head fill recomputes size(1, head + 1) on every iteration (fresh slice + re-sum), so extending the head is O(H²) per file under coverage; keep a running total.
packages/opencode/src/altimate/review/ai-context-card.ts 95 Seeds are in neither MODEL_KINDS nor DECLARING_KINDS, so cardsFor returns [] for every seed while modelCard resolves seed. nodes — a changed seed never gets a card.
Files Reviewed (9 files)
  • packages/opencode/src/altimate/review/ai-context-card.ts - 1 issue
  • packages/opencode/src/altimate/review/ai-review.ts - 2 issues
  • packages/opencode/src/altimate/review/orchestrate.ts - no issues
  • packages/opencode/src/altimate/review/run.ts - no issues
  • packages/opencode/src/altimate/review/runner.ts - no issues
  • packages/opencode/src/altimate/review/verdict.ts - no issues
  • packages/opencode/test/altimate/fixtures/review-ai-prompt-default.txt - no issues
  • packages/opencode/test/altimate/fixtures/review-ai-prompt-input.ts - no issues
  • packages/opencode/test/altimate/review-ai-inputs.test.ts - no issues

Note: the per-call size bound raised earlier is now fixed (batchChars defaults to 120,000 under coverage). The PR description still omits the repository's required "Cost side effects" section; when coverage is enabled the lane sends up to 30 files and splits them into several model calls, so that section should be filled in.

Fix these issues in Kilo Cloud

Previous Review Summary (commit b07b80d)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit b07b80d)

Status: 4 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 3
Issue Details (click to expand)

WARNING

File Line Issue
packages/opencode/src/altimate/review/ai-review.ts 30 Coverage lets one call reach ~660k chars (~165k tokens): 16k diff + 6k file per file, 30-file quotas, batchChars still Infinity. Overruns 128k-context models and fails the lane on the large PRs the mode targets.

SUGGESTION

File Line Issue
packages/opencode/src/altimate/review/ai-review.ts 144 changedLineCount duplicates countChangedLines in diff-filter.ts and counts +++/--- headers as changed lines for header-less diffs.
packages/opencode/src/altimate/review/ai-review.ts 338 Head fill recomputes size(1, head + 1) each iteration: O(H²) with a fresh slice per step; keep a running total.
packages/opencode/src/altimate/review/ai-context-card.ts 95 Seeds are in neither MODEL_KINDS nor DECLARING_KINDS, so modelCard's seed. branch (runner.ts:543) is unreachable and changed seeds get no card.

Note: the PR description omits the repository's required "Cost side effects" section (the automated issue-compliance check already flagged this). The lane is opt-in and bounded by file quotas and character budgets, but it does raise per-review token spend ~1.25x when enabled, so that section should be filled in.

Fix these issues in Kilo Cloud

Files Reviewed (9 files)
  • packages/opencode/src/altimate/review/ai-context-card.ts - 1 issue
  • packages/opencode/src/altimate/review/ai-review.ts - 3 issues
  • packages/opencode/src/altimate/review/orchestrate.ts - no issues
  • packages/opencode/src/altimate/review/run.ts - no issues
  • packages/opencode/src/altimate/review/runner.ts - no issues
  • packages/opencode/src/altimate/review/verdict.ts - no issues
  • packages/opencode/test/altimate/fixtures/review-ai-prompt-default.txt - no issues
  • packages/opencode/test/altimate/fixtures/review-ai-prompt-input.ts - no issues
  • packages/opencode/test/altimate/review-ai-inputs.test.ts - no issues

Reviewed by deepseek-v4.1-flash-1017 · Input: 173.1K · Output: 11.4K · Cached: 578.9K

Review guidance: REVIEW.md from base branch feat/review-recall-rules

@hkaltai hkaltai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: approve, with minor follow-ups. Panel: Claude, Codex and lead reviewer, covering all 9 changed files.

The core promise holds: with the flags unset the prompt is byte-identical to the base (confirmed independently by two reviewers against the snapshot), and the 31 tests in review-ai-inputs.test.ts pass. All findings concern behavior behind the opt-in flags and advisory context quality, so none is CRITICAL or MAJOR.

Counts: CRITICAL 0, MAJOR 0, MINOR 8 (inline), NIT 10.

Most worthwhile follow-ups: bound the coverage-mode call size (batchChars default), make compiled-SQL windowing robust when added lines are Jinja-only, and resolve model/macro cards by project and path rather than by first name match.

Nits

  • ai-review.ts:51 num requires n > 0: ALTIMATE_REVIEW_AI_QUOTA_TEST=0 silently becomes 4, so a bucket cannot be disabled. Quoted: return v && Number.isFinite(n) && n > 0 ? n : d. Fix: separate parser allowing 0 for quotas.
  • excerptFile window excerpt (ai-review.ts:327 area): when the first context window exceeds the budget, the beginning is kept and the changed anchor can be dropped (350-char lines, change at line 100, 6,000 budget). Fix: reserve space for changed lines first, then expand context.
  • runner.ts:640-649: a macro name containing __ splits at the first __, so an unrelated get_x__y is listed as a variant of y. Fix: only treat default/adapter prefixes or dispatchers as variants.
  • ai-review.ts:240: preamble longer than budget emits an empty "(hunk truncated)" but reports hunksSent: 1. Fix: record 0 when maxChars - used <= 0.
  • ai-review.ts:313: excerptFile can exceed maxChars by about 40 chars (trailing marker not reserved when no head window). Fix: reserve it.
  • ai-context-card.ts MODEL_KINDS lacks seed although the ModelCard doc says "model, snapshot or seed". Fix: add seed or fix the doc.
  • Comment at ai-review.ts:563 says cards stop once the call holds 6,000 chars, but a too-large card is skipped and later smaller ones are still added. Reword.
  • planAiInput doc says "Pure apart from reading env", but env is a parameter.
  • Naming: AiReviewFile.card (string) vs AiInputRecord.card (boolean); fileChars in renderFile is a budget (fileBudget); CoverageArg = boolean | AiCoverageRender | undefined is a test-only tri-state; local const MARKER = 40 could be module-level MARKER_RESERVE.
  • Missing tests: dry-run+replay+verify, duplicate model names across packages, macro basename/path collisions, YAML cards via the real runner (not a mocked graphParents), Jinja-only changes on compiled SQL, bounded size under coverage, and rendering of quota-unsent files in the "Not reviewed" block.

@@ -263,13 +646,20 @@ export async function runAiReview(input: AiReviewInput): Promise<AiReviewResult>
const replay = aiReplayPath()
if (replay) return runAiReplay(input, replay)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Replay returns before the dry-run check

if (replay) return runAiReplay(input, replay)

With ALTIMATE_REVIEW_AI_DRYRUN=1, ALTIMATE_REVIEW_AI_REPLAY=<envelope> and ALTIMATE_REVIEW_AI_VERIFY=1, replay returns before dry-run is evaluated, so a stored envelope with AI findings still reaches the verifier (a model call) and no ai.input is recorded. Replay is an eval-only path, hence MINOR, but it breaks the no-model-call promise of dry-run. Fix: evaluate dry-run (and plan the input) before dispatching replay, and add a regression test. (Codex)

* lines (8+ non-space chars, no Jinja) are found by text, and "unlocated" otherwise.
*/
function locateAddedLines(lines: string[], rows: Array<{ line: number; text: string; kind: string }>): number[] | "unlocated" | undefined {
const eligible = rows.filter((r) => r.kind === "+" && !JINJA.test(r.text) && r.text.replace(/\s/g, "").length >= 8)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Compiled-SQL windows fall back to raw line numbers when no added line is eligible

const eligible = rows.filter((r) => r.kind === "+" && !JINJA.test(r.text) && r.text.replace(/\s/g, "").length >= 8)

When every added line is Jinja or under 8 non-space chars ({{ ref(..) }} swaps, {% if is_incremental() %}, , id), eligible is empty and the function returns undefined, which excerptFile treats as "raw line numbers match". On compiled SQL where macro expansion shifted lines, windows are then centred on the wrong lines and the unapproximated marker is not emitted. The diff itself is still sent, so this degrades excerpt quality only (opt-in flag). Fix: let the caller say whether f.sql is compiled, or also check context rows; return "unlocated" if nothing can be verified. Add a Jinja-only test on compiled SQL. (Claude)

maxFiles: num(env.ALTIMATE_REVIEW_AI_MAX_FILES, MAX_FILES),
maxFileChars: num(env.ALTIMATE_REVIEW_AI_MAX_FILE_CHARS, MAX_DIFF_CHARS),
batchChars: num(env.ALTIMATE_REVIEW_AI_BATCH_CHARS, Infinity),
coverage:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Coverage mode enlarges the single call with no bound

batchChars: num(env.ALTIMATE_REVIEW_AI_BATCH_CHARS, Infinity),

Coverage raises the file cap to 30 and adds a per-diff budget, while batchChars stays Infinity. A 40-large-model PR was measured at about 242k chars by default vs about 658k with coverage, which can exceed context or the 300 s timeout and return no AI findings at all. The unbounded single call is pre-existing, but coverage makes it markedly worse. Fix: default batchChars to a finite value when coverage is on, or cap total chars and push overflow into unsent. (Claude)

const nodes = Object.entries<any>(raw.nodes ?? {})
const isModel = (id: string) => id.startsWith("model.")
const hit =
nodes.find(([id, n]) => isModel(id) && n?.name === name) ??

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] modelCard picks the first same-named model across packages

nodes.find(([id, n]) => isModel(id) && n?.name === name) ??

A dependency package model with the same name listed earlier in the manifest supplies materialization, keys, columns and graph facts for the changed project model (reproduced by both external reviewers). The card is presented as authoritative. Fix: prefer metadata.project_name (as macroCard does) and/or match by path or unique id. (Codex, Claude)

const byName = macros.filter(([, m]) => m?.name === name)
const path = cardOpts?.path
const hit =
byName.find(([, m]) => m.package_name === project) ??

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] macroCard resolves by file basename before path, and not within the project

byName.find(([, m]) => m.package_name === project) ??
          byName[0] ??

The lookup name is the file basename, not necessarily the macro defined in it. macros/helpers.sql defining foo can get the card of an unrelated helpers macro, or a package macro (e.g. dbt_utils.date_spine) with its callers and variants. The path fallback is also not restricted to the project package. Fix: match by path within the project first, then name within the project, then any package. (Codex, Claude)

return macro ? [{ name: macro.name, macro }] : []
}
if (!DECLARING_KINDS.has(kind)) return []
const parents = (await runner.graphParents?.([f.path])) ?? []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] YAML declaration cards depend on test-node parents, not patch_path

const parents = (await runner.graphParents?.([f.path])) ?? []

graphParents links a YAML file to nodes via tests, while models are tied to their YAML by patch_path. A schema YAML with no generic tests gets no card even though the manifest has its models' config; removing the last test removes the card. The orchestrator test mocks graphParents, which hides this. Fix: resolve declarations through patch_path. (Codex, Claude)

: fileChars < maxChars
? truncate(f.sql, fileChars)
: excerptFile(f.sql, f.diff, maxChars).text
if (f.sql.length > fileChars) stats = { ...stats, truncated: true }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Ledger reports truncated:false when MAX_FILE_CHARS < 2000

if (f.sql.length > fileChars) stats = { ...stats, truncated: true }

With ALTIMATE_REVIEW_AI_MAX_FILE_CHARS=1000 and 1,500-char SQL, fileChars is widened to 2000 but excerptFile(..., maxChars) still truncates, so the record says truncated:false while the prompt contains (truncated). Fix: compare against the budget actually applied (Math.min(maxChars, ...)). (Claude)

...(manifestMissing && !anyManifest
? modelFiles.map((f) => ({ path: f.path, reason: "no dbt manifest: lineage, config and graph checks skipped" }))
: []),
...aiUnsent.map((p) => ({ path: p, reason: "not sent to the AI (file quota)" })),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Quota-omitted files listed under Not reviewed although engine lanes checked them

...aiUnsent.map((p) => ({ path: p, reason: "not sent to the AI (file quota)" })),

The ledger means "every changed file the review did not look at", but these files went through the deterministic lanes; on a large PR dozens of engine-checked files show up in that block. The default maxFiles cap records nothing, so behaviour is inconsistent. Fix: report as a separate note or field. (Claude)

RaghwendraSingh and others added 4 commits November 20, 2026 12:19
…ext card

All switches are off by default; with them unset the AI prompt is
byte-identical to before (pinned by a snapshot captured before the change).

- `ALTIMATE_REVIEW_AI_COVERAGE=1`: per-kind file quotas (models 12, macros 8,
  tests 4, yml/config 6; `ALTIMATE_REVIEW_AI_QUOTA_{MODEL,MACRO,TEST,YML}`),
  most-changed files first, unused quota redistributed up to the total.
  Over-budget diffs keep whole hunks and name omitted hunks by line range;
  over-budget files send +/-20-line windows around changed lines plus the file
  head, with `… (lines a–b omitted)` markers. Files left out go to
  `scope.notReviewed` as "not sent to the AI (file quota)".
- `ai.input` on the envelope records per file what was sent (chars, truncated,
  hunks sent/total, card) under coverage, context cards or dry run.
- `ALTIMATE_REVIEW_AI_DRYRUN=1`: the lane plans its input, records `ai.input`
  and returns `skipped` without calling a model. `planAiInput` is the pure
  planner `runAiReview` now sends from.
- `ALTIMATE_REVIEW_AI_CONTEXT_CARD=1` (needs a manifest): a `dbt context (from
  manifest):` block under each file header, from new non-throwing runner
  methods `modelCard` and `macroCard`; yml and singular tests get the cards of
  the models they test. At most 600 chars per file and 6,000 per call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… SQL

Under `ALTIMATE_REVIEW_AI_COVERAGE=1`, a model's current file is often
dbt-compiled SQL, whose line numbers drift from the raw diff once Jinja
expands. When the added lines are not at their raw line numbers, `excerptFile`
now finds each added line (8+ non-space chars, no Jinja) by its trimmed text,
scanning forward, and centres the windows there. If fewer than half are found
it keeps the head cut with a marker saying windows would be approximate. Raw
content keeps raw line numbers; the default prompt is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…header

Under `ALTIMATE_REVIEW_AI_COVERAGE=1` only (the default prompt is unchanged):

- The diff section gets its own budget, `ALTIMATE_REVIEW_AI_COVERAGE_DIFF_CHARS`
  (default 16,000), instead of the 6,000-char file budget. Hunks stay whole
  and in order; only hunks past that budget are listed as omitted. Multi-hunk
  macro diffs that sent 7 of 20 hunks now go whole.
- When a long diff (over the file budget) is sent with every hunk, the
  current-file section shrinks to a 2,000-char head; otherwise the file keeps
  its +/-20-line windows within the file budget.
- Fix: an excerpted diff replaced the whole file section (header, card and
  current file) because the excerpt's `text` overwrote the section text.
  `ai.input` chars, hunksSent and hunksTotal now describe what is really sent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- `aiInputBudget`: with `ALTIMATE_REVIEW_AI_COVERAGE=1`, `batchChars` now defaults
  to 120k chars instead of `Infinity`, so 30 quota files with 16k diffs no longer
  build one ~660k-char call. `ALTIMATE_REVIEW_AI_BATCH_CHARS` still overrides it;
  with coverage off the default stays `Infinity` (prompt snapshot unchanged).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Thanks for updating your PR! It now meets our contributing guidelines. 👍

@github-actions

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants