Repository navigation
feat(review): opt-in AI input coverage, hunk excerpts and a dbt manifest context card - #1771
RaghwendraSingh wants to merge 4 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
There was a problem hiding this comment.
13 issues found across 9 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/test/altimate/fixtures/review-ai-prompt-input.ts">
<violation number="1" location="packages/opencode/test/altimate/fixtures/review-ai-prompt-input.ts:12">
P3: The hunk header counts in `bigDiff` don't match the hunk body: the old side has 3 lines and the new side has `size + 2`, but the header says `size + 2` and `size + 4`. Use the real counts so this fixture is a valid unified diff for any future hunk-window test.</violation>
</file>
<file name="packages/opencode/src/altimate/review/ai-review.ts">
<violation number="1" location="packages/opencode/src/altimate/review/ai-review.ts:391">
P2: A diff between `coverage.diffChars` and `maxChars` bypasses `excerptDiff`, so a lower `ALTIMATE_REVIEW_AI_COVERAGE_DIFF_CHARS` override is ignored. Compare against the effective limit, such as `Math.min(maxChars, coverage.diffChars)`.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/ai-review.ts:421">
P3: Batch sizing counts every card, but `buildBatchMessage` omits cards after the 6,000-character per-call cap. Card-heavy inputs can therefore split into unnecessary model calls; apply the same cumulative card cap when estimating batches.</violation>
<violation number="3" location="packages/opencode/src/altimate/review/ai-review.ts:649">
P2: `planAiInput` now runs outside the `try` in `runAiReview`, so an exception while rendering the prompt escapes the lane. The doc comment promises this function never throws. Move `const plan = planAiInput(input)` inside the `try`, or wrap the planning in its own guard that returns an `error` status.</violation>
</file>
<file name="packages/opencode/src/altimate/review/runner.ts">
<violation number="1" location="packages/opencode/src/altimate/review/runner.ts:570">
P2: Column tests disappear from the card for manifests without `attached_node`; fall back to `depends_on.nodes` and `test_metadata.kwargs.column_name` when those fields are absent.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/runner.ts:603">
P3: `text()` keeps only `partition_by.field` and drops `granularity`. Include the granularity in the rendered value, since the card is meant to show partitioning facts the diff does not.</violation>
<violation number="3" location="packages/opencode/src/altimate/review/runner.ts:627">
P2: `macroCard` tries the name before the changed file's path. For a file like macros/sf/hash.sql, the card then describes the dispatcher `hash` from another file, not the macro this PR changed. Try an exact `original_file_path` match first, then fall back to name.</violation>
<violation number="4" location="packages/opencode/src/altimate/review/runner.ts:639">
P2: These `.slice()` caps run before rendering, so the card's `+N more` marker can never show for callers, columns or siblings. A macro with 40 callers reads as having 10. Return the full lists and let `renderContextCard`'s `list()` cap them, or keep the true totals.</violation>
</file>
<file name="packages/opencode/src/altimate/review/ai-context-card.ts">
<violation number="1" location="packages/opencode/src/altimate/review/ai-context-card.ts:95">
P2: Seed files never get a context card. `MODEL_KINDS` leaves out `seed`, although `ModelCard` and `modelCard()` are documented to cover seeds. Add `seed` to the set, and drop the `kind === "model"` filter in the yml/test branch if seed parents should be carded there too.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/ai-context-card.ts:122">
P3: The comment says a yml card covers 'the models it declares or tests', but `graphParents([f.path])` only returns models that generic tests in that yml depend on. A yml that declares a model without tests gets no card. Either fix the comment to say 'tests', or resolve declared models separately.</violation>
<violation number="3" location="packages/opencode/src/altimate/review/ai-context-card.ts:123">
P2: Schema and test changes attached to seed or snapshot nodes lose their related context card because this filter accepts only models. Include the resource kinds supported by `modelCard`.</violation>
</file>
<file name="packages/opencode/src/altimate/review/orchestrate.ts">
<violation number="1" location="packages/opencode/src/altimate/review/orchestrate.ts:2477">
P3: Seed files never receive the context card this new interface promises, because `attachContextCards` excludes `seed` from its cardable kinds. Add `seed` to the helper's model-card kinds.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/orchestrate.ts:2549">
P2: These quota-skipped files were still checked by the deterministic lanes, but the ledger shows them under "Not reviewed". Consider a separate section or label (for example "Not sent to AI") so the header does not overstate coverage.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| const replay = aiReplayPath() | ||
| if (replay) return runAiReplay(input, replay) | ||
| const startedAt = Date.now() | ||
| const plan = planAiInput(input) |
There was a problem hiding this comment.
P2: planAiInput now runs outside the try in runAiReview, so an exception while rendering the prompt escapes the lane. The doc comment promises this function never throws. Move const plan = planAiInput(input) inside the try, or wrap the planning in its own guard that returns an error status.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 649:
<comment>`planAiInput` now runs outside the `try` in `runAiReview`, so an exception while rendering the prompt escapes the lane. The doc comment promises this function never throws. Move `const plan = planAiInput(input)` inside the `try`, or wrap the planning in its own guard that returns an `error` status.</comment>
<file context>
@@ -263,13 +646,20 @@ export async function runAiReview(input: AiReviewInput): Promise<AiReviewResult>
const replay = aiReplayPath()
if (replay) return runAiReplay(input, replay)
const startedAt = Date.now()
+ const plan = planAiInput(input)
+ const { files, budget } = plan
+ const dryRun = aiDryRunEnabled()
</file context>
| ...(manifestMissing && !anyManifest | ||
| ? modelFiles.map((f) => ({ path: f.path, reason: "no dbt manifest: lineage, config and graph checks skipped" })) | ||
| : []), | ||
| ...aiUnsent.map((p) => ({ path: p, reason: "not sent to the AI (file quota)" })), |
There was a problem hiding this comment.
P2: These quota-skipped files were still checked by the deterministic lanes, but the ledger shows them under "Not reviewed". Consider a separate section or label (for example "Not sent to AI") so the header does not overstate coverage.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 2549:
<comment>These quota-skipped files were still checked by the deterministic lanes, but the ledger shows them under "Not reviewed". Consider a separate section or label (for example "Not sent to AI") so the header does not overstate coverage.</comment>
<file context>
@@ -2532,6 +2546,7 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
...(manifestMissing && !anyManifest
? modelFiles.map((f) => ({ path: f.path, reason: "no dbt manifest: lineage, config and graph checks skipped" }))
: []),
+ ...aiUnsent.map((p) => ({ path: p, reason: "not sent to the AI (file quota)" })),
]
const seen = new Set<string>()
</file context>
| const byName = macros.filter(([, m]) => m?.name === name) | ||
| const path = cardOpts?.path | ||
| const hit = | ||
| byName.find(([, m]) => m.package_name === project) ?? |
There was a problem hiding this comment.
P2: macroCard tries the name before the changed file's path. For a file like macros/sf/hash.sql, the card then describes the dispatcher hash from another file, not the macro this PR changed. Try an exact original_file_path match first, then fall back to name.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/runner.ts, line 627:
<comment>`macroCard` tries the name before the changed file's path. For a file like macros/sf/hash.sql, the card then describes the dispatcher `hash` from another file, not the macro this PR changed. Try an exact `original_file_path` match first, then fall back to name.</comment>
<file context>
@@ -531,6 +532,129 @@ export function createDispatcherRunner(opts: DispatcherRunnerOptions): ReviewRun
+ const byName = macros.filter(([, m]) => m?.name === name)
+ const path = cardOpts?.path
+ const hit =
+ byName.find(([, m]) => m.package_name === project) ??
+ byName[0] ??
+ (path
</file context>
| .filter(([id, n]) => id.startsWith("model.") && asArray<string>(n?.depends_on?.macros).includes(uid)) | ||
| .map(([, n]) => String(n.name)) | ||
| .sort((a, b) => Number(changed.has(b)) - Number(changed.has(a))) | ||
| .slice(0, 10) |
There was a problem hiding this comment.
P2: These .slice() caps run before rendering, so the card's +N more marker can never show for callers, columns or siblings. A macro with 40 callers reads as having 10. Return the full lists and let renderContextCard's list() cap them, or keep the true totals.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/runner.ts, line 639:
<comment>These `.slice()` caps run before rendering, so the card's `+N more` marker can never show for callers, columns or siblings. A macro with 40 callers reads as having 10. Return the full lists and let `renderContextCard`'s `list()` cap them, or keep the true totals.</comment>
<file context>
@@ -531,6 +532,129 @@ export function createDispatcherRunner(opts: DispatcherRunnerOptions): ReviewRun
+ .filter(([id, n]) => id.startsWith("model.") && asArray<string>(n?.depends_on?.macros).includes(uid))
+ .map(([, n]) => String(n.name))
+ .sort((a, b) => Number(changed.has(b)) - Number(changed.has(a)))
+ .slice(0, 10)
+ const macroName = String(macro.name)
+ const sep = macroName.indexOf("__")
</file context>
| return text.slice(0, maxChars - 1) + "…" | ||
| } | ||
|
|
||
| const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot"]) |
There was a problem hiding this comment.
P2: Seed files never get a context card. MODEL_KINDS leaves out seed, although ModelCard and modelCard() are documented to cover seeds. Add seed to the set, and drop the kind === "model" filter in the yml/test branch if seed parents should be carded there too.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-context-card.ts, line 95:
<comment>Seed files never get a context card. `MODEL_KINDS` leaves out `seed`, although `ModelCard` and `modelCard()` are documented to cover seeds. Add `seed` to the set, and drop the `kind === "model"` filter in the yml/test branch if seed parents should be carded there too.</comment>
<file context>
@@ -0,0 +1,134 @@
+ return text.slice(0, maxChars - 1) + "…"
+}
+
+const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot"])
+const DECLARING_KINDS = new Set(["schema_yml", "source_yml", "test"])
+
</file context>
| const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot"]) | |
| const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot", "seed"]) |
| const out: string[] = [] | ||
| for (let h = 0; h < hunks; h++) { | ||
| const start = 10 + h * 100 | ||
| out.push(`@@ -${start},${size + 2} +${start},${size + 4} @@`) |
There was a problem hiding this comment.
P3: The hunk header counts in bigDiff don't match the hunk body: the old side has 3 lines and the new side has size + 2, but the header says size + 2 and size + 4. Use the real counts so this fixture is a valid unified diff for any future hunk-window test.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/fixtures/review-ai-prompt-input.ts, line 12:
<comment>The hunk header counts in `bigDiff` don't match the hunk body: the old side has 3 lines and the new side has `size + 2`, but the header says `size + 2` and `size + 4`. Use the real counts so this fixture is a valid unified diff for any future hunk-window test.</comment>
<file context>
@@ -0,0 +1,64 @@
+ const out: string[] = []
+ for (let h = 0; h < hunks; h++) {
+ const start = 10 + h * 100
+ out.push(`@@ -${start},${size + 2} +${start},${size + 4} @@`)
+ out.push(` select ${h} as ctx_before`)
+ for (let i = 0; i < size; i++) out.push(`+ , coalesce(col_${h}_${i}, 'unknown_value_${i}') as renamed_column_${h}_${i}`)
</file context>
| out.push(`@@ -${start},${size + 2} +${start},${size + 4} @@`) | |
| out.push(`@@ -${start},3 +${start},${size + 2} @@`) |
| materialized: text(config.materialized), | ||
| uniqueKey: text(config.unique_key), | ||
| incrementalStrategy: text(config.incremental_strategy), | ||
| partitionBy: text(config.partition_by), |
There was a problem hiding this comment.
P3: text() keeps only partition_by.field and drops granularity. Include the granularity in the rendered value, since the card is meant to show partitioning facts the diff does not.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/runner.ts, line 603:
<comment>`text()` keeps only `partition_by.field` and drops `granularity`. Include the granularity in the rendered value, since the card is meant to show partitioning facts the diff does not.</comment>
<file context>
@@ -531,6 +532,129 @@ export function createDispatcherRunner(opts: DispatcherRunnerOptions): ReviewRun
+ materialized: text(config.materialized),
+ uniqueKey: text(config.unique_key),
+ incrementalStrategy: text(config.incremental_strategy),
+ partitionBy: text(config.partition_by),
+ clusterBy: text(config.cluster_by),
+ contractEnforced: config.contract?.enforced === true || node.contract?.enforced === true,
</file context>
| return macro ? [{ name: macro.name, macro }] : [] | ||
| } | ||
| if (!DECLARING_KINDS.has(kind)) return [] | ||
| const parents = (await runner.graphParents?.([f.path])) ?? [] |
There was a problem hiding this comment.
P3: The comment says a yml card covers 'the models it declares or tests', but graphParents([f.path]) only returns models that generic tests in that yml depend on. A yml that declares a model without tests gets no card. Either fix the comment to say 'tests', or resolve declared models separately.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-context-card.ts, line 122:
<comment>The comment says a yml card covers 'the models it declares or tests', but `graphParents([f.path])` only returns models that generic tests in that yml depend on. A yml that declares a model without tests gets no card. Either fix the comment to say 'tests', or resolve declared models separately.</comment>
<file context>
@@ -0,0 +1,134 @@
+ return macro ? [{ name: macro.name, macro }] : []
+ }
+ if (!DECLARING_KINDS.has(kind)) return []
+ const parents = (await runner.graphParents?.([f.path])) ?? []
+ const names = [...new Set(parents.filter((p) => p.kind === "model").map((p) => p.name))].slice(0, 3)
+ const cards = await Promise.all(names.map(async (name) => ({ name, model: await runner.modelCard?.(name) })))
</file context>
| aiFiles.push({ path: f.path, status: f.status, model: modelNameFromPath(f.path), kind: f.kind, diff: f.diff, sql: contents[i] }), | ||
| ) | ||
| touch(aiFiles) | ||
| if (input.aiContextCard && anyManifest) await attachContextCards(aiFiles, input.runner) |
There was a problem hiding this comment.
P3: Seed files never receive the context card this new interface promises, because attachContextCards excludes seed from its cardable kinds. Add seed to the helper's model-card kinds.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 2477:
<comment>Seed files never receive the context card this new interface promises, because `attachContextCards` excludes `seed` from its cardable kinds. Add `seed` to the helper's model-card kinds.</comment>
<file context>
@@ -2466,6 +2474,7 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
aiFiles.push({ path: f.path, status: f.status, model: modelNameFromPath(f.path), kind: f.kind, diff: f.diff, sql: contents[i] }),
)
touch(aiFiles)
+ if (input.aiContextCard && anyManifest) await attachContextCards(aiFiles, input.runner)
const context = input.aiGraphContext ? await aiGraphContext(aiFiles.map((f) => f.path), input.runner, getContent) : undefined
// The verifier reads engine SQL first (the same SQL the engine lanes saw), then compiled output.
</file context>
| let size = 0 | ||
| for (const f of files) { | ||
| const n = fileSection(f, budget.maxFileChars).length | ||
| const n = fileSection(f, budget.maxFileChars, budget.coverage).length |
There was a problem hiding this comment.
P3: Batch sizing counts every card, but buildBatchMessage omits cards after the 6,000-character per-call cap. Card-heavy inputs can therefore split into unnecessary model calls; apply the same cumulative card cap when estimating batches.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 421:
<comment>Batch sizing counts every card, but `buildBatchMessage` omits cards after the 6,000-character per-call cap. Card-heavy inputs can therefore split into unnecessary model calls; apply the same cumulative card cap when estimating batches.</comment>
<file context>
@@ -105,7 +418,7 @@ export function batchAiFiles(files: AiReviewFile[], budget: AiInputBudget): AiRe
let size = 0
for (const f of files) {
- const n = fileSection(f, budget.maxFileChars).length
+ const n = fileSection(f, budget.maxFileChars, budget.coverage).length
if (cur.length && size + n > budget.batchChars) {
batches.push(cur)
</file context>
| /** Head-side lines of context kept around each changed line in a file excerpt. */ | ||
| const EXCERPT_CONTEXT_LINES = 20 | ||
| /** Characters of a file's diff section under coverage (hunks beyond it are listed as omitted). */ | ||
| const COVERAGE_DIFF_CHARS = 16_000 |
There was a problem hiding this comment.
WARNING: Coverage mode can build a single ~660k-char call with no per-call cap
With ALTIMATE_REVIEW_AI_COVERAGE=1 the diff section can be COVERAGE_DIFF_CHARS (16,000) per file plus up to 6,000 of current file, while the new quotas allow 30 files (12 models + 8 macros + 4 tests + 6 yml) and batchChars still defaults to Infinity (line 57), so batchAiFiles keeps them all in one batch. One call can reach ~30 x 22k = 660k chars (~165k tokens) versus <= 20 x 12k = 240k before. That overruns a 128k-token model's context, reviewBatch throws, every batch fails and the lane reports error with no findings - on exactly the large PRs this mode targets. Consider defaulting batchChars to a bounded value when coverage is on, or scaling the per-file budgets against a per-call total.
| const BUCKET_ORDER = [...new Set(KIND_ORDER.map((k) => BUCKET_OF[k]))] | ||
|
|
||
| /** Added and removed lines of a unified diff (file headers before the first hunk excluded). */ | ||
| export function changedLineCount(diff: string | undefined): number { |
There was a problem hiding this comment.
SUGGESTION: Duplicates the existing countChangedLines
diff-filter.ts:329 already exports countChangedLines (imported from the same module for walkDiffLines) for this exact job, and it correctly ignores +++/--- file headers. This copy counts +++ b/... and --- a/... as changed lines whenever the diff has no @@ hunk (first === -1, so Math.max(first, 0) === 0), which inflates the most-changed ordering used by selectAiFilesByQuota for header-less diffs. Reuse countChangedLines(f.diff) here instead.
| if (!kept.length) return { text: truncate(content, maxChars), truncated: true } | ||
| // Fill what is left with the file head, before the first window. | ||
| let head = 0 | ||
| while (head + 1 < kept[0][0] && used + size(1, head + 1) + MARKER <= maxChars) head++ |
There was a problem hiding this comment.
SUGGESTION: Quadratic head fill
size(1, head + 1) re-slices and re-sums lines 1..head+1 on every iteration, so extending the head over H lines costs O(H^2) with a fresh array each step (H can approach maxChars for files with many short or blank lines, and this runs once per file under coverage). Accumulate the running length instead, e.g. keep headChars and add lines[head].length + 1 per step, then test used + headChars + MARKER <= maxChars.
| return text.slice(0, maxChars - 1) + "…" | ||
| } | ||
|
|
||
| const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot"]) |
There was a problem hiding this comment.
SUGGESTION: Changed seeds never get a context card
MODEL_KINDS omits "seed" and seeds are not in DECLARING_KINDS, so cardsFor returns [] for every seed file. But runner.modelCard explicitly resolves seed. nodes (runner.ts:543) and ModelCard is documented as facts about "one model, snapshot or seed", so that branch is unreachable through this feature. Either add "seed" to MODEL_KINDS (a changed seed is a legitimate review target) or drop the seed lookup from modelCard.
Code Review SummaryStatus: 3 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)SUGGESTION
Files Reviewed (9 files)
Note: the per-call size bound raised earlier is now fixed ( Fix these issues in Kilo Cloud Previous Review Summary (commit b07b80d)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit b07b80d)Status: 4 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Note: the PR description omits the repository's required "Cost side effects" section (the automated issue-compliance check already flagged this). The lane is opt-in and bounded by file quotas and character budgets, but it does raise per-review token spend ~1.25x when enabled, so that section should be filled in. Fix these issues in Kilo Cloud Files Reviewed (9 files)
Reviewed by deepseek-v4.1-flash-1017 · Input: 173.1K · Output: 11.4K · Cached: 578.9K Review guidance: REVIEW.md from base branch |
hkaltai
left a comment
There was a problem hiding this comment.
Verdict: approve, with minor follow-ups. Panel: Claude, Codex and lead reviewer, covering all 9 changed files.
The core promise holds: with the flags unset the prompt is byte-identical to the base (confirmed independently by two reviewers against the snapshot), and the 31 tests in review-ai-inputs.test.ts pass. All findings concern behavior behind the opt-in flags and advisory context quality, so none is CRITICAL or MAJOR.
Counts: CRITICAL 0, MAJOR 0, MINOR 8 (inline), NIT 10.
Most worthwhile follow-ups: bound the coverage-mode call size (batchChars default), make compiled-SQL windowing robust when added lines are Jinja-only, and resolve model/macro cards by project and path rather than by first name match.
Nits
ai-review.ts:51numrequiresn > 0:ALTIMATE_REVIEW_AI_QUOTA_TEST=0silently becomes 4, so a bucket cannot be disabled. Quoted:return v && Number.isFinite(n) && n > 0 ? n : d. Fix: separate parser allowing 0 for quotas.excerptFilewindow excerpt (ai-review.ts:327area): when the first context window exceeds the budget, the beginning is kept and the changed anchor can be dropped (350-char lines, change at line 100, 6,000 budget). Fix: reserve space for changed lines first, then expand context.runner.ts:640-649: a macro name containing__splits at the first__, so an unrelatedget_x__yis listed as a variant ofy. Fix: only treatdefault/adapter prefixes or dispatchers as variants.ai-review.ts:240: preamble longer than budget emits an empty "(hunk truncated)" but reportshunksSent: 1. Fix: record 0 whenmaxChars - used <= 0.ai-review.ts:313:excerptFilecan exceedmaxCharsby about 40 chars (trailing marker not reserved when no head window). Fix: reserve it.ai-context-card.tsMODEL_KINDSlacksseedalthough theModelCarddoc says "model, snapshot or seed". Fix: add seed or fix the doc.- Comment at
ai-review.ts:563says cards stop once the call holds 6,000 chars, but a too-large card is skipped and later smaller ones are still added. Reword. planAiInputdoc says "Pure apart from readingenv", butenvis a parameter.- Naming:
AiReviewFile.card(string) vsAiInputRecord.card(boolean);fileCharsinrenderFileis a budget (fileBudget);CoverageArg = boolean | AiCoverageRender | undefinedis a test-only tri-state; localconst MARKER = 40could be module-levelMARKER_RESERVE. - Missing tests: dry-run+replay+verify, duplicate model names across packages, macro basename/path collisions, YAML cards via the real runner (not a mocked
graphParents), Jinja-only changes on compiled SQL, bounded size under coverage, and rendering of quota-unsent files in the "Not reviewed" block.
| @@ -263,13 +646,20 @@ export async function runAiReview(input: AiReviewInput): Promise<AiReviewResult> | |||
| const replay = aiReplayPath() | |||
| if (replay) return runAiReplay(input, replay) | |||
There was a problem hiding this comment.
[MINOR] Replay returns before the dry-run check
if (replay) return runAiReplay(input, replay)
With ALTIMATE_REVIEW_AI_DRYRUN=1, ALTIMATE_REVIEW_AI_REPLAY=<envelope> and ALTIMATE_REVIEW_AI_VERIFY=1, replay returns before dry-run is evaluated, so a stored envelope with AI findings still reaches the verifier (a model call) and no ai.input is recorded. Replay is an eval-only path, hence MINOR, but it breaks the no-model-call promise of dry-run. Fix: evaluate dry-run (and plan the input) before dispatching replay, and add a regression test. (Codex)
| * lines (8+ non-space chars, no Jinja) are found by text, and "unlocated" otherwise. | ||
| */ | ||
| function locateAddedLines(lines: string[], rows: Array<{ line: number; text: string; kind: string }>): number[] | "unlocated" | undefined { | ||
| const eligible = rows.filter((r) => r.kind === "+" && !JINJA.test(r.text) && r.text.replace(/\s/g, "").length >= 8) |
There was a problem hiding this comment.
[MINOR] Compiled-SQL windows fall back to raw line numbers when no added line is eligible
const eligible = rows.filter((r) => r.kind === "+" && !JINJA.test(r.text) && r.text.replace(/\s/g, "").length >= 8)
When every added line is Jinja or under 8 non-space chars ({{ ref(..) }} swaps, {% if is_incremental() %}, , id), eligible is empty and the function returns undefined, which excerptFile treats as "raw line numbers match". On compiled SQL where macro expansion shifted lines, windows are then centred on the wrong lines and the unapproximated marker is not emitted. The diff itself is still sent, so this degrades excerpt quality only (opt-in flag). Fix: let the caller say whether f.sql is compiled, or also check context rows; return "unlocated" if nothing can be verified. Add a Jinja-only test on compiled SQL. (Claude)
| maxFiles: num(env.ALTIMATE_REVIEW_AI_MAX_FILES, MAX_FILES), | ||
| maxFileChars: num(env.ALTIMATE_REVIEW_AI_MAX_FILE_CHARS, MAX_DIFF_CHARS), | ||
| batchChars: num(env.ALTIMATE_REVIEW_AI_BATCH_CHARS, Infinity), | ||
| coverage: |
There was a problem hiding this comment.
[MINOR] Coverage mode enlarges the single call with no bound
batchChars: num(env.ALTIMATE_REVIEW_AI_BATCH_CHARS, Infinity),
Coverage raises the file cap to 30 and adds a per-diff budget, while batchChars stays Infinity. A 40-large-model PR was measured at about 242k chars by default vs about 658k with coverage, which can exceed context or the 300 s timeout and return no AI findings at all. The unbounded single call is pre-existing, but coverage makes it markedly worse. Fix: default batchChars to a finite value when coverage is on, or cap total chars and push overflow into unsent. (Claude)
| const nodes = Object.entries<any>(raw.nodes ?? {}) | ||
| const isModel = (id: string) => id.startsWith("model.") | ||
| const hit = | ||
| nodes.find(([id, n]) => isModel(id) && n?.name === name) ?? |
There was a problem hiding this comment.
[MINOR] modelCard picks the first same-named model across packages
nodes.find(([id, n]) => isModel(id) && n?.name === name) ??
A dependency package model with the same name listed earlier in the manifest supplies materialization, keys, columns and graph facts for the changed project model (reproduced by both external reviewers). The card is presented as authoritative. Fix: prefer metadata.project_name (as macroCard does) and/or match by path or unique id. (Codex, Claude)
| const byName = macros.filter(([, m]) => m?.name === name) | ||
| const path = cardOpts?.path | ||
| const hit = | ||
| byName.find(([, m]) => m.package_name === project) ?? |
There was a problem hiding this comment.
[MINOR] macroCard resolves by file basename before path, and not within the project
byName.find(([, m]) => m.package_name === project) ??
byName[0] ??
The lookup name is the file basename, not necessarily the macro defined in it. macros/helpers.sql defining foo can get the card of an unrelated helpers macro, or a package macro (e.g. dbt_utils.date_spine) with its callers and variants. The path fallback is also not restricted to the project package. Fix: match by path within the project first, then name within the project, then any package. (Codex, Claude)
| return macro ? [{ name: macro.name, macro }] : [] | ||
| } | ||
| if (!DECLARING_KINDS.has(kind)) return [] | ||
| const parents = (await runner.graphParents?.([f.path])) ?? [] |
There was a problem hiding this comment.
[MINOR] YAML declaration cards depend on test-node parents, not patch_path
const parents = (await runner.graphParents?.([f.path])) ?? []
graphParents links a YAML file to nodes via tests, while models are tied to their YAML by patch_path. A schema YAML with no generic tests gets no card even though the manifest has its models' config; removing the last test removes the card. The orchestrator test mocks graphParents, which hides this. Fix: resolve declarations through patch_path. (Codex, Claude)
| : fileChars < maxChars | ||
| ? truncate(f.sql, fileChars) | ||
| : excerptFile(f.sql, f.diff, maxChars).text | ||
| if (f.sql.length > fileChars) stats = { ...stats, truncated: true } |
There was a problem hiding this comment.
[MINOR] Ledger reports truncated:false when MAX_FILE_CHARS < 2000
if (f.sql.length > fileChars) stats = { ...stats, truncated: true }
With ALTIMATE_REVIEW_AI_MAX_FILE_CHARS=1000 and 1,500-char SQL, fileChars is widened to 2000 but excerptFile(..., maxChars) still truncates, so the record says truncated:false while the prompt contains (truncated). Fix: compare against the budget actually applied (Math.min(maxChars, ...)). (Claude)
| ...(manifestMissing && !anyManifest | ||
| ? modelFiles.map((f) => ({ path: f.path, reason: "no dbt manifest: lineage, config and graph checks skipped" })) | ||
| : []), | ||
| ...aiUnsent.map((p) => ({ path: p, reason: "not sent to the AI (file quota)" })), |
There was a problem hiding this comment.
[MINOR] Quota-omitted files listed under Not reviewed although engine lanes checked them
...aiUnsent.map((p) => ({ path: p, reason: "not sent to the AI (file quota)" })),
The ledger means "every changed file the review did not look at", but these files went through the deterministic lanes; on a large PR dozens of engine-checked files show up in that block. The default maxFiles cap records nothing, so behaviour is inconsistent. Fix: report as a separate note or field. (Claude)
…ext card
All switches are off by default; with them unset the AI prompt is
byte-identical to before (pinned by a snapshot captured before the change).
- `ALTIMATE_REVIEW_AI_COVERAGE=1`: per-kind file quotas (models 12, macros 8,
tests 4, yml/config 6; `ALTIMATE_REVIEW_AI_QUOTA_{MODEL,MACRO,TEST,YML}`),
most-changed files first, unused quota redistributed up to the total.
Over-budget diffs keep whole hunks and name omitted hunks by line range;
over-budget files send +/-20-line windows around changed lines plus the file
head, with `… (lines a–b omitted)` markers. Files left out go to
`scope.notReviewed` as "not sent to the AI (file quota)".
- `ai.input` on the envelope records per file what was sent (chars, truncated,
hunks sent/total, card) under coverage, context cards or dry run.
- `ALTIMATE_REVIEW_AI_DRYRUN=1`: the lane plans its input, records `ai.input`
and returns `skipped` without calling a model. `planAiInput` is the pure
planner `runAiReview` now sends from.
- `ALTIMATE_REVIEW_AI_CONTEXT_CARD=1` (needs a manifest): a `dbt context (from
manifest):` block under each file header, from new non-throwing runner
methods `modelCard` and `macroCard`; yml and singular tests get the cards of
the models they test. At most 600 chars per file and 6,000 per call.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… SQL Under `ALTIMATE_REVIEW_AI_COVERAGE=1`, a model's current file is often dbt-compiled SQL, whose line numbers drift from the raw diff once Jinja expands. When the added lines are not at their raw line numbers, `excerptFile` now finds each added line (8+ non-space chars, no Jinja) by its trimmed text, scanning forward, and centres the windows there. If fewer than half are found it keeps the head cut with a marker saying windows would be approximate. Raw content keeps raw line numbers; the default prompt is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…header Under `ALTIMATE_REVIEW_AI_COVERAGE=1` only (the default prompt is unchanged): - The diff section gets its own budget, `ALTIMATE_REVIEW_AI_COVERAGE_DIFF_CHARS` (default 16,000), instead of the 6,000-char file budget. Hunks stay whole and in order; only hunks past that budget are listed as omitted. Multi-hunk macro diffs that sent 7 of 20 hunks now go whole. - When a long diff (over the file budget) is sent with every hunk, the current-file section shrinks to a 2,000-char head; otherwise the file keeps its +/-20-line windows within the file budget. - Fix: an excerpted diff replaced the whole file section (header, card and current file) because the excerpt's `text` overwrote the section text. `ai.input` chars, hunksSent and hunksTotal now describe what is really sent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- `aiInputBudget`: with `ALTIMATE_REVIEW_AI_COVERAGE=1`, `batchChars` now defaults to 120k chars instead of `Infinity`, so 30 quota files with 16k diffs no longer build one ~660k-char call. `ALTIMATE_REVIEW_AI_BATCH_CHARS` still overrides it; with coverage off the default stays `Infinity` (prompt snapshot unchanged). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
a6823f9 to
936b24a
Compare
b07b80d to
bf6e861
Compare
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
Issue for this PR
Part of #1792
Type of change
What does this PR do?
Stacked on #1770. Gives the AI lane better input on large PRs. All of it is opt-in: with the flags unset, prompts are byte-identical to before (snapshot test).
ALTIMATE_REVIEW_AI_COVERAGE=1:ALTIMATE_REVIEW_AI_CONTEXT_CARD=1: a short dbt context card per file from the manifest. For models: materialization, unique key, parents, children, exposures, declared columns with their tests, siblings. For macros: callers and dispatch variants.ALTIMATE_REVIEW_AI_DRYRUN=1: builds the AI input and records its size without calling the model, for cost estimates.Results (blind Claude judge)
How did you verify your code works?
bun test test/altimate/review: 725 pass.bunx tsc --noEmit: clean.test/altimate/review-ai-inputs.test.ts: quotas, redistribution, hunk windows, ledger, card rendering, default prompt snapshot.Cost side effects
None by default; every option is opt-in and the default prompts are byte-identical. With
ALTIMATE_REVIEW_AI_COVERAGE=1andALTIMATE_REVIEW_AI_CONTEXT_CARD=1the AI lane's input grows with the PR's changed files (bounded by the per-kind quotas), measured at 1.25-1.32x AI-lane tokens; the number of AI calls is unchanged. The context card is built from the local manifest (no warehouse queries).ALTIMATE_REVIEW_AI_DRYRUN=1makes no model call and records the input size for cost estimates.Screenshots / recordings
Not a UI change.
Checklist
🤖 Generated with Claude Code
Summary by cubic
Adds opt-in flags that give the AI reviewer better input on large PRs while leaving the default prompt byte-identical (pinned by a snapshot test).
What changed
ALTIMATE_REVIEW_AI_COVERAGE=1sends per-kind file quotas (models 12, macros 8, tests 4, yml/config 6), most-changed files first, with unused quota redistributed; quotas are overridable viaALTIMATE_REVIEW_AI_QUOTA_*.ALTIMATE_REVIEW_AI_COVERAGE_DIFF_CHARS) so hunks stay whole; over-budget files send ±20-line windows around changed lines with omission markers, and batches are capped at 120,000 chars per call.scope.notReviewedas "not sent to the AI (file quota)".ALTIMATE_REVIEW_AI_CONTEXT_CARD=1adds a short dbt context card per file from the manifest: materialization and keys, parents/children/exposures, declared columns with tests, and macro callers and variants; yml and singular tests get the card of the model they test.ALTIMATE_REVIEW_AI_DRYRUN=1plans the AI input and records its size without calling the model, for cost estimates.ai.inputmetadata (chars, truncated, hunks sent/total, card) is recorded on the verdict envelope under these flags.In a blind evaluation across 25 repos never used for tuning, coverage plus the context card caught 47 vs 43 defects with fewer wrong findings at about 1.25× the tokens; the gain is within noise, so the flags stay opt-in.
Migration
Written for commit bf6e861. Summary will update on new commits.