Repository navigation
feat(review): opt-in AI input coverage, hunk excerpts and a dbt manifest context card - #1771
RaghwendraSingh wants to merge 5 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
There was a problem hiding this comment.
9 issues found across 9 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/test/altimate/fixtures/review-ai-prompt-input.ts">
<violation number="1" location="packages/opencode/test/altimate/fixtures/review-ai-prompt-input.ts:12">
P3: The hunk header counts in `bigDiff` don't match the hunk body: the old side has 3 lines and the new side has `size + 2`, but the header says `size + 2` and `size + 4`. Use the real counts so this fixture is a valid unified diff for any future hunk-window test.</violation>
</file>
<file name="packages/opencode/src/altimate/review/ai-review.ts">
<violation number="1" location="packages/opencode/src/altimate/review/ai-review.ts:391">
P2: A diff between `coverage.diffChars` and `maxChars` bypasses `excerptDiff`, so a lower `ALTIMATE_REVIEW_AI_COVERAGE_DIFF_CHARS` override is ignored. Compare against the effective limit, such as `Math.min(maxChars, coverage.diffChars)`.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/ai-review.ts:421">
P3: Batch sizing counts every card, but `buildBatchMessage` omits cards after the 6,000-character per-call cap. Card-heavy inputs can therefore split into unnecessary model calls; apply the same cumulative card cap when estimating batches.</violation>
<violation number="3" location="packages/opencode/src/altimate/review/ai-review.ts:649">
P2: `planAiInput` now runs outside the `try` in `runAiReview`, so an exception while rendering the prompt escapes the lane. The doc comment promises this function never throws. Move `const plan = planAiInput(input)` inside the `try`, or wrap the planning in its own guard that returns an `error` status.</violation>
</file>
<file name="packages/opencode/src/altimate/review/runner.ts">
<violation number="1" location="packages/opencode/src/altimate/review/runner.ts:570">
P2: Column tests disappear from the card for manifests without `attached_node`; fall back to `depends_on.nodes` and `test_metadata.kwargs.column_name` when those fields are absent.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/runner.ts:603">
P3: `text()` keeps only `partition_by.field` and drops `granularity`. Include the granularity in the rendered value, since the card is meant to show partitioning facts the diff does not.</violation>
<violation number="3" location="packages/opencode/src/altimate/review/runner.ts:639">
P2: These `.slice()` caps run before rendering, so the card's `+N more` marker can never show for callers, columns or siblings. A macro with 40 callers reads as having 10. Return the full lists and let `renderContextCard`'s `list()` cap them, or keep the true totals.</violation>
</file>
<file name="packages/opencode/src/altimate/review/ai-context-card.ts">
<violation number="1" location="packages/opencode/src/altimate/review/ai-context-card.ts:95">
P2: Seed files never get a context card. `MODEL_KINDS` leaves out `seed`, although `ModelCard` and `modelCard()` are documented to cover seeds. Add `seed` to the set, and drop the `kind === "model"` filter in the yml/test branch if seed parents should be carded there too.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/ai-context-card.ts:123">
P2: Schema and test changes attached to seed or snapshot nodes lose their related context card because this filter accepts only models. Include the resource kinds supported by `modelCard`.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| const replay = aiReplayPath() | ||
| if (replay) return runAiReplay(input, replay) | ||
| const startedAt = Date.now() | ||
| const plan = planAiInput(input) |
There was a problem hiding this comment.
P2: planAiInput now runs outside the try in runAiReview, so an exception while rendering the prompt escapes the lane. The doc comment promises this function never throws. Move const plan = planAiInput(input) inside the try, or wrap the planning in its own guard that returns an error status.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 649:
<comment>`planAiInput` now runs outside the `try` in `runAiReview`, so an exception while rendering the prompt escapes the lane. The doc comment promises this function never throws. Move `const plan = planAiInput(input)` inside the `try`, or wrap the planning in its own guard that returns an `error` status.</comment>
<file context>
@@ -263,13 +646,20 @@ export async function runAiReview(input: AiReviewInput): Promise<AiReviewResult>
const replay = aiReplayPath()
if (replay) return runAiReplay(input, replay)
const startedAt = Date.now()
+ const plan = planAiInput(input)
+ const { files, budget } = plan
+ const dryRun = aiDryRunEnabled()
</file context>
There was a problem hiding this comment.
Not changing: the only caller wraps the lane in try/catch and records state: "error" (orchestrate.ts ~2498), so nothing escapes. Only the "never throws" doc wording is loose.
| .filter(([id, n]) => id.startsWith("model.") && asArray<string>(n?.depends_on?.macros).includes(uid)) | ||
| .map(([, n]) => String(n.name)) | ||
| .sort((a, b) => Number(changed.has(b)) - Number(changed.has(a))) | ||
| .slice(0, 10) |
There was a problem hiding this comment.
P2: These .slice() caps run before rendering, so the card's +N more marker can never show for callers, columns or siblings. A macro with 40 callers reads as having 10. Return the full lists and let renderContextCard's list() cap them, or keep the true totals.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/runner.ts, line 639:
<comment>These `.slice()` caps run before rendering, so the card's `+N more` marker can never show for callers, columns or siblings. A macro with 40 callers reads as having 10. Return the full lists and let `renderContextCard`'s `list()` cap them, or keep the true totals.</comment>
<file context>
@@ -531,6 +532,129 @@ export function createDispatcherRunner(opts: DispatcherRunnerOptions): ReviewRun
+ .filter(([id, n]) => id.startsWith("model.") && asArray<string>(n?.depends_on?.macros).includes(uid))
+ .map(([, n]) => String(n.name))
+ .sort((a, b) => Number(changed.has(b)) - Number(changed.has(a)))
+ .slice(0, 10)
+ const macroName = String(macro.name)
+ const sep = macroName.indexOf("__")
</file context>
There was a problem hiding this comment.
Follow-up: low priority, tracked. We'll pass the totals so the card can say how many it left out.
| return text.slice(0, maxChars - 1) + "…" | ||
| } | ||
|
|
||
| const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot"]) |
There was a problem hiding this comment.
P2: Seed files never get a context card. MODEL_KINDS leaves out seed, although ModelCard and modelCard() are documented to cover seeds. Add seed to the set, and drop the kind === "model" filter in the yml/test branch if seed parents should be carded there too.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-context-card.ts, line 95:
<comment>Seed files never get a context card. `MODEL_KINDS` leaves out `seed`, although `ModelCard` and `modelCard()` are documented to cover seeds. Add `seed` to the set, and drop the `kind === "model"` filter in the yml/test branch if seed parents should be carded there too.</comment>
<file context>
@@ -0,0 +1,134 @@
+ return text.slice(0, maxChars - 1) + "…"
+}
+
+const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot"])
+const DECLARING_KINDS = new Set(["schema_yml", "source_yml", "test"])
+
</file context>
| const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot"]) | |
| const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot", "seed"]) |
There was a problem hiding this comment.
Follow-up: low priority, tracked. We'll either add seed or drop the unreachable seed lookup.
| const out: string[] = [] | ||
| for (let h = 0; h < hunks; h++) { | ||
| const start = 10 + h * 100 | ||
| out.push(`@@ -${start},${size + 2} +${start},${size + 4} @@`) |
There was a problem hiding this comment.
P3: The hunk header counts in bigDiff don't match the hunk body: the old side has 3 lines and the new side has size + 2, but the header says size + 2 and size + 4. Use the real counts so this fixture is a valid unified diff for any future hunk-window test.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/fixtures/review-ai-prompt-input.ts, line 12:
<comment>The hunk header counts in `bigDiff` don't match the hunk body: the old side has 3 lines and the new side has `size + 2`, but the header says `size + 2` and `size + 4`. Use the real counts so this fixture is a valid unified diff for any future hunk-window test.</comment>
<file context>
@@ -0,0 +1,64 @@
+ const out: string[] = []
+ for (let h = 0; h < hunks; h++) {
+ const start = 10 + h * 100
+ out.push(`@@ -${start},${size + 2} +${start},${size + 4} @@`)
+ out.push(` select ${h} as ctx_before`)
+ for (let i = 0; i < size; i++) out.push(`+ , coalesce(col_${h}_${i}, 'unknown_value_${i}') as renamed_column_${h}_${i}`)
</file context>
| out.push(`@@ -${start},${size + 2} +${start},${size + 4} @@`) | |
| out.push(`@@ -${start},3 +${start},${size + 2} @@`) |
There was a problem hiding this comment.
Not changing: the parser doesn't use the counts, and the fixture feeds a byte-identical snapshot that has to stay as it is.
| materialized: text(config.materialized), | ||
| uniqueKey: text(config.unique_key), | ||
| incrementalStrategy: text(config.incremental_strategy), | ||
| partitionBy: text(config.partition_by), |
There was a problem hiding this comment.
P3: text() keeps only partition_by.field and drops granularity. Include the granularity in the rendered value, since the card is meant to show partitioning facts the diff does not.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/runner.ts, line 603:
<comment>`text()` keeps only `partition_by.field` and drops `granularity`. Include the granularity in the rendered value, since the card is meant to show partitioning facts the diff does not.</comment>
<file context>
@@ -531,6 +532,129 @@ export function createDispatcherRunner(opts: DispatcherRunnerOptions): ReviewRun
+ materialized: text(config.materialized),
+ uniqueKey: text(config.unique_key),
+ incrementalStrategy: text(config.incremental_strategy),
+ partitionBy: text(config.partition_by),
+ clusterBy: text(config.cluster_by),
+ contractEnforced: config.contract?.enforced === true || node.contract?.enforced === true,
</file context>
There was a problem hiding this comment.
Not changing for now: the card shows the partition column. Granularity is a nice-to-have, and we may add it with the next card revision.
| let size = 0 | ||
| for (const f of files) { | ||
| const n = fileSection(f, budget.maxFileChars).length | ||
| const n = fileSection(f, budget.maxFileChars, budget.coverage).length |
There was a problem hiding this comment.
P3: Batch sizing counts every card, but buildBatchMessage omits cards after the 6,000-character per-call cap. Card-heavy inputs can therefore split into unnecessary model calls; apply the same cumulative card cap when estimating batches.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 421:
<comment>Batch sizing counts every card, but `buildBatchMessage` omits cards after the 6,000-character per-call cap. Card-heavy inputs can therefore split into unnecessary model calls; apply the same cumulative card cap when estimating batches.</comment>
<file context>
@@ -105,7 +418,7 @@ export function batchAiFiles(files: AiReviewFile[], budget: AiInputBudget): AiRe
let size = 0
for (const f of files) {
- const n = fileSection(f, budget.maxFileChars).length
+ const n = fileSection(f, budget.maxFileChars, budget.coverage).length
if (cur.length && size + n > budget.batchChars) {
batches.push(cur)
</file context>
There was a problem hiding this comment.
Follow-up: low priority, tracked. Sizing over-counts cards, so batches come out smaller than needed, never larger.
| /** Head-side lines of context kept around each changed line in a file excerpt. */ | ||
| const EXCERPT_CONTEXT_LINES = 20 | ||
| /** Characters of a file's diff section under coverage (hunks beyond it are listed as omitted). */ | ||
| const COVERAGE_DIFF_CHARS = 16_000 |
There was a problem hiding this comment.
WARNING: Coverage mode can build a single ~660k-char call with no per-call cap
With ALTIMATE_REVIEW_AI_COVERAGE=1 the diff section can be COVERAGE_DIFF_CHARS (16,000) per file plus up to 6,000 of current file, while the new quotas allow 30 files (12 models + 8 macros + 4 tests + 6 yml) and batchChars still defaults to Infinity (line 57), so batchAiFiles keeps them all in one batch. One call can reach ~30 x 22k = 660k chars (~165k tokens) versus <= 20 x 12k = 240k before. That overruns a 128k-token model's context, reviewBatch throws, every batch fails and the lane reports error with no findings - on exactly the large PRs this mode targets. Consider defaulting batchChars to a bounded value when coverage is on, or scaling the per-file budgets against a per-call total.
There was a problem hiding this comment.
Fixed in bf6e861: with ALTIMATE_REVIEW_AI_COVERAGE=1, batchChars now defaults to 120k chars (about 30k tokens), so 30 quota files split across calls instead of one ~660k-char call. ALTIMATE_REVIEW_AI_BATCH_CHARS still overrides it, and with coverage off the default stays Infinity (the prompt snapshot test is unchanged).
| const BUCKET_ORDER = [...new Set(KIND_ORDER.map((k) => BUCKET_OF[k]))] | ||
|
|
||
| /** Added and removed lines of a unified diff (file headers before the first hunk excluded). */ | ||
| export function changedLineCount(diff: string | undefined): number { |
There was a problem hiding this comment.
SUGGESTION: Duplicates the existing countChangedLines
diff-filter.ts:329 already exports countChangedLines (imported from the same module for walkDiffLines) for this exact job, and it correctly ignores +++/--- file headers. This copy counts +++ b/... and --- a/... as changed lines whenever the diff has no @@ hunk (first === -1, so Math.max(first, 0) === 0), which inflates the most-changed ordering used by selectAiFilesByQuota for header-less diffs. Reuse countChangedLines(f.diff) here instead.
There was a problem hiding this comment.
Fair point; we'll reuse countChangedLines in a follow-up cleanup. Behaviour is the same for git diffs, which always carry @@ headers.
| if (!kept.length) return { text: truncate(content, maxChars), truncated: true } | ||
| // Fill what is left with the file head, before the first window. | ||
| let head = 0 | ||
| while (head + 1 < kept[0][0] && used + size(1, head + 1) + MARKER <= maxChars) head++ |
There was a problem hiding this comment.
SUGGESTION: Quadratic head fill
size(1, head + 1) re-slices and re-sums lines 1..head+1 on every iteration, so extending the head over H lines costs O(H^2) with a fresh array each step (H can approach maxChars for files with many short or blank lines, and this runs once per file under coverage). Accumulate the running length instead, e.g. keep headChars and add lines[head].length + 1 per step, then test used + headChars + MARKER <= maxChars.
There was a problem hiding this comment.
Not changing now: it's bounded by the per-file char budget (a few hundred lines). We'll switch it to a running sum if profiling ever shows it.
| return text.slice(0, maxChars - 1) + "…" | ||
| } | ||
|
|
||
| const MODEL_KINDS = new Set(["model_sql", "python_model", "snapshot"]) |
There was a problem hiding this comment.
SUGGESTION: Changed seeds never get a context card
MODEL_KINDS omits "seed" and seeds are not in DECLARING_KINDS, so cardsFor returns [] for every seed file. But runner.modelCard explicitly resolves seed. nodes (runner.ts:543) and ModelCard is documented as facts about "one model, snapshot or seed", so that branch is unreachable through this feature. Either add "seed" to MODEL_KINDS (a changed seed is a legitimate review target) or drop the seed lookup from modelCard.
There was a problem hiding this comment.
Same root cause as r4626982915; tracked there.
Code Review SummaryStatus: 8 Issues Found | Recommendation: Address before merge The branch was rebased onto a newer base; the PR's own files are unchanged since the last review at Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (10 files)
Fix these issues in Kilo Cloud Previous Review Summaries (3 snapshots, latest commit c99cc33)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit c99cc33)Status: 7 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (10 files)
The previous percent-escaped Fix these issues in Kilo Cloud Previous review (commit bf6e861)Status: 3 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)SUGGESTION
Files Reviewed (9 files)
Note: the per-call size bound raised earlier is now fixed ( Fix these issues in Kilo Cloud Previous review (commit b07b80d)Status: 4 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Note: the PR description omits the repository's required "Cost side effects" section (the automated issue-compliance check already flagged this). The lane is opt-in and bounded by file quotas and character budgets, but it does raise per-review token spend ~1.25x when enabled, so that section should be filled in. Fix these issues in Kilo Cloud Files Reviewed (9 files)
Reviewed by deepseek-v4.1-flash-1017 · Input: 166.9K · Output: 30.2K · Cached: 2.7M Review guidance: REVIEW.md from base branch |
hkaltai
left a comment
There was a problem hiding this comment.
Verdict: approve, with minor follow-ups. Panel: Claude, Codex and lead reviewer, covering all 9 changed files.
The core promise holds: with the flags unset the prompt is byte-identical to the base (confirmed independently by two reviewers against the snapshot), and the 31 tests in review-ai-inputs.test.ts pass. All findings concern behavior behind the opt-in flags and advisory context quality, so none is CRITICAL or MAJOR.
Counts: CRITICAL 0, MAJOR 0, MINOR 8 (inline), NIT 10.
Most worthwhile follow-ups: bound the coverage-mode call size (batchChars default), make compiled-SQL windowing robust when added lines are Jinja-only, and resolve model/macro cards by project and path rather than by first name match.
Nits
ai-review.ts:51numrequiresn > 0:ALTIMATE_REVIEW_AI_QUOTA_TEST=0silently becomes 4, so a bucket cannot be disabled. Quoted:return v && Number.isFinite(n) && n > 0 ? n : d. Fix: separate parser allowing 0 for quotas.excerptFilewindow excerpt (ai-review.ts:327area): when the first context window exceeds the budget, the beginning is kept and the changed anchor can be dropped (350-char lines, change at line 100, 6,000 budget). Fix: reserve space for changed lines first, then expand context.runner.ts:640-649: a macro name containing__splits at the first__, so an unrelatedget_x__yis listed as a variant ofy. Fix: only treatdefault/adapter prefixes or dispatchers as variants.ai-review.ts:240: preamble longer than budget emits an empty "(hunk truncated)" but reportshunksSent: 1. Fix: record 0 whenmaxChars - used <= 0.ai-review.ts:313:excerptFilecan exceedmaxCharsby about 40 chars (trailing marker not reserved when no head window). Fix: reserve it.ai-context-card.tsMODEL_KINDSlacksseedalthough theModelCarddoc says "model, snapshot or seed". Fix: add seed or fix the doc.- Comment at
ai-review.ts:563says cards stop once the call holds 6,000 chars, but a too-large card is skipped and later smaller ones are still added. Reword. planAiInputdoc says "Pure apart from readingenv", butenvis a parameter.- Naming:
AiReviewFile.card(string) vsAiInputRecord.card(boolean);fileCharsinrenderFileis a budget (fileBudget);CoverageArg = boolean | AiCoverageRender | undefinedis a test-only tri-state; localconst MARKER = 40could be module-levelMARKER_RESERVE. - Missing tests: dry-run+replay+verify, duplicate model names across packages, macro basename/path collisions, YAML cards via the real runner (not a mocked
graphParents), Jinja-only changes on compiled SQL, bounded size under coverage, and rendering of quota-unsent files in the "Not reviewed" block.
| @@ -263,13 +646,20 @@ export async function runAiReview(input: AiReviewInput): Promise<AiReviewResult> | |||
| const replay = aiReplayPath() | |||
| if (replay) return runAiReplay(input, replay) | |||
There was a problem hiding this comment.
[MINOR] Replay returns before the dry-run check
if (replay) return runAiReplay(input, replay)
With ALTIMATE_REVIEW_AI_DRYRUN=1, ALTIMATE_REVIEW_AI_REPLAY=<envelope> and ALTIMATE_REVIEW_AI_VERIFY=1, replay returns before dry-run is evaluated, so a stored envelope with AI findings still reaches the verifier (a model call) and no ai.input is recorded. Replay is an eval-only path, hence MINOR, but it breaks the no-model-call promise of dry-run. Fix: evaluate dry-run (and plan the input) before dispatching replay, and add a regression test. (Codex)
There was a problem hiding this comment.
Thanks, good catch. In c99cc33 ("dry run before replay, ...") runAiReview plans the input and checks dry run before dispatching replay, so ALTIMATE_REVIEW_AI_DRYRUN=1 with a replay envelope and the verifier switch returns skipped with ai.input recorded and makes no model call. A regression test covers the three switches together.
There was a problem hiding this comment.
✅ Fixed — ai-review.ts:684-687 now returns the dry-run result before runAiReplay; test 'dry run is honoured before replay' covers DRYRUN+REPLAY+VERIFY. Suite: 42 pass.
| * lines (8+ non-space chars, no Jinja) are found by text, and "unlocated" otherwise. | ||
| */ | ||
| function locateAddedLines(lines: string[], rows: Array<{ line: number; text: string; kind: string }>): number[] | "unlocated" | undefined { | ||
| const eligible = rows.filter((r) => r.kind === "+" && !JINJA.test(r.text) && r.text.replace(/\s/g, "").length >= 8) |
There was a problem hiding this comment.
[MINOR] Compiled-SQL windows fall back to raw line numbers when no added line is eligible
const eligible = rows.filter((r) => r.kind === "+" && !JINJA.test(r.text) && r.text.replace(/\s/g, "").length >= 8)
When every added line is Jinja or under 8 non-space chars ({{ ref(..) }} swaps, {% if is_incremental() %}, , id), eligible is empty and the function returns undefined, which excerptFile treats as "raw line numbers match". On compiled SQL where macro expansion shifted lines, windows are then centred on the wrong lines and the unapproximated marker is not emitted. The diff itself is still sent, so this degrades excerpt quality only (opt-in flag). Fix: let the caller say whether f.sql is compiled, or also check context rows; return "unlocated" if nothing can be verified. Add a Jinja-only test on compiled SQL. (Claude)
There was a problem hiding this comment.
Agreed. In c99cc33, when no added line is eligible (Jinja-only or short changes), locateAddedLines checks the eligible context lines: raw line numbers are used only when those match, otherwise the result is "unlocated" and the head goes out with the "approximate" marker. There is a new test with a {{ dbt_utils.star(...) }} swap on compiled SQL, plus the matching raw-file case, which still gets its window.
There was a problem hiding this comment.
{% if is_incremental() %} around one added line) it still returns undefined and trusts raw line numbers on compiled SQL; see the new inline comment.
| maxFiles: num(env.ALTIMATE_REVIEW_AI_MAX_FILES, MAX_FILES), | ||
| maxFileChars: num(env.ALTIMATE_REVIEW_AI_MAX_FILE_CHARS, MAX_DIFF_CHARS), | ||
| batchChars: num(env.ALTIMATE_REVIEW_AI_BATCH_CHARS, Infinity), | ||
| coverage: |
There was a problem hiding this comment.
[MINOR] Coverage mode enlarges the single call with no bound
batchChars: num(env.ALTIMATE_REVIEW_AI_BATCH_CHARS, Infinity),
Coverage raises the file cap to 30 and adds a per-diff budget, while batchChars stays Infinity. A 40-large-model PR was measured at about 242k chars by default vs about 658k with coverage, which can exceed context or the 300 s timeout and return no AI findings at all. The unbounded single call is pre-existing, but coverage makes it markedly worse. Fix: default batchChars to a finite value when coverage is on, or cap total chars and push overflow into unsent. (Claude)
There was a problem hiding this comment.
This was fixed in a3c5f07 ("bound coverage per-call size"), which landed after the commit you reviewed. With coverage on, batchChars now defaults to 120k characters instead of Infinity, so a 30-file coverage run is split across calls. ALTIMATE_REVIEW_AI_BATCH_CHARS still overrides it, and the default without coverage is unchanged. The test "coverage bounds the per-call size by default" covers the 30 x 16k case.
There was a problem hiding this comment.
✅ Fixed — ai-review.ts:37,67: batchChars defaults to 120,000 under coverage (Infinity otherwise); test asserts the default and that 30 files split into several calls.
| const nodes = Object.entries<any>(raw.nodes ?? {}) | ||
| const isModel = (id: string) => id.startsWith("model.") | ||
| const hit = | ||
| nodes.find(([id, n]) => isModel(id) && n?.name === name) ?? |
There was a problem hiding this comment.
[MINOR] modelCard picks the first same-named model across packages
nodes.find(([id, n]) => isModel(id) && n?.name === name) ??
A dependency package model with the same name listed earlier in the manifest supplies materialization, keys, columns and graph facts for the changed project model (reproduced by both external reviewers). The card is presented as authoritative. Fix: prefer metadata.project_name (as macroCard does) and/or match by path or unique id. (Codex, Claude)
There was a problem hiding this comment.
Thanks, confirmed. modelCard now takes the changed file's path. Since c99cc33 it resolves in this order: the node whose original_file_path is that file within metadata.project_name, then a same-named model, snapshot or seed in the project, then any package. The test has a dependency-package orders listed first, and the card now comes from the project's model.
There was a problem hiding this comment.
✅ Fixed — runner.ts:572-590 resolves by changed path, then this project, then any package; test puts a pkg model first and expects the project one.
| const byName = macros.filter(([, m]) => m?.name === name) | ||
| const path = cardOpts?.path | ||
| const hit = | ||
| byName.find(([, m]) => m.package_name === project) ?? |
There was a problem hiding this comment.
[MINOR] macroCard resolves by file basename before path, and not within the project
byName.find(([, m]) => m.package_name === project) ??
byName[0] ??
The lookup name is the file basename, not necessarily the macro defined in it. macros/helpers.sql defining foo can get the card of an unrelated helpers macro, or a package macro (e.g. dbt_utils.date_spine) with its callers and variants. The path fallback is also not restricted to the project package. Fix: match by path within the project first, then name within the project, then any package. (Codex, Claude)
There was a problem hiding this comment.
Fixed in c99cc33 in the order you suggested: first the macros defined in the changed file within the project (preferring the one named like the file), then the name within the project, then any package. macros/helpers.sql defining foo now gets foo's card instead of a package helpers macro, and a project date_spine wins over dbt_utils.date_spine. Both cases are tested.
There was a problem hiding this comment.
✅ Fixed — runner.ts:671-678 prefers macros defined in the changed file within this project before name lookup; helpers -> foo assertion covers it. (A nested-path suffix collision remains possible but is rare; nit.)
| return macro ? [{ name: macro.name, macro }] : [] | ||
| } | ||
| if (!DECLARING_KINDS.has(kind)) return [] | ||
| const parents = (await runner.graphParents?.([f.path])) ?? [] |
There was a problem hiding this comment.
[MINOR] YAML declaration cards depend on test-node parents, not patch_path
const parents = (await runner.graphParents?.([f.path])) ?? []
graphParents links a YAML file to nodes via tests, while models are tied to their YAML by patch_path. A schema YAML with no generic tests gets no card even though the manifest has its models' config; removing the last test removes the card. The orchestrator test mocks graphParents, which hides this. Fix: resolve declarations through patch_path. (Codex, Claude)
There was a problem hiding this comment.
Good point; the mocked graphParents hid this. c99cc33 adds a runner method, patchedModels(path), which returns the project's models, snapshots and seeds whose patch_path is the yml. A yml's cards are now the union of those and the tested models (still at most 3). The new test uses the real dispatcher runner with no test nodes, and the schema yml still gets its model's card.
There was a problem hiding this comment.
✅ Fixed — new patchedModels (runner.ts:554-569) uses patch_path, unioned with test parents in ai-context-card.ts:123-129; test runs against the real manifest reader without test nodes.
| : fileChars < maxChars | ||
| ? truncate(f.sql, fileChars) | ||
| : excerptFile(f.sql, f.diff, maxChars).text | ||
| if (f.sql.length > fileChars) stats = { ...stats, truncated: true } |
There was a problem hiding this comment.
[MINOR] Ledger reports truncated:false when MAX_FILE_CHARS < 2000
if (f.sql.length > fileChars) stats = { ...stats, truncated: true }
With ALTIMATE_REVIEW_AI_MAX_FILE_CHARS=1000 and 1,500-char SQL, fileChars is widened to 2000 but excerptFile(..., maxChars) still truncates, so the record says truncated:false while the prompt contains (truncated). Fix: compare against the budget actually applied (Math.min(maxChars, ...)). (Claude)
There was a problem hiding this comment.
Thanks for tracing this. In c99cc33 the full-diff file budget is Math.min(COVERAGE_FULL_DIFF_FILE_CHARS, maxChars), so the truncated check uses the budget that was actually applied. A test with ALTIMATE_REVIEW_AI_MAX_FILE_CHARS=500 checks that the prompt shows (truncated) and the record says truncated: true.
There was a problem hiding this comment.
✅ Fixed — ai-review.ts:414 bounds the file head by Math.min(COVERAGE_FULL_DIFF_FILE_CHARS, maxChars); the ledger now reports truncated:true for the small-budget fixture.
| ...(manifestMissing && !anyManifest | ||
| ? modelFiles.map((f) => ({ path: f.path, reason: "no dbt manifest: lineage, config and graph checks skipped" })) | ||
| : []), | ||
| ...aiUnsent.map((p) => ({ path: p, reason: "not sent to the AI (file quota)" })), |
There was a problem hiding this comment.
[MINOR] Quota-omitted files listed under Not reviewed although engine lanes checked them
...aiUnsent.map((p) => ({ path: p, reason: "not sent to the AI (file quota)" })),
The ledger means "every changed file the review did not look at", but these files went through the deterministic lanes; on a large PR dozens of engine-checked files show up in that block. The default maxFiles cap records nothing, so behaviour is inconsistent. Fix: report as a separate note or field. (Claude)
There was a problem hiding this comment.
Agreed, the ledger was the wrong place for them. Since c99cc33 these paths stay on the AI lane status (ai.unsent, now in the envelope schema) and appear as one summary note: "N file(s) were not sent to the advisory AI review (file quota); the deterministic checks still ran on them: ...". They no longer appear under "Not reviewed", and the orchestrator test checks both the note and the ledger.
There was a problem hiding this comment.
✅ Fixed — quota-omitted files are no longer in the Not reviewed ledger; they are carried as ai.unsent (verdict.ts:243) and shown as a separate AI-lane note (format.ts:285-294), with a test. The note wording is inaccurate for seeds (new inline comment).
a6823f9 to
936b24a
Compare
b07b80d to
bf6e861
Compare
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
936b24a to
b234d43
Compare
bf6e861 to
c99cc33
Compare
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
2 similar comments
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
|
Thanks for the approval and the detailed pass. All eight MINORs are addressed. The From the nits, c99cc33 also covers:
Several of your missing tests are now added: dry-run with replay and verify, YAML cards through the real runner, Jinja-only changes on compiled SQL, and the quota note rendering. Left as follow-ups:
These only affect excerpt quality behind the opt-in flags, and I preferred to keep this commit focused. |
| const head = aiUnsent.slice(0, 5).map(codeSpan).join(", ") | ||
| const more = aiUnsent.length > 5 ? ` and ${aiUnsent.length - 5} more` : "" | ||
| notes.push( | ||
| `${aiUnsent.length} file(s) were not sent to the advisory AI review (file quota); the deterministic checks still ran on them: ${head}${more}.`, |
There was a problem hiding this comment.
WARNING: The note asserts the deterministic checks ran on every quota-skipped file, but seed files never reach a deterministic lane. orchestrate.ts touches models (1919), snapshots (2215, 2283), macros (2248), tests/project config (2283), schema yml (2390) and modified dbt_project.yml (2421); no lane touches a .csv seed, and only AI-sent files are touched at 2594. With ALTIMATE_REVIEW_AI_COVERAGE=1, one out-of-quota seed (e.g. ALTIMATE_REVIEW_AI_QUOTA_YML=0, or 40 models + 10 seeds where the spare quota is absorbed by models) is reported here as "deterministic checks still ran", while coverageNotes also renders the ledger entry no checks run for seed files at the full tier (orchestrate.ts:2659-2661) for the same path - the same summary contradicts itself. Use the listed set computed at line 264: make this claim only for aiUnsent paths that are not in the ledger, or say that files without deterministic coverage are listed under "Not reviewed".
| // A quota of 0 disables that bucket. | ||
| const quota = (v: string | undefined, d: number) => { | ||
| const n = Number(v) | ||
| return v && Number.isInteger(n) && n >= 0 ? n : d |
There was a problem hiding this comment.
WARNING: quota() validates with Number() instead of a strict integer, so a whitespace or non-decimal value silently becomes a quota of 0 and disables the bucket: Number(" "), Number("0x0") and Number("0e0") are all 0, v is truthy, and n >= 0 accepts it. ALTIMATE_REVIEW_AI_QUOTA_MODEL=" " in a workflow then sends no model at all, while the neighbouring num() falls back to the default for the same input and the test states invalid values keep the default (-1 does). Conversely "1e3" is accepted as 1000. Require a trimmed decimal integer:
| return v && Number.isInteger(n) && n >= 0 ? n : d | |
| return typeof v === "string" && /^\d+$/.test(v.trim()) && Number.isSafeInteger(Number(v)) ? Number(v) : d |
| used += size(s, end) + MARKER | ||
| } | ||
| } | ||
| break |
There was a problem hiding this comment.
SUGGESTION: One window that does not fit stops the loop, so later windows are dropped while budget is left. Windows are sorted but their sizes vary with line length (and the last one is clipped at EOF): with 80-char lines and maxChars=6000, anchors at 100, 600 and 990 give windows costing about 3.4k, 3.4k and 2.6k - the loop breaks on the second (6.7k > 6.0k) even though the first and third fit in 5.95k, and the head fill below then spends the remainder above the first window. The SQL excerpt therefore omits the changed line at 990 unless the diff excerpt still carries that hunk (it is capped at diffChars). Let later windows that still fit be kept:
| break | |
| continue |
| const budget = aiInputBudget() | ||
| const files = selectAiFiles(input.files, budget.maxFiles) | ||
| if (!files.length) return done([], { state: "skipped", reason: "no changed dbt files to review" }) | ||
| if (!files.length && (dryRun || !replay)) return done([], { state: "skipped", reason: "no changed dbt files to review" }) |
There was a problem hiding this comment.
SUGGESTION: files is the quota-selected list, not the changed-file list, so when a quota excludes every sendable file the lane reports no changed dbt files to review even though files did change and are then listed as quota-skipped in the same summary. With ALTIMATE_REVIEW_AI_COVERAGE=1 ALTIMATE_REVIEW_AI_QUOTA_MACRO=0 and only macros/to_cents.sql changed, ai.unsent holds that file while the reason says there were no changed files. The check also precedes the dry-run branch (686), so a dry run with empty quotas reports the same false reason instead of the dry-run one. Choose the reason from plan.unsent.length and keep the dry-run reason first.
| // Only the files the AI lane will send are reached (its file cap, or the | ||
| // per-kind quotas in coverage mode); the rest stay in the ledger. | ||
| const aiBudget = aiInputBudget() | ||
| touch(aiBudget.coverage ? selectAiFilesByQuota(aiFiles, aiBudget.coverage.quotas).sent : selectAiFiles(aiFiles, aiBudget.maxFiles)) |
There was a problem hiding this comment.
SUGGESTION: This repeats the selection branch owned by planAiInput (ai-review.ts:651-656: coverage means selectAiFilesByQuota(...).sent, otherwise selectAiFiles(..., maxFiles)). The ledger's accuracy depends on this selection matching exactly what the lane sends; if either copy changes, a file is silently marked reviewed here without being sent, or the reverse - the same drift this change fixes for the quota case. Extract one exported selector (e.g. selectAiInputFiles(files, budget) returning { files, unsent }) and call it from both places instead of maintaining the branch twice.
hkaltai
left a comment
There was a problem hiding this comment.
Verdict: approve. Panel: Claude, Codex and lead reviewer, re-review of c99cc33 (previously approved at b07b80d). review-ai-inputs.test.ts passes (42), the full review suite passes (827 per one reviewer) and typecheck is clean. With the flags unset the prompt is still byte-identical (snapshot, also checked against the base by a reviewer).
Counts: CRITICAL 0, MAJOR 0, MINOR 2 new (inline), NIT 6 new.
Status of prior MINOR findings (replied in-thread)
| Finding | Status |
|---|---|
| Replay before dry run | Fixed |
| Compiled-SQL windows, no eligible added line | Partially fixed (see new inline comment) |
| batchChars Infinity under coverage | Fixed |
| modelCard first match across packages | Fixed |
| macroCard basename before path | Fixed |
| YAML cards rely on test parents | Fixed (singular tests on seed/snapshot parents still model-only; nit) |
| Ledger truncated:false | Fixed |
| Quota-omitted files under Not reviewed | Fixed (note wording issue, new inline comment) |
Prior nits: fixed: quota 0 parsing, seed in MODEL_KINDS, card-cap comment, "pure" doc. Not fixed (still nits/non-blocking): first oversized excerpt window can drop its anchor (ai-review.ts:342-351), __ macro-prefix variants (runner.ts:688-695), empty truncated hunk counted as sent (ai-review.ts:249-256), trailing marker not reserved (ai-review.ts:333), naming items.
New nits
ai-review.ts:672const plan = planAiInput(input)sits outside thetryinrunAiReview(and now before replay), so a render failure breaks the "never throws" contract; the orchestrator catches it. Fix: wrap planning or plan after the replay branch.ai-review.ts:684if (!files.length && (dryRun || !replay)) return done([], { state: "skipped", reason: "no changed dbt files to review" }): when every quota is 0 all files are unsent but the reason says no changed files. Fix: distinct reason whenplan.unsentis non-empty.ai-review.ts:59thequotaparser maps" "(whitespace) to 0 and so disables a bucket. Fix: accept only trimmed decimal integer text.runner.ts:44-46pathMatchessuffix match:macros/nested/macros/helpers.sqlvsmacros/helpers.sqlcan resolve to the wrong macro (inFile[0]). Fix: prefer exact project-relative path, then the most specific suffix.ai-context-card.ts:127singular-test parents still filter tokind === "model", so seed/snapshot parents get no card. Fix: include them.ai-context-card.ts:115card is labelled withf.modelrather than the resolved node's name when a path match picks a differently named snapshot. Fix: use the resolved name.runner.ts:625,643,686.slice()caps run before rendering, so the card's+N morenever shows (11 callers read as 10). Fix: return full lists and cap in the renderer.- Comment hygiene:
COVERAGE_BATCH_CHARSdoc hard-codes "30 files ... ~660k" arithmetic that will drift;excerptFiledocstring does not mention the context-line fallback. - Naming unchanged:
AiReviewFile.card(string) vsAiInputRecord.card(boolean);fileCharsis a budget;CoverageArgtri-state test-only; localMARKER;isOther->isSnapshotOrSeed.
| const head = aiUnsent.slice(0, 5).map(codeSpan).join(", ") | ||
| const more = aiUnsent.length > 5 ? ` and ${aiUnsent.length - 5} more` : "" | ||
| notes.push( | ||
| `${aiUnsent.length} file(s) were not sent to the advisory AI review (file quota); the deterministic checks still ran on them: ${head}${more}.`, |
There was a problem hiding this comment.
[MINOR] AI-unsent note claims deterministic checks ran on seeds, which they did not
`${aiUnsent.length} file(s) were not sent to the advisory AI review (file quota); the deterministic checks still ran on them: ${head}${more}.`
Seed files are only touched by the AI lane. With coverage on, a yml quota of 1 and two changed seeds, the summary says the deterministic checks still ran on them: seeds/b.csv while the ledger says no checks run for seed files at the full tier. Two reviewers reproduced this; the new note is false for that case.
Fix: list only unsent paths that were touched by a deterministic lane, or drop the 'deterministic checks still ran' clause; add a test with a quota-omitted seed.
| const eligible = rows.filter((r) => r.kind === "+" && isEligible(r)) | ||
| if (!eligible.length) { | ||
| const context = rows.filter((r) => r.kind !== "+" && isEligible(r)) | ||
| return context.length && !context.every(matchesRaw) ? "unlocated" : undefined |
There was a problem hiding this comment.
[MINOR] Compiled-SQL windows still trust raw line numbers when context is also Jinja/short
return context.length && !context.every(matchesRaw) ? "unlocated" : undefined
When the added lines are ineligible and the context lines are too (e.g. {% if is_incremental() %} / {% endif %} around one added line, or short lines like select, from t), context is empty, so this returns undefined and the raw window is used. On compiled SQL where the line sits ~300 lines lower, the excerpt contains neither the changed text nor the approximate marker. Reproduced by two reviewers. This is the remaining part of the original finding.
Fix: pass an isCompiled flag and return "unlocated" when compiled content differs from raw and neither added nor context lines can be checked; add a test for the no-eligible-context case.
…ext card
All switches are off by default; with them unset the AI prompt is
byte-identical to before (pinned by a snapshot captured before the change).
- `ALTIMATE_REVIEW_AI_COVERAGE=1`: per-kind file quotas (models 12, macros 8,
tests 4, yml/config 6; `ALTIMATE_REVIEW_AI_QUOTA_{MODEL,MACRO,TEST,YML}`),
most-changed files first, unused quota redistributed up to the total.
Over-budget diffs keep whole hunks and name omitted hunks by line range;
over-budget files send +/-20-line windows around changed lines plus the file
head, with `… (lines a–b omitted)` markers. Files left out go to
`scope.notReviewed` as "not sent to the AI (file quota)".
- `ai.input` on the envelope records per file what was sent (chars, truncated,
hunks sent/total, card) under coverage, context cards or dry run.
- `ALTIMATE_REVIEW_AI_DRYRUN=1`: the lane plans its input, records `ai.input`
and returns `skipped` without calling a model. `planAiInput` is the pure
planner `runAiReview` now sends from.
- `ALTIMATE_REVIEW_AI_CONTEXT_CARD=1` (needs a manifest): a `dbt context (from
manifest):` block under each file header, from new non-throwing runner
methods `modelCard` and `macroCard`; yml and singular tests get the cards of
the models they test. At most 600 chars per file and 6,000 per call.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… SQL Under `ALTIMATE_REVIEW_AI_COVERAGE=1`, a model's current file is often dbt-compiled SQL, whose line numbers drift from the raw diff once Jinja expands. When the added lines are not at their raw line numbers, `excerptFile` now finds each added line (8+ non-space chars, no Jinja) by its trimmed text, scanning forward, and centres the windows there. If fewer than half are found it keeps the head cut with a marker saying windows would be approximate. Raw content keeps raw line numbers; the default prompt is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…header Under `ALTIMATE_REVIEW_AI_COVERAGE=1` only (the default prompt is unchanged): - The diff section gets its own budget, `ALTIMATE_REVIEW_AI_COVERAGE_DIFF_CHARS` (default 16,000), instead of the 6,000-char file budget. Hunks stay whole and in order; only hunks past that budget are listed as omitted. Multi-hunk macro diffs that sent 7 of 20 hunks now go whole. - When a long diff (over the file budget) is sent with every hunk, the current-file section shrinks to a 2,000-char head; otherwise the file keeps its +/-20-line windows within the file budget. - Fix: an excerpted diff replaced the whole file section (header, card and current file) because the excerpt's `text` overwrote the section text. `ai.input` chars, hunksSent and hunksTotal now describe what is really sent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- `aiInputBudget`: with `ALTIMATE_REVIEW_AI_COVERAGE=1`, `batchChars` now defaults to 120k chars instead of `Infinity`, so 30 quota files with 16k diffs no longer build one ~660k-char call. `ALTIMATE_REVIEW_AI_BATCH_CHARS` still overrides it; with coverage off the default stays `Infinity` (prompt snapshot unchanged). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…olution, compiled windows, quota note - `runAiReview`: dry run is evaluated (and the input planned) before replay, so `ALTIMATE_REVIEW_AI_DRYRUN=1` with a replay envelope makes no verifier call and records `ai.input`. - `excerptFile`: when no added line is eligible (Jinja-only or short changes), context lines decide whether raw line numbers hold; otherwise the head is sent with the "approximate" marker instead of windows on the wrong lines. - `renderFile`: the full-diff file budget is capped at `maxFileChars`, so the `truncated` record matches what the prompt carries. - `modelCard`: resolves the node defined in the changed file within the project, then by name in the project, then any package. `macroCard`: a macro defined in the changed file within the project first (the one named like the file), then by name in the project, then any package. - Context cards: new `patchedModels` resolves a yml's models through `patch_path`, so a schema yml without generic tests still gets cards; seeds get their own card as the `ModelCard` doc says. - Files over the coverage quotas are reported as an AI-lane note (`ai.unsent` in the envelope) instead of entries in "Not reviewed": the deterministic lanes still checked them. - Coverage quotas accept 0 to disable a bucket (no spare quota flows into it). - Comment fixes: card batch limit wording, `planAiInput` purity. - Restack: the coverage ledger marks reached exactly the files the AI lane sends (per-kind quotas in coverage mode, the file cap otherwise), with a test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
b234d43 to
3d47982
Compare
c99cc33 to
31b384c
Compare
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
1 similar comment
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
Issue for this PR
Part of #1792
Type of change
What does this PR do?
Stacked on #1770. Gives the AI lane better input on large PRs. All of it is opt-in: with the flags unset, prompts are byte-identical to before (snapshot test).
ALTIMATE_REVIEW_AI_COVERAGE=1:ALTIMATE_REVIEW_AI_CONTEXT_CARD=1: a short dbt context card per file from the manifest. For models: materialization, unique key, parents, children, exposures, declared columns with their tests, siblings. For macros: callers and dispatch variants.ALTIMATE_REVIEW_AI_DRYRUN=1: builds the AI input and records its size without calling the model, for cost estimates.Results (blind Claude judge)
How did you verify your code works?
bun test test/altimate/review: 725 pass.bunx tsc --noEmit: clean.test/altimate/review-ai-inputs.test.ts: quotas, redistribution, hunk windows, ledger, card rendering, default prompt snapshot.Cost side effects
None by default; every option is opt-in and the default prompts are byte-identical. With
ALTIMATE_REVIEW_AI_COVERAGE=1andALTIMATE_REVIEW_AI_CONTEXT_CARD=1the AI lane's input grows with the PR's changed files (bounded by the per-kind quotas), measured at 1.25-1.32x AI-lane tokens; the number of AI calls is unchanged. The context card is built from the local manifest (no warehouse queries).ALTIMATE_REVIEW_AI_DRYRUN=1makes no model call and records the input size for cost estimates.Screenshots / recordings
Not a UI change.
Checklist
Changes since review:
c99cc33753addresses the second-round review feedback: dry run evaluated before replay, raw-line windows only when context lines confirm them, full-diff file budget capped atmaxFileChars, project-first model/macro card resolution, optionalai.unsentquota note.🤖 Generated with Claude Code
Summary by cubic
Adds opt-in flags that give the AI reviewer better input on large PRs while leaving the default prompt byte-identical (pinned by a snapshot test).
What changed
ALTIMATE_REVIEW_AI_COVERAGE=1sends per-kind file quotas (models 12, macros 8, tests 4, yml/config 6), most-changed files first, with unused quota redistributed; quotas are overridable viaALTIMATE_REVIEW_AI_QUOTA_*, and a quota of 0 disables that bucket.ALTIMATE_REVIEW_AI_COVERAGE_DIFF_CHARS) that keeps hunks whole; over-budget files send ±20-line windows around changed lines with omission markers, and batches are capped at 120,000 chars per call (ALTIMATE_REVIEW_AI_BATCH_CHARS). When a whole diff fits, the current-file section shrinks to a 2,000-char head.ai.unsentrather than in the "Not reviewed" ledger, since the deterministic lanes still checked them.ALTIMATE_REVIEW_AI_CONTEXT_CARD=1adds a short dbt context card per file from the manifest: materialization and keys, parents/children/exposures, declared columns with tests, and macro callers and variants; yml and singular tests get the card of the model they test.ALTIMATE_REVIEW_AI_DRYRUN=1plans the AI input and records its size without calling the model, for cost estimates; it runs before replay so a dry run with a replay envelope makes no verifier call.ai.inputmetadata (chars, truncated, hunks sent/total, card) is recorded on the verdict envelope under these flags. The coverage ledger marks exactly the files the AI lane sends.In a blind evaluation across 25 repos never used for tuning, coverage plus the context card caught 47 vs 43 defects with fewer wrong findings at about 1.25× the tokens; the gain is within noise, so the flags stay opt-in.
Migration
Written for commit 31b384c. Summary will update on new commits.