Repository navigation
feat(review): --all-projects reviews every dbt project a PR touches in one run - #1773
RaghwendraSingh wants to merge 2 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
There was a problem hiding this comment.
10 issues found across 8 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/src/altimate/review/verdict.ts">
<violation number="1" location="packages/opencode/src/altimate/review/verdict.ts:222">
P3: The per-project AI status bypasses the envelope's fixed AI-state contract by accepting any string. Use the same four-value enum as `ai.state` so malformed or unsupported project statuses cannot enter signed envelopes.</violation>
</file>
<file name="packages/opencode/test/altimate/review-all-projects.test.ts">
<violation number="1" location="packages/opencode/test/altimate/review-all-projects.test.ts:44">
P3: This suite leaks every synthetic git repository it creates. Track fixture roots and remove them in `afterEach` (including failed tests) so repeated CI and watch runs do not accumulate repositories.</violation>
<violation number="2" location="packages/opencode/test/altimate/review-all-projects.test.ts:49">
P2: The new discovery tests do not cover the required ignored directories. Add projects under `dbt_packages/`, `target/`, and `node_modules/`, then assert they are absent from discovered and reviewed roots.</violation>
<violation number="3" location="packages/opencode/test/altimate/review-all-projects.test.ts:166">
P2: The AI aggregation test never verifies addition across nonzero project lanes. Give both projects distinct nonzero calls, cost, and token counts, then assert their sums.</violation>
</file>
<file name="packages/opencode/src/altimate/review/format.ts">
<violation number="1" location="packages/opencode/src/altimate/review/format.ts:76">
P2: The project list is unbounded even though this renderer promises a character budget for the posted comment. With enough touched projects, this line alone can exceed 5,000 characters and the later finding-budget loop cannot trim it; cap the displayed roots and retain the full list in `scope.projects`.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/format.ts:76">
P2: Project roots are repository-controlled strings, but this line inserts them raw into Markdown. A project directory containing a backtick or newline can break the summary's formatting or inject content; render each root with `codeSpan` (or otherwise escape Markdown) before joining them.</violation>
</file>
<file name="packages/opencode/src/altimate/review/git.ts">
<violation number="1" location="packages/opencode/src/altimate/review/git.ts:173">
P2: `--cached` includes tracked files deleted from the working tree, so a fully removed dbt project is still discovered and reviewed from a nonexistent `cwd`; filter the combined list to paths that still exist before discovering projects.</violation>
</file>
<file name="packages/opencode/src/altimate/review/multi-project.ts">
<violation number="1" location="packages/opencode/src/altimate/review/multi-project.ts:118">
P2: `scopeToProject` supports custom `*-paths` outside a project root, but this passes every unassigned file to every project run. Shared-path files can therefore be reviewed repeatedly or remain marked outside after one project reviewed them; assign external layout files to their owning projects before constructing runs.</violation>
<violation number="2" location="packages/opencode/src/altimate/review/multi-project.ts:218">
P2: The merged AI object retains `input` and `verifier` data from only one project while aggregating the top-level usage. With AI coverage or verification enabled, the signed result underreports the other projects' inputs and verifier usage; union or aggregate these fields, or expose them per project.</violation>
</file>
<file name="packages/opencode/src/cli/cmd/review.ts">
<violation number="1" location="packages/opencode/src/cli/cmd/review.ts:111">
P2: `--all-projects` can silently skip a dbt project deleted by the PR because discovery only scans the head tree. Discover projects from both head and merge-base, or retain base roots that own deleted paths.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| findings: [shared], | ||
| tier: "lite", | ||
| mode: "gate", | ||
| ai: { state: "skipped", reason: "x", calls: 0 }, |
There was a problem hiding this comment.
P2: The AI aggregation test never verifies addition across nonzero project lanes. Give both projects distinct nonzero calls, cost, and token counts, then assert their sums.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/review-all-projects.test.ts, line 166:
<comment>The AI aggregation test never verifies addition across nonzero project lanes. Give both projects distinct nonzero calls, cost, and token counts, then assert their sums.</comment>
<file context>
@@ -0,0 +1,225 @@
+ findings: [shared],
+ tier: "lite",
+ mode: "gate",
+ ai: { state: "skipped", reason: "x", calls: 0 },
+ scope: { projectRoot: "a", notReviewed: [{ path: "b/models/x.sql", reason: "outside the dbt project (a)" }] },
+ }),
</file context>
There was a problem hiding this comment.
Partly fixed in 622f430: the new AI-state test sums nonzero calls across two lanes. Distinct nonzero tokens and cost are a low-priority follow-up.
| await g(root, "config", "user.email", "t@test") | ||
| await g(root, "config", "user.name", "t") | ||
| await g(root, "config", "commit.gpgsign", "false") | ||
| const projects = ["proj_a", "proj_b", ...(opts.nested ? ["proj_a/integration_tests"] : [])] |
There was a problem hiding this comment.
P2: The new discovery tests do not cover the required ignored directories. Add projects under dbt_packages/, target/, and node_modules/, then assert they are absent from discovered and reviewed roots.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/review-all-projects.test.ts, line 49:
<comment>The new discovery tests do not cover the required ignored directories. Add projects under `dbt_packages/`, `target/`, and `node_modules/`, then assert they are absent from discovered and reviewed roots.</comment>
<file context>
@@ -0,0 +1,225 @@
+ await g(root, "config", "user.email", "t@test")
+ await g(root, "config", "user.name", "t")
+ await g(root, "config", "commit.gpgsign", "false")
+ const projects = ["proj_a", "proj_b", ...(opts.nested ? ["proj_a/integration_tests"] : [])]
+ for (const p of projects) {
+ await write(root, `${p}/dbt_project.yml`, `name: ${p.replace("/", "_")}\nversion: '1.0'\nprofile: x\n`)
</file context>
There was a problem hiding this comment.
Follow-up, together with r4627041814.
|
|
||
| const lines: string[] = [REVIEW_MARKER, "", `## ${verdictHeadline(env)}`, ""] | ||
| const projects = env.scope?.projects ?? [] | ||
| if (projects.length > 1) lines.push(`Reviewed ${projects.length} dbt projects: ${projects.map((p) => p.root || ".").join(", ")}`, "") |
There was a problem hiding this comment.
P2: The project list is unbounded even though this renderer promises a character budget for the posted comment. With enough touched projects, this line alone can exceed 5,000 characters and the later finding-budget loop cannot trim it; cap the displayed roots and retain the full list in scope.projects.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/format.ts, line 76:
<comment>The project list is unbounded even though this renderer promises a character budget for the posted comment. With enough touched projects, this line alone can exceed 5,000 characters and the later finding-budget loop cannot trim it; cap the displayed roots and retain the full list in `scope.projects`.</comment>
<file context>
@@ -72,6 +72,8 @@ export function renderSummary(env: VerdictEnvelope, opts: RenderOptions = {}): s
const lines: string[] = [REVIEW_MARKER, "", `## ${verdictHeadline(env)}`, ""]
+ const projects = env.scope?.projects ?? []
+ if (projects.length > 1) lines.push(`Reviewed ${projects.length} dbt projects: ${projects.map((p) => p.root || ".").join(", ")}`, "")
if (env.summary.degraded) {
</file context>
There was a problem hiding this comment.
Follow-up: low priority, tracked. We'll cap it at RENDER_CAP with "+N more".
|
|
||
| const lines: string[] = [REVIEW_MARKER, "", `## ${verdictHeadline(env)}`, ""] | ||
| const projects = env.scope?.projects ?? [] | ||
| if (projects.length > 1) lines.push(`Reviewed ${projects.length} dbt projects: ${projects.map((p) => p.root || ".").join(", ")}`, "") |
There was a problem hiding this comment.
P2: Project roots are repository-controlled strings, but this line inserts them raw into Markdown. A project directory containing a backtick or newline can break the summary's formatting or inject content; render each root with codeSpan (or otherwise escape Markdown) before joining them.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/format.ts, line 76:
<comment>Project roots are repository-controlled strings, but this line inserts them raw into Markdown. A project directory containing a backtick or newline can break the summary's formatting or inject content; render each root with `codeSpan` (or otherwise escape Markdown) before joining them.</comment>
<file context>
@@ -72,6 +72,8 @@ export function renderSummary(env: VerdictEnvelope, opts: RenderOptions = {}): s
const lines: string[] = [REVIEW_MARKER, "", `## ${verdictHeadline(env)}`, ""]
+ const projects = env.scope?.projects ?? []
+ if (projects.length > 1) lines.push(`Reviewed ${projects.length} dbt projects: ${projects.map((p) => p.root || ".").join(", ")}`, "")
if (env.summary.degraded) {
</file context>
| if (projects.length > 1) lines.push(`Reviewed ${projects.length} dbt projects: ${projects.map((p) => p.root || ".").join(", ")}`, "") | |
| if (projects.length > 1) lines.push(`Reviewed ${projects.length} dbt projects: ${projects.map((p) => codeSpan(p.root || ".")).join(", ")}`, "") |
There was a problem hiding this comment.
Follow-up: low priority, tracked (codeSpan).
| /** Tracked and untracked (not ignored) files in the working tree, repo-root relative; undefined when git can't list them. */ | ||
| export async function listWorkingTreeFiles(cwd: string): Promise<string[] | undefined> { | ||
| try { | ||
| const out = await git(["ls-files", "--cached", "--others", "--exclude-standard", "--full-name"], cwd) |
There was a problem hiding this comment.
P2: --cached includes tracked files deleted from the working tree, so a fully removed dbt project is still discovered and reviewed from a nonexistent cwd; filter the combined list to paths that still exist before discovering projects.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/git.ts, line 173:
<comment>`--cached` includes tracked files deleted from the working tree, so a fully removed dbt project is still discovered and reviewed from a nonexistent `cwd`; filter the combined list to paths that still exist before discovering projects.</comment>
<file context>
@@ -167,6 +167,16 @@ export async function listFilesAt(cwd: string, ref?: string): Promise<string[] |
+/** Tracked and untracked (not ignored) files in the working tree, repo-root relative; undefined when git can't list them. */
+export async function listWorkingTreeFiles(cwd: string): Promise<string[] | undefined> {
+ try {
+ const out = await git(["ls-files", "--cached", "--others", "--exclude-standard", "--full-name"], cwd)
+ return out.split("\n").filter(Boolean)
+ } catch {
</file context>
There was a problem hiding this comment.
Agreed, low priority (cubic, git.ts:173, --cached lists files deleted from the working tree). Leaving this for a follow-up PR rather than widening this one.
| let env | ||
| try { | ||
| env = await reviewPullRequest({ | ||
| const review = args.allProjects === true ? reviewAllProjects : reviewPullRequest |
There was a problem hiding this comment.
P2: --all-projects can silently skip a dbt project deleted by the PR because discovery only scans the head tree. Discover projects from both head and merge-base, or retain base roots that own deleted paths.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/cli/cmd/review.ts, line 111:
<comment>`--all-projects` can silently skip a dbt project deleted by the PR because discovery only scans the head tree. Discover projects from both head and merge-base, or retain base roots that own deleted paths.</comment>
<file context>
@@ -100,7 +108,8 @@ export const ReviewCommand = cmd({
let env
try {
- env = await reviewPullRequest({
+ const review = args.allProjects === true ? reviewAllProjects : reviewPullRequest
+ env = await review({
...prContext,
</file context>
There was a problem hiding this comment.
Follow-up: low priority, tracked. Discovering roots at the merge-base as well would cover it.
| for (const root of touched) { | ||
| const own = byProject.get(root)! | ||
| process.stderr.write(`ℹ️ reviewing dbt project ${root || "."} (${own.length} changed file(s))\n`) | ||
| const env = await reviewPullRequest({ ...optsFor(root), changedFiles: [...own, ...unassigned], prFiles }) |
There was a problem hiding this comment.
P2: scopeToProject supports custom *-paths outside a project root, but this passes every unassigned file to every project run. Shared-path files can therefore be reviewed repeatedly or remain marked outside after one project reviewed them; assign external layout files to their owning projects before constructing runs.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/multi-project.ts, line 118:
<comment>`scopeToProject` supports custom `*-paths` outside a project root, but this passes every unassigned file to every project run. Shared-path files can therefore be reviewed repeatedly or remain marked outside after one project reviewed them; assign external layout files to their owning projects before constructing runs.</comment>
<file context>
@@ -0,0 +1,226 @@
+ for (const root of touched) {
+ const own = byProject.get(root)!
+ process.stderr.write(`ℹ️ reviewing dbt project ${root || "."} (${own.length} changed file(s))\n`)
+ const env = await reviewPullRequest({ ...optsFor(root), changedFiles: [...own, ...unassigned], prFiles })
+ runs.push({ root, paths: own.map((f) => f.path), env })
+ }
</file context>
There was a problem hiding this comment.
Follow-up: low priority, tracked. In the normal layout they're scoped out as outside each project and de-duplicated on merge; custom *-paths outside a root is the edge case.
| calls: z.number().int().nonnegative().optional(), | ||
| /** Multi-project review (`--all-projects`): each project's AI lane state. */ | ||
| projects: z | ||
| .array(z.object({ root: z.string(), state: z.string(), calls: z.number().int().nonnegative().optional() })) |
There was a problem hiding this comment.
P3: The per-project AI status bypasses the envelope's fixed AI-state contract by accepting any string. Use the same four-value enum as ai.state so malformed or unsupported project statuses cannot enter signed envelopes.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/verdict.ts, line 222:
<comment>The per-project AI status bypasses the envelope's fixed AI-state contract by accepting any string. Use the same four-value enum as `ai.state` so malformed or unsupported project statuses cannot enter signed envelopes.</comment>
<file context>
@@ -214,6 +217,10 @@ export const VerdictEnvelope = z.object({
calls: z.number().int().nonnegative().optional(),
+ /** Multi-project review (`--all-projects`): each project's AI lane state. */
+ projects: z
+ .array(z.object({ root: z.string(), state: z.string(), calls: z.number().int().nonnegative().optional() }))
+ .optional(),
/** Generator findings replayed from an earlier envelope: tokens, cost and calls are that run's. */
</file context>
There was a problem hiding this comment.
Fair nit; we'll tighten it to the lane-state enum in a follow-up. (cubic, verdict.ts:222, ai.projects[].state is z.string())
|
|
||
| /** Two dbt projects (proj_a, proj_b), each with a model and a macro, plus a workflow file; returns the base sha. */ | ||
| async function fixture(opts: { nested?: boolean } = {}): Promise<{ root: string; base: string }> { | ||
| const root = await fs.realpath(await fs.mkdtemp(path.join(tmpdir(), "review-all-projects-"))) |
There was a problem hiding this comment.
P3: This suite leaks every synthetic git repository it creates. Track fixture roots and remove them in afterEach (including failed tests) so repeated CI and watch runs do not accumulate repositories.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/review-all-projects.test.ts, line 44:
<comment>This suite leaks every synthetic git repository it creates. Track fixture roots and remove them in `afterEach` (including failed tests) so repeated CI and watch runs do not accumulate repositories.</comment>
<file context>
@@ -0,0 +1,225 @@
+
+/** Two dbt projects (proj_a, proj_b), each with a model and a macro, plus a workflow file; returns the base sha. */
+async function fixture(opts: { nested?: boolean } = {}): Promise<{ root: string; base: string }> {
+ const root = await fs.realpath(await fs.mkdtemp(path.join(tmpdir(), "review-all-projects-")))
+ await g(root, "init", "-q", "-b", "main")
+ await g(root, "config", "user.email", "t@test")
</file context>
There was a problem hiding this comment.
Fair nit; we'll add cleanup in a follow-up.
| const optsFor = (root: string): ReviewPullRequestOptions => ({ | ||
| ...opts, | ||
| cwd: path.join(gitRootReal, root), | ||
| manifestPath: root === manifestOwner ? manifestAbs : undefined, |
There was a problem hiding this comment.
WARNING: --manifest is silently dropped whenever it does not resolve inside a discovered project root.
repoRel() returns undefined for a manifest outside the git root (the common CI shape --manifest "$RUNNER_TEMP/manifest.json"), and owningProject() returns undefined for a repo-level manifest that sits outside every project. In both cases manifestOwner is undefined, so root === manifestOwner is false for every project (including the "" root project) and every run falls back to auto-discovery. reviewPullRequest honours an absolute --manifest (run.ts:277-279), so adding --all-projects silently degrades every project to lint-only with no diagnostic. Pass the manifest through when only one project is touched, or warn/fail instead of ignoring it.
There was a problem hiding this comment.
Fixed in 622f430: a manifest outside every project (CI's $RUNNER_TEMP/manifest.json) now goes to the touched project when the PR changes only one. When it can't be attributed (several touched projects, or it lies inside an untouched project), a warning says no project uses it. Previously each project silently fell back to lint-only. A test covers both cases.
| const seenIds = new Set<string>() | ||
| const findings = envs.flatMap((e) => e.findings).filter((f) => !seenIds.has(f.id) && !!seenIds.add(f.id)) | ||
|
|
||
| const mode = modeOverride ?? (envs.some((e) => e.mode === "gate") ? "gate" : first.mode) |
There was a problem hiding this comment.
WARNING: Mixed per-project modes can turn a non-blocking project's findings into a blocking verdict.
mode is gate if any project uses gate, and the merged verdict is applyMode(max(idealVerdict), mode). A project whose own policy is the default comment already softened its REQUEST_CHANGES to COMMENT, but its idealVerdict is still REQUEST_CHANGES, so a sibling project configured gate promotes it back to a blocking REQUEST_CHANGES. Merge the post-mode per-project verdicts (envs.map((e) => e.verdict)) and keep applyMode out of the merge. The mergeProjectEnvelopes test only covers the both-gate case, so this is untested.
There was a problem hiding this comment.
Fixed in 622f430: the merged verdict is the worst of the projects' post-mode verdicts, so a comment-mode project's REQUEST_CHANGES no longer blocks because a sibling uses gate mode. Each project's mode is recorded in scope.projects, with a mixed-mode test.
| } | ||
|
|
||
| const merged: VerdictEnvelope = { | ||
| ...first, |
There was a problem hiding this comment.
WARNING: ...first leaves first-project-only fields in the signed merged envelope.
manifestHash (rendered in the comment footer, format.ts:118), policy (including posting, which drives the inline budget and silentWhenClean), scope.projectRoot, generatedAt and tierClassified are never overridden, so a multi-project verdict claims the first project's manifest hash, policy hash and posting policy for a run that used several manifests. That is misleading audit data in a signed artifact and changes how the merged comment is posted. Build the envelope explicitly (or clear these fields) and record per-project manifest/policy detail in scope.projects.
There was a problem hiding this comment.
Same issue as r4626966398; fixed in 622f430.
| cacheRead: a.cacheRead + b.cacheRead, | ||
| })) | ||
| : undefined | ||
| const base = ok ?? first |
There was a problem hiding this comment.
WARNING: base = ok ?? first reports the merged AI lane as ok whenever any project succeeded, dropping a sibling project's error/timeout and its reason.
With proj_a error and proj_b ok, the merged envelope says state: "ok" with no reason; ai.projects[] keeps only state/calls, so the failure is unrecoverable from the signed envelope (the top-level events and format notices no longer show the AI lane degraded). Promote the worst state (error > timeout > skipped/ok) and keep the failing lane's reason, or add a per-project reason.
There was a problem hiding this comment.
Same issue as r4626966388; fixed in 622f430.
| */ | ||
|
|
||
| const PROJECT_FILE = /(^|\/)dbt_project\.ya?ml$/ | ||
| const IGNORED_DIR = /(^|\/)(dbt_packages|target|node_modules)\// |
There was a problem hiding this comment.
WARNING: IGNORED_DIR re-implements a subset of the existing skip list and is incomplete.
diff-filter.ts:13-23 (SKIP_DIR_PATTERNS, used by shouldReview) also excludes dbt_modules/, compiled/, run/, logs/ and .venv/. A committed dbt_modules/<pkg>/dbt_project.yml is therefore discovered as a project: its changed files are assigned to that phantom root instead of the enclosing project, they get their own review run (extra AI lane calls), and the summary lists vendored packages as reviewed dbt projects. Export and reuse SKIP_DIR_PATTERNS/shouldReview instead of keeping a second, divergent list.
There was a problem hiding this comment.
Follow-up: low priority, tracked. We'll reuse SKIP_DIR_PATTERNS from diff-filter.ts so vendored dbt_modules/ projects aren't discovered.
| const manifestOwner = manifestRel !== undefined ? owningProject(manifestRel, roots) : undefined | ||
| const optsFor = (root: string): ReviewPullRequestOptions => ({ | ||
| ...opts, | ||
| cwd: path.join(gitRootReal, root), |
There was a problem hiding this comment.
WARNING: Project roots are discovered from the head tree but each project's cwd is taken from the working tree.
With --head pointing at a ref that is not checked out (a documented CLI mode), a project added or renamed by the PR has no directory at path.join(gitRootReal, root). findDbtProject then returns undefined, so run.ts:312 computes projectRootRel = undefined and scopeToProject treats every file as in scope (the unassigned files, already passed to every project, get re-reviewed by this one) while dbtProjectName/dbtProjectLayout silently degrade. Verify the directory exists (or derive the layout from the ref) and pass the project root explicitly so scoping cannot collapse to undefined.
There was a problem hiding this comment.
Follow-up: low priority, tracked. This only happens with a --head that isn't checked out.
| calls: z.number().int().nonnegative().optional(), | ||
| /** Multi-project review (`--all-projects`): each project's AI lane state. */ | ||
| projects: z | ||
| .array(z.object({ root: z.string(), state: z.string(), calls: z.number().int().nonnegative().optional() })) |
There was a problem hiding this comment.
SUGGESTION: ai.projects[].state is z.string() while the parent ai.state (line 211) is z.enum(["ok", "skipped", "timeout", "error"]).
The weaker field lets a replayed or hand-built signed envelope carry an arbitrary project state, and any future switch on it silently falls through. Reuse the same enum.
There was a problem hiding this comment.
Fair nit; we'll tighten it to the lane-state enum in a follow-up. (kilo, same)
|
|
||
| const lines: string[] = [REVIEW_MARKER, "", `## ${verdictHeadline(env)}`, ""] | ||
| const projects = env.scope?.projects ?? [] | ||
| if (projects.length > 1) lines.push(`Reviewed ${projects.length} dbt projects: ${projects.map((p) => p.root || ".").join(", ")}`, "") |
There was a problem hiding this comment.
SUGGESTION: The project list is unbounded, unlike the other multi-entry renders (RENDER_CAP = 8 at line 96, LEDGER_MAX_PATHS = 30 at line 283).
On a monorepo with many (or long-named) projects this single fixed line can consume most of SUMMARY_BUDGET_CHARS (5,000), after which fit drops every finding bullet into the "N more finding(s)" notice. Cap the roots shown and append (+N more var); wrapping/escaping the roots like the rest of the renderer (codeSpan, ledger escaping) would also be safer.
There was a problem hiding this comment.
Same as r4626966319; tracked.
| // A relative --manifest resolves against --cwd, as in the single-project path. | ||
| const rel = await reviewAllProjects({ cwd: root, base, head: "HEAD", noAi: true, manifestPath: "proj_a/artifacts/manifest.json" }) | ||
| expect(rel.manifestHash).toBeDefined() | ||
| expect((await run("proj_b")).manifestHash).toBeUndefined() |
There was a problem hiding this comment.
SUGGESTION: This assertion cannot observe whether proj_b received the manifest.
The merged envelope's manifestHash is always the first project's, and for this call the first project is proj_a, which has no manifest — so the value is undefined whether or not the manifest was applied to proj_b. Both manifest bodies are also identical, so even a per-project check could not distinguish them. Assert on the owning project's own envelope (reviewPullRequest with that project's cwd + manifest) or expose manifestHash in scope.projects, and use distinct manifest contents. A mixed mode: comment + mode: gate case is also untested and is where the merge logic is wrong.
There was a problem hiding this comment.
Fixed in 622f430: the manifest test now checks each project's manifestHash (and that the top level carries none when they differ), and a mixed comment/gate test was added.
Code Review SummaryStatus: 7 Issues Found | Recommendation: Address before merge Overview
The PR content is unchanged since the last review (the branch was rebased onto a newer base, so only base-branch files differ). I re-verified every finding against current HEAD ( Issue Details (click to expand)CRITICAL
WARNING
SUGGESTION
Files Reviewed (8 files)
Fix these issues in Kilo Cloud Previous Review Summaries (2 snapshots, latest commit 622f430)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit 622f430)Status: 7 Issues Found | Recommendation: Address before merge Overview
The per-project orchestration is now much closer: mixed Issue Details (click to expand)CRITICAL
WARNING
SUGGESTION
Files Reviewed (8 files)
Fix these issues in Kilo Cloud Previous review (commit 1161598)Status: 9 Issues Found | Recommendation: Address before merge Overview
The multi-project orchestration is sound in its core shape (deepest-root assignment, exclusive per-project file sets, unsigned merge then re-sign), and the single-project fast paths are correctly preserved. The problems are in the merge and in the edges of project discovery: the merged envelope inherits first-project-only signed fields, mixed per-project Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (8 files)
Reviewed by deepseek-v4.1-flash-1017 · Input: 133.5K · Output: 34K · Cached: 2.1M Review guidance: REVIEW.md from base branch |
dev-punia-altimate
left a comment
There was a problem hiding this comment.
🤖 Code Review — OpenCodeReview (Gemini) — 5 finding(s)
- 5 anchored to a line (posted inline when the comment stream is on)
- 0 without a line anchor
All findings (full text)
1. packages/opencode/src/altimate/review/git.ts (L173-L174)
[🟠 MEDIUM] Using git ls-files without -z can cause bugs when file paths contain non-ASCII or special characters (like spaces or newlines). By default, Git enables core.quotePath, which encloses such paths in double quotes and C-style escapes them (e.g., folder-with-ümlaut/dbt_project.yml becomes "folder-with-\303\274mlaut/dbt_project.yml").
When the path ends with a double quote \", subsequent operations in multi-project.ts using regex like /(^|\/)dbt_project\.ya?ml$/ will fail to match because the string ends with a quote rather than yml.
To ensure robust path handling, it is highly recommended to use the -z flag and split by \0, which safely separates entries and disables git path quoting. (Note: A similar update would be beneficial for listFilesAt as well.)
Suggested change:
const out = await git(["ls-files", "-z", "--cached", "--others", "--exclude-standard", "--full-name"], cwd)
return out.split("\0").filter(Boolean)
2. packages/opencode/src/altimate/review/multi-project.ts (L70-L72)
[🟠 MEDIUM] The condition rel.startsWith("..") incorrectly matches valid file or directory names that begin with .. (e.g., ..foo), wrongly identifying them as being outside the git root. This causes valid files in the repository to be ignored.
Consider checking for exact match or directory separator to correctly identify paths outside the git root.
Suggested change:
const rel = path.relative(gitRootReal, abs).replace(/\\/g, "/")
if (rel === ".." || rel.startsWith("../") || path.isAbsolute(rel)) return undefined
return rel
3. packages/opencode/src/altimate/review/multi-project.ts (L54-L55)
[🟠 MEDIUM] Inside the loop, byProject.set(root, [...(byProject.get(root) ?? []), f]) creates and spreads a new array for every file assigned to a project. This results in O(n^2) time complexity, which can cause significant performance degradation and high memory allocation when reviewing pull requests that touch a large number of files.
You should use .push() on an existing array to optimize this assignment.
Suggested change:
if (root === undefined) {
unassigned.push(f)
} else {
let filesForRoot = byProject.get(root)
if (!filesForRoot) {
filesForRoot = []
byProject.set(root, filesForRoot)
}
filesForRoot.push(f)
}
4. packages/opencode/src/altimate/review/multi-project.ts (L114-L120)
[🟠 MEDIUM] The await reviewPullRequest(...) calls are executed sequentially inside a for...of loop for each touched project. Since reviews involve network I/O or heavy computation, sequential execution unnecessarily blocks concurrency, leading to longer overall wait times for multi-project pull requests.
Consider using Promise.all to execute the project reviews concurrently.
Suggested change:
const runs: ProjectRun[] = await Promise.all(
touched.map(async (root) => {
const own = byProject.get(root)!
process.stderr.write(`ℹ️ reviewing dbt project ${root || "."} (${own.length} changed file(s))\n`)
const env = await reviewPullRequest({ ...optsFor(root), changedFiles: [...own, ...unassigned], prFiles })
return { root, paths: own.map((f) => f.path), env }
})
)
5. packages/opencode/src/altimate/review/multi-project.ts (L161-L163)
[🟠 MEDIUM] Unassigned files are passed to every project's review. If they fail verification, they will be returned in each project's envelope and subsequently duplicated by envs.flatMap without deduplication. This causes the final envelope to contain duplicated unverified file entries.
Consider deduplicating the unverified entries based on their path and reason.
Suggested change:
const outsideCount = notReviewed.filter((n) => OUTSIDE_PROJECT.test(n.reason)).length
const movedCount = envs.reduce((s, e) => s + (e.scope?.movedUnchangedFiles ?? 0), 0)
const unverifiedRaw = envs.flatMap((e) => e.scope?.unverified ?? [])
const unverified = [...new Map(unverifiedRaw.map((u) => [`${u.path}\0${u.reason}`, u])).values()]
| const out = await git(["ls-files", "--cached", "--others", "--exclude-standard", "--full-name"], cwd) | ||
| return out.split("\n").filter(Boolean) |
There was a problem hiding this comment.
[🟠 MEDIUM] Using git ls-files without -z can cause bugs when file paths contain non-ASCII or special characters (like spaces or newlines). By default, Git enables core.quotePath, which encloses such paths in double quotes and C-style escapes them (e.g., folder-with-ümlaut/dbt_project.yml becomes "folder-with-\303\274mlaut/dbt_project.yml").
When the path ends with a double quote \", subsequent operations in multi-project.ts using regex like /(^|\/)dbt_project\.ya?ml$/ will fail to match because the string ends with a quote rather than yml.
To ensure robust path handling, it is highly recommended to use the -z flag and split by \0, which safely separates entries and disables git path quoting. (Note: A similar update would be beneficial for listFilesAt as well.)
Suggested change:
| const out = await git(["ls-files", "--cached", "--others", "--exclude-standard", "--full-name"], cwd) | |
| return out.split("\n").filter(Boolean) | |
| const out = await git(["ls-files", "-z", "--cached", "--others", "--exclude-standard", "--full-name"], cwd) | |
| return out.split("\0").filter(Boolean) |
There was a problem hiding this comment.
Thanks, this is a real bug. Without -z, git C-quotes non-ASCII paths, so a dbt_project.yml under a directory like projé/ was never discovered. Fixed in 622f430: listWorkingTreeFiles and listFilesAt (as you suggested) both use -z and split on \0, with a test for a non-ASCII project directory on both the head tree and the working tree.
| const rel = path.relative(gitRootReal, abs).replace(/\\/g, "/") | ||
| if (rel.startsWith("..") || path.isAbsolute(rel)) return undefined | ||
| return rel |
There was a problem hiding this comment.
[🟠 MEDIUM] The condition rel.startsWith("..") incorrectly matches valid file or directory names that begin with .. (e.g., ..foo), wrongly identifying them as being outside the git root. This causes valid files in the repository to be ignored.
Consider checking for exact match or directory separator to correctly identify paths outside the git root.
Suggested change:
| const rel = path.relative(gitRootReal, abs).replace(/\\/g, "/") | |
| if (rel.startsWith("..") || path.isAbsolute(rel)) return undefined | |
| return rel | |
| const rel = path.relative(gitRootReal, abs).replace(/\\/g, "/") | |
| if (rel === ".." || rel.startsWith("../") || path.isAbsolute(rel)) return undefined | |
| return rel |
There was a problem hiding this comment.
Good catch, thanks. A directory named ..foo inside the repo was treated as outside it. Fixed in 622f430: the check is now rel === ".." || rel.startsWith("../").
| if (root === undefined) unassigned.push(f) | ||
| else byProject.set(root, [...(byProject.get(root) ?? []), f]) |
There was a problem hiding this comment.
[🟠 MEDIUM] Inside the loop, byProject.set(root, [...(byProject.get(root) ?? []), f]) creates and spreads a new array for every file assigned to a project. This results in O(n^2) time complexity, which can cause significant performance degradation and high memory allocation when reviewing pull requests that touch a large number of files.
You should use .push() on an existing array to optimize this assignment.
Suggested change:
| if (root === undefined) unassigned.push(f) | |
| else byProject.set(root, [...(byProject.get(root) ?? []), f]) | |
| if (root === undefined) { | |
| unassigned.push(f) | |
| } else { | |
| let filesForRoot = byProject.get(root) | |
| if (!filesForRoot) { | |
| filesForRoot = [] | |
| byProject.set(root, filesForRoot) | |
| } | |
| filesForRoot.push(f) | |
| } |
There was a problem hiding this comment.
Thanks, that's fair. It's quadratic only in the number of changed files per project, so it doesn't matter in practice, but push is just as clear. We'll switch it in a follow-up cleanup.
| const runs: ProjectRun[] = [] | ||
| for (const root of touched) { | ||
| const own = byProject.get(root)! | ||
| process.stderr.write(`ℹ️ reviewing dbt project ${root || "."} (${own.length} changed file(s))\n`) | ||
| const env = await reviewPullRequest({ ...optsFor(root), changedFiles: [...own, ...unassigned], prFiles }) | ||
| runs.push({ root, paths: own.map((f) => f.path), env }) | ||
| } |
There was a problem hiding this comment.
[🟠 MEDIUM] The await reviewPullRequest(...) calls are executed sequentially inside a for...of loop for each touched project. Since reviews involve network I/O or heavy computation, sequential execution unnecessarily blocks concurrency, leading to longer overall wait times for multi-project pull requests.
Consider using Promise.all to execute the project reviews concurrently.
Suggested change:
| const runs: ProjectRun[] = [] | |
| for (const root of touched) { | |
| const own = byProject.get(root)! | |
| process.stderr.write(`ℹ️ reviewing dbt project ${root || "."} (${own.length} changed file(s))\n`) | |
| const env = await reviewPullRequest({ ...optsFor(root), changedFiles: [...own, ...unassigned], prFiles }) | |
| runs.push({ root, paths: own.map((f) => f.path), env }) | |
| } | |
| const runs: ProjectRun[] = await Promise.all( | |
| touched.map(async (root) => { | |
| const own = byProject.get(root)! | |
| process.stderr.write(`ℹ️ reviewing dbt project ${root || "."} (${own.length} changed file(s))\n`) | |
| const env = await reviewPullRequest({ ...optsFor(root), changedFiles: [...own, ...unassigned], prFiles }) | |
| return { root, paths: own.map((f) => f.path), env } | |
| }) | |
| ) |
There was a problem hiding this comment.
Thanks for the suggestion. We're keeping these sequential on purpose. Each project review starts its own AI lane (up to two concurrent model calls, plus the optional verifier's tool loop), so running projects in parallel would multiply concurrent LLM calls and rate-limit pressure by the number of projects. They also share the native dispatcher, and their stderr progress lines would interleave into an unreadable CI log. Wall time is usually dominated by the model calls, which already run concurrently within each project.
| const outsideCount = notReviewed.filter((n) => OUTSIDE_PROJECT.test(n.reason)).length | ||
| const movedCount = envs.reduce((s, e) => s + (e.scope?.movedUnchangedFiles ?? 0), 0) | ||
| const unverified = envs.flatMap((e) => e.scope?.unverified ?? []) |
There was a problem hiding this comment.
[🟠 MEDIUM] Unassigned files are passed to every project's review. If they fail verification, they will be returned in each project's envelope and subsequently duplicated by envs.flatMap without deduplication. This causes the final envelope to contain duplicated unverified file entries.
Consider deduplicating the unverified entries based on their path and reason.
Suggested change:
| const outsideCount = notReviewed.filter((n) => OUTSIDE_PROJECT.test(n.reason)).length | |
| const movedCount = envs.reduce((s, e) => s + (e.scope?.movedUnchangedFiles ?? 0), 0) | |
| const unverified = envs.flatMap((e) => e.scope?.unverified ?? []) | |
| const outsideCount = notReviewed.filter((n) => OUTSIDE_PROJECT.test(n.reason)).length | |
| const movedCount = envs.reduce((s, e) => s + (e.scope?.movedUnchangedFiles ?? 0), 0) | |
| const unverifiedRaw = envs.flatMap((e) => e.scope?.unverified ?? []) | |
| const unverified = [...new Map(unverifiedRaw.map((u) => [`${u.path}\0${u.reason}`, u])).values()] |
There was a problem hiding this comment.
Thanks for looking at this. It turns out not to happen in the normal case: scope.unverified only ever holds a project's own in-scope models (orchestrate.ts ~1842-1889). The unassigned files every project receives are scoped out as "outside the dbt project" before any engine lane sees them, and the merge already de-duplicates those notReviewed entries. The one edge case is custom *-paths pointing outside a project root (cubic's r4626966425), which we're tracking; de-duplicating unverified by path is cheap extra hardening we may add with it.
🤖 Code Review — OpenCodeReview (Gemini) — No Issues FoundNo supported files changed. |
1acd5d8 to
31cb5c4
Compare
1161598 to
622f430
Compare
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
| scope: { | ||
| baseRef: first.scope?.baseRef, | ||
| baseSha: first.scope?.baseSha, | ||
| baseSource: first.scope?.baseSource, |
There was a problem hiding this comment.
CRITICAL: The rebuilt scope drops headSha, which permanently disables the stale-run guard for multi-project reviews.
postGitHubReview computes const onCurrentHead = !headSha || !commitId || headSha === commitId (post-github.ts:191-192) from env.scope?.headSha (verdict.ts:140). With the field absent, onCurrentHead is always true, so an older --all-projects run that finishes after a newer push can call dismissStale() and dismiss the newer run's CHANGES_REQUESTED block — the exact case the code comment says must never happen: "A run on an older push (its head is no longer the PR's head) never clears the block a newer push's run may have posted."
Every per-project envelope resolves the same head (run.ts:252, headSha at run.ts:486), so carry it through:
| baseSource: first.scope?.baseSource, | |
| headSha: first.scope?.headSha, |
| const env = await reviewPullRequest({ ...optsFor(root), changedFiles: [...own, ...unassigned], prFiles }) | ||
| runs.push({ root, paths: own.map((f) => f.path), env }) | ||
| } | ||
| return mergeProjectEnvelopes(runs, opts.mode) |
There was a problem hiding this comment.
WARNING: Passing the raw --mode flag as modeOverride can soften the verdict that the base policy refused to relax.
reviewPullRequest ignores --mode comment when the base .altimate/review.yml sets mode: gate (run.ts:278-282), so each project envelope is mode: "gate" with a REQUEST_CHANGES verdict. mergeProjectEnvelopes(runs, opts.mode) then sets the merged mode back to "comment" and applyMode(...) rewrites that REQUEST_CHANGES into COMMENT, yielding a non-blocking verdict and exit code. That defeats the documented tighten-only guard (docs: --mode "cannot turn mode: gate into comment").
Each per-project verdict already has its own effective mode applied, so the override should be dropped (or derived from the envelopes) rather than taken from the raw flag:
| return mergeProjectEnvelopes(runs, opts.mode) | |
| return mergeProjectEnvelopes(runs) |
| source: all.map((p) => p.source).reduce((a, b) => (POLICY_SOURCE_RANK[b] > POLICY_SOURCE_RANK[a] ? b : a)), | ||
| hash: createHash("sha256").update(all.map((p) => p.hash).join("\n")).digest("hex").slice(0, 16), | ||
| changedInPr: changed.length ? changed : undefined, | ||
| posting: postings.every((x) => x === postings[0]) ? all[0].posting : undefined, |
There was a problem hiding this comment.
SUGGESTION: When projects' posting policies differ, the merged envelope records none, and postingPolicy() silently falls back to the permissive defaults.
postingPolicy(env) returns env.policy?.posting ?? DEFAULT_POSTING_POLICY (format.ts:388-389): inlineBudget: 10, inlineSeverityFloor: "warning", silentWhenClean: true. If one project opts out of inline comments (inlineBudget: 0) or raises its severity floor, a sibling at defaults makes posting undefined here and the stricter project's settings are discarded for the posted review. Merge to the most restrictive combination (lowest inlineBudget, highest inlineSeverityFloor) instead of dropping the policy.
A mono-repo PR can change several dbt projects; until now the review only covered the project found from `--cwd` and listed every other project's files as "outside the dbt project". - `reviewAllProjects` (new `multi-project.ts`) collects the changed files once, discovers every `dbt_project.yml` at the head side (skipping `dbt_packages/`, `target/`, `node_modules/`), and assigns each file to the deepest project containing it. Each touched project is reviewed sequentially with `reviewPullRequest` (its own policy, layout and manifest auto-discovery; `--manifest` applies only to the project that contains it), seeing only its own files plus files outside every project. - The results merge into one signed envelope: findings concatenated and de-duplicated by id, the most severe verdict and highest tier, summary recomputed, the not-reviewed ledger unioned (other projects' files are no longer "outside"), engine errors combined, AI usage summed, and per-project roots/verdicts/AI states in `scope.projects` / `ai.projects`. - With 0 or 1 touched projects the single-project envelope is returned. - `altimate review --all-projects` wires it into the CLI; the summary adds a "Reviewed N dbt projects" line only when more than one was reviewed. - `reviewPullRequest` gains an optional `prFiles` (the full PR file list when `changedFiles` is a subset); the default path is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…envelope, AI state, path listing - An explicit `--manifest` outside every project (CI `$RUNNER_TEMP/manifest.json`) now goes to the touched project when the PR touches only one; when it can't be attributed (several touched projects, or it belongs to an untouched one) a warning says no project uses it instead of silently degrading to lint-only. - Merged verdict: the worst of each project's post-mode verdict, so a comment-mode project's REQUEST_CHANGES no longer blocks because a sibling uses gate mode. `scope.projects[]` records each project's `mode`. - Merged envelope is built explicitly instead of spreading the first project: `manifestHash` and `policy` are kept only when every project agrees (otherwise `policy` carries the least-trusted source, a combined hash, the union of `changedInPr` and a shared posting policy); `scope.projectRoot` is dropped; each project's `manifestHash` and `policyHash` go in `scope.projects[]`; `tierClassified` is the highest classified tier. - Merged AI state is the worst project state (`error` > `timeout` > `ok` > `skipped`) with that project's reason, so a sibling's failure is not hidden by another project's `ok`; `ai.projects[]` carries each `reason`. - `listFilesAt` / `listWorkingTreeFiles` use `-z`, so `dbt_project.yml` under non-ASCII directory names is discovered (git C-quoted those paths). - `repoRel` rejects only `..` and `../…`, not in-repo names such as `..foo`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
31cb5c4 to
ddc4333
Compare
622f430 to
5470bfe
Compare
Issue for this PR
Part of #1792
Type of change
What does this PR do?
Stacked on #1772.
altimate review --all-projectsreviews every dbt project a PR touches in one run and posts one combined comment. Before, a monorepo PR needed one run per project.What changed
multi-project.ts:dbt_project.ymlon the head side, skippingdbt_packages/,target/andnode_modules/.scope.projects/ai.projects.Results
How did you verify your code works?
bun test test/altimate/review: 754 pass.bunx tsc --noEmit: clean.test/altimate/review-all-projects.test.ts.Cost side effects
None by default:
--all-projectsis opt-in. When used, the review runs once per touched dbt project, so any enabled AI-lane calls and opt-indbt_preparework scale with the number of touched projects (one run per project, as before, but now in one invocation). Single-project PRs are unchanged.Screenshots / recordings
Not a UI change.
Checklist
🤖 Generated with Claude Code
Summary by cubic
Adds
altimate review --all-projectsso a mono-repo PR touching multiple dbt projects is reviewed in one run and gets one combined comment. Previously, each project needed a separate review run.New Features
dbt_project.ymlthe PR touches (git listing uses-z, so non-ASCII directory names work), assigns each changed file to the deepest containing project, and reviews each with its own policy, layout, and manifest.--manifestapplies only to the project containing it; when it can't be attributed to a touched project, a warning says no project uses it.scope.projects/ai.projects. Shared envelope fields (manifestHash,policy) are kept only when every project agrees.Written for commit 5470bfe. Summary will update on new commits.