Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions packages/opencode/src/altimate/review/format.ts
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ export function renderSummary(env: VerdictEnvelope, opts: RenderOptions = {}): s
return ""

const lines: string[] = [REVIEW_MARKER, "", `## ${verdictHeadline(env)}`, ""]
const projects = env.scope?.projects ?? []
if (projects.length > 1) lines.push(`Reviewed ${projects.length} dbt projects: ${projects.map((p) => p.root || ".").join(", ")}`, "")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The project list is unbounded even though this renderer promises a character budget for the posted comment. With enough touched projects, this line alone can exceed 5,000 characters and the later finding-budget loop cannot trim it; cap the displayed roots and retain the full list in scope.projects.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/format.ts, line 76:

<comment>The project list is unbounded even though this renderer promises a character budget for the posted comment. With enough touched projects, this line alone can exceed 5,000 characters and the later finding-budget loop cannot trim it; cap the displayed roots and retain the full list in `scope.projects`.</comment>

<file context>
@@ -72,6 +72,8 @@ export function renderSummary(env: VerdictEnvelope, opts: RenderOptions = {}): s
 
   const lines: string[] = [REVIEW_MARKER, "", `## ${verdictHeadline(env)}`, ""]
+  const projects = env.scope?.projects ?? []
+  if (projects.length > 1) lines.push(`Reviewed ${projects.length} dbt projects: ${projects.map((p) => p.root || ".").join(", ")}`, "")
 
   if (env.summary.degraded) {
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. We'll cap it at RENDER_CAP with "+N more".

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Project roots are repository-controlled strings, but this line inserts them raw into Markdown. A project directory containing a backtick or newline can break the summary's formatting or inject content; render each root with codeSpan (or otherwise escape Markdown) before joining them.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/format.ts, line 76:

<comment>Project roots are repository-controlled strings, but this line inserts them raw into Markdown. A project directory containing a backtick or newline can break the summary's formatting or inject content; render each root with `codeSpan` (or otherwise escape Markdown) before joining them.</comment>

<file context>
@@ -72,6 +72,8 @@ export function renderSummary(env: VerdictEnvelope, opts: RenderOptions = {}): s
 
   const lines: string[] = [REVIEW_MARKER, "", `## ${verdictHeadline(env)}`, ""]
+  const projects = env.scope?.projects ?? []
+  if (projects.length > 1) lines.push(`Reviewed ${projects.length} dbt projects: ${projects.map((p) => p.root || ".").join(", ")}`, "")
 
   if (env.summary.degraded) {
</file context>
Suggested change
if (projects.length > 1) lines.push(`Reviewed ${projects.length} dbt projects: ${projects.map((p) => p.root || ".").join(", ")}`, "")
if (projects.length > 1) lines.push(`Reviewed ${projects.length} dbt projects: ${projects.map((p) => codeSpan(p.root || ".")).join(", ")}`, "")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked (codeSpan).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: The project list is unbounded, unlike the other multi-entry renders (RENDER_CAP = 8 at line 96, LEDGER_MAX_PATHS = 30 at line 283).

On a monorepo with many (or long-named) projects this single fixed line can consume most of SUMMARY_BUDGET_CHARS (5,000), after which fit drops every finding bullet into the "N more finding(s)" notice. Cap the roots shown and append (+N more var); wrapping/escaping the roots like the rest of the renderer (codeSpan, ledger escaping) would also be safer.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as r4626966319; tracked.


if (env.summary.degraded) {
lines.push(
Expand Down
9 changes: 5 additions & 4 deletions packages/opencode/src/altimate/review/git.ts
Original file line number Diff line number Diff line change
Expand Up @@ -171,8 +171,9 @@ export async function gitRepoRoot(cwd: string): Promise<string | undefined> {
/** Every tracked file at `ref` (default HEAD), repo-root relative; undefined when git can't list it. */
export async function listFilesAt(cwd: string, ref?: string): Promise<string[] | undefined> {
try {
const out = await git(["ls-tree", "-r", "--full-tree", "--name-only", ref || "HEAD"], cwd)
return out.split("\n").filter(Boolean)
// -z: paths verbatim (no C-quoting of non-ASCII or special characters).
const out = await git(["ls-tree", "-r", "-z", "--full-tree", "--name-only", ref || "HEAD"], cwd)
return out.split("\0").filter(Boolean)
} catch {
return undefined
}
Expand All @@ -181,8 +182,8 @@ export async function listFilesAt(cwd: string, ref?: string): Promise<string[] |
/** Files in the index plus untracked, non-ignored files (the working tree), repo-root relative; undefined when git fails. */
export async function listWorkingTreeFiles(cwd: string): Promise<string[] | undefined> {
try {
const out = await git(["ls-files", "--full-name", "--cached", "--others", "--exclude-standard", "--", ":(top)"], cwd)
return out.split("\n").filter(Boolean)
const out = await git(["ls-files", "-z", "--full-name", "--cached", "--others", "--exclude-standard", "--", ":(top)"], cwd)
return out.split("\0").filter(Boolean)
} catch {
return undefined
}
Expand Down
1 change: 1 addition & 0 deletions packages/opencode/src/altimate/review/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ export * from "./format"
export * from "./git"
export * from "./runner"
export * from "./run"
export * from "./multi-project"
export * from "./post-github"
export * from "./schema-context"
export * from "./dbt-patterns"
Expand Down
289 changes: 289 additions & 0 deletions packages/opencode/src/altimate/review/multi-project.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,289 @@
import path from "node:path"
import { realpath } from "node:fs/promises"
import { createHash } from "node:crypto"
import { collectChangedFiles, gitRepoRoot, listFilesAt, listWorkingTreeFiles, resolveReviewBase } from "./git"
import { reviewPullRequest, type ReviewPullRequestOptions } from "./run"
import { applyMode, signEnvelope, summarize, VerdictEnvelope, type RiskTier, type Verdict } from "./verdict"
import type { ChangedFile } from "./diff-filter"

/**
* Multi-project review (`altimate review --all-projects`): a PR in a mono-repo
* can change several dbt projects. Each touched project is reviewed on its own
* (its policy, manifest and layout) and the results are merged into one signed
* verdict envelope, so the PR gets one comment.
*/

const PROJECT_FILE = /(^|\/)dbt_project\.ya?ml$/
const IGNORED_DIR = /(^|\/)(dbt_packages|target|node_modules)\//

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: IGNORED_DIR re-implements a subset of the existing skip list and is incomplete.

diff-filter.ts:13-23 (SKIP_DIR_PATTERNS, used by shouldReview) also excludes dbt_modules/, compiled/, run/, logs/ and .venv/. A committed dbt_modules/<pkg>/dbt_project.yml is therefore discovered as a project: its changed files are assigned to that phantom root instead of the enclosing project, they get their own review run (extra AI lane calls), and the summary lists vendored packages as reviewed dbt projects. Export and reuse SKIP_DIR_PATTERNS/shouldReview instead of keeping a second, divergent list.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. We'll reuse SKIP_DIR_PATTERNS from diff-filter.ts so vendored dbt_modules/ projects aren't discovered.


const VERDICT_RANK: Record<Verdict, number> = { APPROVE: 0, COMMENT: 1, REQUEST_CHANGES: 2 }
const TIER_RANK: Record<RiskTier, number> = { trivial: 0, lite: 1, full: 2 }
const OUTSIDE_PROJECT = /^outside the dbt project/
/** Reason for a file outside every reviewed project (the projects are listed in `scope.projects`). */
const OUTSIDE_ALL = "outside the dbt projects reviewed"

/** Repo-relative roots ("" = repo root) of every dbt project at the head side. */
export async function discoverDbtProjects(gitRoot: string, head?: string): Promise<string[]> {
const files = (head ? await listFilesAt(gitRoot, head) : await listWorkingTreeFiles(gitRoot)) ?? []
const roots = new Set<string>()
for (const f of files) {
if (!PROJECT_FILE.test(f) || IGNORED_DIR.test(f)) continue
roots.add(path.posix.dirname(f) === "." ? "" : path.posix.dirname(f))
}
return [...roots].sort()
}

/** The deepest project root containing the repo-relative path `p`, or undefined. */
export function owningProject(p: string, roots: string[]): string | undefined {
let best: string | undefined
for (const r of roots) {
if (r !== "" && !p.startsWith(r + "/")) continue
if (best === undefined || r.length > best.length) best = r
}
return best
}

/** Assign each changed file to the deepest project containing it (new path first, then the old path of a rename). */
export function assignToProjects(
files: ChangedFile[],
roots: string[],
): { byProject: Map<string, ChangedFile[]>; unassigned: ChangedFile[] } {
const byProject = new Map<string, ChangedFile[]>()
const unassigned: ChangedFile[] = []
for (const f of files) {
const root = owningProject(f.path, roots) ?? (f.oldPath ? owningProject(f.oldPath, roots) : undefined)
if (root === undefined) unassigned.push(f)
else byProject.set(root, [...(byProject.get(root) ?? []), f])
Comment on lines +55 to +56

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🟠 MEDIUM] Inside the loop, byProject.set(root, [...(byProject.get(root) ?? []), f]) creates and spreads a new array for every file assigned to a project. This results in O(n^2) time complexity, which can cause significant performance degradation and high memory allocation when reviewing pull requests that touch a large number of files.

You should use .push() on an existing array to optimize this assignment.

Suggested change:

Suggested change
if (root === undefined) unassigned.push(f)
else byProject.set(root, [...(byProject.get(root) ?? []), f])
if (root === undefined) {
unassigned.push(f)
} else {
let filesForRoot = byProject.get(root)
if (!filesForRoot) {
filesForRoot = []
byProject.set(root, filesForRoot)
}
filesForRoot.push(f)
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that's fair. It's quadratic only in the number of changed files per project, so it doesn't matter in practice, but push is just as clear. We'll switch it in a follow-up cleanup.

}
return { byProject, unassigned }
}

async function canonical(p: string): Promise<string> {
try {
return await realpath(p)
} catch {
return path.resolve(p)
}
}

/** `abs` relative to the repo root in POSIX form, or undefined when outside it. */
function repoRel(gitRootReal: string, abs: string): string | undefined {
const rel = path.relative(gitRootReal, abs).replace(/\\/g, "/")
if (rel === ".." || rel.startsWith("../") || path.isAbsolute(rel)) return undefined
return rel
Comment on lines +71 to +73

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🟠 MEDIUM] The condition rel.startsWith("..") incorrectly matches valid file or directory names that begin with .. (e.g., ..foo), wrongly identifying them as being outside the git root. This causes valid files in the repository to be ignored.

Consider checking for exact match or directory separator to correctly identify paths outside the git root.

Suggested change:

Suggested change
const rel = path.relative(gitRootReal, abs).replace(/\\/g, "/")
if (rel.startsWith("..") || path.isAbsolute(rel)) return undefined
return rel
const rel = path.relative(gitRootReal, abs).replace(/\\/g, "/")
if (rel === ".." || rel.startsWith("../") || path.isAbsolute(rel)) return undefined
return rel

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks. A directory named ..foo inside the repo was treated as outside it. Fixed in 622f430: the check is now rel === ".." || rel.startsWith("../").

}

export async function reviewAllProjects(opts: ReviewPullRequestOptions): Promise<VerdictEnvelope> {
const gitRoot = (await gitRepoRoot(opts.cwd)) ?? opts.cwd
const gitRootReal = await canonical(gitRoot)

// Collect the PR's changed files once, against the same merge-base reviewPullRequest resolves.
const collected =
opts.changedFiles ??
(await (async () => {
const resolved = await resolveReviewBase({ cwd: opts.cwd, base: opts.base, head: opts.head })
return collectChangedFiles({ base: resolved.sha, head: opts.head, cwd: opts.cwd })
})())

const roots = await discoverDbtProjects(gitRoot, opts.head)
const { byProject, unassigned } = assignToProjects(collected, roots)
const touched = roots.filter((r) => byProject.has(r))
if (touched.length === 0) return reviewPullRequest(opts)

// An explicit --manifest belongs to the project that contains it; the others auto-discover.
const manifestAbs = opts.manifestPath
? await canonical(path.isAbsolute(opts.manifestPath) ? opts.manifestPath : path.join(opts.cwd, opts.manifestPath))
: undefined
const manifestRel = manifestAbs ? repoRel(gitRootReal, manifestAbs) : undefined
const manifestOwner = manifestRel !== undefined ? owningProject(manifestRel, roots) : undefined
// A manifest outside every project (CI's `$RUNNER_TEMP/manifest.json`) can't be matched by
// path; it goes to the touched project when there is only one.
const manifestFor = (root: string) =>
root === manifestOwner || (manifestOwner === undefined && touched.length === 1) ? manifestAbs : undefined
const optsFor = (root: string): ReviewPullRequestOptions => ({
...opts,
cwd: path.join(gitRootReal, root),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Project roots are discovered from the head tree but each project's cwd is taken from the working tree.

With --head pointing at a ref that is not checked out (a documented CLI mode), a project added or renamed by the PR has no directory at path.join(gitRootReal, root). findDbtProject then returns undefined, so run.ts:312 computes projectRootRel = undefined and scopeToProject treats every file as in scope (the unassigned files, already passed to every project, get re-reviewed by this one) while dbtProjectName/dbtProjectLayout silently degrade. Verify the directory exists (or derive the layout from the ref) and pass the project root explicitly so scoping cannot collapse to undefined.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. This only happens with a --head that isn't checked out.

manifestPath: manifestFor(root),
})

if (touched.length === 1) {
// Invoked from inside the touched project: exactly the single-project review.
const cwdRel = repoRel(gitRootReal, await canonical(opts.cwd))
if (cwdRel !== undefined && owningProject(cwdRel + "/", roots) === touched[0]) return reviewPullRequest(opts)
}
if (manifestAbs && !touched.some((r) => manifestFor(r))) {
const where = manifestOwner === undefined ? "is outside every dbt project" : `belongs to ${manifestOwner || "."}, which this PR does not change`
process.stderr.write(
`⚠️ --manifest ${opts.manifestPath} ${where}, so no reviewed project uses it; each project auto-discovers its own. Place the manifest inside the project it describes (e.g. its target/), or review that project on its own.\n`,
)
}
if (touched.length === 1) return reviewPullRequest(optsFor(touched[0]))

// Each project sees its own files plus the files outside every project (so
// those are listed as not reviewed); other projects' files are left out.
const prFiles = collected.map((f) => f.path)
const runs: ProjectRun[] = []
for (const root of touched) {
const own = byProject.get(root)!
process.stderr.write(`ℹ️ reviewing dbt project ${root || "."} (${own.length} changed file(s))\n`)
const env = await reviewPullRequest({ ...optsFor(root), changedFiles: [...own, ...unassigned], prFiles })

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: scopeToProject supports custom *-paths outside a project root, but this passes every unassigned file to every project run. Shared-path files can therefore be reviewed repeatedly or remain marked outside after one project reviewed them; assign external layout files to their owning projects before constructing runs.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/multi-project.ts, line 118:

<comment>`scopeToProject` supports custom `*-paths` outside a project root, but this passes every unassigned file to every project run. Shared-path files can therefore be reviewed repeatedly or remain marked outside after one project reviewed them; assign external layout files to their owning projects before constructing runs.</comment>

<file context>
@@ -0,0 +1,226 @@
+  for (const root of touched) {
+    const own = byProject.get(root)!
+    process.stderr.write(`ℹ️  reviewing dbt project ${root || "."} (${own.length} changed file(s))\n`)
+    const env = await reviewPullRequest({ ...optsFor(root), changedFiles: [...own, ...unassigned], prFiles })
+    runs.push({ root, paths: own.map((f) => f.path), env })
+  }
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. In the normal layout they're scoped out as outside each project and de-duplicated on merge; custom *-paths outside a root is the edge case.

runs.push({ root, paths: own.map((f) => f.path), env })
}
Comment on lines +125 to +131

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🟠 MEDIUM] The await reviewPullRequest(...) calls are executed sequentially inside a for...of loop for each touched project. Since reviews involve network I/O or heavy computation, sequential execution unnecessarily blocks concurrency, leading to longer overall wait times for multi-project pull requests.

Consider using Promise.all to execute the project reviews concurrently.

Suggested change:

Suggested change
const runs: ProjectRun[] = []
for (const root of touched) {
const own = byProject.get(root)!
process.stderr.write(`ℹ️ reviewing dbt project ${root || "."} (${own.length} changed file(s))\n`)
const env = await reviewPullRequest({ ...optsFor(root), changedFiles: [...own, ...unassigned], prFiles })
runs.push({ root, paths: own.map((f) => f.path), env })
}
const runs: ProjectRun[] = await Promise.all(
touched.map(async (root) => {
const own = byProject.get(root)!
process.stderr.write(`ℹ️ reviewing dbt project ${root || "."} (${own.length} changed file(s))\n`)
const env = await reviewPullRequest({ ...optsFor(root), changedFiles: [...own, ...unassigned], prFiles })
return { root, paths: own.map((f) => f.path), env }
})
)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion. We're keeping these sequential on purpose. Each project review starts its own AI lane (up to two concurrent model calls, plus the optional verifier's tool loop), so running projects in parallel would multiply concurrent LLM calls and rate-limit pressure by the number of projects. They also share the native dispatcher, and their stderr progress lines would interleave into an unreadable CI log. Wall time is usually dominated by the model calls, which already run concurrently within each project.

return mergeProjectEnvelopes(runs, opts.mode)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Passing the raw --mode flag as modeOverride can soften the verdict that the base policy refused to relax.

reviewPullRequest ignores --mode comment when the base .altimate/review.yml sets mode: gate (run.ts:278-282), so each project envelope is mode: "gate" with a REQUEST_CHANGES verdict. mergeProjectEnvelopes(runs, opts.mode) then sets the merged mode back to "comment" and applyMode(...) rewrites that REQUEST_CHANGES into COMMENT, yielding a non-blocking verdict and exit code. That defeats the documented tighten-only guard (docs: --mode "cannot turn mode: gate into comment").

Each per-project verdict already has its own effective mode applied, so the override should be dropped (or derived from the envelopes) rather than taken from the raw flag:

Suggested change
return mergeProjectEnvelopes(runs, opts.mode)
return mergeProjectEnvelopes(runs)

}

/** One project's review: its repo-relative root, the changed files assigned to it, and its envelope. */
export interface ProjectRun {
root: string
paths: string[]
env: VerdictEnvelope
}

/** Merge per-project envelopes into one signed envelope (see reviewAllProjects). */
export function mergeProjectEnvelopes(
runs: ProjectRun[],
modeOverride?: VerdictEnvelope["mode"],
): VerdictEnvelope {
const first = runs[0].env
const envs = runs.map((r) => r.env)

const seenIds = new Set<string>()
const findings = envs.flatMap((e) => e.findings).filter((f) => !seenIds.has(f.id) && !!seenIds.add(f.id))

const mode = modeOverride ?? (envs.some((e) => e.mode === "gate") ? "gate" : first.mode)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Mixed per-project modes can turn a non-blocking project's findings into a blocking verdict.

mode is gate if any project uses gate, and the merged verdict is applyMode(max(idealVerdict), mode). A project whose own policy is the default comment already softened its REQUEST_CHANGES to COMMENT, but its idealVerdict is still REQUEST_CHANGES, so a sibling project configured gate promotes it back to a blocking REQUEST_CHANGES. Merge the post-mode per-project verdicts (envs.map((e) => e.verdict)) and keep applyMode out of the merge. The mergeProjectEnvelopes test only covers the both-gate case, so this is untested.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 622f430: the merged verdict is the worst of the projects' post-mode verdicts, so a comment-mode project's REQUEST_CHANGES no longer blocks because a sibling uses gate mode. Each project's mode is recorded in scope.projects, with a mixed-mode test.

const worst = (vs: Verdict[]) => vs.reduce((a, b) => (VERDICT_RANK[b] > VERDICT_RANK[a] ? b : a))
const idealVerdict = worst(envs.map((e) => e.idealVerdict))
// Each project's verdict already has its own mode applied: a comment-mode project never blocks.
const verdict = applyMode(worst(envs.map((e) => e.verdict)), mode)
const tier = envs.map((e) => e.tier).reduce((a, b) => (TIER_RANK[b] > TIER_RANK[a] ? b : a))
const tierReasons = [...new Set(envs.flatMap((e) => e.tierReasons ?? []))]

// A path some project reviewed is not "outside the dbt project" for the PR.
const reviewed = new Set(runs.flatMap((r) => r.paths))
const notReviewed: Array<{ path: string; reason: string }> = []
const seenEntry = new Set<string>()
for (const e of envs) {
for (const n of e.scope?.notReviewed ?? []) {
const outside = OUTSIDE_PROJECT.test(n.reason)
if (outside && reviewed.has(n.path)) continue
const key = outside ? `outside\0${n.path}` : `${n.path}\0${n.reason}`
if (seenEntry.has(key)) continue
seenEntry.add(key)
notReviewed.push(outside ? { path: n.path, reason: OUTSIDE_ALL } : n)
}
}
const outsideCount = notReviewed.filter((n) => OUTSIDE_PROJECT.test(n.reason)).length
const movedCount = envs.reduce((s, e) => s + (e.scope?.movedUnchangedFiles ?? 0), 0)
const unverified = envs.flatMap((e) => e.scope?.unverified ?? [])
Comment on lines +175 to +177

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🟠 MEDIUM] Unassigned files are passed to every project's review. If they fail verification, they will be returned in each project's envelope and subsequently duplicated by envs.flatMap without deduplication. This causes the final envelope to contain duplicated unverified file entries.

Consider deduplicating the unverified entries based on their path and reason.

Suggested change:

Suggested change
const outsideCount = notReviewed.filter((n) => OUTSIDE_PROJECT.test(n.reason)).length
const movedCount = envs.reduce((s, e) => s + (e.scope?.movedUnchangedFiles ?? 0), 0)
const unverified = envs.flatMap((e) => e.scope?.unverified ?? [])
const outsideCount = notReviewed.filter((n) => OUTSIDE_PROJECT.test(n.reason)).length
const movedCount = envs.reduce((s, e) => s + (e.scope?.movedUnchangedFiles ?? 0), 0)
const unverifiedRaw = envs.flatMap((e) => e.scope?.unverified ?? [])
const unverified = [...new Map(unverifiedRaw.map((u) => [`${u.path}\0${u.reason}`, u])).values()]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking at this. It turns out not to happen in the normal case: scope.unverified only ever holds a project's own in-scope models (orchestrate.ts ~1842-1889). The unassigned files every project receives are scoped out as "outside the dbt project" before any engine lane sees them, and the merge already de-duplicates those notReviewed entries. The one edge case is custom *-paths pointing outside a project root (cubic's r4626966425), which we're tracking; de-duplicating unverified by path is cheap extra hardening we may add with it.


const engineErrors = new Map<string, { method: string; reason: string; count: number }>()
for (const err of envs.flatMap((e) => e.engineErrors ?? [])) {
const key = `${err.method}\0${err.reason}`
const prev = engineErrors.get(key)
engineErrors.set(key, prev ? { ...prev, count: prev.count + err.count } : { ...err })
}

// Per-project values are recorded in `scope.projects`; the top level carries one only when every project agrees.
const shared = <T>(vals: T[]): T | undefined =>
vals.every((v) => JSON.stringify(v) === JSON.stringify(vals[0])) ? vals[0] : undefined
const classified = envs.map((e) => e.tierClassified)
const merged: VerdictEnvelope = {
version: first.version,
verdict,
idealVerdict,
mode,
tier,
tierReasons: tierReasons.length ? tierReasons : undefined,
tierForced: first.tierForced,
tierClassified: classified.every((t) => t !== undefined)
? classified.reduce((a, b) => (TIER_RANK[b!] > TIER_RANK[a!] ? b : a))
: undefined,
findings,
summary: summarize(findings, envs.some((e) => e.summary.degraded)),
engine: first.engine,
manifestHash: shared(envs.map((e) => e.manifestHash)),
staleManifest: envs.some((e) => e.staleManifest) ? true : undefined,
engineErrors: engineErrors.size ? [...engineErrors.values()] : undefined,
policy: mergePolicy(envs),
scope: {
baseRef: first.scope?.baseRef,
baseSha: first.scope?.baseSha,
baseSource: first.scope?.baseSource,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CRITICAL: The rebuilt scope drops headSha, which permanently disables the stale-run guard for multi-project reviews.

postGitHubReview computes const onCurrentHead = !headSha || !commitId || headSha === commitId (post-github.ts:191-192) from env.scope?.headSha (verdict.ts:140). With the field absent, onCurrentHead is always true, so an older --all-projects run that finishes after a newer push can call dismissStale() and dismiss the newer run's CHANGES_REQUESTED block — the exact case the code comment says must never happen: "A run on an older push (its head is no longer the PR's head) never clears the block a newer push's run may have posted."

Every per-project envelope resolves the same head (run.ts:252, headSha at run.ts:486), so carry it through:

Suggested change
baseSource: first.scope?.baseSource,
headSha: first.scope?.headSha,

outOfScopeFiles: outsideCount || undefined,
movedUnchangedFiles: movedCount || undefined,
unverified: unverified.length ? unverified : undefined,
notReviewed,
projects: runs.map((r) => ({
root: r.root,
files: r.paths.length,
verdict: r.env.verdict,
mode: r.env.mode,
manifestHash: r.env.manifestHash,
policyHash: r.env.policy?.hash,
})),
},
ai: mergeAi(runs),
generatedAt: first.generatedAt,
}
return signEnvelope(VerdictEnvelope.parse(merged))
}

/** Least-trusted source first: a policy the PR controls outranks the defaults, which outrank the base branch's. */
const POLICY_SOURCE_RANK = { base: 0, default: 1, working_tree: 2 } as const

/** One project's policy when all agree; otherwise the least-trusted source, a hash of every
* project's hash, every setting the PR changes, and the posting policy only if shared. */
function mergePolicy(envs: VerdictEnvelope[]): VerdictEnvelope["policy"] {
const policies = envs.map((e) => e.policy)
if (policies.some((p) => !p)) return undefined
const all = policies as NonNullable<VerdictEnvelope["policy"]>[]
if (all.every((p) => JSON.stringify(p) === JSON.stringify(all[0]))) return all[0]
const changed = [...new Set(all.flatMap((p) => p.changedInPr ?? []))]
const postings = all.map((p) => JSON.stringify(p.posting))
return {
source: all.map((p) => p.source).reduce((a, b) => (POLICY_SOURCE_RANK[b] > POLICY_SOURCE_RANK[a] ? b : a)),
hash: createHash("sha256").update(all.map((p) => p.hash).join("\n")).digest("hex").slice(0, 16),
changedInPr: changed.length ? changed : undefined,
posting: postings.every((x) => x === postings[0]) ? all[0].posting : undefined,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: When projects' posting policies differ, the merged envelope records none, and postingPolicy() silently falls back to the permissive defaults.

postingPolicy(env) returns env.policy?.posting ?? DEFAULT_POSTING_POLICY (format.ts:388-389): inlineBudget: 10, inlineSeverityFloor: "warning", silentWhenClean: true. If one project opts out of inline comments (inlineBudget: 0) or raises its severity floor, a sibling at defaults makes posting undefined here and the stricter project's settings are discarded for the posted review. Merge to the most restrictive combination (lowest inlineBudget, highest inlineSeverityFloor) instead of dropping the policy.

}
}

function mergeAi(runs: Array<{ root: string; env: VerdictEnvelope }>): VerdictEnvelope["ai"] {
const lanes = runs.filter((r) => r.env.ai)
if (!lanes.length) return undefined
const first = lanes[0].env.ai!
const ok = lanes.find((r) => r.env.ai!.state === "ok")?.env.ai
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
const sum = (pick: (a: NonNullable<VerdictEnvelope["ai"]>) => number | undefined) => {
const vals = lanes.map((r) => pick(r.env.ai!)).filter((v): v is number => v !== undefined)
return vals.length ? vals.reduce((a, b) => a + b, 0) : undefined
}
const tokenLanes = lanes.map((r) => r.env.ai!.tokens).filter((t) => t !== undefined)
const tokens = tokenLanes.length
? tokenLanes.reduce((a, b) => ({
input: a.input + b.input,
inputTotal: a.inputTotal + b.inputTotal,
output: a.output + b.output,
reasoning: a.reasoning + b.reasoning,
cacheRead: a.cacheRead + b.cacheRead,
}))
: undefined
// The worst state wins (a sibling's error or timeout is not hidden by another project's ok),
// with that project's reason; `skipped` (nothing to send) ranks below `ok`.
const worst = lanes.reduce((a, b) => (AI_STATE_RANK[b.env.ai!.state] > AI_STATE_RANK[a.env.ai!.state] ? b : a))
const state = worst.env.ai!.state
const reason = worst.env.ai!.reason
const base = ok ?? first

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: base = ok ?? first reports the merged AI lane as ok whenever any project succeeded, dropping a sibling project's error/timeout and its reason.

With proj_a error and proj_b ok, the merged envelope says state: "ok" with no reason; ai.projects[] keeps only state/calls, so the failure is unrecoverable from the signed envelope (the top-level events and format notices no longer show the AI lane degraded). Promote the worst state (error > timeout > skipped/ok) and keep the failing lane's reason, or add a per-project reason.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as r4626966388; fixed in 622f430.

return {
...base,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The merged AI object retains input and verifier data from only one project while aggregating the top-level usage. With AI coverage or verification enabled, the signed result underreports the other projects' inputs and verifier usage; union or aggregate these fields, or expose them per project.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/multi-project.ts, line 218:

<comment>The merged AI object retains `input` and `verifier` data from only one project while aggregating the top-level usage. With AI coverage or verification enabled, the signed result underreports the other projects' inputs and verifier usage; union or aggregate these fields, or expose them per project.</comment>

<file context>
@@ -0,0 +1,226 @@
+    : undefined
+  const base = ok ?? first
+  return {
+    ...base,
+    model: base.model ?? first.model,
+    calls: sum((a) => a.calls),
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. Usage is summed correctly; the per-file input records and verifier detail should be unioned or kept per project.

state,
reason: state === "ok" ? base.reason : reason && lanes.length > 1 ? `${worst.root || "."}: ${reason}` : reason,
model: base.model ?? first.model,
calls: sum((a) => a.calls),
tokens,
cost: sum((a) => a.cost),
durationMs: sum((a) => a.durationMs),
projects: lanes.map((r) => ({ root: r.root, state: r.env.ai!.state, reason: r.env.ai!.reason, calls: r.env.ai!.calls })),
}
}

const AI_STATE_RANK: Record<NonNullable<VerdictEnvelope["ai"]>["state"], number> = { skipped: 0, ok: 1, timeout: 2, error: 3 }
4 changes: 3 additions & 1 deletion packages/opencode/src/altimate/review/run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ export interface ReviewPullRequestOptions {
/** G2 — [EXPERIMENTAL] bypass the tier classifier with the supplied tier.
* Envelope carries `tierForced: true` when used. Debug / bench only. */
forceTier?: "trivial" | "lite" | "full"
/** Every file the PR changes, when `changedFiles` is a subset of it (multi-project review). */
prFiles?: string[]
}

/** dbt adapter_type → core SQL dialect. Mostly identity; a few aliases. */
Expand Down Expand Up @@ -461,7 +463,7 @@ export async function reviewPullRequest(opts: ReviewPullRequestOptions): Promise
const specs = projectGrepSpecs(projectRootRel, dirs, glob)
return specs ? grepFixedAt(gitRoot, opts.head, pattern, specs) : Promise.resolve(undefined)
},
prFiles: collected.map((f) => f.path),
prFiles: opts.prFiles ?? collected.map((f) => f.path),
prTitle: opts.prTitle,
prBody: opts.prBody,
explainTier: opts.explainTier,
Expand Down
29 changes: 28 additions & 1 deletion packages/opencode/src/altimate/review/verdict.ts
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,21 @@ export const ReviewScope = z.object({
* project, moved, not a dbt file, no check at this tier, no manifest), with
* the reason; also repeats the `unverified` entries. */
notReviewed: z.array(z.object({ path: z.string(), reason: z.string() })).default([]),
/** Multi-project review (`--all-projects`): each dbt project reviewed, its
* changed-file count and its own verdict. */
projects: z
.array(
z.object({
root: z.string(),
files: z.number().int().nonnegative(),
verdict: Verdict,
mode: ReviewMode.optional(),
/** The project's own manifest and policy (the top level keeps one only when all agree). */
manifestHash: z.string().optional(),
policyHash: z.string().optional(),
}),
)
.optional(),
})
export type ReviewScope = z.infer<typeof ReviewScope>

Expand Down Expand Up @@ -222,6 +237,17 @@ export const VerdictEnvelope = z.object({
/** Estimated USD cost from the model's price table. */
cost: z.number().nonnegative().optional(),
calls: z.number().int().nonnegative().optional(),
/** Multi-project review (`--all-projects`): each project's AI lane state. */
projects: z
.array(
z.object({
root: z.string(),
state: z.string(),
reason: z.string().optional(),
calls: z.number().int().nonnegative().optional(),
}),
)
.optional(),
/** Generator findings replayed from an earlier envelope: tokens, cost and calls are that run's. */
replayed: z.boolean().optional(),
/** The replayed envelope did not list every generator finding. */
Expand Down Expand Up @@ -379,7 +405,8 @@ export interface BuildEnvelopeInput {
engineErrors?: VerdictEnvelope["engineErrors"]
}

function summarize(findings: Finding[], degraded: boolean): VerdictEnvelope["summary"] {
/** Severity tally of `findings` for the envelope summary. */
export function summarize(findings: Finding[], degraded: boolean): VerdictEnvelope["summary"] {
const tally: Record<Severity, number> = { critical: 0, warning: 0, suggestion: 0 }
for (const f of findings) tally[f.severity]++
return { critical: tally.critical, warning: tally.warning, suggestion: tally.suggestion, degraded }
Expand Down
Loading
Loading