Repository navigation
fix(review): an honest PR comment: real lint-only banner, AI lane status, PR intent, project_dir - #1720
fix(review): an honest PR comment: real lint-only banner, AI lane status, PR intent, project_dir#1720RaghwendraSingh wants to merge 3 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
There was a problem hiding this comment.
6 issues found across 11 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/src/altimate/telemetry/index.ts">
<violation number="1" location="packages/opencode/src/altimate/telemetry/index.ts:1606">
P3: `ai_state` is typed as `string`, but the AI lane only produces `ok | skipped | timeout | error`. Narrow the type to that union so the telemetry contract matches the envelope schema and typos are caught at compile time.</violation>
</file>
<file name="packages/opencode/test/altimate/review-ci-honesty.test.ts">
<violation number="1" location="packages/opencode/test/altimate/review-ci-honesty.test.ts:157">
P3: This test creates a temp directory and never removes it, so every run leaves a `review-event-*` directory behind. Remove it in a `finally` block (`fs.rm(dir, { recursive: true, force: true })`) or use the repo's tmpdir fixture with auto cleanup.</violation>
</file>
<file name="packages/opencode/src/altimate/review/ai-review.ts">
<violation number="1" location="packages/opencode/src/altimate/review/ai-review.ts:66">
P2: `safeReason` still exposes credential formats that do not use the four dashed prefixes, so provider failures can disclose secrets in PR comments. Reuse the repository's credential/error redactor or mask credential-labeled values and credential-bearing URLs before truncating the reason.
(Based on your team's feedback about masking credential-stem error values.) [a5c5a10f-6a4a-4e42-8c2e-f8e01b48683d]</violation>
</file>
<file name="packages/opencode/src/altimate/review/orchestrate.ts">
<violation number="1" location="packages/opencode/src/altimate/review/orchestrate.ts:1418">
P2: Empty-scope reviews now emit an AI `skipped` status and coverage note even though there is no review scope or AI lane to skip. Guard the AI-status block on a non-empty review scope so empty reviews leave `ai` undefined.</violation>
</file>
<file name="github/review/action.yml">
<violation number="1" location="github/review/action.yml:189">
P2: A misspelled `project_dir` silently broadens a monorepo review to every changed file. Validate that the directory contains `dbt_project.yml`/`.yaml` before adding `--cwd`, or make the review fail closed when project discovery fails.</violation>
</file>
<file name="packages/opencode/src/altimate/review/format.ts">
<violation number="1" location="packages/opencode/src/altimate/review/format.ts:144">
P2: The PR comment does not report the AI lane’s available model and duration: successful runs show only the model, while failed or timed-out runs show only state and reason. Include model and duration in the rendered AI status.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| return raw | ||
| .replace(/\s+/g, " ") | ||
| .replace(/(bearer\s+)[^\s]+/gi, "$1***") | ||
| .replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***") |
There was a problem hiding this comment.
P2: safeReason still exposes credential formats that do not use the four dashed prefixes, so provider failures can disclose secrets in PR comments. Reuse the repository's credential/error redactor or mask credential-labeled values and credential-bearing URLs before truncating the reason.
(Based on your team's feedback about masking credential-stem error values.) [a5c5a10f-6a4a-4e42-8c2e-f8e01b48683d]
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 66:
<comment>`safeReason` still exposes credential formats that do not use the four dashed prefixes, so provider failures can disclose secrets in PR comments. Reuse the repository's credential/error redactor or mask credential-labeled values and credential-bearing URLs before truncating the reason.
(Based on your team's feedback about masking credential-stem error values.) [a5c5a10f-6a4a-4e42-8c2e-f8e01b48683d]</comment>
<file context>
@@ -31,6 +32,39 @@ export interface AiReviewInput {
+ return raw
+ .replace(/\s+/g, " ")
+ .replace(/(bearer\s+)[^\s]+/gi, "$1***")
+ .replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***")
+ .slice(0, 200)
}
</file context>
There was a problem hiding this comment.
Fixed in 4d9a88a. safeReason now runs the shared redactSecrets masker (Authorization headers, Basic/Bearer values, key=value secrets, URL userinfo, provider-prefixed tokens) before collapsing whitespace. Tests added.
| // headless-without-model) — the review degrades to deterministic-only. | ||
| if (lanes.has("ai_review") && input.aiReview) { | ||
| let aiStatus: AiLaneStatus | undefined | ||
| if (!lanes.has("ai_review")) { |
There was a problem hiding this comment.
P2: Empty-scope reviews now emit an AI skipped status and coverage note even though there is no review scope or AI lane to skip. Guard the AI-status block on a non-empty review scope so empty reviews leave ai undefined.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 1418:
<comment>Empty-scope reviews now emit an AI `skipped` status and coverage note even though there is no review scope or AI lane to skip. Guard the AI-status block on a non-empty review scope so empty reviews leave `ai` undefined.</comment>
<file context>
@@ -1410,7 +1414,12 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
// headless-without-model) — the review degrades to deterministic-only.
- if (lanes.has("ai_review") && input.aiReview) {
+ let aiStatus: AiLaneStatus | undefined
+ if (!lanes.has("ai_review")) {
+ aiStatus = { state: "skipped", reason: `not part of the ${tier} tier` }
+ } else if (!input.aiReview) {
</file context>
There was a problem hiding this comment.
Not changing: the note is accurate ("no changed model SQL to review") and tells the reader why the lane didn't run.
| ALTIMATE_REVIEW_SIGNING_KEY: ${{ inputs.signing_key }} | ||
| run: | | ||
| args=(--mode "$IN_MODE" --manifest "$IN_MANIFEST" --severity "$IN_SEVERITY") | ||
| [[ -n "$IN_PROJECT_DIR" && "$IN_PROJECT_DIR" != "." ]] && args+=(--cwd "$IN_PROJECT_DIR") |
There was a problem hiding this comment.
P2: A misspelled project_dir silently broadens a monorepo review to every changed file. Validate that the directory contains dbt_project.yml/.yaml before adding --cwd, or make the review fail closed when project discovery fails.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At github/review/action.yml, line 189:
<comment>A misspelled `project_dir` silently broadens a monorepo review to every changed file. Validate that the directory contains `dbt_project.yml`/`.yaml` before adding `--cwd`, or make the review fail closed when project discovery fails.</comment>
<file context>
@@ -181,6 +186,7 @@ runs:
ALTIMATE_REVIEW_SIGNING_KEY: ${{ inputs.signing_key }}
run: |
args=(--mode "$IN_MODE" --manifest "$IN_MANIFEST" --severity "$IN_SEVERITY")
+ [[ -n "$IN_PROJECT_DIR" && "$IN_PROJECT_DIR" != "." ]] && args+=(--cwd "$IN_PROJECT_DIR")
[[ -n "$IN_BASE" ]] && args+=(--base "$IN_BASE")
[[ -n "$IN_HEAD" ]] && args+=(--head "$IN_HEAD")
</file context>
| [[ -n "$IN_PROJECT_DIR" && "$IN_PROJECT_DIR" != "." ]] && args+=(--cwd "$IN_PROJECT_DIR") | |
| if [[ -n "$IN_PROJECT_DIR" && "$IN_PROJECT_DIR" != "." ]]; then | |
| project_path="${GITHUB_WORKSPACE:-$PWD}/$IN_PROJECT_DIR" | |
| if [[ ! -f "$project_path/dbt_project.yml" && ! -f "$project_path/dbt_project.yaml" ]]; then | |
| echo "::error::project_dir must contain dbt_project.yml or dbt_project.yaml" | |
| exit 1 | |
| fi | |
| args+=(--cwd "$IN_PROJECT_DIR") | |
| fi |
There was a problem hiding this comment.
Follow-up: low priority, tracked. The action should check that dbt_project.y(a)ml exists before --cwd.
| notes.push(`${env.scope.outOfScopeFiles} changed file(s) outside the dbt project were not reviewed.`) | ||
| if (env.ai && env.ai.state !== "ok") { | ||
| const what = { skipped: "skipped", timeout: "timed out", error: "failed" }[env.ai.state] | ||
| notes.push(`Advisory AI review ${what}${env.ai.reason ? `: ${env.ai.reason}` : ""}.`) |
There was a problem hiding this comment.
P2: The PR comment does not report the AI lane’s available model and duration: successful runs show only the model, while failed or timed-out runs show only state and reason. Include model and duration in the rendered AI status.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/format.ts, line 144:
<comment>The PR comment does not report the AI lane’s available model and duration: successful runs show only the model, while failed or timed-out runs show only state and reason. Include model and duration in the rendered AI status.</comment>
<file context>
@@ -110,11 +111,41 @@ export function renderSummary(env: VerdictEnvelope): string {
+ notes.push(`${env.scope.outOfScopeFiles} changed file(s) outside the dbt project were not reviewed.`)
+ if (env.ai && env.ai.state !== "ok") {
+ const what = { skipped: "skipped", timeout: "timed out", error: "failed" }[env.ai.state]
+ notes.push(`Advisory AI review ${what}${env.ai.reason ? `: ${env.ai.reason}` : ""}.`)
+ }
+ return notes.length ? [...notes.map((n) => `> ℹ️ ${n}`), ""] : []
</file context>
There was a problem hiding this comment.
Not changing: the model is already shown in the footer; duration is in the envelope and telemetry and isn't useful in the comment.
| /** Models the engine could not verify (no compiled SQL). */ | ||
| unverified_models?: number | ||
| /** Advisory AI lane outcome: ok | skipped | timeout | error. */ | ||
| ai_state?: string |
There was a problem hiding this comment.
P3: ai_state is typed as string, but the AI lane only produces ok | skipped | timeout | error. Narrow the type to that union so the telemetry contract matches the envelope schema and typos are caught at compile time.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/telemetry/index.ts, line 1606:
<comment>`ai_state` is typed as `string`, but the AI lane only produces `ok | skipped | timeout | error`. Narrow the type to that union so the telemetry contract matches the envelope schema and typos are caught at compile time.</comment>
<file context>
@@ -1594,10 +1594,20 @@ export namespace Telemetry {
+ /** Models the engine could not verify (no compiled SQL). */
+ unverified_models?: number
+ /** Advisory AI lane outcome: ok | skipped | timeout | error. */
+ ai_state?: string
+ ai_duration_ms?: number
+ /** Where the review base came from (flag, github_event, default_branch, …). */
</file context>
| ai_state?: string | |
| ai_state?: "ok" | "skipped" | "timeout" | "error" |
There was a problem hiding this comment.
Not changing: telemetry fields are kept as strings for schema stability. The values come from the AiLaneState union at the source.
|
|
||
| describe("PR title and description reach the intent check", () => { | ||
| test("GitHub: read from the pull_request event", async () => { | ||
| const dir = await fs.mkdtemp(path.join(tmpdir(), "review-event-")) |
There was a problem hiding this comment.
P3: This test creates a temp directory and never removes it, so every run leaves a review-event-* directory behind. Remove it in a finally block (fs.rm(dir, { recursive: true, force: true })) or use the repo's tmpdir fixture with auto cleanup.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/review-ci-honesty.test.ts, line 157:
<comment>This test creates a temp directory and never removes it, so every run leaves a `review-event-*` directory behind. Remove it in a `finally` block (`fs.rm(dir, { recursive: true, force: true })`) or use the repo's tmpdir fixture with auto cleanup.</comment>
<file context>
@@ -0,0 +1,188 @@
+
+describe("PR title and description reach the intent check", () => {
+ test("GitHub: read from the pull_request event", async () => {
+ const dir = await fs.mkdtemp(path.join(tmpdir(), "review-event-"))
+ const ev = path.join(dir, "event.json")
+ await fs.writeFile(ev, JSON.stringify({ pull_request: { title: "Add fct_orders", body: "## Spec\n- grain: order_id" } }))
</file context>
There was a problem hiding this comment.
Not changing in this PR: test-hygiene nit, to be cleaned up with the fixture.
| if (env.summary.degraded) { | ||
| lines.push( | ||
| "> ⚙️ **Lint-only run** — no dbt manifest/warehouse was available, so lineage, equivalence and", | ||
| "> ⚙️ **Lint-only run** — no changed model resolved against a dbt manifest, so lineage, equivalence and", |
There was a problem hiding this comment.
WARNING: The banner now asserts a cause that is only one of the two ways summary.degraded becomes true.
orchestrate.ts:1120 sets runDegraded = modelFiles.length > 0 ? !anyManifest : reviewable.length === 0, so an empty reviewable set also renders this banner. That happens for a PR whose only changed files are non-dbt (.md, .json, … are dropped by filterChangedFiles → shouldReview), and there the advice "Wire manifest_path" cannot change anything. The comment then misstates why checks were skipped, which is the class of problem this PR is fixing. Branch the wording on the actual cause, or give the empty-reviewable case its own note.
There was a problem hiding this comment.
Fixed later in #1721 (runDegraded = modelFiles.length > 0 && !anyManifest).
| const head = unverified.slice(0, 5).map((u) => `\`${u.path}\``).join(", ") | ||
| const more = unverified.length > 5 ? ` and ${unverified.length - 5} more` : "" | ||
| notes.push( | ||
| `${unverified.length} model(s) not verified by the engine — ${unverified[0].reason}: ${head}${more}. ` + |
There was a problem hiding this comment.
SUGGESTION: Only the first model's reason is reported for the whole set.
unverified entries carry two distinct reasons (orchestrate.ts:1108 head-side vs 1110 base-side), so a PR that mixes gaps gets every path labelled with unverified[0].reason. With unverified[0] a base-side entry, the line even tells the user to run dbt compile for models that were only missing target-base. Group by reason or use a neutral phrase ("no compiled SQL for the head or base").
There was a problem hiding this comment.
Fixed later in #1723 (grouped by reason).
| notes.push(`${env.scope.outOfScopeFiles} changed file(s) outside the dbt project were not reviewed.`) | ||
| if (env.ai && env.ai.state !== "ok") { | ||
| const what = { skipped: "skipped", timeout: "timed out", error: "failed" }[env.ai.state] | ||
| notes.push(`Advisory AI review ${what}${env.ai.reason ? `: ${env.ai.reason}` : ""}.`) |
There was a problem hiding this comment.
SUGGESTION: env.ai.reason is interpolated into a > ℹ️ blockquote verbatim.
safeReason in ai-review.ts collapses whitespace, but the orchestrate catch path does not (see the comment there), so a multi-line error message breaks out of the blockquote and can inject arbitrary markdown into the posted PR comment. The footer at lines 114-115 has the same shape: env.scope.baseRef and env.ai.model are placed in single-backtick spans without the fence-length logic this file already applies to tierReasons (lines 64-74, added for exactly this reason). Reuse that escaping for these values.
There was a problem hiding this comment.
safeReason already collapses whitespace for lane-produced reasons. The remaining catch-path reason and backtick escaping are tracked as a low-priority follow-up.
| if (lanes.has("ai_review") && input.aiReview) { | ||
| let aiStatus: AiLaneStatus | undefined | ||
| if (!lanes.has("ai_review")) { | ||
| aiStatus = { state: "skipped", reason: `not part of the ${tier} tier` } |
There was a problem hiding this comment.
SUGGESTION: This reason can be false.
lanes is input.config.reviewers.length ? input.config.reviewers : TIER_LANES[tier] (line 1175), so the branch also fires when a user restricts lanes with reviewers: in .altimate/review.yml. A full-tier run with reviewers: ["sql_quality"] then reports "not part of the full tier" in the PR comment, which is a misstatement of the same kind the PR set out to remove. Distinguish "excluded by the reviewers config" from "not part of the tier".
There was a problem hiding this comment.
Follow-up: low priority, tracked. The reason should branch on config.reviewers.length when reviewers: excludes ai_review.
| // A failed AI lane must never fail the review. | ||
| } catch (err) { | ||
| // A failed AI lane must never fail the review — but it must say so. | ||
| aiStatus = { state: "error", reason: String((err as Error)?.message ?? err).slice(0, 200) } |
There was a problem hiding this comment.
SUGGESTION: This reason reaches the signed envelope and the PR comment unmasked.
Unlike safeReason in ai-review.ts, it is not whitespace-collapsed, not credential-masked, and only sliced to 200 chars — while AiLaneStatus.reason is documented as "never contains credentials". runAiReview does not throw today, so this is the fallback for any injected lane, but the contract should hold on every path: reuse the masking helper (or sanitize the message here) rather than embedding the raw error.
There was a problem hiding this comment.
Follow-up: low priority, tracked (same as the cubic thread on this line).
| - name: Compile the base into target-base/ | ||
| run: | | ||
| git worktree add ../base "origin/${{ github.base_ref }}" | ||
| (cd ../base && dbt deps && dbt compile --target-path "$GITHUB_WORKSPACE/target-base") |
There was a problem hiding this comment.
SUGGESTION: This snippet does not work in the monorepo layout the paragraph below documents.
It writes target-base/ to the repository root and runs dbt compile from the worktree root. The resolver reads <dbt root>/target-base/compiled/<project>/… (compiled.ts:117, with opts.cwd = dbtRoot from run.ts:363), and dbt compile needs dbt_project.yml, so with project_dir: dbt both steps fail or land where the review never looks — equivalence and column-breakage silently stay off. Add a monorepo variant, e.g. (cd ../base/dbt && dbt deps && dbt compile --target-path "$GITHUB_WORKSPACE/dbt/target-base"), and note the path must sit next to the project's manifest.json.
There was a problem hiding this comment.
Follow-up: low priority, tracked (same as the cubic docs thread).
|
|
||
| describe("PR title and description reach the intent check", () => { | ||
| test("GitHub: read from the pull_request event", async () => { | ||
| const dir = await fs.mkdtemp(path.join(tmpdir(), "review-event-")) |
There was a problem hiding this comment.
SUGGESTION: This temp directory is never removed.
test/AGENTS.md ("Temporary Directory Fixture") asks tests to use tmpdir() / tmpdirScoped() from fixture/fixture.ts so cleanup is automatic; a raw fs.mkdtemp leaks one directory per run under the parallel suite. Use await using tmp = await tmpdir({ init: ... }) and write event.json inside tmp.path.
There was a problem hiding this comment.
Not changing in this PR: test-hygiene nit.
Code Review SummaryStatus: 5 Issues Found | Recommendation: Address before merge Overview
Fix these issues in Kilo Cloud Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (12 files)
Verified on Previous Review Summary (commit 1fda92a)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit 1fda92a)Status: 7 Issues Found | Recommendation: Address before merge Overview
Fix these issues in Kilo Cloud Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (11 files)
Verified on Reviewed by deepseek-v4.1-flash-1017 · Input: 144.2K · Output: 42.2K · Cached: 4M Review guidance: REVIEW.md from base branch |
dev-punia-altimate
left a comment
There was a problem hiding this comment.
🤖 Code Review — OpenCodeReview (Gemini) — 3 finding(s)
- 3 anchored to a line (posted inline when the comment stream is on)
- 0 without a line anchor
All findings (full text)
1. packages/opencode/src/altimate/review/run.ts (L382-L385)
[🔵 LOW] The checklist prohibits the use of nested ternary expressions. Please refactor this to avoid nesting, perhaps by computing the disabled reason in a separate variable before using it here.
Suggested change:
cliVersion: opts.cliVersion ?? Installation.VERSION,
aiReview: opts.noAi || config.ai === false ? undefined : runAiReview,
aiDisabledReason: (function() {
if (opts.noAi) return "disabled by --no-ai"
if (config.ai === false) return "disabled by `ai: false` in .altimate/review.yml"
return undefined
})(),
aiModel: opts.aiModel,
2. packages/opencode/src/altimate/review/ai-review.ts (L124-L129)
[🔴 HIGH] Moving buildUserMessage outside the try...catch block introduces the risk of unhandled exceptions. If buildUserMessage throws an error (e.g., due to an unexpected value or formatting error), it will crash the function, violating the JSDoc contract that "a review must not crash because the AI layer is unavailable".
Consider moving the message generation and timeout setup inside the try block to ensure all potential errors are caught and a proper failure status is returned.
Suggested change:
const controller = new AbortController()
let timeout: ReturnType<typeof setTimeout> | undefined
let timeoutMs = 0
let modelLabel: string | undefined
try {
const message = buildUserMessage({ ...input, files })
timeoutMs = aiTimeoutMs(message.length)
timeout = setTimeout(() => controller.abort(), timeoutMs)
3. packages/opencode/src/altimate/review/ai-review.ts (L61-L68)
[🟠 MEDIUM] While this effectively sanitizes bearer tokens and typical API key patterns, it may not cover other sensitive credentials occasionally found in network errors, such as Basic <base64> auth headers or credentials embedded within URLs (e.g., https://user:pass@host/...).
Consider expanding the regex to cover these patterns for more robust redaction.
Suggested change:
function safeReason(err: unknown): string {
const raw = err instanceof Error ? err.message : String(err)
return raw
.replace(/\s+/g, " ")
.replace(/(bearer\s+|basic\s+)[^\s]+/gi, "$1***")
.replace(/:\/\/[^/]*@[^/]+/gi, "://***@")
.replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***")
.slice(0, 200)
}
| cliVersion: opts.cliVersion ?? Installation.VERSION, | ||
| aiReview: opts.noAi || config.ai === false ? undefined : runAiReview, | ||
| aiDisabledReason: opts.noAi ? "disabled by --no-ai" : config.ai === false ? "disabled by `ai: false` in .altimate/review.yml" : undefined, | ||
| aiModel: opts.aiModel, |
There was a problem hiding this comment.
[🔵 LOW] The checklist prohibits the use of nested ternary expressions. Please refactor this to avoid nesting, perhaps by computing the disabled reason in a separate variable before using it here.
Suggested change:
| cliVersion: opts.cliVersion ?? Installation.VERSION, | |
| aiReview: opts.noAi || config.ai === false ? undefined : runAiReview, | |
| aiDisabledReason: opts.noAi ? "disabled by --no-ai" : config.ai === false ? "disabled by `ai: false` in .altimate/review.yml" : undefined, | |
| aiModel: opts.aiModel, | |
| cliVersion: opts.cliVersion ?? Installation.VERSION, | |
| aiReview: opts.noAi || config.ai === false ? undefined : runAiReview, | |
| aiDisabledReason: (function() { | |
| if (opts.noAi) return "disabled by --no-ai" | |
| if (config.ai === false) return "disabled by `ai: false` in .altimate/review.yml" | |
| return undefined | |
| })(), | |
| aiModel: opts.aiModel, |
There was a problem hiding this comment.
Thanks, agreed. Fixed in 4d9a88a: the AI-disabled reason is now computed in a small if/else if block before the runReview call.
| const message = buildUserMessage({ ...input, files }) | ||
| const controller = new AbortController() | ||
| const timeout = setTimeout(() => controller.abort(), AI_TIMEOUT_MS) | ||
| const timeoutMs = aiTimeoutMs(message.length) | ||
| const timeout = setTimeout(() => controller.abort(), timeoutMs) | ||
| let modelLabel: string | undefined | ||
| try { |
There was a problem hiding this comment.
[🔴 HIGH] Moving buildUserMessage outside the try...catch block introduces the risk of unhandled exceptions. If buildUserMessage throws an error (e.g., due to an unexpected value or formatting error), it will crash the function, violating the JSDoc contract that "a review must not crash because the AI layer is unavailable".
Consider moving the message generation and timeout setup inside the try block to ensure all potential errors are caught and a proper failure status is returned.
Suggested change:
| const message = buildUserMessage({ ...input, files }) | |
| const controller = new AbortController() | |
| const timeout = setTimeout(() => controller.abort(), AI_TIMEOUT_MS) | |
| const timeoutMs = aiTimeoutMs(message.length) | |
| const timeout = setTimeout(() => controller.abort(), timeoutMs) | |
| let modelLabel: string | undefined | |
| try { | |
| const controller = new AbortController() | |
| let timeout: ReturnType<typeof setTimeout> | undefined | |
| let timeoutMs = 0 | |
| let modelLabel: string | undefined | |
| try { | |
| const message = buildUserMessage({ ...input, files }) | |
| timeoutMs = aiTimeoutMs(message.length) | |
| timeout = setTimeout(() => controller.abort(), timeoutMs) |
There was a problem hiding this comment.
Thanks, fixed in 4d9a88a. Message building and the timeout setup now sit inside the try, so runAiReview keeps its never-throws contract. Before this, a throw wouldn't have crashed the review: the caller in orchestrate.ts catches it and records the lane as state: error. So the impact was a misreported lane status, not a crash, but the contract should hold locally.
| function safeReason(err: unknown): string { | ||
| const raw = err instanceof Error ? err.message : String(err) | ||
| return raw | ||
| .replace(/\s+/g, " ") | ||
| .replace(/(bearer\s+)[^\s]+/gi, "$1***") | ||
| .replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***") | ||
| .slice(0, 200) | ||
| } |
There was a problem hiding this comment.
[🟠 MEDIUM] While this effectively sanitizes bearer tokens and typical API key patterns, it may not cover other sensitive credentials occasionally found in network errors, such as Basic <base64> auth headers or credentials embedded within URLs (e.g., https://user:pass@host/...).
Consider expanding the regex to cover these patterns for more robust redaction.
Suggested change:
| function safeReason(err: unknown): string { | |
| const raw = err instanceof Error ? err.message : String(err) | |
| return raw | |
| .replace(/\s+/g, " ") | |
| .replace(/(bearer\s+)[^\s]+/gi, "$1***") | |
| .replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***") | |
| .slice(0, 200) | |
| } | |
| function safeReason(err: unknown): string { | |
| const raw = err instanceof Error ? err.message : String(err) | |
| return raw | |
| .replace(/\s+/g, " ") | |
| .replace(/(bearer\s+|basic\s+)[^\s]+/gi, "$1***") | |
| .replace(/:\/\/[^/]*@[^/]+/gi, "://***@") | |
| .replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***") | |
| .slice(0, 200) | |
| } |
There was a problem hiding this comment.
Thanks, this was a real gap. Fixed in 4d9a88a: safeReason now reuses the repo's redactSecrets (from installation), which masks Authorization: headers, Basic/Bearer values, user:pass@/token userinfo in URLs, api_key=...-style assignments and provider-prefixed tokens. Redaction runs before whitespace is collapsed, so a header match stops at its own line. New tests check that Basic, URL, key=value and bearer credentials are masked and that a plain error message is left alone.
🤖 Code Review — OpenCodeReview (Gemini) — 3 finding(s)
All findings (full text)1.
|
hkaltai
left a comment
There was a problem hiding this comment.
Verdict: approve, with minor follow-ups. Panel: Claude, Codex, lead. No CRITICAL or MAJOR findings. 0 CRITICAL, 0 MAJOR, 6 MINOR (inline), 6 NIT. Claude's run of the six review test files (172) and typecheck passed.
Findings above were each verified against the code. The verdict logic is unchanged by the banner fix (degraded does not feed the verdict), so this cannot flip approve/block.
[MINOR] (ai-review.ts:185, outside the diff hunks) Parser failure reported as state ok
const parsed = (((parseRes.data ?? {}) as Record<string, unknown>).findings as any[]) ?? []
The core parser can return { success: false, data: {}, error } instead of throwing. success is ignored and findings default to [], so a failed parse is reported as ok with no findings, which is exactly the silent failure this PR aims to remove.
Fix: check parseRes.success and that findings is an array; otherwise return state: "error" with a masked reason.
Also worth doing: docs/docs/reference/telemetry.md review_run row does not list the new fields (undecided, unverified_models, ai_state, ai_duration_ms, base_source, out_of_scope_files) and the changed meaning of degraded. Tests are missing for runAiReview itself (timeout with partial text, empty text, missing prompt, bad model), safeReason, and the project_dir to --cwd wiring.
Nits
orchestrate.ts:1418-1419/format.tsAI note: every trivial-tier review now shows "Advisory AI review skipped: not part of the trivial tier"; consider keeping that in the envelope only.verdict.ts:153durationMs: z.number().int().nonnegative(): an injected lane returning a fractional duration makesVerdictEnvelope.parsethrow; round it before building the envelope.format.ts:114(env.scope?.baseRef ? ` · base \`${env.scope.baseRef}\...) ``: branch names may contain backticks; escape them.orchestrate.ts:1411-1416comment still says the lane is "Skipped when no aiReview fn is injected ... deterministic-only"; update to mentionaiStatusreporting.ai-review.tsaiTimeoutMsdoc says "capped at 5 minutes", but the prompt limits make the cap effectively unreachable (about 183 s); say so or lower the cap.ai-review.ts:60safeReason: name suggests validation;redactErrorForCommentwould be clearer.
| return raw | ||
| .replace(/\s+/g, " ") | ||
| .replace(/(bearer\s+)[^\s]+/gi, "$1***") | ||
| .replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***") |
There was a problem hiding this comment.
[MINOR] Secret masking in safeReason is incomplete
.replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***")
Only bearer ... and sk|pk|key|token followed by -/_ are masked. ?key=AIza..., api_key=..., x-api-key: ..., ghp_..., AKIA... all pass through (confirmed by running the regex). The reason lands in a public PR comment and the signed envelope. Scenario: a provider error echoes a request URL containing ?key=... and it is posted on the PR.
Fix: add patterns for key=/token= query and header values, AIza..., gh[pousr]_..., AKIA..., JWTs, or mask any 24+ char key-like run; add unit tests.
There was a problem hiding this comment.
Thanks. Fixed in fix(review): address review comments: credential redaction in AI-lane reasons, never-throw AI lane. safeReason now runs the shared redactSecrets (key/token query and header values, provider-prefixed tokens, Basic/Bearer, URL userinfo) before collapsing whitespace, with tests.
| const chosen = input.model ? Provider.parseModel(input.model) : await Provider.defaultModel() | ||
| model = await Provider.getModel(chosen.providerID, chosen.modelID) | ||
| } catch (err) { | ||
| return done([], { state: "skipped", reason: `no usable model: ${safeReason(err)}`, model: input.model }) |
There was a problem hiding this comment.
[MINOR] Explicit --ai-model that fails to resolve reports skipped
return done([], { state: "skipped", reason: `no usable model: ${safeReason(err)}`, model: input.model })
A typo or missing credentials for an explicitly requested model looks the same as --no-ai in the comment and in ai_state telemetry. Fix: use error when input.model was supplied; keep skipped for no resolvable default.
There was a problem hiding this comment.
Agreed, thanks. Fixed in fix(review): address review feedback: AI parse failure, explicit model error, masked lane reason, per-reason unverified notes, docs: an explicitly requested model that can't be resolved is error, and only a missing default is skipped.
| // A failed AI lane must never fail the review. | ||
| } catch (err) { | ||
| // A failed AI lane must never fail the review — but it must say so. | ||
| aiStatus = { state: "error", reason: String((err as Error)?.message ?? err).slice(0, 200) } |
There was a problem hiding this comment.
[MINOR] Thrown-lane reason is unmasked and multi-line
aiStatus = { state: "error", reason: String((err as Error)?.message ?? err).slice(0, 200) }
Bypasses safeReason, so credentials in the message are published and newlines can break the > quote. Only reachable for injected lanes, but it is the defence-in-depth path. Fix: export and reuse safeReason.
There was a problem hiding this comment.
Thanks. Fixed in the same commit: the catch in runReview goes through safeReason, so the reason is masked, on one line and bounded. There's a test with a multi-line bearer header.
| const head = unverified.slice(0, 5).map((u) => `\`${u.path}\``).join(", ") | ||
| const more = unverified.length > 5 ? ` and ${unverified.length - 5} more` : "" | ||
| notes.push( | ||
| `${unverified.length} model(s) not verified by the engine — ${unverified[0].reason}: ${head}${more}. ` + |
There was a problem hiding this comment.
[MINOR] Unverified note shows the first model's reason for all models
`${unverified.length} model(s) not verified by the engine — ${unverified[0].reason}: ${head}${more}. `
Reasons differ (no compiled SQL for head vs base, orchestrate.ts:1108-1110). If a.sql lacks the base compile and b.sql the head compile, the comment says both lack the base compile. Fix: group by reason, one line each.
There was a problem hiding this comment.
Thanks. Fixed in the same commit: unverified models are grouped by reason, one line each, with a test. #1723 extends the same grouping to failed files and abstentions.
| ```yaml | ||
| - name: Compile the base into target-base/ | ||
| run: | | ||
| git worktree add ../base "origin/${{ github.base_ref }}" |
There was a problem hiding this comment.
[MINOR] Base-compile recipe: wrong baseline and monorepo path
git worktree add ../base "origin/${{ github.base_ref }}"
(1) This checks out the target-branch tip, but the review diffs against the merge-base with the PR head. If the target advanced, equivalence is compared against a different baseline (a formatting-only PR can get a false critical). Use $(git merge-base origin/<base> HEAD). (2) target-base/ is resolved under the dbt project root, so with project_dir the snippet compiles from and writes to the wrong folder; compile from ../base/<project_dir> with --target-path "$GITHUB_WORKSPACE/<project_dir>/target-base".
There was a problem hiding this comment.
Both points are right, thanks. Fixed in the same commit: the recipe checks out $(git merge-base origin/<base> HEAD) and shows the project_dir form (cd ../base/<project_dir>, --target-path "$GITHUB_WORKSPACE/<project_dir>/target-base"). dbt_prepare in #1726 does the same.
…tus, PR intent, `project_dir` - Lint-only now means no changed model resolved against a manifest; an undecidable finding no longer flips the whole run. Undecidable findings, unverified models, out-of-scope files and AI-lane problems each get their own line in the summary. - The advisory AI lane reports `ok | skipped | timeout | error` with a reason, model and duration on the signed envelope (`ai`) and in the comment; it never fails silently. The timeout scales with prompt size (60 s + 1 s per 2K chars, capped at 5 min). `--ai-model provider/model` selects the model explicitly. - The CLI passes the PR title and description to the intent check (GitHub `pull_request` event, GitLab `CI_MERGE_REQUEST_TITLE`/`_DESCRIPTION`). - The action gains `project_dir` (→ `--cwd`) for monorepos. - Docs: what each input unlocks (fetch depth, manifest, compiled SQL, `target-base/`, `catalog.json`) and how to compile the base. - `review_run` telemetry adds `undecided`, `unverified_models`, `ai_state`, `ai_duration_ms`, `base_source`, `out_of_scope_files`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… reasons, never-throw AI lane - `safeReason` now runs the shared `redactSecrets` masker before collapsing whitespace, so `Basic` auth values, `Authorization:` headers, `user:pass@` URL userinfo and `api_key=...` secrets no longer reach the public PR comment. Exported and covered by tests. - `runAiReview` builds the user message and arms the timeout inside the `try`, keeping its never-throws contract. - `run.ts`: the AI-disabled reason is computed before the call instead of in a nested ternary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l error, masked lane reason, per-reason unverified notes, docs - `runAiReview`: core's in-band parse failure (`success: false`, or no findings array) is reported as `state: "error"`, never an "ok" run with no findings. - `runAiReview`: an explicitly requested `--ai-model` that cannot be resolved is an `error`; only a missing default model is `skipped`. - `runReview`: a thrown AI lane's reason goes through `safeReason` (masked, one line, bounded). - `coverageNotes`: unverified models are grouped by reason, one line each, so a missing head compile and a missing base compile are not reported as the same thing. - Docs: the base-compile recipe checks out the merge-base (what the review diffs against) and shows the `project_dir` form; the telemetry reference lists the new `review_run` fields and the narrower meaning of `degraded`. - Tests: masked multi-line lane reason, per-reason unverified notes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1fda92a to
e8a893e
Compare
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 40673273 | Triggered | Base64 Basic Authentication | 4d9a88a | packages/opencode/test/altimate/review-ci-honesty.test.ts | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
There was a problem hiding this comment.
1 existing issue remains and 4 new issues found across 7 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="docs/docs/usage/dbt-pr-review.md">
<violation number="1" location="docs/docs/usage/dbt-pr-review.md:218">
P3: This `run:` step interpolates `${{ github.base_ref }}` directly into the shell script, which is the injection pattern `github/review/action.yml` avoids. Users copy this recipe, so pass the value through `env:` and reference `$BASE_REF` instead.</violation>
<violation number="2" location="docs/docs/usage/dbt-pr-review.md:218">
P2: This command computes the base from `HEAD`, but the preceding default `actions/checkout` on `pull_request` checks out GitHub’s synthetic merge commit; the merge-base therefore becomes the target branch tip, not the PR’s branch point. Check out `github.event.pull_request.head.sha` (and make the review use that same head), or pass that SHA explicitly here before compiling the base.</violation>
</file>
<file name="packages/opencode/src/altimate/review/ai-review.ts">
<violation number="1" location="packages/opencode/src/altimate/review/ai-review.ts:149">
P2: Only a missing default should be `skipped`. Any `Provider.defaultModel()` failure, including a config or provider-load error, is reported as `skipped`. Consider separating the default-model path so real failures report `error`.</violation>
</file>
<file name="docs/docs/reference/telemetry.md">
<violation number="1" location="docs/docs/reference/telemetry.md:76">
P2: This row documents a `degraded` condition that the review currently does not report: any parseable manifest makes `degraded` false, even when the changed model is absent from it. Track whether each changed model actually resolves against the manifest, and update the stale note below that still says undecidable findings set `degraded`.</violation>
</file>
Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.
View guided diff | Turn on auto-fix | Re-trigger cubic
| // An explicitly requested model that cannot be resolved is an error, not | ||
| // the same as `--no-ai`; only a missing default is a skip. | ||
| return done([], { | ||
| state: input.model ? "error" : "skipped", |
There was a problem hiding this comment.
P2: Only a missing default should be skipped. Any Provider.defaultModel() failure, including a config or provider-load error, is reported as skipped. Consider separating the default-model path so real failures report error.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 149:
<comment>Only a missing default should be `skipped`. Any `Provider.defaultModel()` failure, including a config or provider-load error, is reported as `skipped`. Consider separating the default-model path so real failures report `error`.</comment>
<file context>
@@ -137,7 +143,13 @@ export async function runAiReview(input: AiReviewInput): Promise<AiReviewResult>
+ // An explicitly requested model that cannot be resolved is an error, not
+ // the same as `--no-ai`; only a missing default is a skip.
+ return done([], {
+ state: input.model ? "error" : "skipped",
+ reason: `no usable model: ${safeReason(err)}`,
+ model: input.model,
</file context>
| | `first_prompt_sent` | The user's first typed message in an onboarding session, and the first typed message in a linked project or a session pinned to a workspace, once per workspace for each machine and Altimate instance. A slash command sent as a command, as the TUI and `run` send it, is excluded, so the hidden `/onboard-connect` submission does not count. The VS Code chat sends a command as its expanded text, so there a command counts. A subagent's prompt never counts. `origin` says which rule sent it: `onboarding` for an onboarding session's first typed prompt, bound or not, and `workspace` for the workspace's row. So a later prompt in an onboarding session, after its project was linked, is a `workspace` row. | | ||
| | `onboarding_abandoned` | The CLI exited during a first run without connecting. `last_stage` is the furthest point reached: `started`, `model_picker`, `provider_setup`, `altimate_base_confirm`, or `gateway_auth`. (`connected` is a funnel position but never a `last_stage` — reaching it means the run completed, which is not an abandonment.) Only emitted for a genuine first run — opening `/connect` as an existing user does not enter the funnel, and abandonment after setup completes is out of scope by definition. Emitted on the exit path under a bounded flush, so the measured rate is a lower bound — see [Delivery & Reliability](#delivery--reliability). | | ||
| | `review_run` | A dbt/SQL review completed or failed — `invocation` (`cli` for `altimate-code review`, `tool` for the `dbt_pr_review` tool), status, duration, and on success the verdict, the pre-gating verdict, mode, risk tier, and finding counts by severity and by category. No file paths, model or column names, finding titles or bodies, SQL, diff content, or repository/branch/PR names. | | ||
| | `review_run` | A dbt/SQL review completed or failed — `invocation` (`cli` for `altimate-code review`, `tool` for the `dbt_pr_review` tool), status, duration, and on success the verdict, the pre-gating verdict, mode, risk tier, finding counts by severity and by category, `degraded` (lint-only: no reviewable files, or no changed model resolved against a manifest; an undecidable finding alone no longer sets it), `undecided` (findings the engine could not decide), `unverified_models` (models without compiled SQL), `ai_state` and `ai_duration_ms` (the advisory AI lane's outcome), `base_source` (where the review base came from, e.g. `flag` or `default_branch`) and `out_of_scope_files` (changed files outside the dbt project). No file paths, model or column names, finding titles or bodies, SQL, diff content, or repository/branch/PR names. | |
There was a problem hiding this comment.
P2: This row documents a degraded condition that the review currently does not report: any parseable manifest makes degraded false, even when the changed model is absent from it. Track whether each changed model actually resolves against the manifest, and update the stale note below that still says undecidable findings set degraded.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/docs/reference/telemetry.md, line 76:
<comment>This row documents a `degraded` condition that the review currently does not report: any parseable manifest makes `degraded` false, even when the changed model is absent from it. Track whether each changed model actually resolves against the manifest, and update the stale note below that still says undecidable findings set `degraded`.</comment>
<file context>
@@ -73,7 +73,7 @@ We collect the following categories of events:
| `first_prompt_sent` | The user's first typed message in an onboarding session, and the first typed message in a linked project or a session pinned to a workspace, once per workspace for each machine and Altimate instance. A slash command sent as a command, as the TUI and `run` send it, is excluded, so the hidden `/onboard-connect` submission does not count. The VS Code chat sends a command as its expanded text, so there a command counts. A subagent's prompt never counts. `origin` says which rule sent it: `onboarding` for an onboarding session's first typed prompt, bound or not, and `workspace` for the workspace's row. So a later prompt in an onboarding session, after its project was linked, is a `workspace` row. |
| `onboarding_abandoned` | The CLI exited during a first run without connecting. `last_stage` is the furthest point reached: `started`, `model_picker`, `provider_setup`, `altimate_base_confirm`, or `gateway_auth`. (`connected` is a funnel position but never a `last_stage` — reaching it means the run completed, which is not an abandonment.) Only emitted for a genuine first run — opening `/connect` as an existing user does not enter the funnel, and abandonment after setup completes is out of scope by definition. Emitted on the exit path under a bounded flush, so the measured rate is a lower bound — see [Delivery & Reliability](#delivery--reliability). |
-| `review_run` | A dbt/SQL review completed or failed — `invocation` (`cli` for `altimate-code review`, `tool` for the `dbt_pr_review` tool), status, duration, and on success the verdict, the pre-gating verdict, mode, risk tier, and finding counts by severity and by category. No file paths, model or column names, finding titles or bodies, SQL, diff content, or repository/branch/PR names. |
+| `review_run` | A dbt/SQL review completed or failed — `invocation` (`cli` for `altimate-code review`, `tool` for the `dbt_pr_review` tool), status, duration, and on success the verdict, the pre-gating verdict, mode, risk tier, finding counts by severity and by category, `degraded` (lint-only: no reviewable files, or no changed model resolved against a manifest; an undecidable finding alone no longer sets it), `undecided` (findings the engine could not decide), `unverified_models` (models without compiled SQL), `ai_state` and `ai_duration_ms` (the advisory AI lane's outcome), `base_source` (where the review base came from, e.g. `flag` or `default_branch`) and `out_of_scope_files` (changed files outside the dbt project). No file paths, model or column names, finding titles or bodies, SQL, diff content, or repository/branch/PR names. |
| `review_post_outcome` | Whether a review was published to GitHub — `not_requested`, `not_attempted`, `target_unresolved`, `full`, `partial`, or `summary_failed`, plus duration. Emitted on the **CLI path only** — the `dbt_pr_review` tool completes reviews but never publishes, so a `review_run` with `invocation: tool` has no post event and that is not a failure. Within the CLI path there is exactly one per **completed** review: a review that failed emits `review_run: failed` and no post event, so absence there means the review failed rather than that an event was lost. `not_attempted` is publication requested but never reached (a bad `--output` path, a stdout write error). No repository, PR, or comment content. |
| `rule_proposal_shown` | A proposed rule was raised: shown as a card, or stored for `learn proposals` when no attached client shows cards (`stored`), or raised again for the same wording (`repeat`) — what raised it (`remember`, `shown_rule`, `recurrence`), the kind of change, whether it is a "Save as" choice, linked/owner, the buttons offered, whether it weakens verification, evidence session count, whether it came from a correction, how many paths it applies to, and its length. The card id is random. No rule text, quotes, paths or globs. |
</file context>
| - name: Compile the base into target-base/ | ||
| run: | | ||
| # The review diffs against the merge-base, not the target branch tip. | ||
| git worktree add ../base "$(git merge-base "origin/${{ github.base_ref }}" HEAD)" |
There was a problem hiding this comment.
P2: This command computes the base from HEAD, but the preceding default actions/checkout on pull_request checks out GitHub’s synthetic merge commit; the merge-base therefore becomes the target branch tip, not the PR’s branch point. Check out github.event.pull_request.head.sha (and make the review use that same head), or pass that SHA explicitly here before compiling the base.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/docs/usage/dbt-pr-review.md, line 218:
<comment>This command computes the base from `HEAD`, but the preceding default `actions/checkout` on `pull_request` checks out GitHub’s synthetic merge commit; the merge-base therefore becomes the target branch tip, not the PR’s branch point. Check out `github.event.pull_request.head.sha` (and make the review use that same head), or pass that SHA explicitly here before compiling the base.</comment>
<file context>
@@ -214,10 +214,14 @@ Compiling the base (optional, for equivalence and column-breakage):
run: |
- git worktree add ../base "origin/${{ github.base_ref }}"
+ # The review diffs against the merge-base, not the target branch tip.
+ git worktree add ../base "$(git merge-base "origin/${{ github.base_ref }}" HEAD)"
(cd ../base && dbt deps && dbt compile --target-path "$GITHUB_WORKSPACE/target-base")
</file context>
</details>
| - name: Compile the base into target-base/ | ||
| run: | | ||
| # The review diffs against the merge-base, not the target branch tip. | ||
| git worktree add ../base "$(git merge-base "origin/${{ github.base_ref }}" HEAD)" |
There was a problem hiding this comment.
P3: This run: step interpolates ${{ github.base_ref }} directly into the shell script, which is the injection pattern github/review/action.yml avoids. Users copy this recipe, so pass the value through env: and reference $BASE_REF instead.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/docs/usage/dbt-pr-review.md, line 218:
<comment>This `run:` step interpolates `${{ github.base_ref }}` directly into the shell script, which is the injection pattern `github/review/action.yml` avoids. Users copy this recipe, so pass the value through `env:` and reference `$BASE_REF` instead.</comment>
<file context>
@@ -214,10 +214,14 @@ Compiling the base (optional, for equivalence and column-breakage):
run: |
- git worktree add ../base "origin/${{ github.base_ref }}"
+ # The review diffs against the merge-base, not the target branch tip.
+ git worktree add ../base "$(git merge-base "origin/${{ github.base_ref }}" HEAD)"
(cd ../base && dbt deps && dbt compile --target-path "$GITHUB_WORKSPACE/target-base")
</file context>
</details>
| test("Basic auth, URL userinfo, key=value and bearer values are masked", () => { | ||
| const err = new Error( | ||
| "request to https://alice:s3cretPass@proxy.example.com/v1 failed\n" + | ||
| "Authorization: Basic YWxpY2U6czNjcmV0UGFzcw==\n" + |
There was a problem hiding this comment.
WARNING: This literal trips the repository's secret scanner.
Basic YWxpY2U6czNjcmV0UGFzcw== is only base64 of alice:s3cretPass (repeated at line 210), but GitGuardian cannot know it is fake and has opened a Base64 Basic Authentication incident on this PR (commit 4d9a88a), so that status keeps reporting a secret on a public repo. Build the header at runtime instead, which keeps the same coverage without a scanner-shaped literal:
const basic = Buffer.from("alice:s3cretPass").toString("base64")
const err = new Error(
"request to https://alice:s3cretPass@proxy.example.com/v1 failed\n" +
`Authorization: Basic ${basic}\n` +
"api_key=abcd1234efgh and Bearer eyJhbGciOiJIUzI1NiJ9.payload",
)and keep the expect(out).not.toContain(basic) assertion.
| choices: ["trivial", "lite", "full"] as const, | ||
| describe: "[EXPERIMENTAL / bench debug] override the tier classifier — verdict envelope carries tierForced: true", | ||
| }) | ||
| .option("ai-model", { |
There was a problem hiding this comment.
SUGGESTION: The new flag is missing from the CLI reference.
docs/docs/usage/dbt-pr-review.md documents every other flag in its Options table (lines 109-120, including --no-ai), but --ai-model is absent, so the only documented way to choose the advisory model is the PR description. Add a row such as | --ai-model <provider/model> | Explicit model for the advisory AI lane (default: the configured default model). |.
|
@hkaltai, re your review: Thanks for the approval. One commit on top addresses the follow-ups:
|
Changes since review: Review feedback commit: AI-lane parse failures and model errors are reported explicitly, the lane reason is credential-masked, unverified models get one note per reason, and the docs are updated.
Issue for this PR
Closes #1792
Stacked on #1719; please review and merge that one first. Takes ideas from the closed #1241, not its code.
Type of change
What does this PR do?
The review comment could misstate what it checked:
degraded, so every manifest-backed run said "Lint-only run — no dbt manifest". Now it shows only when no changed model resolved against a manifest. Undecidable findings, models without compiled SQL, and files outside the project each get their own line.[]on every failure, so an expired key, a timeout, or a missing model looked like "no AI findings". It now reportsok | skipped | timeout | errorwith a reason (credentials masked), the model, and duration. These land in a new signedaifield and in the comment. The timeout now scales with prompt size (60 s + 1 s per 2K chars, capped at 5 min) instead of a fixed 60 s.--ai-model provider/modelpicks the model explicitly.prTitle/prBodyexisted but nothing passed them. The CLI now reads them from the GitHubpull_requestevent, or from GitLab'sCI_MERGE_REQUEST_TITLE/_DESCRIPTION.project_dirinput, mapped to--cwd, for monorepos.target-base/,catalog.json) and a snippet for compiling the base.review_rungainsundecided,unverified_models,ai_state,ai_duration_ms,base_source,out_of_scope_files.Open question, deliberately not changed:
--no-aistill auto-registers the free tier, which is a network call. A release test asserts that on purpose, so the decision is the maintainers'. Changing it is a one-line guard.How did you verify your code works?
test/altimate/review-ci-honesty.test.ts(13 tests):bun typecheckis clean.Cost side effects
None. The AI lane's timeout ceiling can rise from 60 s to 5 min on very large prompts, but the number of model calls doesn't change (still at most one per review).
Screenshots / recordings
N/A (CLI).
Checklist
🤖 Generated with Claude Code
Summary by cubic
Makes the dbt PR review comment candid about what it checked. Previously a single undecidable finding labeled any manifest-backed run as "Lint-only", the advisory AI lane returned
[]on every failure, and no PR intent reached the AI check.Review honesty
ok | skipped | timeout | errorwith a reason, model, and duration on a new signedaienvelope field and in the comment; it never fails silently.redactSecretsmasker (basic auth, bearer tokens, URL userinfo,key=...secrets) before reaching the public comment, and the lane never throws.success: false) is flagged aserror, and an unresolvable explicit--ai-modelerrors while a missing default only skips.--ai-model provider/modelselects the model explicitly.pull_requestevent or GitLab'sCI_MERGE_REQUEST_TITLE/_DESCRIPTIONfor the intent check.Inputs, docs, and telemetry
project_dirinput maps to--cwdfor monorepos;manifest_pathresolves relative to it.review_runtelemetry gainsundecided,unverified_models,ai_state,ai_duration_ms,base_source, andout_of_scope_files.target-base/, including aproject_dirform.Deliberately unchanged:
--no-aistill auto-registers the free tier, which is a network call; a release test asserts that, so the decision is the maintainers'. Model call count is unchanged (still at most one per review); only the timeout ceiling rises, to 5 minutes on very large prompts.Written for commit e8a893e. Summary will update on new commits.