Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/docs/reference/telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ We collect the following categories of events:
| `activation_job_selected` / `first_job_completed` | Which activation job the user started and, where observable, finished. In onboarding, completion is reported only for the job that was actually selected, so the two form a coherent pair. `first_job_completed` is also sent once per workspace for each machine and Altimate instance, for the first turn in a linked project, or in a session pinned to a workspace by the IDE extension, that had a tool call that didn't fail and ended `completed`. `origin` says which: `onboarding` or `workspace`. **Derived** — see the note below. |
| `first_prompt_sent` | The user's first typed message in an onboarding session, and the first typed message in a linked project or a session pinned to a workspace, once per workspace for each machine and Altimate instance. A slash command sent as a command, as the TUI and `run` send it, is excluded, so the hidden `/onboard-connect` submission does not count. The VS Code chat sends a command as its expanded text, so there a command counts. A subagent's prompt never counts. `origin` says which rule sent it: `onboarding` for an onboarding session's first typed prompt, bound or not, and `workspace` for the workspace's row. So a later prompt in an onboarding session, after its project was linked, is a `workspace` row. |
| `onboarding_abandoned` | The CLI exited during a first run without connecting. `last_stage` is the furthest point reached: `started`, `model_picker`, `provider_setup`, `altimate_base_confirm`, or `gateway_auth`. (`connected` is a funnel position but never a `last_stage` — reaching it means the run completed, which is not an abandonment.) Only emitted for a genuine first run — opening `/connect` as an existing user does not enter the funnel, and abandonment after setup completes is out of scope by definition. Emitted on the exit path under a bounded flush, so the measured rate is a lower bound — see [Delivery & Reliability](#delivery--reliability). |
| `review_run` | A dbt/SQL review completed or failed — `invocation` (`cli` for `altimate-code review`, `tool` for the `dbt_pr_review` tool), status, duration, and on success the verdict, the pre-gating verdict, mode, risk tier, and finding counts by severity and by category. No file paths, model or column names, finding titles or bodies, SQL, diff content, or repository/branch/PR names. |
| `review_run` | A dbt/SQL review completed or failed — `invocation` (`cli` for `altimate-code review`, `tool` for the `dbt_pr_review` tool), status, duration, and on success the verdict, the pre-gating verdict, mode, risk tier, finding counts by severity and by category, `degraded` (lint-only: no reviewable files, or no changed model resolved against a manifest; an undecidable finding alone no longer sets it), `undecided` (findings the engine could not decide), `unverified_models` (models without compiled SQL), `ai_state` and `ai_duration_ms` (the advisory AI lane's outcome), `base_source` (where the review base came from, e.g. `flag` or `default_branch`) and `out_of_scope_files` (changed files outside the dbt project). No file paths, model or column names, finding titles or bodies, SQL, diff content, or repository/branch/PR names. |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This row documents a degraded condition that the review currently does not report: any parseable manifest makes degraded false, even when the changed model is absent from it. Track whether each changed model actually resolves against the manifest, and update the stale note below that still says undecidable findings set degraded.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/docs/reference/telemetry.md, line 76:

<comment>This row documents a `degraded` condition that the review currently does not report: any parseable manifest makes `degraded` false, even when the changed model is absent from it. Track whether each changed model actually resolves against the manifest, and update the stale note below that still says undecidable findings set `degraded`.</comment>

<file context>
@@ -73,7 +73,7 @@ We collect the following categories of events:
 | `first_prompt_sent` | The user's first typed message in an onboarding session, and the first typed message in a linked project or a session pinned to a workspace, once per workspace for each machine and Altimate instance. A slash command sent as a command, as the TUI and `run` send it, is excluded, so the hidden `/onboard-connect` submission does not count. The VS Code chat sends a command as its expanded text, so there a command counts. A subagent's prompt never counts. `origin` says which rule sent it: `onboarding` for an onboarding session's first typed prompt, bound or not, and `workspace` for the workspace's row. So a later prompt in an onboarding session, after its project was linked, is a `workspace` row. |
 | `onboarding_abandoned` | The CLI exited during a first run without connecting. `last_stage` is the furthest point reached: `started`, `model_picker`, `provider_setup`, `altimate_base_confirm`, or `gateway_auth`. (`connected` is a funnel position but never a `last_stage` — reaching it means the run completed, which is not an abandonment.) Only emitted for a genuine first run — opening `/connect` as an existing user does not enter the funnel, and abandonment after setup completes is out of scope by definition. Emitted on the exit path under a bounded flush, so the measured rate is a lower bound — see [Delivery & Reliability](#delivery--reliability). |
-| `review_run` | A dbt/SQL review completed or failed — `invocation` (`cli` for `altimate-code review`, `tool` for the `dbt_pr_review` tool), status, duration, and on success the verdict, the pre-gating verdict, mode, risk tier, and finding counts by severity and by category. No file paths, model or column names, finding titles or bodies, SQL, diff content, or repository/branch/PR names. |
+| `review_run` | A dbt/SQL review completed or failed — `invocation` (`cli` for `altimate-code review`, `tool` for the `dbt_pr_review` tool), status, duration, and on success the verdict, the pre-gating verdict, mode, risk tier, finding counts by severity and by category, `degraded` (lint-only: no reviewable files, or no changed model resolved against a manifest; an undecidable finding alone no longer sets it), `undecided` (findings the engine could not decide), `unverified_models` (models without compiled SQL), `ai_state` and `ai_duration_ms` (the advisory AI lane's outcome), `base_source` (where the review base came from, e.g. `flag` or `default_branch`) and `out_of_scope_files` (changed files outside the dbt project). No file paths, model or column names, finding titles or bodies, SQL, diff content, or repository/branch/PR names. |
 | `review_post_outcome` | Whether a review was published to GitHub — `not_requested`, `not_attempted`, `target_unresolved`, `full`, `partial`, or `summary_failed`, plus duration. Emitted on the **CLI path only** — the `dbt_pr_review` tool completes reviews but never publishes, so a `review_run` with `invocation: tool` has no post event and that is not a failure. Within the CLI path there is exactly one per **completed** review: a review that failed emits `review_run: failed` and no post event, so absence there means the review failed rather than that an event was lost. `not_attempted` is publication requested but never reached (a bad `--output` path, a stdout write error). No repository, PR, or comment content. |
 | `rule_proposal_shown` | A proposed rule was raised: shown as a card, or stored for `learn proposals` when no attached client shows cards (`stored`), or raised again for the same wording (`repeat`) — what raised it (`remember`, `shown_rule`, `recurrence`), the kind of change, whether it is a "Save as" choice, linked/owner, the buttons offered, whether it weakens verification, evidence session count, whether it came from a correction, how many paths it applies to, and its length. The card id is random. No rule text, quotes, paths or globs. |
</file context>

| `review_post_outcome` | Whether a review was published to GitHub — `not_requested`, `not_attempted`, `target_unresolved`, `full`, `partial`, or `summary_failed`, plus duration. Emitted on the **CLI path only** — the `dbt_pr_review` tool completes reviews but never publishes, so a `review_run` with `invocation: tool` has no post event and that is not a failure. Within the CLI path there is exactly one per **completed** review: a review that failed emits `review_run: failed` and no post event, so absence there means the review failed rather than that an event was lost. `not_attempted` is publication requested but never reached (a bad `--output` path, a stdout write error). No repository, PR, or comment content. |
| `rule_proposal_shown` | A proposed rule was raised: shown as a card, or stored for `learn proposals` when no attached client shows cards (`stored`), or raised again for the same wording (`repeat`) — what raised it (`remember`, `shown_rule`, `recurrence`), the kind of change, whether it is a "Save as" choice, linked/owner, the buttons offered, whether it weakens verification, evidence session count, whether it came from a correction, how many paths it applies to, and its length. The card id is random. No rule text, quotes, paths or globs. |
| `rule_proposal_answered` | A card was answered or closed (`send`, `approve`, `approve_anyway`, `edit`, `dismiss`, `save_local`, `save_memory`, `expired`, `closed`) — `ok` or why it did not go through (`busy`, `not_available`, `verification_flagged`, `invalid_text`, `refused` by the workspace, `failed`), what happened (`sent`, `queued`, `approved`, `approve_failed`, `approve_retry`, `target_changed`, `exists`, `dismissed`, `saved_memory_project`, `saved_memory_global`, `saved_local`, or `saved_unanswered` when a card the person asked for closed unanswered and was saved as before cards), wording edits so far, and time since the card was shown. No rule text. |
Expand Down
37 changes: 35 additions & 2 deletions docs/docs/usage/dbt-pr-review.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,38 @@ jobs:
# model_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
```

### What the review needs (and what each input unlocks)

| Input | How to produce it | Unlocks |
|---|---|---|
| Full git history of the PR's target branch | `actions/checkout` with `fetch-depth: 0` | Correct base: the diff starts at the merge-base with the PR's target branch. A target branch that is not fetched is an error, never a silent fallback to `main`. |
| `target/manifest.json` | `dbt compile` (or `dbt parse` for lint + lineage only) | Blast radius, lineage breakage, tiering. Without it the run is lint-only. |
| `target/compiled/…` | `dbt compile` | Engine checks on the head (lint, grade, PII, grain). |
| `target-base/compiled/…` | Compile the base ref into `target-base/` (below) | Equivalence proofs and column-breakage for modified models. |
| `target/catalog.json` | `dbt docs generate` | Complete warehouse columns for column-lineage breakage. |

Compiling the base (optional, for equivalence and column-breakage):

```yaml
- name: Compile the base into target-base/
run: |
# The review diffs against the merge-base, not the target branch tip.
git worktree add ../base "$(git merge-base "origin/${{ github.base_ref }}" HEAD)"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This command computes the base from HEAD, but the preceding default actions/checkout on pull_request checks out GitHub’s synthetic merge commit; the merge-base therefore becomes the target branch tip, not the PR’s branch point. Check out github.event.pull_request.head.sha (and make the review use that same head), or pass that SHA explicitly here before compiling the base.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/docs/usage/dbt-pr-review.md, line 218:

<comment>This command computes the base from `HEAD`, but the preceding default `actions/checkout` on `pull_request` checks out GitHub’s synthetic merge commit; the merge-base therefore becomes the target branch tip, not the PR’s branch point. Check out `github.event.pull_request.head.sha` (and make the review use that same head), or pass that SHA explicitly here before compiling the base.</comment>

<file context>
@@ -214,10 +214,14 @@ Compiling the base (optional, for equivalence and column-breakage):
         run: |
-          git worktree add ../base "origin/${{ github.base_ref }}"
+          # The review diffs against the merge-base, not the target branch tip.
+          git worktree add ../base "$(git merge-base "origin/${{ github.base_ref }}" HEAD)"
           (cd ../base && dbt deps && dbt compile --target-path "$GITHUB_WORKSPACE/target-base")

</file context>


</details>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This run: step interpolates ${{ github.base_ref }} directly into the shell script, which is the injection pattern github/review/action.yml avoids. Users copy this recipe, so pass the value through env: and reference $BASE_REF instead.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/docs/usage/dbt-pr-review.md, line 218:

<comment>This `run:` step interpolates `${{ github.base_ref }}` directly into the shell script, which is the injection pattern `github/review/action.yml` avoids. Users copy this recipe, so pass the value through `env:` and reference `$BASE_REF` instead.</comment>

<file context>
@@ -214,10 +214,14 @@ Compiling the base (optional, for equivalence and column-breakage):
         run: |
-          git worktree add ../base "origin/${{ github.base_ref }}"
+          # The review diffs against the merge-base, not the target branch tip.
+          git worktree add ../base "$(git merge-base "origin/${{ github.base_ref }}" HEAD)"
           (cd ../base && dbt deps && dbt compile --target-path "$GITHUB_WORKSPACE/target-base")

</file context>


</details>

(cd ../base && dbt deps && dbt compile --target-path "$GITHUB_WORKSPACE/target-base")
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: This snippet does not work in the monorepo layout the paragraph below documents.

It writes target-base/ to the repository root and runs dbt compile from the worktree root. The resolver reads <dbt root>/target-base/compiled/<project>/… (compiled.ts:117, with opts.cwd = dbtRoot from run.ts:363), and dbt compile needs dbt_project.yml, so with project_dir: dbt both steps fail or land where the review never looks — equivalence and column-breakage silently stay off. Add a monorepo variant, e.g. (cd ../base/dbt && dbt deps && dbt compile --target-path "$GITHUB_WORKSPACE/dbt/target-base"), and note the path must sit next to the project's manifest.json.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked (same as the cubic docs thread).

```

With `project_dir`, compile from that folder and write next to its own
`target/`: `(cd ../base/<project_dir> && dbt deps && dbt compile --target-path "$GITHUB_WORKSPACE/<project_dir>/target-base")`.

**Monorepos:** set `project_dir` to the folder holding `dbt_project.yml`. Only
changes inside it are reviewed, `manifest_path` is resolved relative to it, and
changes elsewhere are counted in the comment.

**What the comment tells you it could not check:** findings the engine could not
decide, models without compiled SQL, files outside the project, and whether the
advisory AI lane ran, was skipped, timed out or failed (with the reason). The
same facts are in the envelope (`scope`, `ai`) and the `review_run` telemetry.

### Model & credentials for the advisory lane

The deterministic engine (lineage, equivalence, PII, grade, lint — the only layer
Expand Down Expand Up @@ -356,8 +388,9 @@ reviewer does **not** re-implement Jinja — it consumes dbt's own compiled outp
from `target/compiled/<project>/…` (HEAD) and `target-base/compiled/…` (BASE),
produced by `dbt compile`. To enable full equivalence verdicts, compile both
the base and head refs in CI (the base into `target-base/`, the Recce
convention). Without compiled SQL the engine lanes fall back to raw and stay
*undecidable* (never fabricated) — the `dbt-patterns` lane still runs.
convention). A model without compiled SQL is never sent to the engine as raw
Jinja: it is listed as **not verified** (with the reason) in the comment and in
the envelope's `scope.unverified`, and the `dbt-patterns` lane still runs.

## What it checks — deterministic rule catalog

Expand Down
8 changes: 7 additions & 1 deletion github/review/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,12 @@ inputs:
description: "comment (never blocks) | gate (blocks the merge on REQUEST_CHANGES)"
required: false
default: "comment"
project_dir:
description: "Directory of the dbt project (the one holding dbt_project.yml), relative to the repository root. Only changes inside it are reviewed — set it in a monorepo. `manifest_path` is resolved relative to it."
required: false
default: "."
manifest_path:
description: "Path to the compiled dbt manifest.json. Run `dbt compile` before this step for the full (non-degraded) verdict."
description: "Path to the compiled dbt manifest.json, relative to `project_dir`. Run `dbt compile` before this step for the full (non-degraded) verdict; `dbt docs generate` (catalog.json) and a base compile into `target-base/` enable column-lineage breakage and equivalence proofs."
required: false
default: "target/manifest.json"
severity_threshold:
Expand Down Expand Up @@ -170,6 +174,7 @@ runs:
# from `${{ }}` would be a shell-injection vector. Build an arg array instead.
env:
IN_MODE: ${{ inputs.mode }}
IN_PROJECT_DIR: ${{ inputs.project_dir }}
IN_MANIFEST: ${{ inputs.manifest_path }}
IN_SEVERITY: ${{ inputs.severity_threshold }}
IN_BASE: ${{ inputs.base }}
Expand All @@ -181,6 +186,7 @@ runs:
ALTIMATE_REVIEW_SIGNING_KEY: ${{ inputs.signing_key }}
run: |
args=(--mode "$IN_MODE" --manifest "$IN_MANIFEST" --severity "$IN_SEVERITY")
[[ -n "$IN_PROJECT_DIR" && "$IN_PROJECT_DIR" != "." ]] && args+=(--cwd "$IN_PROJECT_DIR")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: A misspelled project_dir silently broadens a monorepo review to every changed file. Validate that the directory contains dbt_project.yml/.yaml before adding --cwd, or make the review fail closed when project discovery fails.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At github/review/action.yml, line 189:

<comment>A misspelled `project_dir` silently broadens a monorepo review to every changed file. Validate that the directory contains `dbt_project.yml`/`.yaml` before adding `--cwd`, or make the review fail closed when project discovery fails.</comment>

<file context>
@@ -181,6 +186,7 @@ runs:
         ALTIMATE_REVIEW_SIGNING_KEY: ${{ inputs.signing_key }}
       run: |
         args=(--mode "$IN_MODE" --manifest "$IN_MANIFEST" --severity "$IN_SEVERITY")
+        [[ -n "$IN_PROJECT_DIR" && "$IN_PROJECT_DIR" != "." ]] && args+=(--cwd "$IN_PROJECT_DIR")
         [[ -n "$IN_BASE" ]] && args+=(--base "$IN_BASE")
         [[ -n "$IN_HEAD" ]] && args+=(--head "$IN_HEAD")
</file context>
Suggested change
[[ -n "$IN_PROJECT_DIR" && "$IN_PROJECT_DIR" != "." ]] && args+=(--cwd "$IN_PROJECT_DIR")
if [[ -n "$IN_PROJECT_DIR" && "$IN_PROJECT_DIR" != "." ]]; then
project_path="${GITHUB_WORKSPACE:-$PWD}/$IN_PROJECT_DIR"
if [[ ! -f "$project_path/dbt_project.yml" && ! -f "$project_path/dbt_project.yaml" ]]; then
echo "::error::project_dir must contain dbt_project.yml or dbt_project.yaml"
exit 1
fi
args+=(--cwd "$IN_PROJECT_DIR")
fi

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. The action should check that dbt_project.y(a)ml exists before --cwd.

[[ -n "$IN_BASE" ]] && args+=(--base "$IN_BASE")
[[ -n "$IN_HEAD" ]] && args+=(--head "$IN_HEAD")
[[ "$IN_POST" == "true" ]] && args+=(--post)
Expand Down
105 changes: 90 additions & 15 deletions packages/opencode/src/altimate/review/ai-review.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,15 @@ import { Agent } from "@/agent/agent"
import { MessageV2 } from "@/session/message-v2"
import { MessageID, SessionID } from "@/session/schema"
import { Log } from "@/altimate/util/log"
import { redactSecrets } from "@/installation"
import { Dispatcher } from "../native"
import { type Finding, type ReviewCategory, type Severity, makeFinding } from "./finding"
import { readStreamText } from "../llm/stream-text"

const log = Log.create({ service: "ai-review" })

const AI_TIMEOUT_MS = 60_000
const AI_TIMEOUT_BASE_MS = 60_000
const AI_TIMEOUT_MAX_MS = 300_000
const MAX_DIFF_CHARS = 6_000 // per file, keep the prompt bounded
const MAX_FILES = 20

Expand All @@ -31,6 +33,42 @@ export interface AiReviewInput {
grounding: Finding[]
prTitle?: string
prBody?: string
/** Explicit `provider/model`; omit to use the configured default model. */
model?: string
}

/** What happened in the advisory lane, reported on the envelope and in the comment. */
export type AiLaneState = "ok" | "skipped" | "timeout" | "error"
export interface AiLaneStatus {
state: AiLaneState
/** Why the lane was skipped or failed (never contains credentials). */
reason?: string
/** `provider/model` the lane ran with. */
model?: string
durationMs?: number
}
export interface AiReviewResult {
findings: Finding[]
status: AiLaneStatus
}

/** Timeout scaled to the prompt: 60 s plus 1 s per 2,000 characters, capped at 5 minutes. */
export function aiTimeoutMs(promptChars: number): number {
return Math.min(AI_TIMEOUT_MAX_MS, AI_TIMEOUT_BASE_MS + Math.ceil(promptChars / 2_000) * 1_000)
}

/** Error text safe to show in a PR comment: one line, bounded, with
* credentials masked (authorization headers, `Basic`/`Bearer` values,
* `key=value` secrets, `user:pass@` URL userinfo, provider-prefixed tokens).
* Redaction runs before whitespace is collapsed so a header match stops at
* its own line. */
export function safeReason(err: unknown): string {
const raw = err instanceof Error ? err.message : String(err)
return redactSecrets(raw)
.replace(/\s+/g, " ")
.replace(/(bearer\s+)[^\s]+/gi, "$1***")
.replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: safeReason still exposes credential formats that do not use the four dashed prefixes, so provider failures can disclose secrets in PR comments. Reuse the repository's credential/error redactor or mask credential-labeled values and credential-bearing URLs before truncating the reason.

(Based on your team's feedback about masking credential-stem error values.) [a5c5a10f-6a4a-4e42-8c2e-f8e01b48683d]

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 66:

<comment>`safeReason` still exposes credential formats that do not use the four dashed prefixes, so provider failures can disclose secrets in PR comments. Reuse the repository's credential/error redactor or mask credential-labeled values and credential-bearing URLs before truncating the reason.

(Based on your team's feedback about masking credential-stem error values.) [a5c5a10f-6a4a-4e42-8c2e-f8e01b48683d]</comment>

<file context>
@@ -31,6 +32,39 @@ export interface AiReviewInput {
+  return raw
+    .replace(/\s+/g, " ")
+    .replace(/(bearer\s+)[^\s]+/gi, "$1***")
+    .replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***")
+    .slice(0, 200)
 }
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 4d9a88a. safeReason now runs the shared redactSecrets masker (Authorization headers, Basic/Bearer values, key=value secrets, URL userinfo, provider-prefixed tokens) before collapsing whitespace. Tests added.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Secret masking in safeReason is incomplete

.replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***")

Only bearer ... and sk|pk|key|token followed by -/_ are masked. ?key=AIza..., api_key=..., x-api-key: ..., ghp_..., AKIA... all pass through (confirmed by running the regex). The reason lands in a public PR comment and the signed envelope. Scenario: a provider error echoes a request URL containing ?key=... and it is posted on the PR.

Fix: add patterns for key=/token= query and header values, AIza..., gh[pousr]_..., AKIA..., JWTs, or mask any 24+ char key-like run; add unit tests.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Fixed in fix(review): address review comments: credential redaction in AI-lane reasons, never-throw AI lane. safeReason now runs the shared redactSecrets (key/token query and header values, provider-prefixed tokens, Basic/Bearer, URL userinfo) before collapsing whitespace, with tests.

.slice(0, 200)
}

/**
Expand Down Expand Up @@ -73,23 +111,47 @@ function buildUserMessage(input: AiReviewInput): string {

/**
* Run the LLM reviewer lane. Returns advisory findings (severity ≤ warning,
* clamped by core), or [] if no model / core is available or the call fails —
* a review must never crash because the AI layer is unavailable.
* clamped by core) and a status saying whether the lane ran, was skipped, timed
* out or failed. Never throws — a review must not crash because the AI layer is
* unavailable — but never fails silently either: the status reaches the
* envelope and the PR comment.
*/
export async function runAiReview(input: AiReviewInput): Promise<Finding[]> {
export async function runAiReview(input: AiReviewInput): Promise<AiReviewResult> {
const startedAt = Date.now()
const done = (findings: Finding[], status: Omit<AiLaneStatus, "durationMs">): AiReviewResult => ({
findings,
status: { ...status, durationMs: Date.now() - startedAt },
})
const files = input.files.filter((f) => f.status !== "deleted" && (f.diff || f.sql))
if (!files.length) return []
if (!files.length) return done([], { state: "skipped", reason: "no changed model SQL to review" })

const controller = new AbortController()
const timeout = setTimeout(() => controller.abort(), AI_TIMEOUT_MS)
let timeoutMs = AI_TIMEOUT_BASE_MS
let timeout: ReturnType<typeof setTimeout> | undefined
let modelLabel: string | undefined
try {
const message = buildUserMessage({ ...input, files })
timeoutMs = aiTimeoutMs(message.length)
timeout = setTimeout(() => controller.abort(), timeoutMs)
// Prompt comes from the compiled core, not this file.
const promptRes = await Dispatcher.call("altimate_core.review_ai_prompt", {})
const system = ((promptRes.data ?? {}) as Record<string, unknown>).prompt as string | undefined
if (!system) return []

const defaultModel = await Provider.defaultModel()
const model = await Provider.getModel(defaultModel.providerID, defaultModel.modelID)
if (!system) return done([], { state: "error", reason: "the core reviewer prompt is unavailable" })

let model: Awaited<ReturnType<typeof Provider.getModel>>
try {
const chosen = input.model ? Provider.parseModel(input.model) : await Provider.defaultModel()
model = await Provider.getModel(chosen.providerID, chosen.modelID)
} catch (err) {
// An explicitly requested model that cannot be resolved is an error, not
// the same as `--no-ai`; only a missing default is a skip.
return done([], {
state: input.model ? "error" : "skipped",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Only a missing default should be skipped. Any Provider.defaultModel() failure, including a config or provider-load error, is reported as skipped. Consider separating the default-model path so real failures report error.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 149:

<comment>Only a missing default should be `skipped`. Any `Provider.defaultModel()` failure, including a config or provider-load error, is reported as `skipped`. Consider separating the default-model path so real failures report `error`.</comment>

<file context>
@@ -137,7 +143,13 @@ export async function runAiReview(input: AiReviewInput): Promise<AiReviewResult>
+      // An explicitly requested model that cannot be resolved is an error, not
+      // the same as `--no-ai`; only a missing default is a skip.
+      return done([], {
+        state: input.model ? "error" : "skipped",
+        reason: `no usable model: ${safeReason(err)}`,
+        model: input.model,
</file context>

reason: `no usable model: ${safeReason(err)}`,
model: input.model,
})
}
modelLabel = `${model.providerID}/${model.id}`

const agent: Agent.Info = {
name: "dbt-ai-reviewer",
Expand Down Expand Up @@ -119,18 +181,29 @@ export async function runAiReview(input: AiReviewInput): Promise<Finding[]> {
abort: controller.signal,
sessionID: user.sessionID,
retries: 1,
messages: [{ role: "user", content: buildUserMessage({ ...input, files }) }],
messages: [{ role: "user", content: message }],
})
const text = await readStreamText(stream)
if (!text) return []
if (controller.signal.aborted)
return done([], { state: "timeout", reason: `no answer within ${Math.round(timeoutMs / 1000)} s`, model: modelLabel })
if (!text) return done([], { state: "error", reason: "the model returned no text", model: modelLabel })

// Parse + clamp in core (the prompt-injection-resistant, advisory-only
// contract). Returns already-validated, severity-clamped, file-checked items.
const parseRes = await Dispatcher.call("altimate_core.review_ai_parse", {
text,
valid_files: files.map((f) => f.path),
})
const parsed = (((parseRes.data ?? {}) as Record<string, unknown>).findings as any[]) ?? []
// The core parser reports failure in-band (`success: false`) rather than
// throwing; that is an error, never an "ok" run with no findings.
const parsedRaw = ((parseRes.data ?? {}) as Record<string, unknown>).findings
if (parseRes.success === false || !Array.isArray(parsedRaw))
return done([], {
state: "error",
reason: `could not parse the model's answer: ${safeReason(parseRes.error ?? "no findings list")}`,
model: modelLabel,
})
const parsed = parsedRaw as any[]
const byFile = new Map(files.map((f) => [f.path, f]))

const out: Finding[] = []
Expand Down Expand Up @@ -165,10 +238,12 @@ export async function runAiReview(input: AiReviewInput): Promise<Finding[]> {
}
}
log.info("ai review complete", { findings: out.length })
return out
return done(out, { state: "ok", model: modelLabel })
} catch (err) {
log.error("ai review failed", { error: err })
return []
if (controller.signal.aborted)
return done([], { state: "timeout", reason: `no answer within ${Math.round(timeoutMs / 1000)} s`, model: modelLabel })
return done([], { state: "error", reason: safeReason(err), model: modelLabel })
} finally {
clearTimeout(timeout)
}
Expand Down
Loading
Loading