Skip to content

feat(review): AI evidence line, suggestions and base-branch repo instructions (needs core 0.10) - #1774

Draft
RaghwendraSingh wants to merge 3 commits into
feat/review-ai-verifierfrom
feat/review-ai-remit
Draft

RaghwendraSingh wants to merge 3 commits into
feat/review-ai-verifierfrom
feat/review-ai-remit

Conversation

@RaghwendraSingh

@RaghwendraSingh RaghwendraSingh commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Issue for this PR

Part of #1792

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Draft, not for merge yet. Stacked on #1769. It needs the @altimateai/altimate-core release that parses the new AI output fields (0.10.0, PR in the core repo). Against core 0.9.0 the new fields default and behaviour is unchanged.

What changed

  • Reads the AI reviewer's evidence_line, pre_existing, suggestion and cites and maps them to the finding: quoted line, pre-existing tag, a GitHub suggestion block when valid, and (per <file>).
  • Repo review instructions (REVIEW.md, .altimate/REVIEW.md, AGENTS.md, CLAUDE.md at the root and above each changed path) are read from the base branch and given to the AI as conventions, so a PR can't change its own instructions. Capped at 4k chars per file and 12k in total.
  • AI findings that quote a line but give no line number are anchored to the diff line they quote.

Results so far

  • Widening the AI's remit to macro, test and yml bugs did not help, so that part was dropped:
    • Tuning repos: +2 catches (95% CI −3 to +7).
    • Held-out repos: −1 catch, with 8% wrong comments.
  • This PR keeps only the output fields, repo instructions and anchoring. Measuring those alone is still to do: one tuning-repo run before the core release.

How did you verify your code works?

  • bun test test/altimate/review: 678 pass.
  • bunx tsc --noEmit: clean.

Cost side effects

No new model calls, warehouse queries or cloud resources. When the AI lane is enabled, base-branch repo instructions add at most 12k characters (about 3k tokens) of input per AI call (4k per file), and the new output fields add a few output tokens per finding.

Screenshots / recordings

Not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR
  • I checked for cost side effects (model calls, queries or cloud resources that grow with user data) and described them, or wrote "none"

🤖 Generated with Claude Code


Summary by cubic

Maps the AI reviewer's newer output fields onto findings and supplies the reviewer with the repo's own review instructions read from the base branch. Requires altimate-core 0.10.0; against 0.9.0 the new fields default to absent and behavior is unchanged.

  • Reads evidence_line, pre_existing, suggestion and cites from the AI reviewer, mapping them to the finding's quoted line, pre-existing tag, GitHub suggestion block, and a (per <file>) body suffix; malformed suggestions are dropped, not fatal.
  • Reads REVIEW.md, .altimate/REVIEW.md, AGENTS.md and CLAUDE.md at the repo root and each ancestor directory of a changed path from the base commit, so a PR can't rewrite its own instructions; each file is capped at 4k chars, 12k total.
  • AI findings that quote a line without a line number are anchored to the matching diff line.
  • Widening the AI remit to macro, test and yml bugs did not improve catch rate and was dropped.

Written for commit 58b9300. Summary will update on new commits.

View guided diff Turn on auto-fix

@coderabbitai

coderabbitai Bot commented Nov 19, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

RaghwendraSingh and others added 3 commits November 20, 2026 12:21
…, suggestion and cites

Newer cores return `evidence_line`, `pre_existing`, `suggestion` and
`cites` on each AI finding. `aiOptionalFields` maps them onto the finding
(`evidenceLine`, `preExisting`, `suggestion`, and a `(per \`<file>\`)`
body suffix for `cites`). Each defaults to absent, so the lane works
unchanged against the pinned core 0.9.0. A malformed suggestion is
dropped rather than failing the finding's schema check.

The existing pipeline already handles the rest: `anchorToDiff` only ever
sets `preExisting`, so an AI-set flag survives anchoring, and AI
suggestions pass through `validateSuggestions` before posting. Tests cover
both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e base branch

- `loadRepoInstructions` reads `REVIEW.md`, `.altimate/REVIEW.md`,
  `AGENTS.md` and `CLAUDE.md` at the repo root and at each ancestor
  directory of a changed path, from the BASE commit via `showAtRef`, so a
  PR cannot rewrite the instructions it is reviewed under. Each file is cut
  to 4,000 chars and the set to 12,000; the root comes first.
- `run.ts` loads them only when the AI lane runs and a git base resolved;
  `OrchestrateInput.repoInstructions` carries them to `AiReviewInput`.
- `buildUserMessage` adds a "REPO REVIEW INSTRUCTIONS" section per call,
  listing only files whose directory contains a file in that call, and
  asks the model to set `cites` when a finding relies on one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…umber

When the AI reviewer returns `evidence_line` without `line`, the quoted
text now goes into `evidence.result.line`, so `anchorByEvidence` places
the finding on the matching diff line, as it does for rule findings.
The logic lives in a small exported `aiEvidenceResult` helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Thanks for updating your PR! It now meets our contributing guidelines. 👍

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant