Repository navigation
2.4.0: verified reviews, specialist /audit, supply-chain scanning, closed trigger holes - #34
Conversation
β¦nd closed trigger holes Security - Only collaborators with write access can start a run from a comment, @mention or issue; everyone else is ignored before any model call. - Fork pull requests are checked out by number (refs/pull/N/head) and Forge never pushes a follow-up or a CI fix to a fork. - REVIEW.md, FORGE.md, AGENTS.md, security guidance and skills are read from the base branch, so a pull request cannot rewrite its own review rules. - /secrets on a pull request scans the pull request's code instead of the default branch. - Dependabot alerts on a review are limited to manifests the PR touches and never block it. Review - Every finding is re-checked by a second agent before it is posted. - review_depth now works: light (one pass), standard (find, then verify), deep (three focused reviewers, then verify). The spend cap is shared. Scanning - New dependency scanner: changed lockfiles checked against OSV.dev for CVEs and malicious releases, new install scripts, off-registry sources and missing integrity hashes (dep_scan). Audit - /audit runs up to eight specialist slices adapted from pentest-skills, then one verifier with known false-positive shapes, and reports a coverage map. - semgrep, gitleaks, trivy, osv-scanner, bandit and checkov run when installed, with no token or model key in their environment; the Action image bundles them pinned and checksum-verified (external_scanners). - audit_skills: owner/repo@<sha> loads slices from another repository. Evals - forge eval --model-review scores the model review against planted bugs and safe look-alikes.
- Hero: a live, looping Forge review (scan, find, verify, post, fixed) over an animated grid and glow; a smaller three-line headline. - New sections: an agent-flow diagram with step cards, a hub of what Forge reads, checks and posts (with provider logos), a comparison with Claude Code Review and Cursor Bugbot drawn only from their public docs, a replayed supply-chain check, and what makes Forge safe on public repositories. - "What it does" and "Skills" become interactive players with real run logs and each skill's enforced tool allowlist. - Landing split into one component per section under sections/landing. - Docs: the audit section rewritten for the specialist team and scanners; a review depth and verification tab; new configuration keys. - Tests hold the page's claims to the code: scanner list, review depths, skill names and tool access.
β¦lines - New "Review & security" step: review depth, what fails the scan check, dependency scan, bundled scanners on /audit, and audit skills pinned to a commit (validated as owner/repo@<40-character SHA). - The generated workflow emits each input only when it differs from the Action's default; a test checks every input exists in action.yml. - /how-to added to the skill list. - WorkflowBuilder.tsx (711 lines) split into one component per form group under workflow/builder/; generateWorkflow.ts split into validate, agentYml and yaml helpers, keeping its exports.
- simple-git 3 -> 4 (critical; it clones the repositories Forge reviews). - openai 4 -> 7 and zod 3 -> 4 (high). The tool schemas sent to the model are hand-written JSON, unchanged; zod only validates arguments. - js-yaml 4.3.2, and overrides pinning patched proxy-addr, body-parser, nanoid, undici, source-map-js and vite within their current majors. - web: npm audit fix, 0 vulnerabilities. Remaining: 27 moderate advisories inside probot 13, fixed only by probot 14.
- Findings keep their explanation, suggested fix and start line (secrets redacted, capped), through a new additive migration. - /api/findings/list pages with an offset and leaves the text out, so a page stays small; /api/findings/detail returns one finding in full. - Findings page: 50 per page, a "Top rules" panel with readable CWE names, and a drawer with the explanation, the fix (copyable), links to the line on GitHub and to the run that found it. - Charts: eight distinct series hues (no two orange), donut legends that wrap instead of truncating model names, wider bar labels. - Site-wide: accent tokens move to ember in both themes; toggles, the active nav link, form labels and the scroll bar stop borrowing syntax colours; selected chips use dark text on the accent for contrast.
- Each review records the head commit it covered in a hidden marker on Forge's review comment. On the next push, Forge asks GitHub which files changed since and reviews the pull request's changes in those files only. - When none of the PR's files moved (or the head has not moved), the previous review stands: no clone, no model call. - A force-push, a missing marker, a compare error or a capped file list falls back to a full review. Opening, readiness, an invitation and /review are always full. - Thread resolution is scoped to the files a partial review read, so it never closes a thread in a file it did not look at. Threads now carry their path.
π‘οΈ ShipIT Forge β security scanβ Nothing found. No committed credentials, no infrastructure or workflow misconfiguration, and nothing matching the source-code security rules, and no vulnerable or malicious dependency. What was scanned
All of it is deterministic: the same commit gives the same answer, every time, with no model call and no token cost. The model review judges what is reachable and worth worrying about; this pass judges nothing and simply never gets bored. The dependency check sends package names and versions, and nothing else, to api.osv.dev. Scanned 154 files in the files this pull request changes. No model call β this scan is deterministic and costs nothing. |
|
π ShipIT Forge β what this run looked at:
The scans are deterministic β no model call, no token cost β and run before the review, so their findings survived even if the model run stopped early. Findings arrive as inline comments, plus one security-scan comment grouped by rule. π ShipIT Forge reviewed this PR β π¬ commented (no blocking issues)No findings. π Reviewed by one reviewer; nothing to verify. |
There was a problem hiding this comment.
ShipIT Forge review
β No blocking issues found. I reviewed the changed files and ran:
- π‘οΈ Security checks β SSRF, injection (SQL/command/template), broken auth/authz, hardcoded secrets, unsafe deserialization, path traversal, weak crypto.
- π§ Code review β correctness, error handling, missing tests, clarity.
Nothing to flag. This is a comment, not an approval β ShipIT Forge never approves PRs; a human reviewer should approve and merge.
Why
Forge posted whatever its first pass suspected, let anyone who could comment trigger it, read review rules from the pull request itself, and had no dependency or supply-chain checks. This brings it to the shape of the managed reviewers (find, verify, post) while closing the security holes found along the way.
Security
@mentionor issue.refs/pull/N/head); Forge never pushes a follow-up or CI fix to a fork.REVIEW.md,FORGE.md,AGENTS.md, security guidance and skills come from the base branch./secretson a PR scans the PR's code (it scanned the default branch and passed the PR's check).Review
review_depth(light / standard / deep) now works, within one spend cap.Scanning and audit
/audit: up to eight specialist slices adapted from pentest-skills, a verifier, a coverage map; semgrep, gitleaks, trivy, osv-scanner, bandit and checkov bundled in the Action image (pinned, checksum-verified, no token in their environment).audit_skills: owner/repo@<sha>overrides the slices.Site and dashboard
Behaviour changes to note
agent-fix.review_depth: lightrestores the old behaviour.Tests
Not covered
refs/pullcheckout, the compare call for incremental review, loading privateaudit_skills.--model-review) was built but not run (no credentials).