Repository navigation
Fix/captain fact precision - #55
Merged
Merged
Conversation
…generated lock files Two of the outcome-labels study's top follow-ups (docs/experiments.md): contract.api fired one fact per deleted public declaration, so one mass-refactor PR (cli/cli#12289) alone fired 2,628 facts and starved every other fact type of the PR comment's five slots. contract.env fired one fact per new variable name and never excluded generated files, so one generated GitHub Actions lock file (issue-triage.lock.yml) produced 10 of run 1's 14 fires. Both now aggregate into one fact per file with a count, the way change.safety_removed already counts per PR. contract.env also skips *.lock.yml (and similar generated files) entirely. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…s/manifests/tests The outcome-labels study (docs/experiments.md, "coupling.missing") measured this fact against 1,010 merged PRs' real review comments and later fixes: 0 of 45 fires engaged a reviewer and the later-fix rate was no higher than the PR's other files (0.92x). Every fire was on a CI workflow, a manifest or a test, the same noise pattern the field test found at 18-of-18 two orders of magnitude earlier. The fact is now always weight "info" (never "strong"), so it can no longer by itself raise a PR's captain priority, and it never fires at all when the changed side (as opposed to the missing partner) is a CI workflow (.github/workflows/**), a manifest, or a test, superseding the field test's narrower test-only downgrade. It still appears in the full AI-facing brief. Decision and reasoning recorded in docs/captain.md's coupling.missing section and section 8. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…sion follow-ups The three detector fixes just committed were derived from run 2's own 1,010-PR corpus, so that corpus cannot validate them. Pre-registers a held-out corpus (two repos never in run 2's six, never error-path/opendrs: pallets/click, yargs/yargs), the decision rule (structural correctness plus no-regression on the shown-facts lift), and the sanity-check-only reference to run 2's numbers, before any held-out fetch or brief runs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Picks up the contract.api, contract.env and coupling.missing detector fixes. Bundle rebuilt after a clean npm ci (worker rule 13): no symlinked node_modules in this worktree. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…follow-ups — all three held Ran the pre-registered held-out check (yargs/yargs, expressjs/express; pallets/click swapped out mid-run after it hit the spec's own >20% non-reconstructible stop rule from a default-branch rename, recorded before any result was read) on 89 PRs never scored by the standing eval, built twice against main's dist/ and this branch's. All three fixes held: zero structural violations (no file fires more than one aggregated contract.api/contract.env fact, no fire on a generated *.lock.yml, no coupling.missing fire on a CI workflow/manifest/test or at weight "strong"); the 3 pre-fix coupling.missing fires were all on .github/workflows/ci.yml and all eliminated with the captain priority table byte-for-byte unchanged. contract.api and contract.env had only 0-1 fire each on this corpus, so the lift (precision) question stays open on run 2's own numbers, not newly answered here -- reported honestly rather than overclaimed. No fix reverted. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
🟢 Diff risk 0.16 · green · human review6 files · +340 −67 · Captain: quick confirm
Review brief for AI reviewers (opendrs)Score details
Policy
opendrs 0.5.0 · default model · github.com/error-path/opendrs Auto-approval: not approved: decision is human-review; self-modification guard: the PR changes action/dist/index.js. |
|
Caution Review failedThe pull request is closed. Finishing Touches
Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-ups 1–3 from the outcome study (#54). contract.api and contract.env now fire once per file with a count; generated *.lock.yml files are skipped. coupling.missing becomes info-only and never fires on CI workflows, manifests or tests (decision in docs/captain.md). Score, tier and gate unchanged (verify-gate test green). Validated on a held-out set pre-registered before fetch (yargs + express, 89 PRs, never in run 2, never opendrs): all three fixes held. contract.api/env had too few fires there for a lift verdict, as reported. Bundle rebuilt; full check green in a CI-like env.