Skip to content

feat: squabble verify-satisfied — the definition of done for a PR (exit 5 if not) - #119

Merged
hyperpolymath merged 7 commits into
fix/rollup-neutral-skipped-passfrom
feat/verify-satisfied
Oct 1, 2026
Merged

hyperpolymath merged 7 commits into
fix/rollup-neutral-skipped-passfrom
feat/verify-satisfied

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Adds squabble verify-satisfied <owner>/<repo> <pr>, one implementation of "is this PR actually done?" for the D2 Stop hook and for humans. JSON goes to stdout and a readable list to stderr. Exit 5 means agent work remains; 0/2/3/4 keep their existing meanings.

⚠ Stacked on #114 (base fix/rollup-neutral-skipped-pass). When #114 merges, retarget this PR to main before merging. Otherwise GitHub closes it when the base branch is deleted.

What counts as done

# Condition Agent item when unmet
1 Not CONFLICTING; mergeability computed; not draft; not BEHIND resolve in-file / re-run / mark ready / update
2 No required context Failed or Missing. Pending is fine; skipped/neutral satisfy but are listed as evidence-free fix the check or the ruleset
2b No red non-required check, unless a PR-body line names it with an issue link (the 2026-09-15 ruling: a finding becomes an issue, not a blocker, but someone must have looked). An acknowledged red moves to notes and is still shown fix it, or file an issue and name it in the body
3 Zero unresolved review threads (outdated ones count); no review-bot CHANGES_REQUESTED; no review bot still pending act on or resolve; wait for the bot
4 Code-scanning alerts the PR introduces not evaluated yet and said so in notes → #117
5+6 Automerge armed with SQUASH wherever GitHub allows arming arm / re-arm with squash
7 Every [.rules[].type] on the base, not just required checks. merge_queue refuses rather than passing vacuously; unknown types go to notes —

Held by a human (listed, not agent work): human approval, a required deployment, and a CLEAN PR, which GitHub refuses to arm automerge on, so it waits for a human merge. Change requests made only by review bots are agent work, not human-held.

Shared rules, so fetch and verify-satisfied can't disagree

  • CheckRun::from_github maps rollup vocabulary. An unknown completed conclusion is Failed.
  • CheckRun::for_context decides a required context from all same-named runs, and the worst run wins. Both readers used first-match before, so a passing twin could mask a failing one depending on rollup order.

Evidence

  • Tests: core 103, cli 44, forge 16, plus the rest, all green. Mutants killed: first-match ordering (2 red), the non-required loop disabled (1), the link requirement dropped (1), the context match dropped (1), digit-after-# dropped (1), ctx_done = true in pagination (2).
  • Live control 1, both arms (fix(gate): SKIPPED/NEUTRAL required checks satisfy, as Skipped #114): with no acknowledgement it exits 5 naming rust-ci / Cargo check + clippy + fmt. After a body line citing style: make rust-ci green on main (fmt, needless lifetime, dead fn) #116 it exits 0, and the acknowledged red is still shown in notes.
  • Live control 3 (standards#1076): exit 5, correctly (Debt ratchet red on main; SonarCloud Code Analysis required but never emitted → standards#1081).
  • Live control 4 (hypatia#883, another session's PR, read only): exit 5 with the same 5 unresolved threads an independent GraphQL count finds.
  • The "armed + required pending ⇒ exit 0" arm is unit-tested only; I found no live PR in that state to exercise it.

Not in this PR

Inherited red

🤖 Generated with Claude Code

https://claude.ai/code/session_0136eszqrQ53Kj7aBH1D4rXK

hyperpolymath and others added 4 commits September 30, 2026 12:02
`squabble verify-satisfied <owner/repo> <pr>` answers "has the agent
finished its part?" — distinct from "can this merge?". Agent items
(conflict, draft, failing/missing required check, unresolved thread,
bot CHANGES_REQUESTED, pending review bot, automerge unarmed or not
squash, merge queue) make it exit 5; human-held items (required
approval, deployment, a CLEAN PR GitHub will not automerge) are listed
but do not.

- squabble-core::done: the pure evaluator, 16 tests.
- CheckRun::from_github: one mapping of GitHub status/conclusion,
  shared by fetch and the new reader.
- squabble-forge::pr_done: one GraphQL query, two independently
  paginated connections (contexts, threads), fail-closed on errors or
  more than 20 pages; schema-validated against the snapshot.
- fetch::base_gate: required contexts (rulesets ∪ classic) plus every
  ruleset rule type; empty is a real answer here, not NoGate.

Check 4 (new code-scanning alerts) is not evaluated yet and says so in
every verdict's notes rather than passing silently.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136eszqrQ53Kj7aBH1D4rXK
… same-named run decides

Control 1 (cicd-squabbler#114) read DONE while `rust-ci / Cargo check +
clippy + fmt` was red: the check is not required, so the evaluator never
looked at it. "All the checkers have run" means their output was dealt
with, not that the ruleset is satisfied. New `Item::CheckFailed`; skipped
and neutral are not items. Re-run live: #114 now exits 5 naming the check,
#116 (its cure) exits 0.

`squabble fetch` and `verify-satisfied` both took the FIRST same-named run,
so a passing twin could mask a failing one depending on rollup order. One
shared rule, `CheckRun::for_context`: the worst run decides.

Mutants killed: first-match ordering (2 tests red), the new loop disabled
(1 test red).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136eszqrQ53Kj7aBH1D4rXK
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136eszqrQ53Kj7aBH1D4rXK
…ue link in the PR body

The previous commit made a red non-required check agent work with no way
out but green, which inverts the owner's 2026-09-15 ruling: a new
scanner finding becomes an issue with acceptance criteria, not a merge
blocker. The ask was "checked and dismissed/acted on", not "green".

A check run has no dismiss action, so the acknowledgement lives in the
PR body: a line naming the context AND carrying an issue/PR reference
(#N, /issues/N, /pull/N). It then moves to `notes` (still visible), not
dropped. Naming the check without a link does not count.

`body` added to pr_done.graphql and PrFacts (String; absent = no acks,
fail-closed). Mutants killed: link requirement dropped, context match
dropped, digit-after-# check dropped (1 test red each).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136eszqrQ53Kj7aBH1D4rXK
@coderabbitai

coderabbitai Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 1af77fe2-bda5-43a0-9ac5-6660921808c7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

The base gate feeds only the evidence-free listing once a PR is merged or
closed, never an agent item; pinned by a new core test (killed by a mutant
that routes every state through the open-PR path). Skipping the call keeps
the commonest done-claim off the REST secondary limit this PAT shares with
every other session -- live, 'landed hyperpolymath/hypatia#880' failed
closed on exactly that 403.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136eszqrQ53Kj7aBH1D4rXK
@hyperpolymath

Copy link
Copy Markdown
Owner Author

Pushed 0870d63 (signed): verify-satisfied skips the base-gate REST read for a merged or closed PR. The base gate feeds only the evidence-free listing there, never an agent item — pinned by the new core test the_base_gate_never_decides_a_closed_or_merged_verdict, killed by a mutant that routes every state through the open-PR path. Motivation, measured live: landed hyperpolymath/hypatia#880 failed closed on a REST secondary 403 (core quota 5000) at exactly that call. Workspace tests green (core 104, cli 44, forge 16+64).

This binary now backs two hooks in the owner's ~/.claude/settings.json (Stop: a result: line naming a not-done PR blocks; PreToolUse: non-squash merges denied on Bash and both MCP merge tools; 48-check mutant harness).

…base is

Owner ruling 2026-09-30: merge commits stay allowed (proof PRs keep their
commit series; GitHub signs the merge commit). Rebase is the form that
replays commits unsigned and breaks required_signatures, so REBASE is now
the only wrong automerge method. The test pins both halves; a mutant
dropping "MERGE" from the accepted set turns it red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136eszqrQ53Kj7aBH1D4rXK
@hyperpolymath

Copy link
Copy Markdown
Owner Author

58d1af6: a MERGE automerge is no longer agent work; only REBASE is.

The owner ruled on 2026-09-30: allow merge commits, never rebase. A merge commit is GitHub-signed and is kept for proof PRs. Rebase replays commits unsigned (119 of 286 unsigned default-branch commits).

  • done.rs: WrongMergeMethod fires only on REBASE, and its description now reads "re-arm with SQUASH (or MERGE for a proof PR)".
  • Test renamed to a_rebase_automerge_is_agent_work_but_a_merge_commit_is_not. It asserts REBASE → wrong_merge_method and MERGE → no agent items. A mutant that drops MERGE from the accepted set turns it red.
  • Workspace tests are green. The clippy/fmt failures are pre-existing in squabble-fight on main and are not from this change.

The matching changes are the merge-gate hook (local, 54/54 mutant harness) and hypatia#884 (strategist never picks :rebase; merged).

🤖 Generated with Claude Code

https://claude.ai/code/session_0136eszqrQ53Kj7aBH1D4rXK

@coderabbitai

coderabbitai Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

Autopilot could not be updated. Open Coding to check access and billing.

@hyperpolymath
hyperpolymath merged commit faa0752 into fix/rollup-neutral-skipped-pass Oct 1, 2026
30 checks passed
@hyperpolymath
hyperpolymath deleted the feat/verify-satisfied branch October 1, 2026 17:32
hyperpolymath added a commit that referenced this pull request Oct 1, 2026
Resolve crates/squabble-cli/src/main.rs as a union: main added
`board`/`inbox-sweep` (exit 6, #123/#124) where this branch added
`verify-satisfied` (exit 5, #119). Both modules, dispatch arms, usage
lines and exit-code docs are kept; codes listed in numeric order.

cargo fmt --check, clippy -D warnings and cargo test --workspace (279) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W5CoaksP2Bg21HpDCgFgwS
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant