Skip to content

fix(gate): tick after refusal checks, skip act-item rows, refuse silent untick - #3003

Merged
mfittko merged 25 commits into
mainfrom
issue-2898
Nov 1, 2026
Merged

mfittko merged 25 commits into
mainfrom
issue-2898

Conversation

@mfittko

@mfittko mfittko commented Nov 1, 2026 •

Copy link
Copy Markdown
Owner

Objective

Closes #2898

Ticks in PR bodies now follow the evidence rules. upsert-checkpoint-verdict ticks only after every refusal check and skips rows an open act item names. edit-pr refuses a body that unticks or removes live [x] rows.

This PR replaces #2960 (closed). The branch is the same; the gate history restarts at round 1 by operator decision (second repro on issue 3002: a rebase orphaned the cap-round baseline). The docs step 5 wording act item from the old head (round 4) is already in this branch.

In scope

  • scripts/github/upsert-checkpoint-verdict.mjs: tick after the refusal checks, skip rows an open act item names. A spec id (ac:<n>, dod:<n>) skips its resolved row when the PR body spec digest matches the pinned spec; every row of its category is skipped only when the id cannot be resolved against the pinned spec, or when the resolved row text matches no unchecked PR row.
  • scripts/github/edit-pr.mjs: refuse a body that unticks or removes checked rows; add --allow-untick.
  • skills/docs/acceptance-criteria-verification.md step 5: contract text matches the new order.

Durable fix

  • Original repro: PR 2891 pre_approval round 1. A refused upsert-checkpoint-verdict call (--verdict clean against a findings_present ledger) still edited the PR body 8 seconds before the refusal. A separate edit-pr call reset 15 ticked rows to [ ] with ok:true.
  • Rule home: ACCEPT-CRITERIA-VERIFY-AND-REFLECT in skills/docs/acceptance-criteria-verification.md (step 5).
  • Agent remedy: the edit-pr refusal names the rows and --allow-untick as the deliberate path.
  • Regression tests (test/github/upsert-checkpoint-verdict.test.mjs, test/github/edit-pr.test.mjs) must fail on the pre-fix source; a worker confirms this against origin/main.

Acceptance criteria

  • A refused verdict post changes no PR or issue body.
  • The tick never flips a row that an open act item names.
  • The source comments match the behavior.
  • edit-pr.mjs refuses a silent untick.
  • A deliberate untick still works.
  • Ticks that the tooling sanctions are unaffected.

Definition of done

  • A test runs upsert-checkpoint-verdict with a --verdict clean that contradicts the ledger. It asserts the call exits non-zero and that the PR body edit stub was never called.

  • A test posts a findings_present pre_approval verdict with an open act item on dod:1 and reviewer-verified labels that include dod:1. It asserts that dod:1 stays [ ] and the other verified labels are ticked.

  • The comments at the old lines 2754 and 2819 still hold. The two tests above cover them.

  • A test with a live body of 15 [x] rows and a new body with them as [ ] asserts exit non-zero, nothing edited, and the error names the rows.

  • The same test with --allow-untick asserts the edit goes through.

  • The existing tick-verified-checkboxes and upsert-checkpoint-verdict tick tests pass unchanged.

  • Merge condition, checked at merge: Every AC row has its evidence in a merged PR.

  • The edit-pr.mjs USAGE text documents --allow-untick.

  • A changes fragment exists.

Validation

Targeted check: bun test test/github/upsert-checkpoint-verdict.test.mjs test/github/edit-pr.test.mjs. CI verify-suite on the PR head. Size budget 400 weighted logic LOC; the figure is re-measured after each push.

dod:5 disclosure: the only edit to an existing tick test is the gh issue view call-index fixture shift (failIssueViewAfter and failIssueViewCount) in test/github/upsert-checkpoint-verdict.test.mjs. The deferred live re-fetch moves the failing call by one index. Its assertions are unchanged. Restoring the old fixture makes the test "a failed post-tick issue re-fetch cannot drop the unverified issue AC item" fail with HTTP 502 from fetchIssueBody.

Non-goals

  • Changing which labels reviewers verify, or the reviewer approvals format.
  • Issue-side tick behavior beyond keeping it after the refusal checks.
  • Freezing the PR-body spec (issue 2715) or reusing reviewer evidence across body edits (issue 2476).

🤖 Generated with Claude Code

https://claude.ai/code/session_01R7yetT95WedZayUQMysoHv

Known residuals

Tick-integrity checks key rows by AC/DoD scope plus label (CHECKLIST-SCOPE-ONE-MODEL, shared fence-aware checklistLineScopes). The following pre-PR review items stay open and are not claimed as covered by the shared parser:

  • The label-text skip is one-directional.
  • The ac:<n> skip does not cover issue-body rows.
  • checkedCriteria semantics.
  • waive-adr-tripwire can untick silently.
  • The PR-body re-read uses the issue view.
  • Rows outside AC and DoD share one null scope, so a same-label tick swap between two non-checklist sections is accepted.

mfittko and others added 14 commits November 1, 2026 10:05
…t untick in edit-pr

Refs #2898

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetS2hFz48x4dfrpt8j7
…ch test

Refs #2898

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetS2hFz48x4dfrpt8j7
…n the live body, count duplicate unticks

Refs #2898

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetS2hFz48x4dfrpt8j7
…terion

Refs #2898

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetS2hFz48x4dfrpt8j7
…n the dry-run artifact

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetS2hFz48x4dfrpt8j7
…match every checkbox form in edit-pr

Refs #2898

- upsert-checkpoint-verdict: run the deferred tick writes after the desiredBody id guard
- ac:<n>/dod:<n> skip only the PR row the pinned spec resolves to; unresolved ids skip the category
- edit-pr untick guard reads the same checkbox grammar as the tick tool

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetS2hFz48x4dfrpt8j7
… pinned spec

Refs #2898

- tickReviewerVerifiedItems compares the PR body spec digest with the pinned specDigest
- no pin or another pinned spec falls back to the whole-category skip

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetS2hFz48x4dfrpt8j7
…ites, AC-less pinned body

Refs #2898

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetS2hFz48x4dfrpt8j7
…rites

Refs #2898

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetS2hFz48x4dfrpt8j7
Refs #2898

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetT95WedZayUQMysoHv
@mfittko mfittko self-assigned this Nov 1, 2026

@mfittko mfittko left a comment •

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate review: draft_gate

Reviewed head SHA: dd03b14748e8c934d6edc10fcfb241592fb16220
Verdict: clean
Converged: fixer d07ad39a74fce0ebd5699c1e64331055fdcc6fc4 delta sha256:5e32c0cd117aa29faec5f8626f33844ceb868aa184686dc2b0832f5d6f6aeb34

Findings summary: Round 1 converged: 6 low/nit act items fixed

Next action: Mark ready for review

Comment thread changes/2898-tick-integrity.md Resolved
Refs #2898

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetT95WedZayUQMysoHv
@mfittko
mfittko marked this pull request as ready for review November 1, 2026 15:21
Copilot AI balanced review requested due to automatic review settings November 1, 2026 15:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The script can post a clean verdict using stale dry-run checks after a concurrent edit adds an unchecked AC/DoD row, leaving the new requirement without a blocker.

5 open issues
What changed in this PR

This PR addresses #2898 by tightening checklist evidence handling in gate verdict posting and protecting checked PR-body rows from accidental removal or unticking.

Changes:

  • Dry-runs verified ticks before verdict composition and defers writes until after refusal checks.
  • Adds exclusions for rows named by open act items and an explicit --allow-untick override.
  • Adds regression coverage and updates the canonical contract, Claude mirror, and changelog.
File Description
test/​github/​upsert-checkpoint-verdict.test.mjs Tests deferred ticks, act-item exclusions, refusal paths, and concurrent edits.
test/​github/​edit-pr.test.mjs Tests untick protection, duplicate labels, checkbox formats, and the override.
skills/​docs/​acceptance-criteria-verification.md Documents dry-run composition, deferred writes, and exclusion rules.
scripts/​github/​upsert-checkpoint-verdict.mjs Implements deferred tick writes and act-item filtering.
scripts/​github/​edit-pr.mjs Adds checked-row preservation checks and --allow-untick.
changes/​2898-tick-integrity.md Records the checklist-integrity fixes.
.claude/​skills/​docs/​acceptance-criteria-verification.md Mirrors the updated verification contract for Claude.

Review effort: Balanced

Comment thread scripts/github/edit-pr.mjs Resolved
Comment thread scripts/github/edit-pr.mjs Resolved
mfittko and others added 3 commits November 1, 2026 16:33
…ks, live checklist preview

Refs #2898

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetT95WedZayUQMysoHv
…left the body

Refs #2898

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetT95WedZayUQMysoHv

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Checklist-scope bugs can tick excluded DoD rows and allow a clean verdict with unverified rows after concurrent heading edits, while incorrect fence parsing can bypass untick protection.

3 open issues
5 resolved since last review

Review effort: Balanced

Comment thread scripts/github/edit-pr.mjs Resolved
mfittko and others added 3 commits November 1, 2026 16:57
… untick guard (Refs #2898)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetT95WedZayUQMysoHv
…e; narrow CHECKLIST-SCOPE-ONE-MODEL to the PR body (Refs #2898)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetT95WedZayUQMysoHv
…ix source (Refs #2898)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetT95WedZayUQMysoHv
mfittko

This comment was marked as outdated.

mfittko and others added 2 commits November 1, 2026 17:28
…-SCOPE-ONE-MODEL

Refs #2898

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7yetT95WedZayUQMysoHv
mfittko

This comment was marked as outdated.

@mfittko mfittko left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate review: pre_approval_gate

Reviewed head SHA: c4d852dcc3fc21bf07b8a67efbb777e3d6d12e9b
Verdict: clean
Size-budget outcome: pass
Size-budget T1 slice: not touched
Size-budget waiver: none

Findings summary: Round 3: 0 high, 0 medium, 6 low, 4 nit; judge act 0, defer 6, reject 4 (low_fix_until_cap)

Details

Deferred (3)

  • agents/review.agent.md:69 low (contradiction-lens): Still open at this head (prior act items b9ef425d98304018 and 9bcd2cbbdbad2742): agents/review.agent.md:69 says the pre_approval_gate verdict poster 'ticks exactly these labels before it composes the verdict', and accepted ADR docs/decisions/0100-reviewer-verified-items-drive-the-pre-approval-tick.md:19 says upsert-checkpoint-verdict.mjs 'ticks the ledger's verifiedItems before it composes the verdict'. The changed step 5 (skills/docs/acceptance-criteria-verification.md:28) now says the poster computes the ticked bodies as a dry run before composing, writes each body only after every refusal check directly before the post, and leaves act-named labels unticked. Neither file is in this diff. — judge: defer - low_fix_until_cap (GATE-EXEC-LOW-FIX-UNTIL-CAP)
  • scripts/github/edit-pr.mjs:315 low (dry): Prior act items 20128590ae14a6d0 and 3c2b761ed6034b68 are still open at this head. The checkbox row grammar lives in three hand-copied regexes: CHECKBOX_ROW here, CHECKBOX_RE in tick-verified-checkboxes.mjs, and the inline regex in upsert-checkpoint-verdict.mjs:2375. Only a comment asserts that they match. The untick guard and the live-checklist preview are fail-closed checks over the rows the tick flips, so a later edit to one copy desynchronizes which rows each check sees. — judge: defer - low_fix_until_cap (GATE-EXEC-LOW-FIX-UNTIL-CAP)
  • scripts/github/upsert-checkpoint-verdict.mjs:2366 low (dry): Prior act items 9ff81b50dbad1a37 and 91282d01a1470c3e are still open at this head. In the byCategory predicate, indices.has(index) from checklistSectionLineIndices is redundant because checklistLineScopes returns ac or dod only inside those sections. Two section classifiers still decide the PR-body tick scope, which the CHECKLIST-SCOPE-ONE-MODEL text this PR adds forbids. — judge: defer - low_fix_until_cap (GATE-EXEC-LOW-FIX-UNTIL-CAP)

Rejected (3)

  • skills/docs/acceptance-criteria-verification.md:28 nit (contradiction-lens): Still open at this head (prior act items 72c451f5be9dbf47 and 34d8edf29bf02579): the step 5 heading 'Automatic tick before composing' contradicts the step body, which says the edit is written only after every refusal check and directly before the verdict is posted. — judge: reject - low_fix_until_cap (GATE-EXEC-LOW-FIX-UNTIL-CAP)
  • scripts/github/edit-pr.mjs:195 nit (dry): Prior act items 6eeca2b78d9d0160 and f134e606ebec5874 are still open at this head. The --allow-untick value parse repeats the --allow-cross-issue expression at line 204 verbatim. — judge: reject - low_fix_until_cap (GATE-EXEC-LOW-FIX-UNTIL-CAP)
  • test/github/edit-pr.test.mjs:489 nit (dry): Prior act items 34fc60ddac183d81 and 0087a09ff05ab48d are still open at this head. The module-level attemptEdit helper at line 502 is redefined as a local attempt at lines 489 and 545, and line 479 has a third variant. — judge: reject - low_fix_until_cap (GATE-EXEC-LOW-FIX-UNTIL-CAP)

Gate evidence note: Copilot review rounds exhausted (2/2); current head has zero unresolved threads and green CI, so pre_approval_gate fallback is allowed without another Copilot re-request.

Next action: Final approval is the operator's

@mfittko
mfittko merged commit 76cd5e5 into main Nov 1, 2026
20 of 22 checks passed
@mfittko
mfittko deleted the issue-2898 branch November 1, 2026 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(gate): upsert-checkpoint-verdict ticks before its refusal checks and ticks rows an open act item names; edit-pr silently unticks

2 participants