Skip to content

fix(ipr): re-apply a signature on the new head when a cross-repo push races it - #8256

Merged
bokelley merged 1 commit into
mainfrom
fix/ipr-signature-retry-reapply
Oct 17, 2026
Merged

bokelley merged 1 commit into
mainfrom
fix/ipr-signature-retry-reapply

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Summary

Several AAO repos call ipr-check-callable.yml, and every one of them writes the shared signatures/ipr-signatures.json on adcp's main. Actions concurrency groups are per repo, so adcp-ipr-signature-write serializes writes within one caller repo but not across repos.

When two signatures landed at the same time, the second push was rejected and check-and-record.mjs ran git pull --rebase. Both commits append to the end of the same JSON array, so the rebase conflicted and the record job failed. I reproduced the conflict with two shallow clones of a local bare remote.

Fix

scripts/ipr/check-and-record.mjs now records through recordSignatureWithRetry:

  • On a rejected push, it fetches origin/main, runs reset --hard FETCH_HEAD, re-reads the ledger, and adds the entry again. It does not replay the old commit.
  • If the author is already on the fresh ledger (for example, they signed another repo's PR at the same moment), it returns without writing.
  • It makes up to 5 attempts, with exponential backoff (1s, 2s, 4s, 8s) plus up to 1s of jitter. After the last attempt it throws the final push error, and the job fails without posting success.

Unchanged from #8224:

  • The ledger format, which is still written by writeSignatures.
  • The remote-URL guard.
  • The status behaviour: after the write, the commented head and the PR's latest head are both marked success.

The handler now uses one ledger.record(entry, message) call in place of write + commit. The git ops take a directory, so tests can point them at a temp repo. The workflow comments and governance/ipr-bot-setup.md now describe the new retry instead of the rebase.

Tests

New cases in tests/ipr-check-and-record.test.mjs. The first two run real git against a local bare remote with two shallow clones of the same commit:

  • Concurrent append. Another repo's run pushes first. This run's first push is rejected, and it re-applies on the new head. The remote ends with all three entries in the writeSignatures format, and the head gets success.
  • Already signed on retry. The same author was recorded concurrently from another repo. There is no second commit and no duplicate entry, and both heads still get success.
  • Retry exhaustion. Every push is rejected. The loop makes the bounded number of attempts, syncs and re-applies before each retry, and backs off between them.
  • Failed recording. No success status is posted.

The existing handler tests were updated for the record interface.

Verification

  • npm run typecheck: pass
  • npm run test:unit: pass
  • npm run test:server-unit -- --shard=N/3: 1/3 4612 passed; 2/3 3311 passed, 31 skipped; 3/3 4687 passed
  • No changeset: check-changeset-protocol-scope.cjs --has-protocol-scoped-changes reports no protocol-scoped changes.

🤖 Generated with Claude Code

… races it

Several AAO repos call ipr-check-callable.yml, and each writes the shared
signatures/ipr-signatures.json on adcp's main. Actions concurrency groups
are per repo, so the write group does not serialize writes across repos.
When two signatures were recorded at the same time, the second push was
rejected and the script ran `git pull --rebase`. Both commits append to
the end of the same JSON array, so the rebase conflicted and the record
job failed.

On a rejected push, the script now resets to the fresh remote head and
applies the signature to that ledger again instead of replaying its
commit. If the author is already recorded there (for example, they signed
on another repo's PR at the same moment), it skips the write. It retries
up to 5 times with exponential backoff and jitter, then fails with the
last push error.

The ledger format and the post-write status behaviour from #8224 are
unchanged: the commented head and the PR's latest head are still marked
success after the write.

Tests run real git against a local bare remote. They cover a concurrent
append from another repo, the same author recorded concurrently, retry
exhaustion with backoff, and no success status when recording fails.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Escalate to human review

Escalate — gated workflow files require human/CODEOWNERS approval.

This PR modifies .github/workflows/ipr-agreement.yml and .github/workflows/ipr-check-callable.yml, both of which match the gated-path glob .github/workflows/** (gated_paths: true). The current review_decision is REVIEW_REQUIRED, not APPROVED, so the hard approval gate (decision rule row 2) is not satisfied and the outcome cannot be approve.

The change itself is clean on the merits: the reviewer reports no Critical/High/Medium findings. It replaces the rebase-retry loop in check-and-record.mjs with a reset-and-reapply approach (recordSignatureWithRetry + createGitLedgerOps), correctly fixing the cross-repo append conflict on the shared ledger JSON. Logic is idempotent (findSignature guards every write), fails closed on retry exhaustion, and is tested against a real local bare remote. The only non-blocking note is a pre-existing lack of explicit timeout on git fetch/push, bounded by the concurrency group and job timeout and not worsened here.

Escalation reasons

  • .github/workflows/ipr-agreement.yml and .github/workflows/ipr-check-callable.yml are under the hard approval gate (.github/workflows/**) and review_decision is REVIEW_REQUIRED. Human/CODEOWNERS approval is required before merge.

Once a human/CODEOWNERS approval lands (review_decision: APPROVED), this PR would fall through to a clean approve given no blocking findings.

Why human review

  • Gated paths touched without approval: .github/workflows/ipr-agreement.yml and .github/workflows/ipr-check-callable.yml match .github/workflows/** (gated_paths: true) while review_decision is REVIEW_REQUIRED — human/CODEOWNERS approval required.
  • This PR touches a path under a hard, non-overridable approval gate (.github/workflows/ipr-agreement.yml (modified) matches .github/workflows/**; .github/workflows/ipr-check-callable.yml (modified) matches .github/workflows/**) and the current GitHub review decision is 'REVIEW_REQUIRED', not APPROVED. This is a hard gate enforced in code — Ladon cannot auto-approve until a human/CODEOWNERS approval is recorded, regardless of how clean the diff is.

@aao-secretariat aao-secretariat Bot added the ladon/needs-human-review Ladon has escalated this PR for human review. label Oct 16, 2026
@bokelley
bokelley merged commit 4907c42 into main Oct 17, 2026
33 checks passed
@bokelley
bokelley deleted the fix/ipr-signature-retry-reapply branch October 17, 2026 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ladon/needs-human-review Ladon has escalated this PR for human review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant