Skip to content

fix(ci): retry the exact C2PA install fallback failure once - #7520

Open
bokelley wants to merge 2 commits into
mainfrom
conductor/bound-c2pa-ci-install-retry
Open

bokelley wants to merge 2 commits into
mainfrom
conductor/bound-c2pa-ci-install-retry

Conversation

@bokelley

@bokelley bokelley commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

CI's root npm ci now runs through .github/scripts/npm-ci.mjs. It runs a normal npm ci and retries once in one case only: npm exits 1 with the contiguous Linux-x64 C2PA block, which is the prebuilt-binary download failing, the Rust fallback starting, and the published Cargo manifest failing to find a workspace root. The release URL in that block must be built from the exact @contentauth/c2pa-node version in the root package-lock.json. Anything else fails without a retry and keeps npm's exit status or signal: missing or invalid lock metadata, a version mismatch, other npm errors or paths, a log-capture failure, or cancellation.

Why now. This failure took out four shards of #8804's first Build Check attempt on 2026-11-08. Other shards in the same run installed fine, and a rerun passed.

Refresh (2026-11-08)

  • The five commits are squashed into one and rebased onto main fafe043b. The tree before the rebase matched the old head d4e9bb0d exactly, and the rebase applied with no conflicts.
  • main now locks c2pa-node 0.10.0. I checked the published tarball: its scripts/postinstall.cjs is byte-identical to 0.9.7's, its Cargo.toml still has rust-version.workspace = true, and the 0.10.0 release asset URL resolves. So the fingerprint still applies with the version substituted.
  • New tests: 0.10.0 in the version matrix and mixed-version pairs, plus a fixture with the verbatim unhandled-rejection tail npm printed after the signature on docs(agents): pin the base SHA and lease exactly when working in parallel worktrees #8804. That tail must still retry once.
  • I re-listed the install sites on current main. All 12 lifecycle-enabled root installs, across 7 workflows, are covered. These still call npm ci directly, on purpose:
    • apps/web and the checked-out sdk/, which have separate package graphs;
    • release.yml and deploy.yml, which publish and deploy;
    • --ignore-scripts installs, which never run this postinstall.

Review notes

  • Bounded retry. At most two npm ci runs, with an integer delay of 1,000–5,000 ms between them. A delay outside that range throws before a second install. Each retry is a full reinstall, so the second download happens about 40–105 s after the first, judging by the 36–100 s install times on docs(agents): pin the base SHA and lease exactly when working in parallel worktrees #8804. A sustained outage fails both attempts, the same as today.
  • Cancellation. SIGINT, SIGTERM and SIGHUP abort the wait and are forwarded to npm's detached process group, which includes the lifecycle shell and cargo. The wrapper then re-raises the signal. A cancelled install is never retried.
  • Logs. npm output streams live, as before. Full copies go to a fresh adcp-npm-ci-* directory (0700, files 0600) under RUNNER_TEMP. Nothing is uploaded, and there is no environment or config dump. A test checks that a planted env secret never appears.
  • Changeset scope. training-agent-storyboards.yml counts as protocol-scoped. The classifier exempts only the exact run: npm ci → run: node .github/scripts/npm-ci.mjs substitution in that file, and any other edit there is still scoped. The classifier now reads base content at the merge-base, matching its three-dot diff. --has-protocol-scoped-changes reports none for this PR, so no changeset is needed.
  • Without fix(ci): isolate current creative storyboards #7523. This PR doesn't depend on fix(ci): isolate current creative storyboards #7523. fix(ci): isolate current creative storyboards #7523 also edits the classifier and will need a refresh onto main after this lands.

Validation on 3c98bfc1

  • node --test .github/scripts/npm-ci.test.cjs: 74/74 pass.
  • node tests/changeset-protocol-scope.test.cjs passes.
  • server/tests/unit/external-smoke.test.ts and matched-v4-provider-billing-reconciliation.test.ts: 61/61 pass.
  • tests/test-coverage-registration.test.cjs and tests/release-workflow-immutability.test.cjs pass.

I haven't seen a real 0.10.0 download failure, only the matching 0.9.7 failure plus the identical installer source.

Brian authored this PR, so a different maintainer has to approve it.

🤖 Generated with Claude Code

bokelley and others added 2 commits November 8, 2026 20:45
Squash of the five reviewed commits on this branch (982c602, bb811c5,
553cdb6, a7bf7db, d4e9bb0), rebased onto current main.

Root `npm ci` in CI goes through .github/scripts/npm-ci.mjs, which retries
once, after 1-5 s of jitter, only when npm exits 1 with the contiguous
Linux-x64 C2PA release-download / workspace-fallback block for the version
the root lockfile installs. Cancellation aborts the wait and is forwarded to
npm's process group; complete logs stay in a private runner-local directory.
The changeset scope check exempts only the exact install-command substitution
in the storyboard workflow and reads base content at the merge-base.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tail

main now locks @contentauth/c2pa-node 0.10.0, whose postinstall.cjs is
byte-identical to 0.9.7's and whose Cargo manifest still inherits workspace
fields. Adds 0.10.0 to the version matrix and mixed-version pairs, and a
fixture with the verbatim unhandled-rejection tail npm printed after the
signature in #8804's failed install, which must still retry once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bokelley
bokelley force-pushed the conductor/bound-c2pa-ci-install-retry branch from d4e9bb0 to 3c98bfc Compare November 8, 2026 20:45
@bokelley
bokelley marked this pull request as ready for review November 8, 2026 20:46

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Escalate to human review

Escalate — gated paths touched without required approval.

This PR adds a CI-only npm ci wrapper to handle C2PA prebuilt-download/workspace-fallback failures. The reviewer found no actionable issues (no protocol surface touched, fail-closed retry logic well-tested at 74/74 + scope tests, changeset scope correct). The diff itself looks clean.

However, the PR modifies seven files under .github/workflows/**, which are declared gated paths (gated_paths: true). The required review status is REVIEW_REQUIRED, not APPROVED. Per decision table row 2, a PR touching gated paths cannot be auto-approved until a human/CODEOWNERS approval is recorded.

Escalation reasons

  • Modifies gated workflow files (.github/workflows/broken-links.yml, build-check.yml, external-smoke.yml, migration-smoke-test.yml, training-agent-storyboards.yml, validate-schema-bundle.yml, verify-matched-v4-runtime-attestation.yml) under hard approval gate .github/workflows/**; review_decision is REVIEW_REQUIRED. Human/CODEOWNERS review required before merge.

No blocking code findings — this is purely the deterministic gated-path approval gate. Once a human approval lands, a clean re-run falls through to approve.

Why human review

  • Modifies gated workflow files under .github/workflows/** (broken-links.yml, build-check.yml, external-smoke.yml, migration-smoke-test.yml, training-agent-storyboards.yml, validate-schema-bundle.yml, verify-matched-v4-runtime-attestation.yml) while review_decision is REVIEW_REQUIRED, not APPROVED — human/CODEOWNERS review required (decision table row 2).
  • This PR touches a path under a hard, non-overridable approval gate (.github/workflows/broken-links.yml (modified) matches .github/workflows/**; .github/workflows/build-check.yml (modified) matches .github/workflows/**; .github/workflows/external-smoke.yml (modified) matches .github/workflows/**; .github/workflows/migration-smoke-test.yml (modified) matches .github/workflows/**; .github/workflows/training-agent-storyboards.yml (modified) matches .github/workflows/**; .github/workflows/validate-schema-bundle.yml (modified) matches .github/workflows/**; .github/workflows/verify-matched-v4-runtime-attestation.yml (modified) matches .github/workflows/**) and the current GitHub review decision is 'REVIEW_REQUIRED', not APPROVED. This is a hard gate enforced in code — Ladon cannot auto-approve until a human/CODEOWNERS approval is recorded, regardless of how clean the diff is.

@aao-secretariat aao-secretariat Bot added the ladon/needs-human-review Ladon has escalated this PR for human review. label Nov 8, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ladon/needs-human-review Ladon has escalated this PR for human review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant