Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions .github/scripts/npm-ci.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# CI npm install retry

Run `node .github/scripts/npm-ci.mjs` from the repository root on Linux x64.
It executes ordinary `npm ci` and permits **one retry only** when exit 1 and
the contiguous npm stderr block identify the exact release download of the C2PA
version locked in the root `package-lock.json` (`packages["node_modules/@contentauth/c2pa-node"].version`),
the Rust fallback, and the missing workspace manifest. The release URL and
filename are built from that one exact `x.y.z` version. A missing, unreadable,
malformed or non-exact (range, prerelease, leading-zero) lockfile version fails
closed: no retry, and npm's original exit status is preserved. Other versions
(including a stderr version that differs from the lock, or a URL whose tag and
filename disagree), platforms, partial signatures, additional npm error
codes/paths, and signals do not retry.
A second failure remains a job failure with its own exit status or signal.
Before the sole retry, wait a uniformly randomized integer delay of 1,000–5,000
milliseconds. Cancellation aborts that wait and prevents the second install.
Tests inject randomness and a clock through the module API; the CLI always uses
the bounded production jitter and timer, with no environment override.

The matcher comes from [main job 104032687845](https://github.com/adcontextprotocol/adcp/actions/runs/34860948128/job/104032687845)
on 2026-09-14. The same release installed in other jobs of that run, and
[#7511's failed install](https://github.com/adcontextprotocol/adcp/actions/runs/34857319525/job/104020125889)
[passed unchanged](https://github.com/adcontextprotocol/adcp/actions/runs/34857319525/job/104028425924).
Upstream returns false on a non-2xx download without logging the HTTP status;
the fallback then fails because the published Cargo manifest inherits workspace
fields without a workspace root. The logs cannot establish the exact HTTP code.
The same block, followed by the installer's unhandled-rejection trace, failed
four shards of [#8804's first Build Check attempt](https://github.com/adcontextprotocol/adcp/actions/runs/42472661157/attempts/1)
on 2026-11-08 with 0.9.7 locked; the rerun passed.
The published 0.9.7, 0.9.9 and 0.10.0 Cargo manifests still inherit workspace
fields, and 0.10.0 ships the same `postinstall.cjs` as 0.9.7, so the underlying
fallback bug is not fixed upstream; the matcher follows the locked version
instead of a hardcoded one. Keep the match narrow; reassess/remove
this workaround when upstream fixes the installer. This is not a general npm
retry policy.

Both attempts stream stdout/stderr live, with backpressure, and retain separate
complete files in a unique `adcp-npm-ci-*` directory under `RUNNER_TEMP` (or the
OS temporary directory). The path is printed, directory permissions are 0700,
and files are 0600. Capture or live-forwarding failure disables retry. If a live
log consumer disconnects, the wrapper continues draining npm into the private
logs and preserves npm's outcome. No environment/config dump,
shell evaluation, log replay, or artifact upload is added. GitHub's existing
masking applies to the live streams; runner-local files are raw npm output and
must not be uploaded as public artifacts. They live until runner cleanup.
SIGINT/SIGTERM/SIGHUP are forwarded to npm's process group, including lifecycle
children, and the wrapper preserves signal termination even if npm traps it.

The helper covers 12 root installs: Build Check (4 job definitions), migration
smoke (2), training storyboards (2), broken links, external smoke, schema PR
bundle construction, and the untrusted runtime-attestation build. Existing workflow gates, environment
variables, permissions and timeouts remain in force. `release.yml` and
`deploy.yml` stay outside this CI-only change, including release verification.
The separate `apps/web` and checked-out `sdk` packages are excluded, as is the
existing `sdk-response-conformance.yml` install with scripts disabled (which
cannot execute this postinstall). No dependency, lockfile or install policy changes.

Offline tests run before Build Check's install and can also be run locally:

```sh
node --test .github/scripts/npm-ci.test.cjs
node tests/changeset-protocol-scope.test.cjs
```

No protocol changeset is appropriate for this operational change. The scope
check exempts only the exact install-command substitution in the otherwise
protocol-scoped storyboard workflow; any other edit there remains scoped.

External smoke isolation has landed on `main`; its install step adopts this
helper with its `PUPPETEER_SKIP_DOWNLOAD` environment and permissions unchanged.
Draft #7514 owns PostgreSQL test barriers. This helper only addresses the shared
dependency-install failure.
168 changes: 168 additions & 0 deletions .github/scripts/npm-ci.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
#!/usr/bin/env node
// CI-only, Linux x64: one normal npm ci retry for the C2PA release-download/workspace fallback failure of the lockfile's version.
import { spawn } from 'node:child_process';
import { randomInt } from 'node:crypto';
import { setTimeout as delay } from 'node:timers/promises';
import { fileURLToPath } from 'node:url';
import { createReadStream, mkdtempSync, openSync, closeSync, readFileSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { createInterface } from 'node:readline';
import { Writable } from 'node:stream';
import { pipeline } from 'node:stream/promises';

// The only version that may match is the one the root lockfile installs. Any
// missing, unreadable or non-exact-semver metadata yields null (no retry).
function lockedC2paVersion() {
try {
const lock = JSON.parse(readFileSync(join(process.cwd(), 'package-lock.json'), 'utf8'));
const version = lock?.packages?.['node_modules/@contentauth/c2pa-node']?.version;
return typeof version === 'string' && /^(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)$/.test(version) ? version : null;
} catch {
return null;
}
}

async function matchesFailure(log) {
const version = lockedC2paVersion();
if (!version) return false;
const root = join(process.cwd(), 'node_modules/@contentauth/c2pa-node');
// Match npm's contiguous stderr error block, including the package path,
// exact release/version/platform, download-to-build transition and manifest.
// Checking for a release alone does NOT establish a failed download.
const expected = [
'npm error code 1',
`npm error path ${root}`,
'npm error command failed',
'npm error command sh -c node scripts/postinstall.cjs',
"npm error Detected { arch: 'x64', platform: 'linux' }",
`npm error Checking for a release at: https://github.com/contentauth/c2pa-js/releases/download/%40contentauth%2Fc2pa-node%40${version}/c2pa-node_x86_64-unknown-linux-gnu-v${version}.zip`,
'npm error 🦀 Building Rust...',
`npm error ERROR: Error: Command failed: npx cargo-cp-artifact -nc "${root}/dist/index.node" -- cargo build --message-format=json-render-diagnostics --release --manifest-path="${root}/Cargo.toml"`,
`npm error error: failed to parse manifest at \`${root}/Cargo.toml\``,
'npm error',
'npm error Caused by:',
"npm error error inheriting `rust-version` from workspace root manifest's `workspace.package.rust-version`",
'npm error',
'npm error Caused by:',
'npm error failed to find a workspace root',
'npm error Did not copy "cdylib:c2pa-node"',
];
let index = 0;
let codes = 0;
let paths = 0;
for await (const raw of createInterface({ input: createReadStream(log), crlfDelay: Infinity })) {
const line = raw.trimEnd();
if (line.startsWith('npm error code ')) codes++;
if (line.startsWith('npm error path ')) paths++;
if (index === 0 && line !== expected[0]) continue;
if (index < expected.length) {
if (line !== expected[index]) return false;
index++;
}
}
return index === expected.length && codes === 1 && paths === 1;
}

// Injectable clock/randomness for offline tests; the CLI always uses these defaults.
export async function runNpmCi({ jitter = randomInt, sleep = delay } = {}) {
const outputs = [process.stdout, process.stderr];
const failedOutputs = new Set();
const outputHandlers = outputs.map(output => () => failedOutputs.add(output));
// A disconnected live-log consumer must not crash the wrapper or stop capture.
outputs.forEach((output, index) => output.on('error', outputHandlers[index]));
const logDir = mkdtempSync(join(process.env.RUNNER_TEMP || tmpdir(), 'adcp-npm-ci-'));
console.error(`npm ci logs (runner-local): ${JSON.stringify(logDir)}`);
const backoff = new AbortController();
let child;
let interrupted;
const signals = ['SIGINT', 'SIGTERM', 'SIGHUP'];
function forwardSignal(signal) {
interrupted ??= signal;
backoff.abort();
if (child?.pid) {
try {
// npm's lifecycle shell and Cargo must receive cancellation too.
process.kill(-child.pid, signal);
} catch (error) {
if (error.code !== 'ESRCH') throw error;
}
}
}
const handlers = signals.map(signal => () => forwardSignal(signal));
signals.forEach((signal, index) => process.on(signal, handlers[index]));

try {
for (let attempt = 1; attempt <= 2; attempt++) {
if (interrupted) break;
let captureFailed = false;
const files = ['stdout', 'stderr'].map(stream => join(logDir, `attempt-${attempt}.${stream}.log`));
const fds = files.map(file => openSync(file, 'wx', 0o600));
// Fixed executable/arguments, no shell or flags; inherit ordinary npm config.
child = spawn('npm', ['ci'], { detached: true, stdio: ['inherit', 'pipe', 'pipe'] });
const result = new Promise(resolve => {
child.once('error', error => resolve({ code: error.code === 'ENOENT' ? 127 : 126 }));
child.once('close', (code, signal) => resolve({ code, signal }));
});
const copies = [child.stdout, child.stderr].map((input, index) => pipeline(input, new Writable({
write(chunk, encoding, callback) {
try {
writeFileSync(fds[index], chunk);
} catch {
captureFailed = true;
}
// Backpressure and completion callbacks preserve live, untruncated output.
const output = outputs[index];
if (failedOutputs.has(output)) return callback();
output.write(chunk, encoding, error => {
if (error) failedOutputs.add(output);
// Always keep draining npm into the private log, even after EPIPE.
callback();
});
},
})));
const copying = Promise.allSettled(copies);
const { code, signal } = await result;
const copied = await copying;
child = undefined;
fds.forEach(fd => closeSync(fd));
captureFailed ||= copied.some(copy => copy.status === 'rejected');
if (captureFailed || failedOutputs.size) console.error('npm ci log capture/forwarding failed; retry disabled.');
interrupted ??= signal;
process.exitCode = code ?? 1;
if (interrupted || code !== 1 || attempt === 2 || captureFailed || failedOutputs.size || !(await matchesFailure(files[1]))) break;
const milliseconds = jitter(1000, 5001);
if (!Number.isInteger(milliseconds) || milliseconds < 1000 || milliseconds > 5000) {
throw new RangeError('CI install retry delay must be between 1000 and 5000 ms.');
}
console.error(`Known locked C2PA download/workspace fallback failure; waiting ${milliseconds} ms before retry.`);
try {
await sleep(milliseconds, undefined, { signal: backoff.signal });
} catch (error) {
if (!interrupted || error.name !== 'AbortError') throw error;
}
if (interrupted || failedOutputs.size) break;
// Observe an async write failure before starting another detached install.
await new Promise(resolve => process.stderr.write('retrying normal npm ci once (attempt 2/2).\n', error => {
if (error) failedOutputs.add(process.stderr);
resolve();
}));
if (interrupted || failedOutputs.size) break;
}
} finally {
// Drain diagnostics while error/signal handlers are still installed. The
// next event-loop turn also lets write callbacks' pending error events fire.
await Promise.all(outputs.map(output => new Promise(resolve => {
if (output.destroyed) resolve();
else output.write('', resolve);
})));
await new Promise(resolve => setImmediate(resolve));
outputs.forEach((output, index) => output.off('error', outputHandlers[index]));
signals.forEach((signal, index) => process.off(signal, handlers[index]));
}
if (interrupted) process.kill(process.pid, interrupted);
}

if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
await runNpmCi();
}
Loading
Loading