Adopt check-supply-chain: verify pin truth and register agreement

[sgort]

ttl-editor now carries scripts/check-supply-chain.mjs, a preflight that closes the two gaps SECURITY-PIPELINE.md has recorded since the pinning work landed. This repository should adopt it.

Upstream: sgort/ttl-editor#75 (merged) · sgort/ttl-editor#86 (a fix this repository found) · sgort/ttl-editor#76 (wiring it into the audit job) · sgort/ronl-business-api has the same issue open.

What it checks that zizmor cannot

zizmor validates pin format — that uses: names a 40-character SHA. It cannot say the SHA is the right one, so a wrong or hostile digest carrying a plausible # v4.4.0 comment passes zizmor, Prettier and review alike.

1. Pin truth — each digest is resolved against the GitHub API and compared with the version its comment claims.
2. Register agreement — the Pinned table is compared with the workflows: digests, versions, (×N) multiplicities, and the totals headline where one exists.

This repository found two bugs in it

Worth leading with, because it is the reason adoption here is straightforward now and would not have been a week ago. Run against this repository's acc, the script reported two findings — and both were the script's fault, not yours:

• actions/checkout is pinned at two digests here — 11d5960a… (v4.4.0) in three workflows and a37ce912… (v3.7.0) in four, mid-upgrade — and SECURITY-PIPELINE.md correctly records both rows. The script keyed register rows by action alone, so the second overwrote the first and every workflow on the other digest read as disagreeing.
• Azure/static-web-apps-deploy is recorded as v1 (branch head), which is more informative than the bare v1 the workflow comment carries — the whole point of that pin being that v1 names both a 2021 tag and a 2024 branch head. Exact-string comparison called it a mismatch.

Neither could surface in ttl-editor, which pins every action once and annotates nothing — the same shape as the defect the script exists to catch, a check that only ever ran where its assumptions hold. Fixed in ttl-editor#86; rows are matched by digest and versions compared on the leading token.

It runs clean here now

check-supply-chain: 23 pinned reference(s) across 5 action(s) in .github/workflows/
  note: Azure/static-web-apps-deploy@v1 verified against the branch, per its recorded exception

OK — digests, version comments and the register all agree.

Adoption lands green.

And this repository is why the register half matters

SECURITY-PIPELINE.md in ttl-editor cites what happened here: five keys used as JSON comments in renovate.json were rejected as invalid configuration and Renovate stopped opening pull requests as a precaution — and nothing in CI noticed. The repository looked green while half its supply-chain policy was inert.

That specific failure is caught by the renovate-config-validator --strict step, not by this script. They are the two halves of the same argument: pinning without working automated updates decays into an unpatched tree, and a register that has drifted from the workflows describes a policy nobody is actually running. Worth adopting both if the validator step is not here yet.

Adoption

Copy scripts/check-supply-chain.mjs verbatim — take it from acc after #86 merges, or the two false positives above come with it — and add:

"check-supply-chain": "node scripts/check-supply-chain.mjs"

No configuration is repo-specific. The script reads SECURITY-PIPELINE.md and .github/workflows/ relative to the working directory, so run it from the repository root.

Then decide whether it gates

Same decision as ttl-editor#76, and a real trade-off: adding it to the audit job puts a network dependency inside a blocking gate, so a required check could fail on an API outage or rate limit. --offline checks format and register agreement without resolving pins.

1. npm run check-supply-chain only — zero risk, catches drift where someone is already looking.
2. A step in the existing audit job with continue-on-error: true, flipped to blocking after a week or two — mirroring how the gate itself was introduced. Needs GITHUB_TOKEN: ${{ github.token }}; contents: read suffices for public refs.
3. Blocking immediately — defensible since it is green today, but with no evidence yet about flakiness.

A step in the existing job rather than a new job: the ruleset requires the check named audit, so a step is covered automatically, while a new job would need adding to the ruleset first and would silently not block until it was.

One thing to decide alongside

The two checkout digests are a live upgrade, not a steady state — four workflows are still on v3.7.0. The script is happy either way, since the register records both. But once they converge, the second register row should go with them, and the script will say so as a note (SECURITY-PIPELINE.md lists actions/checkout at a37ce912… (v3.7.0), which no workflow currently uses) rather than a finding.

Acceptance

• npm run check-supply-chain exits 0 on acc.
• A planted wrong digest with a plausible version comment fails, naming the action and every digest the register records for it.
• A digest changed without updating SECURITY-PIPELINE.md fails.
• A Renovate pull request that bumps a digest and rewrites the version comment still passes — confirm against a real Renovate PR rather than assuming, since this is the case that would make the check hostile to routine updates if it were wrong.

[sgort]

Cross-links, now that both replication issues exist:

• Pilot: sgort/ttl-editor#75 — merged, the script itself
• The fix this repository surfaced: sgort/ttl-editor#86
• Wiring decision: sgort/ttl-editor#76
• Sibling: sgort/ronl-business-api#81
• Related here: CI: check-format runs in no workflow #45 (check-format runs in no workflow) and CI: the backend runs no tests on a pull request #46 (the backend runs no tests on a pull request) — same class: a standard that nothing actually executes

[sgort]

Adopted in #77 as a non-blocking step in the audit job. Three of the four acceptance boxes pass; the fourth does not, and it is the interesting one.

Boxes 1–3 pass

npm run check-supply-chain exits 0 on acc — 23 references across 5 actions, register agrees, with the Azure/static-web-apps-deploy branch-head note.

A planted wrong digest with a plausible comment fails and names every digest the register records for that action:

[register] actions/checkout: workflow pins deadbeef1234… (v4.4.0) but
           SECURITY-PIPELINE.md records only 11d5960a3267… (v4.4.0), a37ce9120846… (v3.7.0)
[pin-truth] actions/checkout: comment claims v4.4.0, which resolves to
            11d5960a3267… — but the workflow pins deadbeef1234…

Confirming the ttl-editor#86 fix on the split-pin shape this repository has and ttl-editor does not.

Box 4 does not pass

A Renovate pull request that bumps a digest and rewrites the version comment still passes — confirm against a real Renovate PR rather than assuming.

Confirmed against a real one, and it fails. Fetching the tree of #66 (actions/checkout → v7.0.1) and running the check against it:

[register] actions/checkout: workflow pins 3d3c42e5aac5… (v7.0.1) but
           SECURITY-PIPELINE.md records only 11d5960a3267… (v4.4.0), a37ce9120846… (v3.7.0)

Pin truth passes. Renovate's digest and its rewritten comment agree with each other and with GitHub — the half the box was worried about is fine. What fails is register agreement, because Renovate maintains workflow pins and never touches SECURITY-PIPELINE.md. The check is right; the register is stale.

Worth separating those two, because the box conflates them: the check is not hostile to Renovate's rewrite, it is intolerant of the register lagging behind it. That is the drift it exists to catch, arriving by the most routine route there is.

The decision

The register is updated on the Renovate pull request's branch, before merging — not after.

The distinction is load-bearing rather than stylistic. The check runs on the pull request, so a register fixed after the merge leaves the check red for that pull request's whole life, and makes the step impossible to promote to blocking: no Renovate bump could ever present a green result to merge on. Recorded in SECURITY-PIPELINE.md under Keeping this register true, alongside the #66 output as evidence.

For the other repositories

sgort/ronl-business-api has the same issue open. Two things transfer:

1. Box 4 will fail there too — it is a property of Renovate not touching the register, not of anything specific here. Adopt non-blocking, and settle the register-update habit before promoting.
2. Check the split-pin shape before adopting. The false positives ttl-editor#86 fixed only surface where an action is pinned at two digests or the register annotates a version (v1 (branch head)). Worth running once locally before wiring it into a gate.

Left open deliberately

The two checkout rows are a live upgrade, not a resting state. #66 converges all seven references to v7.0.1, so both rows collapse into one when it merges — that is the first real exercise of the policy above, and the register edit belongs on #66's branch.