ci: adopt check-supply-chain as a non-blocking step in the audit job - #77
Merged
Merged
Conversation
zizmor validates pin FORMAT — that a `uses:` names a 40-character SHA. It cannot say the SHA is the right one, so a wrong or hostile digest carrying a plausible "# v4.4.0" comment passes zizmor, Prettier and review alike. check-supply-chain.mjs closes that gap: it resolves every digest against the GitHub API and compares SECURITY-PIPELINE.md's register with the workflows — digests, versions and multiplicities. Copied verbatim from ttl-editor's acc, after sgort/ttl-editor#86. Taking it before that fix would have imported two false positives this repository is uniquely shaped to trigger: actions/checkout is pinned at two digests here (v4.4.0 in three workflows, v3.7.0 in four, mid-upgrade) and the register correctly records both rows, while Azure/static-web-apps-deploy is recorded as "v1 (branch head)" — more informative than the bare "v1" the workflow comment carries. Rows are now matched by digest and versions compared on the leading token. Left unformatted deliberately: Prettier does not cover scripts/, and reformatting would diverge from upstream for no benefit. ── Why non-blocking, and what it is waiting on ── Not caution alone. Renovate rewrites workflow pins and their version comments together, correctly, and never touches SECURITY-PIPELINE.md. So every action-bump pull request fails the register half until the register is updated by hand. Verified against the real thing rather than assumed. Run against the tree of #66 (actions/checkout to v7.0.1): [register] actions/checkout: workflow pins 3d3c42e5aac5… (v7.0.1) but SECURITY-PIPELINE.md records only 11d5960a3267… (v4.4.0), a37ce9120846… (v3.7.0) Pin truth passes there — Renovate's digest and comment agree with each other and with GitHub. The check is right and the register is stale, which is exactly what it exists to say. Blocking on that would fail a required check on routine dependency updates, which is how gates get resented and then bypassed. The agreed policy, recorded in SECURITY-PIPELINE.md: the register is updated ON the Renovate pull request's branch, before merging. Not afterwards — the check runs on the pull request, so a register fixed after the merge leaves it red for that pull request's whole life and makes this step impossible to promote, because no bump could ever show a green result to merge on. A second reason to wait: this puts a network call inside a job the ruleset requires, so an API outage or rate limit could fail a gate unrelated to the change under review. --offline checks format and register agreement without resolving pins if that proves to be a problem. ── Placement ── A step in the existing audit job, not a new job: the ruleset requires the check named `audit`, so a step is covered automatically, while a new job would silently not block until someone added it to the ruleset. if: always() for the same reason as the config-validator step above it — one run should report on every half of the policy rather than stopping at the first failure. Adds no `uses:` line, so the register's own totals stay accurate at 23 references across 5 actions. ── Verification ── Exits 0 on acc. A planted wrong digest with a plausible version comment fails and names every digest the register records for that action, which is the #86 fix working on this repository's split pin. A digest changed without updating the register fails. The Renovate case is the one that does not pass, documented above and the reason for continue-on-error. semgrep clean over 487 rules; the workflow parses and the step carries continue-on-error: true with if: always().
4 tasks
This was referenced Sep 8, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #73.
Adopts
scripts/check-supply-chain.mjsfrom ttl-editor, addsnpm run check-supply-chain, and wires it into the existingauditjob as a non-blocking step.What it checks that zizmor cannot
zizmor validates pin format — that a
uses:names a 40-character SHA. It cannot say the SHA is the right one, so a wrong or hostile digest carrying a plausible# v4.4.0comment passes zizmor, Prettier and human review alike. This resolves every digest against the GitHub API and comparesSECURITY-PIPELINE.md's register with the workflows: digests, versions, and(×N)multiplicities.Copied verbatim from ttl-editor
accafter ttl-editor#86 — confirmed by checking that the last commit touching the file is that fix. Taking it earlier would have imported the two false positives this repository is uniquely shaped to trigger.Left unformatted deliberately: Prettier doesn't cover
scripts/, and reformatting would diverge from upstream for no benefit.zizmor.ymllikewise already failsprettier --checkidentically onacc, so I haven't reformatted that either.Acceptance
npm run check-supply-chainexits 0 onaccBox 2, since it is the one the upstream fix was about:
Both recorded digests are named. Before ttl-editor#86 the second row overwrote the first and every workflow on the other digest read as disagreeing.
Box 4 is why this is non-blocking
Tested against the real thing, not reasoned about. Fetching the tree of #66 (
actions/checkout→ v7.0.1) and running the check against it:Pin truth passes. Renovate's digest and its rewritten comment agree with each other and with GitHub. The failure is purely register drift — Renovate maintains workflow pins and never touches
SECURITY-PIPELINE.md. The check is right and the register is stale, which is exactly what it exists to say.Blocking on that would fail a required check on routine dependency updates, which is how gates get resented and then bypassed.
The policy this establishes
Recorded in
SECURITY-PIPELINE.mdunder Keeping this register true: when a Renovate pull request bumps an action, the register is updated on that pull request's branch, before merging.Not afterwards — and the distinction is load-bearing. The check runs on the pull request, so a register fixed after the merge leaves the check red for that pull request's entire life, and makes this step impossible to promote to blocking, because no Renovate bump could ever present a green result to merge on.
Promote to blocking once that habit holds.
Expected immediately after merge
#66 and #67 will show the register finding. That is the check working, not a regression. Under the policy above, each needs a
SECURITY-PIPELINE.mdcommit pushed to its branch before merging. #66 is the interesting one: it converges all sevencheckoutreferences to v7.0.1, so the two existing rows collapse into one.Placement
A step in the existing
auditjob rather than a new job: the ruleset requires the check namedaudit, so a step is covered automatically, while a new job would silently not block until someone added it to the ruleset.if: always()for the same reason as the config-validator step above it — one run should report on every half of the policy rather than stopping at the first failure.Adds no
uses:line, so the register's own totals stay accurate at 23 references across 5 actions.Also verified
semgrep clean across 487 rules over the new script and changed files; the workflow parses with 5 audit steps and the new one carrying
continue-on-error: trueandif: always();GITHUB_TOKEN: ${{ github.token }}with the job's existingcontents: read, which suffices for public refs.