Repository navigation
fix(cdn): publish and verify compliance JSONL artifacts - #7509
Conversation
bc2a3b0 to
90f0cc5
Compare
90f0cc5 to
3b6fd2d
Compare
|
RC5 train refresh: stacked the exact current #7510 fence head ( |
|
The predecessor #7510 is now landed on |
There was a problem hiding this comment.
Ladon verdict: Approve
Approve — operational CDN publisher fix, no blocking or medium findings.
This PR adds *.jsonl (application/x-ndjson; charset=utf-8) support to the backfill sync/cp compliance filters, a matching Worker MIME fallback, and a local-bytes compliance enumeration in the cutover verifier. Scope is operational tooling only: scripts/**, workers/artifact-cdn/**, tests, and package.json.
Checks performed:
- No
static/schemas/source/**,docs/reference/**, ordist/**artifacts modified — no wire-shape change, so schema↔docs coherence and released-artifact-immutability rules are not implicated. - Changed scripts are outside PROTOCOL_SCOPED_PATHS, so the absent changeset is correct (operational-only PR).
- Largest change (tests/cdn-artifacts.test.cjs, +421) confirms version gating, path-traversal rejection, and faithful AWS-filter simulation.
Decision-table walk: no critical/high findings (row 1 n/a); gated_paths false (row 2 n/a); high_risk false (rows 3, 5 n/a); no medium findings (rows 4, 8 n/a); no prior escalation (row 6 n/a); no no-auto-approve team match (row 7 n/a). Falls through to row 9 → approve. Note: review_decision is REVIEW_REQUIRED, but gated_paths is false, so row 2 does not fire.
Current RC5-train status (2026-09-23)
Refreshed at exact head
ffaed0439224920fc4a6d2165b40543c5b4ebbeb, stacked on the current #7510 fence head. Relative to that fence, this remains exactly 7 files, +688/-5. The combined release command passed 149/149 publication/fence tests, 9/9 compliance-version tests, 53/53 JSONL publisher/Worker tests, plus promotion and immutability. All review threads are resolved/outdated. This PR intentionally remains Draft and merge-held until #7510 lands; this paragraph supersedes older head/status details below.The deploy/release artifact publisher omitted nested compliance JSON Lines files. Both compliance upload filters now include
*.jsonlwithapplication/x-ndjson; charset=utf-8, and the cutover verifier checks pinned compliance files against exact local bytes. Draft only; auto-merge is off.Hard merge-order dependency — #7510 MUST land first
RELEASE-SECURITY / PREDECESSOR HOLD remains. Superseded #7510
bf980c03was blocked by current-collaborator permission and stale-generated-PR provenance/race defects. The replacement candidate is now fetched at32496b54e8fcdef46a12b077a9426eb7193ac522, treed8f8ae9ccc870bcdd50b33aa102f8cdec5d54d75, on main336546b0dc2b1af9f12b955bddcfd74e002a23f9. This task verified all nine range-diff entries, all 15 touched blobs and the complete patch are identical to reviewed predecessor01f90fec. Independent approval remains limited to the Draft lease push. This task freshly verified terminal CI on32496b54: 55 successful CheckRuns plus one successful IPR status (56 successful rollup contexts), one expected Draft advisory skip, and zero pending/failing contexts. The owner reports nativebmilekicandEmmaLouise2018requests renewed at 21:29:46 UTC after green, with new timeline events verified; requests are not human approval. Earlier head CI, including the historical541d258dCodeQL failure and its01f90fecfixture correction, does not qualify this rebased head. That is not human approval or landing; #7509 stays held unchanged. Priorafaef188/bf980c03CI/reviews do not qualify this replacement or clear the successor hold.The fetched candidate registers
tests/changesets-fence.test.cjsalongside ordering and introducesscripts/fence-changesets.cjs. Futuretest:release-workflowcomposition must retain the Changesets fence, ordering, CDN-artifact and Worker suites exactly once each, together with existing release checks. The historical command below is not the future union. No predecessor code is absorbed into held #7509. Existing fixtures provide exactly four GitHub assets; their full permission/provenance/conditional-write contract still needs requalification on actual landed main.Recovery authority is unqualified. The owner reports that strict merge-provenance validation quarantines original rc.3 target
71f9cd5414454e94ccfef87ce25777ead6fad228. The previously described ordinary rc.3 recovery-dispatch route is withdrawn; do not treat the original target or that route as usable recovery authority. No bypass or exception has been added. Any later recovery needs a separately reviewed provenance/recovery plan and explicit authorization. All recovery, deployment and publication operations remain held; this update requests no readiness, merge or live operation.HOLD: #7509 MUST NOT land before human-reviewed #7510. Keep this PR Draft with auto-merge off. Even green CI on both PRs and a clean synthetic combined-tree review do not remove this hard dependency or establish that #7509 is safe to land first. This update requests neither readiness nor merge.
At exact base
36a86d6c4ae5c42e5e2e783d95bdb97b5cb6f614,.github/workflows/deploy.yml:508still invokes the legacy live bulk backfill with--build-latest. This PR adds JSONL to that publisher's compliance enumeration. Landing it first would let automatic deploy create the four omitted historical rc.0–rc.3 JSONL objects without explicit recovery authorization. Merely coordinating deployment is insufficient; the publication gate must land first.Required sequence:
32496b54e8fcdef46a12b077a9426eb7193ac522(or fully requalify any subsequent replacement). Its independent Draft-only review and exact CI are now complete, but neither is human approval or landing. Supersededbf980c03must not land. Human maintainers must approve and land the repaired fix(release): fence publication and recover superseded releases #7510 first. It remains Draft and unmerged, so the prerequisite main state does not exist. No readiness or merge is requested by this task.3b6fd2d124c691370d97b8e57d0fa23bc0a2563f; do not rebase or push. Then fetch and rebase onto the resulting current main. Resolve package registration against the actual landed command, retaining the new Changesets fence, ordering, CDN-artifact and Worker suites once each alongside existing checks; preserve the JSONL patch and distinct historical-authority fixtures.RELEASE_SHArejection before AWS. Repeat the exact focused/release/current-main tests, all required CI and the immutable-artifact/inventory audit. Obtain fresh independent exact-head review and human review. Current3b6fd2d1CI and synthetic composition evidence are historical inputs to that later review, not substitutes for it.Exact review inputs
36a86d6c4ae5c42e5e2e783d95bdb97b5cb6f614(current fetched main is336546b0dc2b1af9f12b955bddcfd74e002a23f9; no fix(cdn): publish and verify compliance JSONL artifacts #7509 rebase or current-main qualification has occurred)3b6fd2d124c691370d97b8e57d0fa23bc0a2563fcab6a6579f08aec4d6a887911840ddf277ab31b7git diff 36a86d6c4ae5c42e5e2e783d95bdb97b5cb6f614...3b6fd2d124c691370d97b8e57d0fa23bc0a2563f— seven files, +688/-5.8af531fc6c8119ef68c8a06b812e3c0f1f5c2c4387140fbed0f9e74ea6f2b467, identical to reviewed90f0cc5eon98fa207c.98fa207conto36a86d6c. All five range-diff entries are=; all seven touched final file blobs are identical, including all four distinct historical-tag fixtures. Main's intervening fix(training-agent): keep shared creative repair reads coherent #7516/fix(addie): preserve certification workflows and report tool failures #7515/fix(certification): route si_* tools proactively to /si tenant #7512 server changes do not overlap this PR. No production patch changed.967b0f16-b0b4-4fac-9936-a83534ed3ac3owns the review. Prior implementation, fixture and composition reviews remain historical evidence.refs/heads/conductor/fix-jsonl-artifact-publication:90f0cc5ed60d13ac1fdb57a982cefdf0d4a82d77. Main and remote PR head were rechecked immediately before the successful authorized lease push; the remote now points to the exact reviewed rebased head above. Post-push main remains36a86d6c.Scope and integration
Both
deploy.ymlandrelease.ymlcall the shared backfill helper. This PR adds only two JSONL filter lines to that publisher. Existing extension handling, schema indexes/pointers, protocol tuple behavior, and cache policies remain unchanged by this PR. The Worker adds a JSONL MIME fallback when R2 metadata is absent; alias/fallback and true-404 behavior are unchanged.The verifier recursively enumerates every regular file in local semver compliance trees without an extension allowlist. It requires HTTP 200, no redirects, and exact bytes; JSONL additionally requires its MIME and immutable cache headers. Repeatable
--compliance-versionchecks only selected local trees against the candidate, without reference discovery or mutable aliases. Default verification retains schema/protocol discovery checks; matching remote errors now fail. This is an operator verification path, not a new release workflow gate.Build/compliance and tarball enumeration already include JSONL. All four existing tarballs contain the correct bytes. Other publishers handle protocol/PR-bundle tuples or Git/Mintlify documentation, not individual compliance objects. The committed semver compliance inventory contains YAML, JSON, Markdown and JSONL; root-level compiled
.js/.d.ts/.mapfiles are not release assets. The completedistsubtree is identical at the original base, current base and head:16839080cf3ce1c9f1e140080e14988f56eacddf. No artifact regeneration or changeset is needed.Draft #7510 owns publication gates, invocation modes, freshness, concurrency and recovery. Its fetched prerequisite candidate is
32496b54e8fcdef46a12b077a9426eb7193ac522, treed8f8ae9ccc870bcdd50b33aa102f8cdec5d54d75, based on336546b0dc2b1af9f12b955bddcfd74e002a23f9. This task verified all nine rebased patches and 15 touched blobs match01f90fec, and the full-index patch remains 153,720 bytes with SHA-2568dbd5f38d01cedb350fafa2f17756557d1dc5ec22e2f4455c107726e0c98e4ad. The owner reports no overlap with 16 upstream paths, unchanged dist/2,097 rc.3 paths, current-base 149 ordering/fence plus nine compliance tests, policy 3/docs 55/broader 30, both typechecks and static checks passing, and independent approval only for Draft lease push. #7509 remains unchanged on its older reviewed base.The owner reports prospective
32496b54+ held3b6fd2d1composition at tree4fa9b60077678e392941dfcb32cfc873e02bd21f: only the package conflict, each ordering/fence/CDN/Worker suite once, 2,411 inputs verified and 99,714 tree entries. The actual combined command reportedly passed 149 ordering/fence, 53 JSONL/Worker and nine compliance tests plus registration 1. This is prospective evidence only, not #7509 readiness or qualification on actual landed main. Earlier prospective trees, including01f90fec's90669daabb446d3bdb1ec35fd7da34aa312ef22b, remain historical.Predecessor push-hook disclosure: the owner reports the default local matrix encountered an SDK rc.3 discovery mismatch; a CI candidate-mode signals replay passed, then the long sales replay was stopped locally. The owner used the documented
ADCP_SKIP_STORYBOARD_MATRIX=1advisory-only skip; version/scope/docs hooks passed. No source/dependency/floor workaround or--no-verifywas used. The full fresh current/frozen GitHub storyboard gate has now passed; the owner reports all 22 storyboard jobs succeeded. New-head runs are Build 34896518549, Storyboards 34896518945, Migrations 34896519083 and CodeQL 34896518499; terminal CI is now verified green on this exact head, including CodeQL. The owner reports all 13 Actions runs on attempt 1 (12 successes and one expected skip), zero new CodeQL alerts/annotations, and human request renewal only after green. This closes the predecessor CI wait, not #7509's hard successor HOLD. No GitHub workflow rerun/cancel was performed for this predecessor update.Historical independent composition evidence below applies specifically to fetched peer
afaef18867f8eee3d20224412c29570d4e8c770a, tree6ebff5dcc23008948ca37c2dc4d9959eb925aa1d; its four rebased patches and 12 touched blobs matched peer60647c7a.That historical combination of
3b6fd2d1andafaef188resolves to tree84e928b998a693e06f1c0bf81c1257aabc3a8436. The then-only merge conflict waspackage.json; its historical resolved command was the following, with the peer's indirect ordering-suite require removed. This command predates the new Changesets fence suite and must not be copied as the future combined registration:Each suite runs exactly once, and both newly added root suites are directly registered. All 2,673 materialized files match the combined tree; its index retains all 99,706 tracked entries. The actual combined
npm run test:release-workflowpassed: 34 ordering, 53 JSONL/Worker and nine published-compliance tests, plus existing immutability/promotion checks, with no skipped or failed tests. Registration passed 1/1.For that historical
afaef188combination, the independent reviewer verified both fetched heads, peer range/blob identities, package resolution, preservation of all new-base files, and equality with the tested tree, with no findings. The #7510 owner independently computed the same tree and ran the full combined command with the same 34/53/9 and registration 1/1 passes, verifying all 2,406 files in their materialized checkout with the full tracked inventory retained. The different materialized counts reflect different sparse-checkout selections.The prior
90f0cc5e+60647c7acombination on98fawas independently validated as tree03d22d189f6fcdbca478645fd042518449deee8b; the current result above supersedes that historical integration evidence. The one-line package resolution has not been applied to either source PR and must be preserved during normal human-reviewed integration. The #7510-first hard dependency and subsequent actual-main revalidation/review remain required; this synthetic composition does not permit landing #7509 first. Old-writer quiescence and separate recovery authorization also remain required.Historical omissions
Exactly four committed semver compliance JSONL files exist, with the same relative path
test-vectors/reporting-reconciliation/rows.jsonl:3.2.0-rc.0compliance/3.2.0-rc.0/test-vectors/reporting-reconciliation/rows.jsonl0a3672608d323c615e7ea73c9e8a62306510cb763.2.0-rc.1compliance/3.2.0-rc.1/test-vectors/reporting-reconciliation/rows.jsonla42e86df500b176c8d9d1d6640dc7edbf2eb21a63.2.0-rc.2compliance/3.2.0-rc.2/test-vectors/reporting-reconciliation/rows.jsonl929ceb16090b2f11ad20348b889780097173b00a3.2.0-rc.3compliance/3.2.0-rc.3/test-vectors/reporting-reconciliation/rows.jsonl71f9cd5414454e94ccfef87ce25777ead6fad228Every blob is 156 bytes, SHA-256
735610f303d6d91aa618df3aa6d7f18bcf5dfa52bdbd9ef551293e6bd14c92d9. The independent incident audit reported true 404 for all four. Local inventory/digests were verified; this environment's public probes returned 403 for both fixtures and existing indexes, so the 404 finding is attributed to that independent audit. Recheck existence before recovery.The original rc.3 incident audit covered 2,093 schema/compliance paths: 1,612 schemas + 481 compliance, with 2,092 exact matches and one missing JSONL. The signed protocol tuple adds four files: 2,097 total, not 2,093 including the tuple.
Current external release state — reconcile before repair
The original cancellation/no-tag incident is historical. A later external execution of release run 34844149713, attempt 4 completed successfully. This task independently read its API metadata:
triggering_actor=bokelley,run_started_at=2026-09-14T16:38:30Z,updated_at=2026-09-14T18:15:47Z, run head71f9cd5414454e94ccfef87ce25777ead6fad228; release job 104093354472 completed successfully at 18:15:46Z.The coordinating read-only audit reports that the job used the old gate, accepted #7485's final-head
e7d42approval, created the public release at 18:11:49Z, uploaded its assets through 18:11:58Z, then invoked legacy unscoped bulk R2 backfill. That audit confirms the tag targets71fand downloaded all four GitHub assets, comparing each byte-for-byte with unchanged committeddist/protocol; all four matched. These asset comparisons were performed by the coordinator, not repeated by this task. Their evidence files aresecurity-current-original-release-{run.json,jobs.json,job.log}andsecurity-current-rc3-{tag.txt,release.json,assets-evidence.json}.At the earlier attempt-4 follow-up, no fresh full R2/CDN audit was claimed. The coordinator now reports a subsequent direct R2 rc.3 audit: 2,096 of 2,097 paths match, the sole missing path is
compliance/3.2.0-rc.3/test-vectors/reporting-reconciliation/rows.jsonl, with zero differing objects and zero audit errors. This covers the 2,093 committed schema/compliance paths plus four protocol files. This task records that supplied audit without claiming to have repeated it. The older rc.0–rc.2 missing-key observations remain historical until separately rechecked.The coordinator separately reports legacy rc.4 state at abbreviated candidate
15fcfand a routing audit: tested direct R2 rc.4 index keys and tarball returned 404, while public/shadow schema routes and public compliance returned rc.3 bytes. These are separate object-existence and routing observations; no R2 overwrite is inferred. Do not treat a public fallback response as proof that the corresponding rc.4 objects exist.Reconcile current tag/release/asset provenance, all relevant committed schema/compliance and protocol paths, and any still-active old writers before proposing further repair. Matching GitHub tuple bytes and success under the old gate do not qualify
71funder the repaired provenance rules or lift its quarantine. The #7510-first hard predecessor, later #7509 requalification, separate historical one-key authorization and all recovery holds remain in force. Do not infer permission for replay, overwrite or recovery from this external event.Validation and limits
36a86d6cbase: release-workflow passes, including 49 focused tests and four expected fenced-only skips on the legacy publisher. Immutable-artifact, registration policy 1/1, scope, Bash syntax, ShellCheck 0.11, Node syntax and diff checks pass. Both TypeScript checks pass. All seven test files changed by the three intervening main commits pass 139/139.RELEASE_SHAand currentTESTED_SHA. Using current main asRELEASE_SHAmust fail before any AWS call.If-None-Match: *creation with exact bytes/MIME/cache, repeat/no-op, and differing-byte refusal. All other extensions, nested files, latest-only behavior, untracked exclusion, verification failures and Worker GET/HEAD behavior remain covered. Existing committed JSONL blobs are reused without duplicate checked-in fixtures.--versionenforces a one-key write limit.af1: full server suite 9,371 pass/31 skip; broader signing/bundle/index/reporting/doc-nav tests 31/31; original regression tests fail against base scripts. Those broad-suite results are historical evidence, not current-base CI approval. The five initial test-harness CodeQL findings were fixed through trusted response preloading and directory-entry traversal; CodeQL passed on superseded head90f0cc5e; CodeQL also passes on rebased head3b6fd2d1.36a86d6cpassed at3b6fd2d124c691370d97b8e57d0fa23bc0a2563f(not qualification on current main or the landed predecessor): 54 successful checks, one advisorycode_reviewskip, zero pending or failed checks. Build Check, production image and migrations, and storyboards all completed successfully on this exact head, attempt 1. CodeQL passes with no new findings; the five old comments are outdated. The migration job passed dependency installation and actual migrations. The extended wait was for queued GitHub-hosted runners; this rebase needed no workflow rerun, cancellation or dependency workaround. Normal push checks passed, with the documented advisory local storyboard skip and GitHub CI authoritative.bmilekic's formal review request was renewed after this exact head's CI passed. Main and fix(cdn): publish and verify compliance JSONL artifacts #7509 head were rechecked at that renewal; the subsequent fix(release): fence publication and recover superseded releases #7510 follow-up is tracked separately above. The PR remains Draft with auto-merge off; human approval, the package resolution during integration, old-writer quiescence and separately authorized recovery/publication remain gates. The prior90f0cc5eresult of 56 successes and one skip was on98faafter an authorized failed-job rerun; it remains historical evidence, not the current-head CI result.npm ci, before compilation or migration/test execution.@contentauth/c2pa-node@0.9.4fell back to a Rust build whose packaged manifest cannot find its workspace root. Lockfile, dependency declarations, install lifecycle scripts and migration workflow match98faexactly. A targeted offline Cargo manifest-parser reproduction using the NPM archive verified against the lockfile failed identically for base and candidate (exit 101 while inheritingworkspace.package.rust-version). This reproduces the fallback manifest defect, not the original download failure or a full basenpm ci. Other exact-head installs and the corresponding main migration job passed in the coordinating audit; the unchanged-head rerun now passes too. The incident is classified as external binary-install volatility, not a candidate or migration regression. No dependency workaround or source change was made, and no workflow was cancelled.Maintainer recovery handoff — no live authorization
#7510 must land first; #7509 must remain unmerged until it has been rebased and freshly validated/reviewed on that resulting main as specified above. Automatic deploy on the current base still uses the legacy bulk path, so landing #7509 first would itself expose the four historical JSONL keys without recovery approval. After the prerequisite and revalidation, account for and quiesce all old-code writers before any separately authorized recovery. The coordinating audit reports old af1 release run 34847610745 completed and updated #7508 to a later candidate; the historical #7508 regression is not a claim about that newer head. Leave #7508 under separate review and preserve the recovery HOLD.
71f9cd5414454e94ccfef87ce25777ead6fad228; it is not qualified dispatch authority. Re-audit its generated-head/merge-base provenance and human approval under the repaired permission checks. No bypass or exception is authorized. Any future recovery needs a separately reviewed plan and explicit maintainer authorization; retain the same-package-version and approval contract without weakening it for older releases.release.ymlreplays from current rc.3 main. A separately reviewed, explicitly authorized backfill must use current reviewed code and enforce conditional creation of only that version's one confirmed missing JSONL key. Require its existing published tag/GitHub prerelease at the expected target, correct metadata, exact local/tagged compliance subtree and JSONL bytes, and all four unchanged GitHub tuple bytes. The authority checker can use freshTESTED_SHA,PUBLICATION_BRANCH=mainand the matching historicalRELEASE_SHA.--version VERSION --skip-latestscopes the entire version, not one key; an execution boundary must enforce the narrower authorization. Any other missing object requires a separate recovery plan.application/x-ndjson; charset=utf-8andpublic, max-age=31536000, immutable. Normal latest/Worker deployment is a separate authorized path; the JSONL upload metadata works with the existing Worker.Read-only compliance verification:
Separate ordering defect
Deploy 34845112730 checked out
71f9cd5414454e94ccfef87ce25777ead6fad228and progressively exposed rc.3 before the release/tag workflow. PR #7485 had exact-head human approval ate7d42cbe5d7b3d05c3fcdcaa6f568b08ba131439at 12:34:18 UTC; that historical observation does not establish current collaborator authority or valid generated-commit provenance; the original71ftarget now requires re-audit under the release-security hold. The last main-SHA check started at 12:47:03; R2 publication ran 13:06:30–13:15:05, with rc.3 schema sync at 13:09:17 and compliance starting 13:12:15, without a fresh main fence before R2 use. #7510 owns that ordering/freshness fix.This #7509 task performed no PR merge, deploy, release, tag, signing, CDN/R2 mutation, workflow cancellation or branch-protection change. Its only direct workflow action was the explicitly user-authorized rerun of the failed migration job on the unchanged earlier PR head; approved Draft pushes/body updates and coordination are recorded above. The external original-release attempt 4 was observed read-only and was not initiated by this task. Committed released artifact bytes and history are unchanged.
Open workspace in Conductor