Skip to content

fix(release): republish the root schema discovery pointers on every deploy - #8324

Merged
bokelley merged 2 commits into
mainfrom
fix/schema-discovery-pointer-publication
Oct 19, 2026
Merged

bokelley merged 2 commits into
mainfrom
fix/schema-discovery-pointer-publication

Conversation

@bokelley

@bokelley bokelley commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Problem

https://adcontextprotocol.org/schemas/index.json reports latest_stable 3.2.1, v3.1 → 3.1.24, v3.0 → 3.0.26, while 3.2.5, 3.1.29 and 3.0.28 are released. The document's own description tells file-based consumers to resolve latest_stable or an alias instead of listing directories, so those consumers get schemas several releases old.

Root cause

scripts/backfill-cdn-artifacts.sh is the only writer to the artifact R2 bucket, and no publication mode ever uploaded schemas/index.json or schemas/latest.json:

Caller Flags Publishes the root pointers?
deploy.yml (every main merge) --latest-only No — both uploads were gated behind latest_only -eq 0
release.yml (every release) --version V --skip-latest No — skip_latest excludes them
bulk backfill (none) Would have, but live bulk backfill is disabled

The --latest-only gate arrived with the publication fence in #7510 (2026-09-22). Confirmed in deploy run 39184396496's predecessor 39184268136: the step uploaded only schemas/latest, compliance/latest, protocol/latest.*. The live objects carry Last-Modified: Thu, 01 Oct 2026 07:38:25 GMT.

This was already known and written down — .agents/shortcuts/cut-minor-ga.md step 17 said "No script path refreshes them today" and prescribed either a tooling PR or a tracked manual one-off. The 2026-10-01 one-off is what wrote the 3.2.1 copy still being served. This PR is that tooling PR.

Fix

Routine deployment now publishes both documents, under the authority rule in RELEASING.md that it must never advertise a version release authority has not published. scripts/build-schema-discovery-pointers.cjs enforces that mechanically rather than by review:

  • versions[] comes from one delimited schemas/ listing. A truncated listing is refused.
  • An alias may only resolve to a release whose schemas/<version>/index.json object is confirmed with a HEAD. A prefix exists from release publication's first object, and that publication is progressive and per-object, so prefix existence alone would let a half-uploaded release win an alias under an immutable cache policy. Each line is walked newest-first and stops at the first confirmation — 4 HEADs against production today.
  • The source must agree with its own version list (and with the expected $id/$schema/channel), so a checkout that skipped build:schemas cannot re-freeze the pointers.
  • Every URL in the output is derived from a validated version string. Nothing but two prose fields is copied from the source document.
  • No pointer may regress — except onto a release the source marks withdrawn/unpublished, the one legitimate backwards move. Without that exemption, the next RELEASE_STATUS_OVERRIDES entry on a live alias would wedge every deploy.

scripts/verify-schema-discovery.mjs then fetches the canonical public URLs and fails the deploy unless they serve byte-for-byte what the run uploaded. Uploading is not serving.

--aws-dry-run computes and prints both documents against the live bucket without uploading, for incident preview.

What this produces today

Simulated against the live bucket state:

served : 3.2.1  {"v3":"3.2.1","v3.2":"3.2.1","v3.1":"3.1.24","v3.0":"3.0.26","v2":"2.5.3","v2.5":"2.5.3"}
publish: 3.2.5  {"v3":"3.2.5","v3.2":"3.2.5","v3.1":"3.1.29","v3.0":"3.0.26","v2":"2.5.3","v2.5":"2.5.3"}
::warning Schema versions missing from the CDN::3.0.28, 3.0.27 ...

That matches the CDN worker's own dynamic /schemas/ discovery exactly, with no pointer regression.

Second, independent gap: v3.0 stays at 3.0.26

schemas/3.0.27/ and schemas/3.0.28/ are not on the CDN at all. /schemas/3.0.28/index.json returns HTTP 200 serving adcp_version: 3.0.26 through the worker's pinned-fallback, so advertising v3.0 → 3.0.28 would hand consumers a path whose payload disagrees with the version they asked for. The builder therefore pins the line to 3.0.26 and emits the warning above naming the gap.

Recovering those two releases is a separate maintainer action: recover-protocol-cdn.yml is protocol-tuple-only (as RELEASING.md already states), so there is no existing safe dispatch for schema/compliance directories. Out of scope here — widening that workflow is a publication-authority change deserving its own reviewed PR.

Expert review

code-reviewer and security-reviewer both ran before this PR opened. Their findings drove the design above: the HEAD confirmation tier (prefix ≠ published), the output allowlist (source URLs reached the public document verbatim), the withdrawal exemption (deploy wedge), byte-exact verification over a canonical URL (a field-subset check over a cache-busted URL could pass on a worker fallback), IsTruncated handling, requiring the served copy so the regression guard can't silently disable itself, and a working --aws-dry-run preview.

Tests

tests/schema-discovery-pointers.test.cjs (24 tests) covers the builder and verifier, including the alias/prefix distinction, the withdrawal exemption, prototype-key handling on the attacker-reachable published index, and byte-vs-pointer mismatch. tests/cdn-artifacts.test.cjs and tests/release-publication-ordering.test.cjs execute the real shell publisher against an isolated AWS boundary and assert the published pointer bytes; both fail if the fix is reverted.

Local: npm run test:release-workflow green, precommit gate green (4596 server unit tests + typecheck).

Hook note: pushed with ADCP_SKIP_STORYBOARD_MATRIX=1, the documented skip. The matrix triggered only because package.json is in STORYBOARD_PATHS and I added two npm scripts; nothing here touches training-agent or compliance source. It exceeded 30 minutes on this sandbox. training-agent-storyboards.yml is the authoritative gate.

No changeset: check-changeset-protocol-scope.cjs reports no protocol-scoped changes (publication orchestration is operational).

Closes #8318

🤖 Generated with Claude Code


Open workspace in Conductor

…eploy

`schemas/index.json` and `schemas/latest.json` are the documents file-based
consumers are told to resolve, and no publication step uploaded them. Routine
deploys call `backfill-cdn-artifacts.sh --latest-only`, which gated both
uploads behind `latest_only -eq 0`; `release.yml` calls
`--version V --skip-latest`, which also skips them; live bulk backfill, the
only remaining writer, is disabled. They froze at whatever the last manual
one-off wrote on 2026-10-01 while 3.2.2 through 3.2.5 and 3.1.25 through
3.1.29 shipped past them.

Routine deployment now publishes them, under the authority rule that it must
never advertise a version release authority has not published:

- `versions[]` comes from one delimited `schemas/` listing; a truncated
  listing is refused.
- An alias may only resolve to a release whose `schemas/<version>/index.json`
  object is confirmed with a HEAD. A prefix exists from release publication's
  first object, and that publication is progressive and per-object, so prefix
  existence alone would let a half-uploaded release win an alias under an
  immutable cache policy. Each line is walked newest-first and stops at the
  first confirmation.
- The source document must agree with its own version list, so a checkout
  that skipped `build:schemas` cannot re-freeze the pointers.
- Every URL in the output is derived from a validated version string.
- No pointer may regress, except onto a release the source marks withdrawn or
  unpublished.

`verify-schema-discovery.mjs` then fetches the canonical public URLs and fails
the deploy unless they serve byte-for-byte what the run uploaded.

A release line whose newest patch never reached the CDN stays pinned to the
newest patch that is actually fetchable, with a warning naming the gap. That
is live today: `schemas/3.0.27/` and `schemas/3.0.28/` are absent from R2, so
`/schemas/3.0.28/index.json` serves 3.0.26 bytes through the worker's pinned
fallback and `v3.0` correctly stays at 3.0.26 until those releases are
recovered.

Closes #8318

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Escalate to human review

Escalate — gated paths require human/CODEOWNERS review.

This PR is operational/deploy tooling fixing #8318 (frozen root /schemas/{index,latest}.json pointers). The reviewer found no Critical/High/Medium findings; the diff is mechanically clean. However, the PR touches files under a hard, deterministic approval gate and review_decision is REVIEW_REQUIRED, not APPROVED.

Escalation reasons

  • .agents/shortcuts/cut-minor-ga.md (modified) matches gated path .agents/**
  • .github/workflows/deploy.yml (modified) matches gated path .github/workflows/**

Per decision rule row 2 (gated_paths: true AND review_decision ≠ APPROVED), this PR cannot be auto-approved. Human/CODEOWNERS review is required to satisfy the gate; once review_decision becomes APPROVED, the clean diff can fall through to approval.

No blocking findings. One Low nit (not flagged): retiredVersions(source) computed twice in buildPublishablePointers (L699, L706).

Why human review

  • .agents/shortcuts/cut-minor-ga.md (modified) matches gated path .agents/** — human/CODEOWNERS review required
  • .github/workflows/deploy.yml (modified) matches gated path .github/workflows/** — human/CODEOWNERS review required
  • review_decision is REVIEW_REQUIRED, not APPROVED, so the hard approval gate is not satisfied
  • This PR touches a path under a hard, non-overridable approval gate (.agents/shortcuts/cut-minor-ga.md (modified) matches .agents/**; .github/workflows/deploy.yml (modified) matches .github/workflows/**) and the current GitHub review decision is 'REVIEW_REQUIRED', not APPROVED. This is a hard gate enforced in code — Ladon cannot auto-approve until a human/CODEOWNERS approval is recorded, regardless of how clean the diff is.

@aao-secretariat aao-secretariat Bot added the ladon/needs-human-review Ladon has escalated this PR for human review. label Oct 19, 2026
Review nit on #8324: buildPublishablePointers derived it twice, once for the
regression exemption and once for the dropped-entry report.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Escalate to human review

Escalate — gated paths touched and review not yet approved.

This PR modifies files under hard, non-overridable approval gates, and the current GitHub review decision is REVIEW_REQUIRED, not APPROVED. Per decision table row 2, the gate is not satisfied and Ladon cannot auto-approve regardless of how clean the diff is.

Escalation reasons

  • .agents/shortcuts/cut-minor-ga.md (modified) matches gated path .agents/** — human/CODEOWNERS review required.
  • .github/workflows/deploy.yml (modified) matches gated path .github/workflows/** — human/CODEOWNERS review required.
  • review_decision is REVIEW_REQUIRED, not APPROVED, so the hard approval gate remains unsatisfied.

No blocking or medium findings. The reviewer confirms the current delta is a behavior-preserving refactor in scripts/build-schema-discovery-pointers.cjs (hoisting a pure, deterministic retiredVersions(source) call) with no new findings; the prior pass was clean. The only reason this does not approve is the outstanding gated-path approval requirement — once a human/CODEOWNERS approval is recorded, this falls through to a clean approve.

Why human review

  • .agents/shortcuts/cut-minor-ga.md (modified) matches gated path .agents/** — human/CODEOWNERS review required
  • .github/workflows/deploy.yml (modified) matches gated path .github/workflows/** — human/CODEOWNERS review required
  • review_decision is REVIEW_REQUIRED, not APPROVED, so the hard approval gate is not satisfied
  • This PR touches a path under a hard, non-overridable approval gate (.agents/shortcuts/cut-minor-ga.md (modified) matches .agents/**; .github/workflows/deploy.yml (modified) matches .github/workflows/**) and the current GitHub review decision is 'REVIEW_REQUIRED', not APPROVED. This is a hard gate enforced in code — Ladon cannot auto-approve until a human/CODEOWNERS approval is recorded, regardless of how clean the diff is.

@bokelley
bokelley merged commit e113cfb into main Oct 19, 2026
106 of 108 checks passed
@bokelley
bokelley deleted the fix/schema-discovery-pointer-publication branch October 19, 2026 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ladon/needs-human-review Ladon has escalated this PR for human review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Public schemas/index.json aliases are stale (latest_stable 3.2.1, v3.1 3.1.24, v3.0 3.0.26)

1 participant