[bokelley]

Summary

Fixes #3320. Hosted compliance always runs comply() with an external schema root for the pinned release. Under 15.9.2's served-release rule (#3278 / #3280), that root won over the agent's echoed adcp_version. So a 3.0 target graded a conformant 3.2 get_adcp_capabilities response against 3.0, with two effects:

• every storyboard response_schema check on that task failed (52 steps across 43 storyboards in the 3.0.27 suite);
• discovery raised spurious capabilities_response_schema_invalid notices, which fix(compliance): validate capability notices against the release the agent echoed adcp#8264 currently re-validates after the run.

Rule

For get_adcp_capabilities only, three checks now validate against the echoed release:

• the storyboard response_schema check;
• discoverAgentProfile, which feeds the notices;
• TaskExecutor, which backs getAdcpCapabilities().

They do so when all of these hold, whether or not an external root is active for the pin:

• the echo is the pin's major;
• the echo is listed in the response's own adcp.supported_versions. This condition is new; fix(testing): validate capabilities against the served release; key compliance_testing in the cross-check #3280 deliberately didn't consult it;
• a schema root for the echo is available: one supplied through servedReleaseSchemaRoots, an external root registered or scoped for it, or the SDK's bundled schemas.

Otherwise they validate strictly against the pin. That covers a different major, an unlisted echo, an echo with no schema, and no echo. An echo of the pinned release keeps the exact pin and its root. Other tasks' responses, storyboard selection, the run's adcp_version and grading are untouched.

New API: servedReleaseSchemaRoots

• TestOptions.servedReleaseSchemaRoots?: Readonly<Record<string, string>>, so it is available on comply(), runStoryboard() and runStoryboardStep().
• Keys are AdCP versions ('3.2', '3.2.5', '3.3.0-beta.1'). Values are schema-data roots in the same shape as schemaRoot.
• withServedReleaseSchemaRoots(roots, fn) is exported from @adcp/sdk/testing and validation, for callers outside comply() and the runner. It is scoped with AsyncLocalStorage, like withExternalSchemaRoot.
• Each root is checked against its key with the same check withExternalSchemaRoot uses. That check takes about 1s per full root, so it is memoized per (version, root) for the life of the process. A malformed or mismatched map throws before the run, and so do two keys that name the same release with different roots.
• Storyboard checks compare the echo with the run's pin, so they apply it only when the run has one (adcpVersion, or the storyboard or compliance bundle's adcp_version). comply() always does.
• A supplied root is activated only inside the capabilities validation call (withServedValidationVersion), so it never reaches any other response. It isn't cleared from the validator caches afterwards. Those caches (states, schemaRefValidators) are keyed by bundle key plus root, so switching roots is safe.

I chose a map over a resolver callback because it matches schemaRoot, is serializable, and is all hosted needs. Hosted can then drop its post-run notice re-validation.

Changes

• src/lib/validation/schema-loader.ts: the rule, withServedReleaseSchemaRoots, and internal withServedValidationVersion. resolveServedValidationVersion keeps its signature.
• src/lib/testing/storyboard/validations.ts: the response_schema check on get_adcp_capabilities validates under the selected release. When it picks the echo, it uses the echo for both adcpVersion and responseAdcpVersion, so the reported schema_id/schema_url name the release that was actually validated.
• src/lib/testing/client.ts (discoverAgentProfile) and src/lib/core/TaskExecutor.ts. The TaskExecutor body moved into validateResponseSchemaAgainst, with the same behavior and error handling.
• src/lib/testing/compliance/comply.ts and storyboard/runner.ts: scope the option around the run. In comply() that includes discovery.
• isExternalResponseSchemaAuthoritative, used by the runner's Zod-rejection recovery path, also treats a capabilities response as authoritative when it is validated against an echoed release whose root is external or supplied.
• Docs: docs/guides/VALIDATE-YOUR-AGENT.md.

The changeset is patch, on purpose. This is a conformance bug fix that has to ship as a 15.9.x patch for latest. servedReleaseSchemaRoots is an additive, optional input that exists only to complete the fix for hosted callers. A minor changeset would push the 15.9.x backport to 15.10.0.

Test plan

• New test/lib/capabilities-echoed-release-hosted-validation.test.js (24 tests). Against unpatched main, 17 fail; the 7 that pass are the pin-fallback guards. Coverage:
  • With a 3.0 external root active, a step response_schema check on a same-major echoed 3.2 response passes, and a real 3.2 schema error still fails against 3.2.
  • A different-major echo, an unlisted echo, an echo with no schema and no echo all fail strictly against the 3.0 pin.
  • Non-capabilities tasks are unaffected. get_products echoing 3.2 keeps its pin verdict. With the pin set to 3.2 and a supplied 3.2.5 root in scope, get_products is still validated against the bundled 3.2, and isExternalSchemaRootActive('3.2') stays false.
  • A supplied 3.2.5 root is used: a marker field it requires shows up in the step check, the discovery notices and runStoryboard({ servedReleaseSchemaRoots }).
  • An unbundled echo (3.9) falls back to the pin until a served or external root is supplied for it, then validates against that root.
  • comply() end to end in the hosted shape: version: '3.0.25' plus an external 3.0 schemaRoot, against a live seller echoing 3.2. On main this reproduces the bug: false response_schema failures and capabilities_response_schema_invalid notices. With the fix there are none. With servedReleaseSchemaRoots, the supplied root grades both. A bad map rejects before the run.
  • Live MCP seller: discovery, TaskExecutor strict mode, and runStoryboard with schemaRoot. Also input validation, including conflicting keys, and the @adcp/sdk/testing export.
• Updated test/lib/capabilities-served-release-validation.test.js for the new supported_versions condition and the reversed pin-root precedence.
• Full npm run test:node locally. Every suite passes except the Postgres-backed ones (ECONNREFUSED, no local database), which CI covers. A few other files failed in the full run because the pre-push hook rebuilt dist/ mid-run; all of them pass when re-run in isolation.
• npm run typecheck, npm run lint (0 errors), lint:boundaries, prettier --check, ci:docs-check, build:lib

A 15.9.x backport for latest follows in a separate PR.

🤖 Generated with Claude Code

[aao-secretariat]

Ladon verdict: Approve

Approve — clean, well-tested bugfix for #3320.

This PR fixes hosted compliance grading get_adcp_capabilities against the wrong release: a conformant 3.2 echo was being graded against the 3.0 pin when an external schema root was active. The fix gates the served-validation version on same-major + release-precision + supported_versions listing + a resolvable root, and fails closed to the pin otherwise.

Checked:

• selectServedValidationVersion (schema-loader.ts:586-615) — correct gating, fails closed; removing the old isExternalSchemaRootActive guard is the intended reversal.
• withServedValidationVersion (schema-loader.ts:707-727) — served root scoped to fn, applied only for get_adcp_capabilities at all three call sites (TaskExecutor.ts:140, testing/client.ts:842, validations.ts:548); leaks nothing.
• validateResponseSchemaAgainst (TaskExecutor.ts:161-214) — faithful extraction, normalization/identity-3.3/warn-strict/v2.5 guard all preserved. No witness-not-translator violation.
• New 410-line test covers fallback-to-pin-strict, served-root precedence, and other-tasks-ignore-served-roots.

Medium findings: none.

Non-blocking (low): changeset marked patch for an additive public-API addition — conventionally minor, but moot under SDK 16 beta pre-mode.

No blocking findings, no medium findings, no team gate, gated_paths false, high_risk false. Falls through to row 9.

[aao-secretariat]

Ladon verdict: Approve

Approve — clean subsequent review, prior blockers none, delta is additive.

This is a follow-up to a prior clean approve (head a1f308f). The delta since then is 4 files / ~240 additive lines, all reviewed:

• validations.ts (L719-736): isExternalResponseSchemaAuthoritative now threads optional taskName/taskResult and treats a capabilities echo with an active external/served release root as authoritative. Logic is consistent — bundled echo with no external/served root correctly returns false, served root is scoped only for the boolean check with no leak.
• schema-loader.ts (L1350-1356): new duplicate-release guard in withServedReleaseSchemaRoots fails-closed when two map keys name the same release with different roots.
• types.ts: doc-comment clarification only.
• New test file: covers unbundled-3.9-root, cross-task isolation, duplicate-key, and end-to-end comply() paths.

No Critical/High/Medium findings. Changeset (patch) and new public export were in scope at the prior clean approve and remain sound.

Gate checks:

• No no-auto-approve team match.
• gated_paths: false (review_decision REVIEW_REQUIRED does not trigger row 2 when gated_paths is false).
• high_risk: false.
• No medium findings → rows 4, 5, 8 do not fire; prior decision was approve so row 6 does not apply.

Falls through to row 9 → approve.