Skip to content

fix(testing): backport echoed-release capability validation under an external pin root for SDK 15.9.x - #3323

Merged
bokelley merged 1 commit into
15.9.xfrom
fix/15.9.x-capabilities-echoed-release
Oct 17, 2026
Merged

bokelley merged 1 commit into
15.9.xfrom
fix/15.9.x-capabilities-echoed-release

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Summary

Backport of #3322 to 15.9.x for the latest dist-tag. Fixes #3320.

Hosted compliance always runs comply() with an external schema root for the pinned release. The served-release rule from 15.9.2 (#3287) let that root win over the agent's echoed adcp_version, so a 3.0 target graded a conformant 3.2 get_adcp_capabilities response against 3.0. Every storyboard response_schema check on that task failed (52 steps across 43 storyboards in the 3.0.27 suite), and discovery raised spurious capabilities_response_schema_invalid notices (worked around in adcontextprotocol/adcp#8264).

For get_adcp_capabilities only, the storyboard response_schema check, discoverAgentProfile and TaskExecutor now validate against the echoed release when all of these hold:

  • the echo is the pin's major;
  • it is listed in the response's own adcp.supported_versions;
  • it has a schema root: one supplied through servedReleaseSchemaRoots, an external root registered for it, or the SDK's bundled schemas.

This holds whether or not an external root is active for the pin. Otherwise validation stays strictly on the pin. Other tasks, storyboard selection, the run's adcp_version and grading are unchanged.

There is a new servedReleaseSchemaRoots option (version → schema root) on comply(), runStoryboard() and runStoryboardStep(), and a withServedReleaseSchemaRoots(roots, fn) helper exported from @adcp/sdk/testing. Hosted callers use them to supply their own roots for echoed releases. Those roots are activated only for this capabilities validation. See #3322 for the full design notes.

Differences from #3322

Test plan

  • test/lib/capabilities-echoed-release-hosted-validation.test.js (24 tests). Against unpatched 15.9.x, 17 fail; the 7 that pass are the pin-fallback guards. With the fix all pass, including the end-to-end comply() run in the hosted shape (version: '3.0.25' plus an external 3.0 schemaRoot).
  • test/lib/capabilities-served-release-validation.test.js updated for the new supported_versions condition (17 tests, all pass).
  • npm run typecheck, build:lib
  • Full npm run test:node locally. Every suite passes except the Postgres-backed ones (ECONNREFUSED, no local database), which CI covers.

🤖 Generated with Claude Code

…external pin root for SDK 15.9.x

Backport of #3322. Hosted compliance always runs with an external schema
root for the pinned release, and the served-release rule from 15.9.2 let
that root win over the echo. A 3.0 target therefore graded a conformant
3.2 get_adcp_capabilities response against 3.0: every storyboard
response_schema check on that task failed (52 steps in the 3.0.27 suite),
and discovery raised spurious capabilities_response_schema_invalid notices.

For get_adcp_capabilities only, the storyboard response_schema check,
discoverAgentProfile and TaskExecutor now validate against the echoed
adcp_version when it is the pin's major, is listed in the response's own
adcp.supported_versions, and has a schema root (one supplied for that
release, an external root registered for it, or the SDK's bundled
schemas), whether or not an external root is active for the pin.
Otherwise they validate strictly against the pin. Other tasks, storyboard
selection, the run's adcp_version and grading stay on the pin.

New servedReleaseSchemaRoots option on comply(), runStoryboard() and
runStoryboardStep(), and a withServedReleaseSchemaRoots() scope helper
exported from @adcp/sdk/testing, let a hosted grader supply its own roots
for echoed releases. They are activated only for that capabilities
validation.

Fixes #3320

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — clean backport of #3322 to 15.9.x, fixing #3320.

The #3287 served-release rule let the pin's external schema root win over the agent's echoed adcp_version, causing hosted compliance to grade a 3.2 capabilities response against a 3.0 pin (52 failed steps + spurious notices). The fix drops external-root-wins for get_adcp_capabilities only, selecting the echoed release when same-major + in supported_versions + a schema root exists, scoped via withServedValidationVersion.

Checked:

  • Cache safety: withServedValidationVersion (schema-loader.ts:744) merges the served root without clearStatesForBundle; safe because states/schemaRefValidators are keyed by stateCacheKey(bundleKey, root) — distinct root = distinct entry, no stale reuse.
  • validateResponseSchemaAgainst (TaskExecutor.ts:153) is a verbatim extraction; only version source + scoping change.
  • isExternalResponseSchemaAuthoritative widening (validations.ts:691) is additive; both call sites pass a full ValidationContext.
  • Tests prove the correct root is selected over the bundled default and that served roots never leak across tasks; live-seller end-to-end comply() covered.
  • Changeset present; patch justified for a maintenance-line backport that only selects validation version (no wire-shape change). Witness-not-translator preserved.

No blocking findings, no medium findings, no escalation triggers. gated_paths false, high_risk false, no team gate. Falls through to row 9.

@bokelley
bokelley merged commit 8865b4c into 15.9.x Oct 17, 2026
41 checks passed
@bokelley
bokelley deleted the fix/15.9.x-capabilities-echoed-release branch October 17, 2026 14:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant