[bokelley]

Summary

The daily External smoke schedule workflow has failed on all 21 runs since #7511. The smoke check validated the deployed agent's capabilities against the release the client pinned, but it should validate against the release the agent actually served. The deployed agent is behaving as the spec requires. No agent change and no deploy are needed.

Root cause

• withHostedTestOptions(..., hostedComplianceTarget()) pins adcpVersion: "3.0.27" (the default hosted line, 3.0). It sets no wireAdcpVersion.
• In @adcp/sdk 15.8.1, buildVersionEnvelope (dist/lib/protocols/index.js, about line 125) checks bundleSupportsAdcpVersionField("3.0"). That returns false, so the client sends only the major version. This is the request captured against the public agent:

  {"method":"tools/call","params":{"name":"get_adcp_capabilities","arguments":{"adcp_major_version":3}}}

• docs/reference/versioning.mdx ("Server resolves") says that when only adcp_major_version is sent, the server uses its highest supported release in that major. The agent does this: it answers adcp_version: "3.2", supported_versions includes "3.2", and the response carries the 3.2 features.bidding_policy object and 24 scenarios.
• The same doc ("Server echoes") says buyers SHOULD validate the response against the echoed release, not against their pin. testCapabilityDiscovery instead validates against the pin. It calls validateIncomingResponse(..., schemaAdcpVersion ?? client.getAdcpVersion()) in dist/lib/testing/client.js at line 416. TaskExecutor.validateResponseSchema (dist/lib/core/TaskExecutor.js, line 4400) does the same. This produces the /media_buy/features/bidding_policy and /compliance_testing/scenarios/4..N issues.
• The agent's 3.0 gating works when the caller actually asks for 3.0. With adcp_version: "3.0", the agent echoes "3.0" and leaves out bidding_policy. The smoke probe never asked for 3.0.

This is case (b): the smoke check validated against the wrong version.

Fix (scripts/external-smoke/)

When the echoed adcp_version differs from the pin, the check now:

1. requires the echoed release to be in the same major and listed in adcp.supported_versions;
2. discovers again pinned to that release. The SDK then sends adcp_version: "3.2" and validates strictly against 3.2;
3. requires the pinned probe to keep serving the same release;
4. resolves storyboards from that release's hosted bundle. The 3.0 bundle doesn't know the sales-dooh specialism the agent declares at 3.2.

The check still fails on any schema issue. If the echo matches the pin, or there is no echo, it still validates against the pin, as the spec says for legacy sellers.

Two more failures were hidden behind the schema error. Each would have failed the job on the next run:

• SDK cross-check bug. crossValidateProtocolsAndTools (dist/lib/testing/scenarios/capabilities.js, line 242) lists comply_test_controller under the protocol compliance. But parseCapabilitiesResponse reports the compliance_testing block as the protocol compliance_testing. As a result, every agent that declares the block correctly fails with Tools suggest unreported protocols: compliance (has: [comply_test_controller]). The check now ignores only that exact message, and only when the response actually carries the compliance_testing block. Any other cross-check problem still fails.
• Anonymous OAuth challenge. After fix(training-agent): support organization OAuth sandboxes #8202, the anonymous probe on /sales/mcp gets a 401 Bearer challenge. The SDK turns it into NeedsAuthorizationError ("... requires OAuth authorization ..."), which reaches the check only as a step string. The check now treats that message as auth evidence, so the anonymous probe reports auth-required.

No changeset: this changes only tooling and tests, not the protocol surface.

SDK follow-up (adcp-client, not patched here)

The bugs are unchanged in 15.9.1 (latest).

1. dist/lib/testing/client.js:416 (discoverAgentProfile) and dist/lib/core/TaskExecutor.js:4400 (validateResponseSchema): validate against the release the server echoed (data.adcp_version) when it is a release in the same major that has a bundled schema. Fall back to the pin only when the echo is absent. This matches versioning.mdx step 4.
2. dist/lib/testing/scenarios/capabilities.js:242: change the key compliance: to compliance_testing: to match TOOL_PROTOCOL_MAP and parseCapabilitiesResponse.

Once both are fixed, the re-pin and the cross-check exception in check.ts can be removed.

Not fixed here (separate, low priority)

With an explicit adcp_version: "3.0", /sales/mcp still lists the newer compliance_testing.scenarios, because salesCapabilityScenarios only uses the 3.0 list under storyboardCompat. The spec's "serving an older pin" guidance says new enum values fall under the unknown-value tolerance rule, and the smoke probe never sends that pin. I left it out of this PR.

Test plan

• npx vitest run server/tests/unit/external-smoke.test.ts: 45 passed. 9 of the new tests fail against the old check.ts, including the main case, "validates and resolves against the echoed release instead of the 3.0 pin".
• npm run test:external-smoke against the deployed public agent (read-only discovery, the same calls the daily job makes): Authenticated discovery and storyboard resolution passed. / Anonymous discovery: auth-required.
• The same command against a local training agent (createTrainingAgentRouter on 127.0.0.1): it passes. The old check reproduces the exact CI failure against that local agent.
• npm run typecheck, plus an ad-hoc tsc --noEmit over scripts/external-smoke/*.ts and the test file, which the server tsconfig doesn't include
• npm run test:unit: 1113 passed
• npm run test:server-unit -- --shard=N/3: 4612, 3311 (+31 skipped), and 4688 passed

The workflow runs the script from the checkout, so the next scheduled run after merge should pass without a deploy.

🤖 Generated with Claude Code

[aao-secretariat]

Ladon verdict: Approve

Approve — tooling-only fix to scripts/external-smoke/** plus its unit test.

Checked:

• No findings at any severity (Critical/High/Medium/Low all empty).
• No author no-auto-approve team match.
• gated_paths: false, so the row-2 hard gate does not apply despite review_decision: REVIEW_REQUIRED.
• high_risk: false; no deleted/modified high-risk files.
• No protocol surface touched (static/schemas/source/**, docs/reference/**, dist/**, task/compliance assets untouched), so the omitted changeset is correct and no schema↔docs, oneOf, immutability, or 3.1.x patch-eligibility rules are triggered.
• The new load-bearing path in check.ts (negotiateServedRelease, cross-check waiver, releasePrecision) is fail-closed, bounded by existing discovery timeouts, and fully covered by the new tests.

No blocking or escalation triggers fired; falls through to row 9.