[bokelley]

Optional collection tests currently seed a product and thereby invent the feature they purport to grade. This change resolves native catalog products independently for selectable subsets, default materialization, fixed-bundle enforcement/restatement and structural cross-publisher carriage/subsets. A complete valid no-match is fixture_unsatisfied / not_applicable (feature untested); malformed discovery and broken selected support still fail.

Closes #8254, completing the collection work after reporting #8336 merged. Supersedes proposal #8259 using the merged foundation #3354, authoring/output contract #8376 and already installed SDK16.3.0. Patch changeset; no SDK/dependency/lock/bridge changes, new capability fields or controller refusal semantics.

• Six core scopes resolve independently, including one-pair default/restatement versus two-distinct-total-pair subsets. Fixed accepts omitted or false targeting flags. Separate optional-reader companions preserve persisted subset/default obligations without suppressing core creation.
• Real selectors, collision-checked negatives, protected product identity and commercial terms come from the SAME selected product/price. Whole-price snapshots preserve absent/0/5000 minima. The production caller derives valid budgets and canonical full selector arrays; exact pair-set/product/buy/package/cardinality assertions remain graded.
• Full and standalone use prepared bindings. Standalone returns the actual native root gate for absent wholesale/reader support with the observed inventory, and readback requires trusted successful-create context. SDK grades, wire evidence, failure precedence, disposition rules and required-phase floors are preserved. Hosted labels distinguish untested catalog/reader gaps from executed passes.

Implementation details, exact scenario/selector contracts, authority and historical proposal reconciliation: qualification memo.

Review repairs:

• R1: every executable standalone discovery/profile fallback requires the same complete, verified successful-create history. The separate real absent-reader/wholesale native gate remains intact. Detected protected caller metadata goes directly to the SDK's pre-contact typed denial/private audit, preserving its original authority and evidence semantics.
• R2: a private, purpose-separated integrity proof is issued only after the actual standalone create passes. It binds the exact authored source, seller/account/version, committed buy/package/product and normalized pair set. It is past-result provenance, not an access grant; fresh preparation/current SDK authorization still runs. Both read companions reject failed/unissued/retargeted context. Addie's actual encrypted context-ref handoff keeps pre-call context after failed/skipped collection-reader steps, while the failed response, captures, checks and history remain available as evidence.
• CI registration: seven explicit test paths replace the unsupported shell glob; the existing coverage guard is unchanged.

Validation at cf2ca5de81351c5811068847c069f7e1f54776d6 (base 3e94f5e3):

• Full isolated serial public matrix: 130/130 registered tests passed, 0 failures/skips, natural exit 0 (228.838 seconds). These run through the actual eight YAMLs/production adapter/public SDK. Includes all prior 94 cases and 36 history controls covering every missing comparand, failed/malformed supplied profiles, real fail-once capability recovery, both failed-create boundaries, scope/identity/selector retargeting and protected-metadata denial/private audit.
• Actual Addie handler test file and existing counterpart: 176/176 pass, including four real public-SDK success/failed-create opaque-reference round trips. The old unsafe production context handoff is exercised, not a mocked sealer.
• Root independently reran the unchanged seven-file public command at this head: 130/130 pass, zero failures/cancellations/skips, natural exit 0 (216.93 seconds; log SHA256 75deae4abbaedba310274a5e8e6a576a5f5a9b8d8ef8fce5ca7fe13aab87b650). The unchanged original R1 probe independently exits 0. Fresh code/security/protocol-product delta review resolves R1/R2 with no remaining concrete blocker within the reviewed scope. Frozen 10e88cd failed probes and its independent 94-pass run remain separate; neither prior counts nor agent reviews become human approval.
• An overlapping matrix worker was killed while the cgroup OOM counter increased; that failed/cancelled run is preserved separately. It is not relabeled green. Final qualification uses the subsequent isolated serial execution.
• Normal commit hooks passed: 1,276 root tests; 794 server files / 13,151 passing tests / 31 existing skips; both typechecks and dynamic-import/format-identity/state-change guards pass. This ordinary full hook run used env -u ANTHROPIC_API_KEY -u ADDIE_ANTHROPIC_API_KEY to match CI. Historical live-key route timeouts remain failed evidence; the exact unchanged two-file non-live replay passed 27/27 before the full successful hook run. No hook, timeout, source or test guard was changed.
• Exact-head CI is green: 63 successful checks / 1 normal skip (record) / 0 pending or failed, including Training Agent Storyboards and all canonical/server checks. Canonical 3/4 executed 130/130 collection tests and the actual handler file (90 tests, including four opaque-reference round trips) inside 1,276 passing root tests. Canonical 4/4 passed the unchanged registration guard, 3/3. Normal pre-push checks passed; only the repository-supported local advisory matrix skip was used. No inline comments, conversation comments or unresolved review threads remain at the final read. Exact-head Ladon approval is automation, not human/WG approval. Existing fixture/sample lints and capability/evaluator/reporting evidence retain their scope in the memo.
• Native artifact audit: all eight sales artifacts yield 176/180 clean result rows; 1,202 passing / 4 failing / 106 skipped / 0 N/A steps. All eight collection rows explicitly log fixture_unsatisfied with zero graded steps. These are untested gaps, never positive support evidence. The reporting baseline run 39895814069 has the identical source tree as this PR base: 170/174 clean, 1,222 passing / 4 failing / 106 skipped / 0 N/A steps. The only changed existing result rows remove 12+8 seeded collection pass credits; six new independent rows are gaps. The same four raw failures remain: version_negotiation, inline_creatives_without_sync, provenance_audit_observation, provenance_truth_of_claim. Floors and grading are unchanged. Post-result child teardown telemetry is retained (baseline: 182 SIGKILL after received result; current: 187 plus one natural exit, all received results and no timeouts); this is not a claim that every storyboard child exits naturally.

Bounded coverage and follow-ups:

• Only native wholesale sandbox, non-guaranteed fixed-price CPM products are graded; guaranteed, auction-priced, seeded-only and brief-only support remains untested. Cross-publisher is a structural namespace relation, not ownership/host authorization or verified supply path.
• SDK#3395: legal omitted optional media_buy_status can falsely fail an active read. Positive fixture uses the legal explicit lifecycle form; omission is unqualified, not made mandatory.
• SDK#3396: widened subsets correctly fail, but failed comparison evidence omits required authored context_key; output conformance is not claimed. The existing output MUST is unchanged.
• SDK#3397: a conforming feed-version change requires buyer restart; SDK16.3 does not restart. Mutable-feed qualification remains incomplete, never relabeled seller nonconformance. Genuinely invalid adcp_version is tested separately as a schema failure.
• The three self-contained public repaired-behavior probes are committed under the memo's namespace and naturally FAIL on16.3.0. They preserve unresolved failures separately from green grading regressions, with no expected-failure conversion.
• Existing SDK#3389 keeps supplied fresh-preparation host hooks fail-closed before HTTP (zero callback calls, not callback qualification); SDK#3382 covers bare comply/heartbeat commercial/selector projection. No durable cross-request authority, heterogeneous comply branding or catalog-retirement immunity is claimed. #8416 tracks richer hosted coverage display.

Main merge disposition: ordinary main merge can automatically refresh draft Version Packages #8419 and build/register/sign candidate artifacts through the unchanged release.yml blob 68a2822d8bf89322deea5e6887e2d66cd4061335. Merge remains held for Brian/root's separate disposition of this consequence. This PR does not authorize or execute merge, workflow dispatch, VP mutation, publication, release, maintenance or production writes. The maintainer/root coordinator owns repeated expert review of repaired bytes and the ordinary merge decision.

[aao-secretariat]

Ladon verdict: Comment (human reviewer recommended)

Comment — governance review warranted; no blocking findings.

The reviewer found no Critical/High/Medium findings. The behavior-carrying code (catalog-reporting-runner.ts, member-tools.ts) cleanly generalizes the reporting path to collections, preserves timeouts/abort/finally-based grant release, fails closed on missing prior context, and orders NOT-APPLICABLE labeling correctly (never crediting an absent reader). Changeset is present and correctly scoped (patch; compliance surface + Addie). No schema source, oneOf, or released-artifact changes, so those mandatory rules are not triggered. Error-code allow-list tightening aligns with not-found precedence in error-handling.mdx.

Why a human should look: the reviewer explicitly deferred a governance question to the arbiter — this PR introduces new 3.3 normative obligations (optional product-collection qualification spec) and was authored by a maintainer. New normative requirements on the published protocol surface deserve a human eye to confirm classification (Editorial/Normative-non-breaking vs. Breaking-class) and that any controlling governance precedent is satisfied. The reviewer did not read the four smaller test files or the readback/subset/restatement scenario tails line-by-line (template clones), so coverage on those is asserted, not verified.

Decision-table trace: No critical/high findings (row 1 no). gated_paths is false (row 2 no). high_risk reasons are all (modified) with no (deleted) (row 3 no). No medium findings at all, so rows 4–6 and 8 do not fire. No no-auto-approve team match (row 7 no). With zero findings this would fall through to row 9 (approve); however, review_decision is REVIEW_REQUIRED and the reviewer deferred a live governance/normative-obligation question on a new-spec, maintainer-authored change. Surfacing this for human eyes via comment rather than silently auto-approving a new normative protocol obligation is the conservative, correct posture.

Note: high_risk is true only on (modified) files with no medium-or-higher concerns, so the flag alone does not escalate.

[aao-secretariat]

Ladon verdict: Approve

Approve — clean pass, no blocking findings.

The reviewer traced the full delta: a security-hardening refactor of standalone collection readback. Caller-supplied-context provenance is replaced by an HMAC commitment (catalog-reporting-runner.ts L293-346) minted only on a genuine successful create, bound to seller/authored-storyboard/adcpVersion/sandbox_account/buy-package-product ids/normalized pairs, and verified via timingSafeEqual (L348-365). Every path was walked: assertCollectionReadHistory fires before seller contact on the profile-supplied path; the wrapper execute covers native fallbacks; invalid-caller-map and executeRootGate yield typed denial / skipped gate only — no forged-pass path. Tests (history.test.cjs L445-473, L528-552) assert failed creates carry no proof, never dispatch get_media_buys, and that rebinding invalidates replayed proofs. Spec doc matches code.

Decision table applied top-down:

• Rows 1 (critical/high): no such findings.
• Row 2 (gated_paths): gated_paths is false — does not fire despite review_decision REVIEW_REQUIRED.
• Row 3 (deletion in high-risk): no deletions.
• Rows 4-5 (medium in sensitive/modified high-risk): no medium findings at all.
• Row 6 (sticky escalate): prior decision was comment, not escalate.
• Row 7 (no-auto-approve team): no team match.
• Row 8 (≥3 medium): zero medium findings.
• Row 9: approve.

high_risk is true on two (modified) files, but that is a heuristic LOOK signal; the reviewer surfaced no medium-or-higher concern on either, so the modifications are presumed safe. No schema↔docs drift, changeset present and correctly scoped, no oneOf regression, no artifact mutation.

[EvgenyAndroid]

This closes #8254 the way we hoped: an agent that doesn't offer the feature now grades not_applicable instead of failing on a product the seed invented. It's on main only for now, and our agents are graded on 3.1.29 (sales) and 3.2.6 (signals), so we'll re-check both once it ships in a bundle we're graded on and report anything that doesn't match.

[EvgenyAndroid]

Following up on my earlier comment: hosted grading puts 3.2 agents on 3.2.8, which still has the pre-#8428 and pre-#8336 versions of these storyboards. I don't see a 3.2.x backport, open or merged.

Local run on 2026-10-30: our sales agent (https://sell.nofluffadvisory.com/mcp) with its 3.2 line switched on, runner @adcp/sdk 17.0.0-beta.6, the v3.2.8 bundle, schemas from dist/schemas/3.2.8:

• collection_selection: discovery fails on "Product declares collection_targeting_allowed … Expected true, got undefined" and on the two-collection bundle check. The six steps after it skip as prerequisite_failed.
• owner_sold_channel_carriage: discovery fails on the selector's publisher_domain (expects channel-owner.example). The four steps after it skip.
• advanced_delivery_reporting: the report step fails the time-threshold and format-breakdown checks on the seeded product.

The same agent on the 3.3.0-beta.1 bundle, which carries both PRs, has none of these failures. All three storyboards grade not applicable, which is the outcome #8254 asked for.

So any 3.2 seller that doesn't offer these optional features still fails on products the fixture invents. Could #8428 and the storyboard part of #8336 go to 3.2.x, the way #8555 and #8581 did? Happy to open the backport PRs if that helps.