Skip to content

fix(compliance): stop optional-tool skips failing dependent steps - #8953

Open
Synaeve wants to merge 1 commit into
adcontextprotocol:mainfrom
Synaeve:fix/8506-provenance-audit-missing-tool
Open

Synaeve wants to merge 1 commit into
adcontextprotocol:mainfrom
Synaeve:fix/8506-provenance-audit-missing-tool

Conversation

@Synaeve

@Synaeve Synaeve commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Refs #8506

Problem

When a seller does not advertise an optional tool, the runner skips that step as missing_tool and the step passes. If the skipped step is stateful and other stateful steps share its phase, the runner cascades prerequisite_failed onto them, as runner-output-contract.yaml > cascade_rules.default_cascade requires. Those steps then count as failures, so the storyboard fails for a tool the seller is allowed to omit.

In provenance_audit_observation, the two audit-observation reads after a skipped sync_creatives failed this way. That made Strict Spec Incomplete for sellers without a creative library.

What already landed, and what this PR adds

Piece Status
Storyboard gated on creative.supports_provenance on main Done by #8703
Same gate on 3.1.x Done by #8721
Runner honors an explicit requires_tool before the cascade (single-agent runs) Done by adcontextprotocol/adcp-client#3524, first shipped in @adcp/sdk 17.0.0-beta.22. adcp still pins beta.21.
Hosted grader treats a skipped step whose own requires_tool is unadvertised as not applicable Already in classifyRunStep (#7454). It reaches the audit reads once they name sync_creatives, which takes effect when hosted grading uses a release built from this source (the next 3.3 cut or the 3.2.x backport). Hosted runs on 3.1.29 or 3.2.x bundles do not change before then.
Hosted controller-preflight gaps #8598. Unrelated to this cascade. #8576 remains open.
Audit reads declare sync_creatives (the source half #3524 asks for) This PR
Each audit read in its own phase, so the fix holds on the pinned beta.21 and on any contract-following runner This PR
Same cascade in create_media_buy_async_lifecycle and governance_agent_binding_acceptance, which #3524 does not cover This PR
3.2.x, the only release line where the reported failure remains Follow-up backport; it closes #8506

Changes

provenance_audit_observation (1.1.0)

  • query_directed_audit_observation and query_edited_audit_observation now declare requires_tool: sync_creatives, the tool whose state they read. The storyboard-level requires: [controller] still covers the controller.
  • A controller step that names the tool owning its state already has a precedent: universal/deterministic-testing.yaml:1146 gates force_session_terminated (task: comply_test_controller) on requires_tool: si_initiate_session.
  • Each read runs alone in its own phase, with depends_on: [accept_carveout_claim].
  • Results, identical on beta.21 and beta.22:
    • A seller without sync_creatives (with or without a creative-library declaration) skips the sync and both reads. Its verifier discovery is still graded.
    • A seller with sync_creatives runs the full audit.
    • A rejected sync still cascades prerequisite_failed onto both reads.

create_media_buy_async_lifecycle (1.1.0)

  • get_task_status and list_tasks are optional in 3.x. Each read now runs in its own phase, with depends_on naming only the submit phase, or the submit and completion phases.
  • The completed-task reads keep the force_task_completion gate.
  • A seller without either read surface no longer fails the controller completion. A seller with only list_tasks is graded on it.
  • Verified on beta.22: without this split, these sellers still fail, because the dependent steps do not consume the missing tool's state.

governance_agent_binding_acceptance (1.1.0)

  • The optional comply_test_controller read and the recovery sync_governance each run in their own phase after account setup.
  • A seller without the controller still has its recovery graded. When the controller is present, the read still runs before the resync.
  • Verified on beta.22: without this split, the recovery still fails.

Tests

  • tests/sdk-runner-capability-gates.test.cjs: three runner tests. Each loads its storyboard with the SDK parseStoryboard and runs it against stub sellers, asserting dispatch order and that no step is prerequisite_failed or failed.
    • Provenance: a seller without sync_creatives, one with sync_creatives but no library declaration (full audit), and one whose sync is rejected (both reads prerequisite_failed).
    • Async: sellers with no read, with list_tasks only, and with both reads; also both reads without force_task_completion.
    • Governance: sellers without and with the controller.
    • All three fail on the previous storyboards, with both SDK versions for provenance.
  • tests/provenance-tool-gates.test.cjs: the audit reads must name sync_creatives, and must not share a phase with the sync they depend on.

Tests

  • @adcp/sdk 17.0.0-beta.21 (pinned):
    • tests/sdk-runner-capability-gates.test.cjs 71/71.
    • test:governance-task-slices 55/55.
    • test:auction-only-compliance 182/182.
  • @adcp/sdk 17.0.0-beta.22, loaded from the published tarball without changing the lockfile: tests/sdk-runner-capability-gates.test.cjs 71/71.
  • All scripts/lint-storyboard-*.cjs lints over the source pass.
  • Training agent (sales tenant, 3.3 candidate mode): the three storyboards run clean with full coverage (7, 10 and 5 steps passed).

Changeset

adcontextprotocol patch: compliance source change.

Out of scope / follow-ups

🤖 Generated with Claude Code

provenance_audit_observation
- declare sync_creatives on both audit reads, as adcp-client#3524 expects
- run each audit read in its own phase after the sync

create_media_buy_async_lifecycle
- run each optional get_task_status/list_tasks read in its own phase
- keep the force_task_completion gate on the completed-task reads

governance_agent_binding_acceptance
- isolate the optional controller read from the recovery sync

tests
- runner regressions for sellers with and without each optional tool
- provenance audit reads name the tool whose state they read
@mintlify

mintlify Bot commented Nov 12, 2026

Copy link
Copy Markdown

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit 4729071.

Approve preview

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve

Compliance-grading fix: splits three storyboards so a skipped optional tool no longer cascades prerequisite_failed onto stateful dependents. Each optional read now sits alone in its own phase with explicit depends_on, and the provenance audit reads are re-gated requires_tool: sync_creatives, matching precedent at deterministic-testing.yaml:1148. Verified: no phase left empty by the splits, every depends_on target exists, declaration order preserved, and storyboard-level requires: [controller] still covers the controller.

Changeset is patch on adcontextprotocol for a static/compliance/source/** change — in-scope and non-breaking (loosens grading), consistent with prior compliance fixes (#8703, #8603). No schema/dist/3.1.x surface touched. New tests assert exact dispatch order, correct skip_reason values, and no prerequisite_failed cascade.

No Critical/High/Medium findings. No gated paths, no high-risk flag, no team gate. Falls through to row 9.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant