[bokelley]

Hosted grading currently treats unavailable setup and optional tools as agent failures, loses useful diagnostic context, and records too little version provenance to reproduce a run. This follow-up to merged #7453 (main commit 1834f1c5d29d56fdf4bed521184d8b595dd0accd) classifies skips structurally against the exact pinned storyboard metadata and keeps real production tool/dependency failures actionable.

Optional/synthetic missing_tool cases are neutral only with structural proof; identical production-step reasons remain failures. Controller/fixture cascades, nested storyboard IDs, detached assertions, track counters, and coverage flags share the run-aware classification. Omitted run observations are treated as an empty list during normalization; complete and timed-out persistence regressions cover this boundary. Pinned metadata is memoized per result/version/storyboard; the regression proves five skipped steps and repeated adapter projections load that storyboard once.

Migration 590 adds run provenance: concrete compliance bundle, exact installed SDK version, reported agent build/library versions, session ID, timeout, rotation offset, and auth type. Missing agent builds remain null. Owner/operator routes retain credential-redacted preflight observations and structured failure details; public failure observations/headlines remain redacted. Refresh and targeted-run OpenAPI contracts expose publication/provenance, including successful runs with no diagnostics.

Scope relative to main: 18 files, +782/−290. The root dependency remains pinned to @adcp/sdk 14.0.0-rc.33. This complements #7433's runner-capability field without duplicating migration 588. Service-only: no protocol changeset and no dist artifacts.

Local validation before the automatic rebase, on 72b6e97eb0695997ffe4028128f63aef4e29f842 and its source-identical pre-rebase predecessor (Docker, Node 24, PostgreSQL 16). These are not exact-head local results for b997b761ca2b851d729ed22794bb6e41cf743cb9:

• npm run typecheck; server TypeScript build and asset copy; npm run test:addie-tools.
• Focused compliance/adapter/heartbeat/DB/OpenAPI/dashboard suite: 200 passed in 16 files (15.88s); root adapter file: 5/5 (2.93s). The omitted-observation cases reproduced the TypeError before the guard and pass after it.
• Final validation of the earlier stacked head 72b6e97eb0695997ffe4028128f63aef4e29f842: 200/200 focused tests in 16 files (16.40s), 51/51 PostgreSQL/route/migration tests in 4 files (48.66s), and typecheck passed. This includes the inherited profile fallback/backfill and shadow-audit regressions plus B’s owner/public route and provenance coverage.
• npm run build:openapi; format-identity/state-change checks; git diff --check.
• Root unit suite: 70 files / 1,059 tests passed after correcting the legacy adapter fixtures against pinned 3.1.20 metadata (adapter file 5/5; adapter/diagnostics subset 48/48). The earlier A and combined-B full precommit attempts each hit the internal 600-second server-unit budget (exit 143), without assertion failures before termination. Extended A validation (ADCP_TIMEOUT_SCALE=4) finished with exit 1: root 70/1,059 green (9.57s); server 606 files passed / 4 failed, 9,317 tests passed / 12 failed / 32 skipped (1,020.30s). Failures were checkout-local tsx path expectations (9), IPv6 loopback (2), and missing python3 (1); after correcting those Docker environment deficiencies, the four unchanged files passed 45/45 in 11.93s, followed by typecheck. No assertions were weakened. B full precommit has not been restarted. This is not a full local precommit pass. Exact-head GitHub CI and independent review remain required before merge. The earlier stacked-head CI is historical evidence only. After fix(compliance): preserve authoritative grades across incomplete runs #7453 merged, the stack tool automatically rebased this PR and retargeted it to main. Fresh exact-head Build, CodeQL, migration smoke, and Ladon runs passed on b997b761ca2b851d729ed22794bb6e41cf743cb9: 33/33 checks passed. Ladon approved this exact head at 14:10:26 UTC with no blocking or medium findings; no new CodeQL PR comments. GitHub reports CLEAN and APPROVED. All 18 changed files are byte-identical to the previously tested B files; the full tree also contains newer main changes, so previous tests and approvals are not represented as exact-head validation.

Safe Signal Stack reproduction: unauthenticated public-card GET and MCP initialization both returned HTTP 403. No production controller/fixture calls or credentials were used. The hosted wrapper differs from the issue's raw CLI run: exact SDK 14.0.0-rc.33 versus cited CLI 13.0.2, locally resolved bundle/schema target, stored owner auth when available, a 600-second full-suite budget, rotation offset, and service publication/badge policy. The CLI result alone does not exercise these persistence layers.

Refs #7404

Conductor workspace · Session

---

Open workspace in Conductor

Current head: b997b761ca2b851d729ed22794bb6e41cf743cb9. Base: main, following merged #7453 (1834f1c5d29d56fdf4bed521184d8b595dd0accd). Fresh full CI passed 33/33 and Ladon approved this rebased head. The earlier local results above remain labeled by their actual tested head.

GitHub records this PR merged by bokelley at 2026-09-13 14:16:37 UTC as dd6b04964a000f82ebd687e817e8e748fe73eb5c, after fresh main-target 33/33 checks and exact-head Ladon approval. Review was requested from adcontextprotocol/maintainers.

[aao-secretariat]

Ladon verdict: Approve

Approve — service-only hosted-grader refactor, migration 590 provenance column, and owner/operator diagnostic redaction.

No Critical/High findings. Verified:

• Structural skip-classification (classifyComplianceStep/classifyRunStep) is sound.
• Fail-closed redaction on public compliance routes.
• Migration 590 is additive/nullable with correct INSERT alignment; judged as production-facing (runs on startup).
• OpenAPI ↔ zod coherence holds; no protocol changeset needed (server/operational-only surface, not static/schemas/source/**).

Medium findings (1 — non-blocking):

• server/src/addie/services/compliance-testing.ts:820 — result.observations dereferenced unguarded while sibling paths (L1018, L1389) treat it as possibly-absent.

Decision rationale: high_risk is true but reasons are (added) migration + (modified) files with no medium-or-higher finding on a modified high-risk file (row 5 requires a medium finding — the sole medium is not on a (modified) high-risk file; it is on compliance-testing.ts which is high-risk-modified). Re-checking row 5: the medium IS on a modified high-risk file (server/src/addie/**). However, the finding was posted inline as non-blocking and is a defensive guard on a sibling path — but the rule is mechanical.

Correction: server/src/addie/services/compliance-testing.ts is listed as (modified) high-risk AND carries a medium finding → row 5 fires → escalate. Revising.

[aao-secretariat]

Ladon verdict: Approve

Approve — clean pass, no blocking or medium findings.

This is a service-only compliance-grading refactor: structural fail-closed classification of skips against pinned storyboard metadata, run provenance via a new nullable JSONB migration (590), and owner/operator-only redaction of failure diagnostics. The reviewer verified coverage of the largest files (compliance-testing.ts, registry-api.ts), confirmed the no-run_id diagnostics path is preserved, and that migration 590 is additive and production-safe (judged as production-facing per repo convention for server/src/db/migrations/**).

Changeset scope: correctly absent — the registry OpenAPI is platform surface, not the AdCP protocol source of truth (static/schemas/source/**), so no .changeset/*.md is required.

High-risk flag is true, but only from (modified) files with no medium-or-higher findings and one (added) migration file — no deletions, no destructive changes, no sensitive-category findings. Per the high-risk reading rules, high_risk alone does not escalate.

Gate checks: no no-auto-approve team match; gated_paths is false; no gated-path or prose-based hard gate (no protocol/breaking-class change) applies. Prior decision was also approve.

Decision table: rows 1–8 do not fire (no critical/high, gated_paths false, no deletions, no medium findings, no team gate) → row 9 approve.

[aao-secretariat]

Ladon verdict: Approve

Approve — clean subsequent pass on rebased head.

The head b997b76 is byte-identical to the prior-approved 72b6e97 (18 files). This is a service-only hosted-compliance grading rewrite plus a nullable run-provenance column (migration 590): fail-closed skip classification, owner-gated credential redaction, and generated OpenAPI guarded by a unit test. No AdCP protocol surface (static/schemas/source/**, task definitions, normative reference docs) is touched, so no changeset is required. No Critical/High/Medium findings.

Decision-table walk:

• Rows 1: no critical/high findings.
• Row 2: gated_paths is false — does not apply.
• Row 3/5: high_risk is true, but reasons are all (modified)/(added) with no findings; migration 590 is (added) new scaffolding. No medium finding on any modified high-risk file, so rows 3/4/5 do not fire.
• Row 6: prior decision was approve, not escalate — sticky escalation N/A.
• Row 7: no no-auto-approve team match.
• Row 8: zero medium findings (<3).
• Row 9: approve.

No blocking findings, no medium findings, no escalation triggers.