Skip to content

fix(compliance): preserve authoritative grades across incomplete runs - #7453

Merged
bokelley merged 2 commits into
mainfrom
conductor/fix-compliance-publication
Sep 13, 2026
Merged

bokelley merged 2 commits into
mainfrom
conductor/fix-compliance-publication

Conversation

@bokelley

@bokelley bokelley commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor

A timeout-budget run currently replaces the public grade and storyboard rows, and can revoke badges. Persist its audit evidence first, then return before authoritative materialization, transition notifications, and badge processing. SDK completeness: timed_out is the structural boundary; targeted storyboard runs are also audit-only.

Full runs retain the selected bundle storyboard denominator, including nested storyboard IDs. Migration 589 separates scheduling into next_compliance_check_at, so heartbeat locks, deferrals, interval changes, and manual requeues preserve the last authoritative timestamp. The migration also seeds metadata for previously checked discovered agents with INSERT ON CONFLICT and preserves existing settings and non-NULL schedules. Durable refresh recovery and the dashboard honor the same publication state. Supported-version and declared-specialism fallbacks read only authoritative, non-dry-run profiles; newer incomplete, targeted, and dry-run rows cannot shadow them.

Scope: 20 files, +696/−287. This is the publication layer of the #7404 stack. The follow-up PR contains exact skip classification, provenance, and diagnostic visibility. Neither this PR nor metadata PRs #7405/#7406 alone resolve every report in #7404. No protocol changeset or dist artifacts.

Validation in Docker (Node 24, PostgreSQL 16):

  • npm run typecheck
  • Focused compliance/heartbeat/DB/denominator/dashboard unit tests: 173 passed in 15 files (6.90s).
  • PostgreSQL, refresh-route, and migration/audit regressions: 35 passed in 3 files (38.56s). These include actual migration 589 execution and heartbeat selection for recent/overdue/never-checked discovered agents, idempotent schedule preservation, the profile fallback matrix, audit-only exclusion, incomplete-run persistence, and badge preservation.
  • npm run build:openapi; git diff --check; title and protocol-scope checks.
  • Root unit suite: 70 files / 1,059 tests passed. The earlier A and combined-B full precommit attempts each hit the internal 600-second server-unit budget (exit 143), without assertion failures before termination. The extended ADCP_TIMEOUT_SCALE=4 npm run precommit run finished with exit 1: root 70 files / 1,059 tests passed in 9.57s; server 606 files passed and 4 failed, with 9,317 tests passed, 12 failed, 32 skipped, and one IPv6 listener error, in 1,020.30s. The 12 failures are in unchanged environment-sensitive tests: 9 expect a physical checkout-local tsx path (this worktree used symlinked node_modules), 2 require IPv6 loopback, and 1 requires python3. After installing python3, enabling IPv6 loopback in the test container, and giving the worktree a physical node_modules directory, all four unchanged files passed: 45/45 tests in 11.93s, followed by a passing typecheck. No test assertions were weakened. This is not a full local precommit pass. Old-head GitHub integration shard 2/4 separately exposed a change-caused fixture drift: the shadow-audit test manually created agent_compliance_runs without is_authoritative. The fixture now models the post-589 column and tests both aggregate and per-agent audit-only exclusion. The corrected 35-test integration set and typecheck pass; new exact-head GitHub CI and independent review remain required before merge.

Related: #7433 reserves migration 588 for runner capability version; this stack uses 589/590. Signal Stack unauthenticated public-card and read-only MCP initialization probes both returned HTTP 403 from this workspace. No credentials or controller/fixture calls were used.

Refs #7404

Conductor workspace · Session

Current head: 62803575f6c5f86a213c3af0827235ae4da0cd5b. Follow-up: #7454.

Exact-head GitHub validation: 32/32 checks passed on 62803575f6c5f86a213c3af0827235ae4da0cd5b, including the full Build Check workflow, migration smoke tests, CodeQL, and the normally rerun IPR workflow. Fresh Ladon review approved this head with no findings. Independent review was requested from adcontextprotocol/maintainers at 14:01 UTC. GitHub records the normal merge by bokelley at 14:06:15 UTC as 1834f1c5d29d56fdf4bed521184d8b595dd0accd. The local full-precommit limitation above remains separate from these passing GitHub gates. #7454 was subsequently rebased and retargeted to main for fresh validation.

Refs #7404. Publication, scheduler, denominator, and profile fallback safeguards only; classification and provenance follow in the next stacked PR.
aao-secretariat[bot]
aao-secretariat Bot previously approved these changes Sep 13, 2026

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — compliance run-publication change is clean, no blocking findings.

Checked: scheduling migrated to agent_registry_metadata.next_compliance_check_at with authoritative card/badges gated on is_authoritative + completeness (timed-out/targeted runs persist as immutable audit-only evidence); CHECK-constraint invariant; dead COALESCE removal (check_interval_hours is NOT NULL DEFAULT 12); CREATE OR REPLACE VIEW column-order safety; and the lock/deferral operational path.

Migration 589 is a new (added) file under a high-risk glob — scaffolding, not an in-place edit to an existing sensitive file; no destructive/deletion change. This is a migration/operational PR touching server/**, not the published protocol surface (static/schemas/source/**, docs/reference/**, dist artifacts), so schema↔docs coherence, changeset-scope, oneOf, artifact-immutability, and 3.1.x patch-eligibility gates do not apply. No breaking-class protocol change.

Medium findings

  • server/src/db/migrations/589_compliance_run_publication.sql — backfill only seeds next_compliance_check_at for agents with an existing metadata row, so heartbeat-checked-but-unregistered agents become immediately due after deploy (bounded, self-healing recheck storm).

Decision path: no critical/high findings (row 1 no); gated_paths false (row 2 no); no deletions (row 3 no); the single medium is not data-loss/schema/infra category — it's a bounded operational backfill gap, not a schema-invariant or infra defect (row 4 no); high_risk is true but the only medium is on the newly-added migration, not a (modified) high-risk file (row 5 no); no prior escalation (row 6 no); no team gate (row 7 no); only 1 medium finding (row 8 no) → row 9 approve.

Refs #7404. Seed missing metadata idempotently, preserve owner scheduling, and cover the heartbeat queue plus post-publication shadow-audit schema.
bokelley added a commit that referenced this pull request Sep 13, 2026
…ance

Refs #7404. Stack on #7453 with exact pinned skip classification, memoized metadata, reproducible run provenance, and owner/public diagnostic boundaries.

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — no blocking or medium findings.

This PR adds an idempotent backfill to migration 589 that seeds next_compliance_check_at for discovered agents lacking a registry-metadata row, closing a cutover thundering-herd where recently-checked agents would re-test immediately under the new scheduling query. Seed defaults match the prior COALESCE fallbacks, and two new integration tests exercise the real selection query against the migrated schema.

Checked:

  • Migration 589 is a new file (added) under server/src/db/migrations/**; judged as production-facing (runs on startup) and the backfill is idempotent with defaults matching prior behavior.
  • High-risk flag is true only due to modifications to server/src/addie/** and server/src/routes/** and the new migration — reviewer found no medium-or-higher concerns on any modified file, so the modifications are presumed contract-preserving.
  • No spec-surface changes (static/schemas/source/**, docs/reference/**), so schema↔docs coherence, changeset scope, oneOf discriminator, and artifact-immutability gates do not apply. This is a migration/operational-only change.
  • gated_paths is false; no no-auto-approve team match; prior decision was approve.

Decision table: rows 1–8 do not fire (no critical/high, no medium, gated_paths false, no deleted high-risk file, no team gate, <3 medium). Falls through to row 9 → approve.

@bokelley
bokelley requested a review from a team September 13, 2026 14:01

Copy link
Copy Markdown
Contributor Author

I have read the IPR Policy

@bokelley
bokelley merged commit 1834f1c into main Sep 13, 2026
32 checks passed
@bokelley
bokelley deleted the conductor/fix-compliance-publication branch September 13, 2026 14:06
bokelley added a commit that referenced this pull request Sep 13, 2026
…ance

Refs #7404. Stack on #7453 with exact pinned skip classification, memoized metadata, reproducible run provenance, and owner/public diagnostic boundaries.
bokelley added a commit that referenced this pull request Sep 13, 2026
…ance (#7454)

* fix(compliance): classify hosted skips and expose redacted run provenance

Refs #7404. Stack on #7453 with exact pinned skip classification, memoized metadata, reproducible run provenance, and owner/public diagnostic boundaries.

* fix(compliance): tolerate omitted observations during normalization

Refs #7404. Preserve complete and timed-out run persistence when the SDK result omits observations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant