Skip to content

feat(reporting): issueObserver — notify the seller when a reporting issue opens, escalates or resolves (#3648) - #3654

Merged
bokelley merged 9 commits into
mainfrom
feat/3648-issue-observer
Nov 9, 2026
Merged

bokelley merged 9 commits into
mainfrom
feat/3648-issue-observer

Conversation

@bokelley

@bokelley bokelley commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Closes #3648. A seller can now learn, through a supported callback, when a CONSUMER_STATUS_MISMATCH opens, escalates past consumer_mismatch_escalation_seconds, or resolves. This is the TypeScript side of the Python SDK's issue_observer: storage outbox adcontextprotocol/adcp-client-python#1874, SDK observer adcontextprotocol/adcp-client-python#1875.

const reporting = createReliableReportingService({
  storage, // memoryStorage(), or postgresStorage({ pool }) with migration 0012 applied
  resolveConsumerId,
  consumerMismatchEscalation: { escalationSeconds: 14_400, operationsContact: { email: 'ops@seller.example' } },
  issueObserver: async event => {
    // at least once: deduplicate on event.eventId
    if (event.issue && event.issue.responsibleParty !== 'buyer') await page(event);
  },
});

How the TypeScript ledger maps onto the Python design

The Python outbox hangs off reporting_issue_lifecycle, a persisted per-chain occurrence table with generations and an operator set_issue_state. TypeScript has neither: a mismatch is a pure read-time projection (projectReportingConsumerStatusMismatchV1), and its issueId is hash(obligation, reporting_status_id). So the occurrence here is that projection, with the seller-health suppression removed, as in Python's consumer_statement_conflicts. It is re-derived at the writes that can change it.

Kind Recorded by Same transaction as
opened syncConsumerStatusBatch (a new conflicting leaf) or commitRevision (a restatement makes a leaf conflict, or the first revision anchors one) the statement append or the revision insert, under the account lock
resolved syncConsumerStatusBatch (the superseded leaf's occurrence) or commitRevision (a leaf that no longer conflicts) same
escalated sweepIssueEscalations on runCycle, after re-settling the obligation its own per-obligation transaction under the account lock
  • Identity. An occurrence is (issueId, consumerId). The published issueId names the obligation and the causing reporting_status_id but not the consumer, and two consumers may legally reuse one status id. UNIQUE (issue_id, consumer_id, kind) makes each record exactly once. eventId is rpie_ plus the first 40 hex characters of sha256(canonical_json(["reporting-issue-event-v1", issueId, consumerId, kind])), pinned by a vector computed with Python's canonical JSON. Python hashes [tag, issue_id, kind]; its issue_id is already consumer-scoped (the occurrence key includes consumer_id), so it has no collision.
  • Delivery. At least once. At most one event per occurrence is leased at a time, in recording order (SKIP LOCKED on PostgreSQL). The observer gets a 10 s timeout. Retries back off from 30 s, doubling to 1 h, with no limit. An aborted cycle releases the rest of its batch.
  • Retention. The outbox is also the occurrence ledger, so retention prunes only whole closed occurrences: resolved, every event delivered more than 7 days ago, and unable to be derived again because the causing statement was superseded or the obligation is gone. An open occurrence is never pruned, so a later supersession still resolves it and an escalation window longer than 7 days still escalates. A resolved occurrence whose statement is still current is kept as the anchor, so no later write can record a delivered eventId again. The Python counterpart re-escalates still-open occurrences after its prune: fix(reporting): issue-event retention prune re-escalates still-open occurrences every 7 days adcp-client-python#1885.
  • Failures. A failure becomes a ReportingIssueObserverError with component issue_observer, issue_events or issue_events:snapshot. It goes to the scheduler's onError and into runCycle(...).issueEvents.errors. It never fails a buyer call or the cycle.
  • Rendering. issue is rendered as get_reporting_status showed it at occurredAt: revisions created by then, persisted issues open then, and the escalation window. It is null for resolved, while the read suppressed it, or when the evidence is gone. Rendering never throws. Managed Delivery is left out. principal comes from the obligation on every kind, resolved included, and is omitted only once the obligation is gone.

Escalation and delivery need runCycle. opened and resolved are recorded with the writes either way. Without runCycle (start() or your own scheduler), though, the observer is never called and escalated is never recorded. Python behaves the same: its worker turn both sweeps escalations and delivers.

API

  • @adcp/sdk/reporting/service
    • issueObserver option. It is refused at construction against a store without the capability, and at the first cycle before the table exists.
    • runCycle(...).issueEvents.
    • The scheduler reports failures under phase issue_observer.
    • Re-exports the event and dispatcher types.
  • @adcp/sdk/reporting/ledger (experimental store surface)
    • The ReportingIssueEventStoreV1 capability and supportsReportingIssueEventsV1.
    • Lease and record types.
    • Pure helpers that both stores share: reportingIssueEventIdV1, reportingConsumerMismatchOccurrenceV1, planReportingIssueEventsV1, reportingIssueEscalationV1, renderReportingIssueEventV1.
    • createReportingIssueEventDispatcherV1 and ReportingIssueObserverError.
  • memoryStorage() implements the capability.
  • @adcp/reporting-storage
    • PostgresReportingLedgerStore implements it.
    • Migration 0012 reporting_issue_events: one new table and its indexes, additive. The SQL is exported as REPORTING_ISSUE_EVENTS_MIGRATION.

Storage contract and migration

TypeScript has no STORAGE_CONTRACT constant. The Python side kept its contract at 2 because its 0008 is additive. This change is likewise one additive migration, numbered 0012 because #3651 takes 0011. No existing table, key, function or trigger changes, so workers of the previous release run beside it and record no events. postgresStorage().checkSchema() names 0012 as pending until npx adcp-reporting-storage migrate runs. That matches Python, which refuses to start while a non-contract migration is pending. The semantics stay aligned.

Parity gaps (known, deliberate)

  1. acknowledged / waived are never emitted. They are in the kind union for type parity. The TypeScript ledger has no operator issue-state API (Python's set_issue_state and its bilateral waiver binding). TSDoc and the guide say so.
  2. A seller-side change never clears a TypeScript mismatch: fix(reporting): CONSUMER_STATUS_MISMATCH never resolves through a seller-side change (TS read-path parity with Python's consumer_statement_conflicts) #3653. Python's consumer_statement_conflicts resolves content_mismatch, revision_missing and unreadable when the seller's record comes to agree. The TypeScript read projection is monotone in seller evidence, so only a buyer supersession resolves one. commitRevision already re-settles against the read projection and records resolved when it stops conflicting. That branch is wired and tested at store level, and it will fire once fix(reporting): CONSUMER_STATUS_MISMATCH never resolves through a seller-side change (TS read-path parity with Python's consumer_statement_conflicts) #3653 changes the projection. fix(reporting): CONSUMER_STATUS_MISMATCH never resolves through a seller-side change (TS read-path parity with Python's consumer_statement_conflicts) #3653 is a buyer-visible read change, so it stays out of this PR.
  3. Issue identity differs. TypeScript issueIds are per causing statement; Python's are per chain occurrence with generations. TypeScript's eventId also hashes the consumer. The two SDKs never publish equal ids for one disagreement, so observers should not compare them across SDKs.
  4. The production service supports the observer. TypeScript is ahead here, not divergent. createPostgresReliableReportingProductionService passes issueObserver to the core service, whose runCycle and start() it drives. When the observer is set, it also adds REPORTING_ISSUE_EVENTS_MIGRATION to setup.migrations. Python currently refuses issue_observer with production=, and the parity follow-up that adds it there is feat(reporting): support issue_observer on the production= service path adcp-client-python#1873. A production-service test covers delivery, a failing observer reaching onError without failing the cycle, and redelivery.
  5. Opening timing. Python can also open an occurrence from a status read's settle. TypeScript reads stay read-only: it opens only at the writes above, which are the only events that change the TypeScript projection's conflict set.

Tests

  • test/lib/reporting-issue-observer.test.js (42 tests, memory store, dispatcher, service):
    • The eventId golden vector (computed with Python's canonical JSON over [tag, issueId, consumerId, kind]), and exact instant arithmetic.
    • The planner, and the store-level flows: an agreeing leaf records nothing; a restatement opens in commitRevision; a negative status opens at ingest, or at the first revision commit when filed before any revision; supersession resolves the old occurrence, and opens a new one when the new leaf also conflicts; exact or unchanged replays record nothing.
    • Resolve on restatement: an open occurrence whose leaf agrees after a commit resolves in that commit.
    • Escalation once, at the exact boundary, and never for a resolved occurrence.
    • Leases, in order and one at a time per occurrence; lease expiry with attempt 2; a stale lease can't complete; release backoff; prune; account scoping.
    • Observer failure retried with the same eventId; async timeout; a snapshot-read failure released, not delivered; rendering never throws.
    • Service refusals, and runCycle(...).issueEvents.
    • Review fixes:
      • An open occurrence survives retention and still resolves on a later supersession.
      • An escalation window longer than the retention still escalates, exactly once.
      • A resolved occurrence whose statement is still current is kept, and no later write recreates a pruned occurrence's eventIds.
      • Two consumers reusing one reporting_status_id get independent occurrences: one resolves while the other escalates.
      • principal appears on opened and resolved, and is omitted once the obligation is gone.
      • An aborted cycle releases the rest of its batch.
  • packages/reporting-storage/test/reporting-issue-events-pg.test.js (24 tests, PostgreSQL 16):
    • 0012 is additive and idempotent; with the table absent, ingest and commit still work and record nothing.
    • The same lifecycle flows; exactly-once escalation across 4 concurrent sweeps; disjoint concurrent claims; per-occurrence ordering; lease expiry, backoff, prune, account scoping, and dispatcher delivery.
    • The review fixes on PostgreSQL: closed-occurrence prune, long escalation window, the current-statement anchor, two consumers sharing one status id, and principal.
  • packages/reporting-storage/test/reporting-issue-observer-production-pg.test.js (3 tests): createPostgresReliableReportingProductionService with issueObserver.
    • setup.migrations carries REPORTING_ISSUE_EVENTS_MIGRATION, and the observer starts through applyMigrations alone.
    • An occurrence opens and runCycle delivers it (result.core.issueEvents).
    • A failing observer leaves the cycle resolving and reaches the scheduler's onError as a ReportingIssueObserverError.
    • The next cycle redelivers the same eventId with attempt 2.
  • reporting-storage-migrations.test.js asserts that 0012 is the last migration.
  • Regression, local PostgreSQL 16:
    • SDK reporting suites, including the new file and request-signing-applies-when: 374/374.
    • Storage PG suites: 260/260. That covers the new files, migrations including fix(reporting-storage): allow shared row sets across SDKs #3651's, ingest, ledger, facade, lifecycle, production, notification activity, managed delivery, subscription ledger/receipts/delivery, writer fence, history and compaction.
    • typecheck, typecheck:reporting-storage, lint:boundaries, format:check, ci:doc-links and ci:docs-check are clean.

Changesets

  • @adcp/sdk minor
  • @adcp/reporting-storage minor, with a migration note

🤖 Generated with Claude Code

aao-secretariat[bot]
aao-secretariat Bot previously approved these changes Nov 8, 2026

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — clean additive PR with no blocking or medium findings.

Checked:

  • Adds TypeScript issueObserver for Reliable Reporting (transactional outbox + dispatcher + PG migration 0012), mirroring Python's issue_observer.
  • Core logic verified: resolve-never-before-open in planReportingIssueEventsV1, at-most-one-per-occurrence/recording-order in claimOn (SKIP LOCKED) and the memory head-map, exactly-once via UNIQUE(issue_id,kind) + derived eventId, sub-ms-preserving addReportingInstantOffset.
  • Newly-unconditional commitRevision callback is a no-op change (appendChange gated on changeFeed; port?. short-circuits).
  • Operational readiness solid: 10s observer timeout, capped backoff, errors routed to onError, per-event retry.
  • Both changesets present and correctly scoped minor/additive; package boundary respected.

Gate checks: no no-auto-approve team match; gated_paths false; high_risk false. review_decision is REVIEW_REQUIRED but gated_paths is false, so row 2 does not apply. No critical/high/medium findings — falls through to row 9 (approve).

Non-blocking note: PostgreSQL-specific SQL is exercised only by the env-gated reporting-issue-events-pg.test.js, consistent with the repo's other PG-store tests; the in-memory suite covers shared logic.

@bokelley

bokelley commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

CI note: Unit Tests Fast (1/3) (and the Unit Tests / Test & Build aggregates) fail on 3 tests in test/lib/request-signing-applies-when.test.js (storyboard dispatch (3.2 profile): legacy_hmac_fallback … skips vector 027, added by #3627). That is unrelated to this change: #3652, on the same base 727c947c8, fails the same 3 tests, and they fail locally on this branch without touching request signing. Every other check passes, including Reporting Storage Tests.

aao-secretariat[bot]
aao-secretariat Bot previously approved these changes Nov 8, 2026

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — delta pass over 4 files (changeset note, REPORTING-LEDGER.md, one conditional line in packages/reporting-storage/src/service/index.ts, new PG integration test). No critical/high/medium findings.

Checked:

  • Production-service change adds REPORTING_ISSUE_EVENTS_MIGRATION to applyMigrations only when issueObserver is set — same conditional pattern as rowStorage (0004) and changeFeed (0005), with the migration unconditionally present in the canonical MIGRATIONS list that checkSchema enforces. issueObserver flows to core service via ...rest, gated on issueEventsReady(). Right shape.
  • Changeset is minor on @adcp/reporting-storage per the experimental-schema policy; docs accurate.
  • No witness-not-translator concerns, no transport-client reimplementation, no doc-link drift.

Gate checks: no no-auto-approve team match; gated_paths is false (row 2 does not fire despite review_decision REVIEW_REQUIRED); high_risk false; no deletions in sensitive paths. Prior decision was approve with no findings; nothing new surfaced. Rows 1–8 do not fire → row 9 approve.

bokelley and others added 5 commits November 8, 2026 23:26
…3648)

Record CONSUMER_STATUS_MISMATCH occurrence transitions (opened, escalated,
resolved) in a transactional outbox beside the ledger writes that cause them,
and deliver them to an issueObserver at least once from runCycle. Python
parity with adcp-client-python#1874 and #1875.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sets (#3648)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…memory and PostgreSQL (#3648)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
createPostgresReliableReportingProductionService adds the issue-event migration
to setup.migrations when issueObserver is set; test delivery, onError and
redelivery through the production composition.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…table

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
aao-secretariat[bot]
aao-secretariat Bot previously approved these changes Nov 8, 2026

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — clean, additive issue-observer feature with no blocking or medium findings.

What I checked:

  • No findings at any severity (no critical/high/medium).
  • high_risk: false; gated_paths: false so row 2 does not apply despite review_decision: REVIEW_REQUIRED.
  • No no-auto-approve team match.
  • Witness-not-translator honored: render yields issue: null rather than fabricating upstream data.
  • Changesets correct (sdk minor + reporting-storage minor; no hand-edited package.json version).
  • Storage semantics verified: UNIQUE(issue_id,kind) + ON CONFLICT DO NOTHING for exactly-once across replicas; FOR UPDATE SKIP LOCKED claim; 0012 migration is additive.
  • Operability: 10s observer timeout via Promise.race with timer cleared, lease sizing, backoff, failures routed to onError without failing buyer call/cycle, 7-day prune.
  • Tests exhaustive including gated PostgreSQL integration.

Prior decision was also approve for an earlier head; current run confirms no regressions. Decision table: rows 1–8 do not fire → row 9 approve.

bokelley and others added 3 commits November 9, 2026 00:16
…open one (#3648)

Two consumers may reuse one reporting_status_id on one obligation, and the
published issueId does not name the consumer, so occurrences and event
uniqueness are now (issueId, consumerId[, kind]) and the eventId hashes the
consumer. The outbox doubles as the occurrence ledger, so retention prunes only
whole resolved, fully delivered occurrences: an open occurrence keeps its
opened row for a later resolution and for escalation windows longer than the
retention. Migration 0012 changes in place (unreleased).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#3648)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…abort (#3648)

A resolved occurrence whose statement is still current can be derived again,
so pruning it would let a later write record its delivered eventIds again.
Prune only occurrences that are resolved, fully delivered past the retention,
and whose statement was superseded or whose obligation is gone. The dispatcher
now releases the rest of a claimed batch when the cycle is aborted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… resolved principal, abort (#3648)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — clean pass, no blocking or medium findings.

Checked:

  • Re-keying issue occurrences on (issueId, consumerId): reportingIssueEventIdV1 now hashes [tag, issueId, consumerId, kind], and every occurrence-membership check (plan, open-set, escalation sweep, claim head, memory issueOccurrenceKey) was updated in lockstep, matching the UNIQUE and the new (issue_id, consumer_id, seq) index.
  • Pruning removes only whole closed occurrences (resolved, all events past retention, statement no longer current via currentConsumerLeaf/is_current or obligation gone) and never an open/anchor occurrence; postgres CTE and memory grouping agree.
  • Dispatcher reads the obligation for resolved to carry principal, retries on read throw, returns undefined (no retry) when obligation is genuinely gone, and releases the batch tail on abort.
  • Test coverage is sound: keyed occurrences, occurrence-whole prune, principal-on-resolved (incl. obligation-gone), and abort release.
  • Both changesets present and minor — correct for an experimental, wholly-new surface.

Decision path: no critical/high (row 1 no), gated_paths false (row 2 no), high_risk false (rows 3/5 no), no medium findings (rows 4/8 no), prior decision was approve (row 6 n/a), no team gate (row 7 no) → row 9 approve. Prior run also approved on clean findings; this run remains clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(reporting): issueObserver — notify the seller when a reporting issue opens, escalates or resolves (Python parity)

1 participant