Skip to content

feat(reporting-storage): record issue-lifecycle transitions in a transactional outbox - #1874

Merged
bokelley merged 1 commit into
nextfrom
claude/issue-1258-issue-event-outbox
Nov 8, 2026
Merged

bokelley merged 1 commit into
nextfrom
claude/issue-1258-issue-event-outbox

Conversation

@bokelley

@bokelley bokelley commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Summary

The storage half of #1258: a transactional outbox of issue-lifecycle transitions, which the SDK's upcoming ReliableReportingService(issue_observer=...) reads. The SDK PR is stacked on this one and merges after the storage beta release.

  • Migration 0008 reporting_issue_events (new header -- adcp-upgrade: additive) adds one table and its indexes. No existing table, key, function or trigger changes, and the storage contract stays 2 (the shared _record_rows column lists are untouched).
  • Same-transaction writes. Every write that moves a reporting_issue_lifecycle row records its transition in that transaction: an occurrence opening (ensure, settle intents, replay persistence), being acknowledged or waived (set_issue_state), or resolving (retire_issue, settle intents). An event exists exactly when its transition committed. UNIQUE (issue_id, kind) keeps the record exactly once across replicas. event_id is derived from (issue_id, kind), so every redelivery carries the same id.
  • New store API on PgReportingLedgerStore:
    • issue_events_supported (the capability flag) and issue_events_ready() (has 0008 been applied).
    • sweep_issue_escalations(after=): settles each candidate account under its lock, so a cleared condition resolves instead of escalating, then records escalated with ON CONFLICT DO NOTHING.
    • claim_issue_events: SKIP LOCKED leases, at most one event per occurrence, each occurrence in recording order.
    • complete_issue_event, release_issue_event(retry_after=) and prune_issue_events(older_than=).
    • ReportingIssueEventLease is exported.
  • Unmigrated ledgers. Until 0008 is applied, lifecycle writes record nothing and issue_events_ready() is False. Nothing else changes.

Rolling deploys and 0.1.x pods

python -m adcp migrate now applies an additive migration even while a contract step waits. 0008 therefore lands on an expanded ledger while adcp-reporting-storage 0.1.x/0.2.0b0 (SDK 9.x) pods still run. Those pods neither read nor write the table: the earlier-release validators check required objects only, so an extra reporting_* table is unrelated DDL to them. They record no events for the transitions they make.

The published-wheel rolling test (test_a_rolling_deploy_runs_published_and_new_pods_side_by_side_until_contract, 9.3.1 + 0.1.0b4 and 10.0.0b2 + 0.2.0b0, with and without notifications) now asserts the table exists before its old pods restart, read, write and run worker turns beside it. test_migration_0008_adds_only_the_event_table proves the migration is purely additive: dropping its objects restores exactly the 0001–0007 catalog.

Tests

  • New tests/conformance/reporting/test_reporting_issue_events_pg.py:
    • additivity
    • unmigrated behavior
    • one record per write; idempotent ensure, re-acknowledge and external_ref record nothing
    • rollback records nothing
    • the ingest settle path opens and resolves
    • escalation exactly once across four concurrent sweepers, and never for resolved or waived occurrences
    • a cleared condition resolves instead of escalating
    • claim ordering, leases, release, expiry and stale-lease refusal
    • disjoint concurrent claims
    • pruning
  • Migration unit and PG tests are updated for 0008 and for the held-contract plan (_past_held_contract).
  • The published-wheel upgrade suite is updated for 0008 being applied while 0007 is held.

Docs: docs/reporting-ledger-migration.md (new "Issue-lifecycle events (migration 0008)" section and deploy step) and the storage README.

🤖 Generated with Claude Code

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — clean additive outbox for reporting-storage.

Checked:

  • Semver signal correct: feat(reporting-storage): matches a purely additive change (new ReportingIssueEventLease export, new store methods, additive migration 0008). Nothing removed, renamed, or narrowed on the public surface.
  • Largest file ledger/issue_events.py (362 new lines) verified: claim's per-occurrence ordering is sound (NOT EXISTS seq-gating arm + FOR UPDATE SKIP LOCKED leases at most one event per occurrence in seq order; sorted(..., key=row[-1]) sorts by trailing seq). complete/release gate on lease_token, so a stale holder cannot settle a reclaimed lease.
  • Op-readiness: crashed-worker reclaim via expiring lease, idempotent redelivery via event_id derived from (issue_id, kind), same-transaction writes with short-circuit on unmigrated ledgers.

Two non-blocking latent follow-ups noted by reviewer (dry-run/real _migrate divergence only if a second contract step ever existed — unreachable with single 0007 contract today; sweep_escalations per-account re-read lacks a hard LIMIT but is bounded by open occurrences). Neither rises to medium.

No critical/high/medium findings. No gated paths, no high-risk flag, no author-team gate. Falls through to row 9.

…sactional outbox

Migration 0008 (header "-- adcp-upgrade: additive") adds
reporting_issue_events and its indexes: one new table, nothing else, and the
storage contract stays 2. Every write that moves a reporting_issue_lifecycle
row (ensure opens, set_issue_state acknowledges or waives, the projection
retires as resolved, replay persistence) records its transition there in the
same transaction, so an event exists exactly when its transition committed.
UNIQUE (issue_id, kind) keeps the record exactly once across replicas, and
event_id is derived from (issue_id, kind), so every redelivery of a
transition carries the same id.

PgReportingLedgerStore gains the outbox the issue observer reads
(adcp-client-python#1258): issue_events_supported and issue_events_ready(),
sweep_issue_escalations(after=...) (settles each candidate account under its
lock, then records 'escalated' for occurrences still open or acknowledged past
opened_at + after, ON CONFLICT DO NOTHING), claim_issue_events (SKIP LOCKED
leases, at most one event per occurrence, each occurrence in recording order),
complete_issue_event, release_issue_event(retry_after=...) and
prune_issue_events(older_than=...), both measured on the store's clock.
ReportingIssueEventLease is exported.

Until 0008 is applied, lifecycle writes record nothing and
issue_events_ready() is False; nothing else changes. python -m adcp migrate
now applies an additive migration even while a contract step waits, so 0008
lands on an expanded ledger beside adcp-reporting-storage 0.1.x/0.2.0b0
processes, which neither read nor write the table. The published-wheel
rolling test runs them beside it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bokelley
bokelley force-pushed the claude/issue-1258-issue-event-outbox branch from 021325d to 84f0e56 Compare November 8, 2026 15:42

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — additive issue-lifecycle event outbox for adcp-reporting-storage.

This PR adds a new table (migration 0008), new store methods, and a ReportingIssueEventLease export — all additive and tracked in public_api_snapshot.json, correctly signaled with feat(reporting-storage):. The reviewer verified:

  • Migration-walk contract logic (_migrate/_past_held_contract mirror step-for-step; additive 0008 applies past a held 0007).
  • Exactly-once recording via UNIQUE (issue_id, kind) + event_id derived from the pair.
  • claim ordering safety (issue_events.py:385-388) — the NOT EXISTS guard holds one-per-occurrence/in-order under SKIP_LOCKED across replicas.
  • record_issue_transition_on occurred_at handling (issue_events.py:261-263) correct for acknowledged vs waived/resolved.
  • Replay double-record avoided in status_snapshot.py (final-state INSERT no-ops the follow-on UPDATE).

No critical/high/medium findings. Not high-risk, gated_paths false (and review_decision is APPROVED anyway), no no-auto-approve team match. Prior decision was also approve with no findings. Falls through to row 9.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant