Skip to content

Collect orphaned row objects by listing, fenced against clones (parity with adcp-client#3291 and #3593) #1829

Closed

Description

@bokelley

Problem

Python's write-intent sweep deletes only the objects an intent lists. It cannot see an object whose PUT succeeded before the intent recorded it, intents for a binding no running host configures, or intents lost to a database restore. The design record (specs/shared-sdk-persistence-proposal.md §4.2) tracks a listing collector for them. The JS SDK shipped it in adcp-client #3291 (closes adcp-client #3284) and then fenced it against a cloned or restored ledger database in adcp-client #3593 (closes adcp-client #3292). Python should mirror both, opt-in, with the same semantics and the same shared DDL.

Mirror

Collector (JS #3291)

  • Optional provider capability list (prefix, page token, page size) for the filesystem, GCS, S3 and Azure providers, and a list check in the provider conformance suite when a provider has it. A provider without list keeps working; its binding is reported unsupported.
  • PgReportingLedgerStore.collect_orphaned_row_objects: lists each configured object binding under {prefix}/{namespace_key}/ and deletes only a well-formed row-object key (.jsonl or .jsonl.gz) under that prefix, older than min_age_seconds (default 7 days, floor 1 day), that no non-pruned row set chunk and no write intent names (version- and binding-blind pre-check), under the per-object advisory lock of every binding and the exact-version reference check, then deletes that exact version. read_only bindings are skipped unless include_read_only. dry_run deletes nothing and reports candidates. Deleting requires acknowledge_exclusive_namespace.
  • Budgets and resumability: limit per binding per pass, a page cap, repeated page tokens fail the binding, a failed first page of a resumed cursor drops the cursor, interval_seconds spaces full listings.
  • Opt-in in Python's production maintenance as a store option, next to row_storage and change_feed.

Clone fence (JS #3593)

  • A create-only epoch chain at {prefix}/{namespace_key}/_collector-fence/{epoch}.json, with the database recording (epoch, token) in adcp_reporting_orphan_collector_fences. A real pass claims the namespace before its first delete, re-checks the fence before the first delete of every page and every 25 deletes, and a finished claim always mints a fresh token. Refusals are reported as fenced with a cause. take_over_orphan_collector_fence(acknowledge_previous_owner_gone=True) is the deliberate step after a restore.

Python-specific

  • Python has no versioned migration runner; the fence table goes into the idempotent reporting_row_storage.sql, outside the required-schema manifest so an adopter that never enables the collector is unaffected.
  • Maintenance reporting follows Python's closed integer counters.

Out of scope: #1492 (per-object lock removal), which waits on adcp-client #3281.

Mirrors adcp-client #3291 and #3593. Part of the JS parity set; follows #1498.

Activity

  1. added
    enhancementNew feature or request
    no-triageSkip the Claude triage bot — human or designated agent will handle this issue
    pythonPull requests that update python code
    on Nov 3, 2026
  2. added a commit that references this issue on Nov 3, 2026
    af4f946
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestno-triageSkip the Claude triage bot — human or designated agent will handle this issuepythonPull requests that update python code

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions