feat(document-records): make persistence retries idempotent

docs/DATA_MODEL.md

@@ -19,6 +19,8 @@
 | `candidate_worker_link` | Legacy append-only candidate-to-worker linkage retained for historical reads; new writes use `candidate_worker_conversion_record`. |
 | `candidate_worker_conversion_record` | Governed bitemporal candidate-to-worker conversion bound to the hire decision, person, employment, immutable audit event, and outbox evidence. |
 | `people_mutation_idempotency_record` | Append-only tenant/route/idempotency-key binding to the canonical command digest and first committed created-record identity for governed People writes. |
+| `document_record` | Immutable, tenant-scoped document metadata/evidence snapshot with opaque Person/Employment, artifact, audit, and outbox references; it never stores raw document bytes or free-form HR content. |
+| `document_record_persist_receipt` | Append-only tenant-scoped idempotency receipt binding one persistence key and semantic-command digest to the first committed `document_record` identity, receipt digest, and database-owned recorded time. |
 | `criterion_blueprint` | Job-related performance criterion definition. |
 | `criterion_observation` | Observed criterion result. |
 | `decision_evidence_set` | Versioned evidence-set header whose database-computed digest and membership are sealed by one accountable selection decision. |
@@ -66,6 +68,16 @@ New predictive-validity membership uses `validity_study_case_record` rather than
 
 The owning write port acquires an exact-key transaction-scoped advisory lock and writes the HRIS fact, immutable audit/outbox evidence, and idempotency row inside one PostgreSQL transaction. A rolled-back mutation therefore cannot leave a false replay marker. The relation is append-only, TRUNCATE-protected, tenant-RLS isolated, and uses opaque operational UUIDs. The idempotency key is transport correlation, not HR data or authorization evidence; actor, purpose, human-confirmation and resource authorization remain independently required.
 
+## Document-record persistence and idempotency
+
+`document_record` is the immutable document-metadata system of record for the `document_records` bounded context. It stores only governed metadata and evidence: tenant identity, opaque Person/Employment references, category, uploader/persisting actor references, artifact/source/retention/evidence/application SHA-256 digests, the exact bounded canonical evidence JSON, opaque audit/outbox correlations, business `received_at`, and PostgreSQL-owned `recorded_at`. Raw document bytes, title/free-form HR text, compensation, ratings, credentials, and employment-decision output are excluded from this relation.
+
+`document_record_persist_receipt` is the append-only uncertain-retry authority for that persistence command. Its tenant-qualified business key is `(tenant_record_id, idempotency_key)`. The row binds the opaque idempotency key to the canonical semantic-command digest, the first committed `document_record_id`, the immutable document/audit/outbox references returned by that command, a receipt digest, and the database-owned `recorded_at`. The receipt does not duplicate document bytes, Person/Employment attributes, or other free-form HR content.
+
+First execution writes one `document_record` and one receipt in the same PostgreSQL transaction. A same-key/same-semantic retry returns the original committed identity and original database-owned recorded time without creating another document fact. Reusing the key for a different semantic command fails closed before another document write. Concurrent first attempts serialize on a tenant-qualified transaction-scoped advisory lock. The owner function requires exact tenant context before acquiring that coordination state, runs only under `READ COMMITTED`, canonicalizes digest timestamps in UTC, and is exposed to the application executor as an EXECUTE-only `SECURITY DEFINER` capability; the executor has no direct document/receipt table DML authority.
+
+Both relations are tenant-scoped with forced row-level security and tenant-qualified referential integrity. Receipt UPDATE, DELETE, and TRUNCATE are rejected. This receipt proves creation/retry convergence only; return/destruction completion and recovery-aware deletion remain separate `document_records` lifecycle authority and must not be inferred from a persistence receipt.
+
 ## Audit and outbox normalization
 
 `audit_event_record`, `outbox_delivery_record`, and `outbox_delivery_escalation_record` are deliberately separate relations. The audit relation stores the immutable, PII-minimized canonical CloudEvents representation and its SHA-256 digest. The database allowlists the event shape, verifies event and tenant identifiers, requires accountable human confirmation when `data.high_impact` is true, and recomputes the digest over the exact stored UTF-8 text before accepting the row.

docs/ERD.md

@@ -1,12 +1,14 @@
 # ERD
 
-For readability, the diagram renders representative `tenant_record` scoping edges rather than repeating the same edge for every tenant-owned relation. The authoritative tenant-isolation contract is `docs/DATA_MODEL.md`: **every owned HRIS fact** stores `tenant_record_id`, every cross-table reference is tenant-qualified, and forced row-level security applies independently to every tenant-scoped table. This omission is visual only; it does not weaken the relational or authorization contract for employment, candidate, evidence, decision, validation-link, compensation, transition, audit, outbox, or outbox-escalation entities.
+For readability, the diagram renders representative `tenant_record` scoping edges rather than repeating the same edge for every tenant-owned relation. The authoritative tenant-isolation contract is `docs/DATA_MODEL.md`: **every owned HRIS fact** stores `tenant_record_id`, every cross-table reference is tenant-qualified, and forced row-level security applies independently to every tenant-scoped table. This omission is visual only; it does not weaken the relational or authorization contract for employment, candidate, document, evidence, decision, validation-link, compensation, transition, audit, outbox, or outbox-escalation entities.
 
 ```mermaid
 erDiagram
     tenant_record ||--o{ person_record : scopes
     tenant_record ||--o{ organization_unit : scopes
     tenant_record ||--o{ job_profile : scopes
+    tenant_record ||--o{ document_record : scopes
+    tenant_record ||--o{ document_record_persist_receipt : scopes
     tenant_record ||--o{ audit_event_record : scopes
     tenant_record ||--o{ outbox_delivery_escalation_record : scopes
     person_record ||--o{ person_name_record : has_names
@@ -23,6 +25,7 @@ erDiagram
     position_record ||--o{ assignment_record : assigned_through
     candidate_profile ||--o| candidate_worker_link : may_become
     person_record ||--o{ candidate_worker_link : links_worker
+    document_record ||--o| document_record_persist_receipt : may_have_first_commit_receipt
     job_profile ||--o{ criterion_blueprint : requires
     performance_cycle ||--o{ criterion_observation : schedules
     criterion_blueprint ||--o{ criterion_observation : produces
@@ -53,6 +56,8 @@ Every owned HRIS fact carries `tenant_record_id`. Relationships that cross table
 
 A candidate profile can be linked to at most one worker identity within its tenant. A person identity can have multiple candidate-worker links across reapplications or historical candidate profiles, so the person-side cardinality is one-to-many.
 
+A `document_record` is one immutable document-metadata fact owned by `document_records`. It deliberately keeps Person, Employment, artifact, audit, and outbox correlations as opaque owner-contract references rather than cross-context foreign keys. A persistence command executed through `persist_document_record_once(...)` creates exactly one `document_record_persist_receipt` with its first committed document fact. The receipt has tenant-qualified uniqueness on both `(tenant_record_id, idempotency_key)` and `(tenant_record_id, document_record_id)`, and the latter pair is a foreign key to the document fact, so each receipt identifies exactly one document and each document can have at most one receipt. Migration 0024 does not backfill or require receipts for document rows that already existed before the idempotent write port, so the dataset-level document-to-receipt cardinality is zero-or-one. The receipt stores only replay identity/digests and database-owned time; it does not duplicate document bytes or free-form HR data.
+
 Each criterion observation belongs to one effective-dated performance cycle so reporting periods remain reconstructable across effective and system time.
 
 A high-impact selection decision seals exactly one versioned `decision_evidence_set`. Evidence members are inserted while the set is open; the decision records the set reference and atomically changes that set to sealed. After sealing, neither new evidence members nor a second decision may reuse that evidence set. This prevents post-decision evidence drift while retaining normalized, version-addressable evidence.

docs/OPERABILITY.md

@@ -38,9 +38,25 @@
 - A successful dead-letter transition clears lease metadata, preserves the terminal failure code, moves the queue row to `dead_lettered`, and records tenant-scoped immutable escalation evidence. The escalation row is append-only and cannot be rewritten to make an operational failure disappear.
 - Dead-lettered rows are excluded from normal claiming. Recovery requires an explicit new business/operator action and must not mutate or reopen the terminal historical row.
 - Stable dispatcher worker references remain operational identities. The operator path exists only for the narrower expired-and-exhausted failure mode and does not permit takeover of live or retryable work.
-- Audit/outbox SQL boundaries pin `search_path` to `pg_catalog, public, pg_temp`, the migration revokes `CREATE` on `public` from `PUBLIC`, and project objects remain in the trusted application schema until schema extraction work explicitly moves them. Normal dispatcher/persistence functions remain security-invoker boundaries; the lost-final-worker recovery function is the sole `SECURITY DEFINER` exception and is owned by the hardened NOLOGIN recovery role rather than a login or superuser role.
+- Audit/outbox SQL boundaries pin `search_path` to `pg_catalog, public, pg_temp`, the migration revokes `CREATE` on `public` from `PUBLIC`, and project objects remain in the trusted application schema until schema extraction work explicitly moves them. Ordinary dispatcher/persistence functions remain security-invoker boundaries unless an ADR proves an execute-only capability cannot be preserved that way. The lost-final-worker recovery function and document-record uncertain-retry persistence function are current narrow `SECURITY DEFINER` exceptions; each is owned by its own hardened NOLOGIN/NOBYPASSRLS role rather than a login or superuser role, and its externally assignable executor/operator role has no direct DML on the protected tables.
 - Exponential/backoff policy selection, policy-specific producer configuration, and external delivery receipts remain release blockers before reliable asynchronous delivery is called production-ready; terminal dead-letter/escalation evidence and lost-final-worker recovery are implemented but do not by themselves prove downstream receipt.
 
+### Document-record persistence under uncertain retry
+
+- `persist_document_record_once(...)` is the `document_records` owner boundary for an uncertain-outcome metadata-persistence retry. One transaction binds the immutable `document_record` fact to one append-only `document_record_persist_receipt`; a generic uniqueness violation, elapsed time, or connection loss is never interpreted as success evidence.
+- The callable boundary is execute-only. Migration 0024 rejects pre-existing reserved capability-role names before project mutation, creates `orgmetra_document_persistence_owner` and `orgmetra_document_persistence_executor` as NOLOGIN/NOBYPASSRLS roles, transfers the function to the restricted owner as `SECURITY DEFINER`, removes temporary schema `CREATE` before commit, and grants the executor function EXECUTE but no direct table DML. Production login roles obtain this capability only through explicit purpose-bound membership provisioning. A role-name collision is an operator-review failure, not a reason to reuse unknown role state.
+- The caller must establish `orgmetra.tenant_record_id` and it must exactly match `p_tenant_record_id` before semantic digest calculation, replay lookup, advisory-lock acquisition, or durable write. FORCE RLS remains active for the dedicated function owner and defense in depth rather than permission to acquire another tenant's database-global coordination state.
+- The retry algorithm is intentionally restricted to PostgreSQL `READ COMMITTED`. It relies on a fresh statement snapshot after acquiring the transaction-scoped advisory lock so a waiter can observe the winner's committed receipt. `REPEATABLE READ` and `SERIALIZABLE` calls fail closed rather than pretending that stale-snapshot replay recovery is safe.
+- The transaction-scoped advisory lock covers only replay lookup plus authoritative database writes. OCR, LLM work, object-store transfer, network calls, long-running calculation, and other external work must complete before entering this transaction boundary; no idle external wait is allowed while the lock is held.
+- The semantic command digest is tenant- and purpose-bound and includes the governed persistence inputs. Function-local UTC serialization prevents equivalent `timestamptz` instants from acquiring different replay identities because of caller session timezone.
+- Same tenant/key plus the same semantic digest returns the originally committed document identity, audit/outbox references, semantic digest, receipt digest, and database-owned `recorded_at`. The replay result does not mint fresh authoritative identities or timestamps. Same tenant/key with a different semantic digest fails closed as a conflicting replay.
+- After a transport failure where commit outcome is unknown, recovery resubmits the same purpose-bound idempotency key and the exact same semantic command. A new key is a new command identity and must not be used merely to escape uncertainty. Operators escalate a conflicting replay instead of deleting or rewriting the original receipt.
+- The receipt is append-only, FORCE-RLS protected, and PII-minimized: it carries the opaque idempotency identity, semantic digest, document identity, receipt digest, and original database time rather than document bytes or free-form HR content.
+- Concurrency acceptance must observe the second PostgreSQL backend actually blocked on the first backend's advisory lock and must prove both callers converge to one durable document and one receipt. Fixed sleeps are not serialization evidence.
+- Capability acceptance must prove the owner role has only the table privileges required by the function, the executor has none of those table DML privileges, direct table access as the executor is denied, and function execution still reaches reviewed command validation. A generic role with schema usage only must remain unable to execute the function.
+- Recovery evidence is incomplete if a test client, transaction, or temporary security principal survives the acceptance run. Test-only principals use collision-resistant per-run identities; normal completion verifies strict cleanup, while failure cleanup remains best-effort so it cannot mask the original assertion.
+- This retry receipt proves initial metadata persistence only. Return/destruction completion and recovery-invisibility evidence belong to the separate `document_records` lifecycle authority tracked by #308 and must not be inferred from this receipt.
+
 ### Other dependencies
 
 - Psychometrics Commons unavailable: assessment-result fetches show an unavailable state, not invented scores.

docs/SECURITY.md

@@ -25,6 +25,7 @@
 - A dispatcher lease is an executable capability. Completion and retry require the exact owner of a still-live lease under the active tenant context. Before retry-budget exhaustion, expired ownership must be reclaimed through the guarded claim path; after exhaustion, claim/retry cannot create attempt N+1 and only the exact recorded stable worker reference may use the normal worker terminalization path. If that recorded final worker identity is permanently unavailable, a separately provisioned purpose-bound operator capability may terminalize only an already-expired exhausted lease through the audited recovery function; the operator role has no direct outbox read/write or escalation-insert privilege.
 - Terminal dead-lettering additionally requires an immutable database-owned retry budget, durable exhaustion of that budget, a bounded failure classification, and matching opaque escalation evidence. A dispatcher cannot lower the threshold at finalization, structurally valid direct terminal DML cannot omit the evidence/budget invariant, a foreign worker cannot steal an exhausted row, a dead-lettered row cannot silently re-enter normal dispatch, and its escalation evidence cannot be updated or deleted.
 - Privileged outbox recovery role names are fail-closed deployment identities. Migration 0008 rejects either reserved role name if it already exists instead of inheriting unknown memberships or ACLs; fresh NOLOGIN/NOBYPASSRLS roles are created only after that preflight. The temporary `CREATE` privilege needed for function ownership transfer is granted and revoked inside one transaction, so an interrupted handoff cannot strand schema-creation authority.
+- Document-record uncertain-retry persistence is also an explicit database capability. Migration 0024 rejects pre-existing reserved persistence-role names before changing project objects, owns `persist_document_record_once(...)` with a dedicated NOLOGIN/NOBYPASSRLS `SECURITY DEFINER` role, and grants the externally assignable executor only schema usage plus function `EXECUTE`. The executor receives no direct `SELECT`, `INSERT`, `UPDATE`, `DELETE`, or `TRUNCATE` privilege on `document_record` or `document_record_persist_receipt`, so a service role cannot bypass semantic-digest comparison, advisory-lock serialization, tenant-before-lock validation, or receipt creation with direct DML. The function owner receives schema `CREATE` only for the ownership handoff and loses it before commit.
 - Credentials and passkeys remain in Keyverse or external secret managers.
 - Service database roles cannot query another service's application tables.
 - Client error responses expose a random `support_reference`, never an internal trace/span identifier or encoded infrastructure context.