Cluster record locks: amortized per-record ownership (Phase 1 of #483)

[kriszyp]

Phase 0 (#2462) made the rocksdb-js in-memory key lock the sole authority for table.lock(id), exclusive across one node's worker threads. This branch carries that verb across every node that replicates the database.

The Ricart–Agrawala arbitration rule is gone. table.lock(id) now runs amortized per-record
ownership: an operator-agreed home map supplied by the transport, a rendezvous-hashed home per
key, and a volatile delegation that serves repeat locks from the Phase 0 key lock with zero cluster messages. Releasing the application lock deliberately does not release the delegation — that is the amortization, and the unit suite asserts it directly: 26 locks on one key from one node, spread over 25 seconds of its own clock, cost one round.

	durable commits	frame deliveries	one node down
Ricart–Agrawala, P=12	13	143	every cluster lock blocks
First lock on a cold key	0	2 unicast	only that node's ring share
Repeat lock under a live delegation	0	0	—

docs/record-lock-ownership.md specifies the replacement, and is the substance of this push. Three levels at three very different
rates:

• Home map (rare, operator-agreed, immutable) — per database, (generation, homes[]), published by an administrator and never derived from liveness. It is immutable for the life of its generation: nothing a node observes — an unreachable peer, a restart, a partition — changes it. No consensus runs at any rate. Agreement is that every node holds the same generation, checked by digest before the feature is enabled.
• Home node (derived, free) — within a generation the arbiter for a key is a rendezvous hash over homes[]. One arbiter per key is trivially exclusive, which deletes the entire grant state
  machine: no deferral queues, no (tsR, nodeName) tiebreak, no synthesized grants, no split votes,
  no revocation protocol.
• Delegation (volatile, the amortization) — a node asks a key's home for the exclusive right to admit critical sections on that key for a bounded time. With a live delegation lock()/unlock() are pure Phase 0: the local key lock, zero cluster messages. Releasing the application lock does not release the delegation, so a node writing the same record repeatedly pays one round and then nothing, and the delegate is in practice the last writer.

Steady-state cost goes from P+1 durable commits and P²−1 deliveries per lock to zero of each; a first lock on a cold key is two unicast messages and one RTT to the home; a handoff is one durable release plus P−1 deliveries. A node that is down blocks only its own share of the ring, until an operator publishes a new generation — that price is the design decision, not a caveat, and §4.2 of the note states it in those words.

This is close to what harper-pro#438 filed as its own "Phase 1 — single-owner delegation", and to what #483 describes in prose ("if an exclusive lock is held by one other node, the lock can be requested from that node"). The home map is deliberately its own recordLockHomes generation rather than server.shards: a shard map is a residency directive, so reusing it would couple arbitration to data placement and require sharding configuration, which one customer uses.

Product and architecture tour

What a cluster lock() promises now

Is exclusion-only the right guarantee to ship, given what it leaves for the caller to know?

Before	After
Ricart–Agrawala: every participant must grant, so one unreachable peer blocks every cluster lock in the database, and each acquisition is P+1 durable commits and up to P²−1 frame deliveries — 13 and 143 at P=12.	One arbiter per key. An unreachable node blocks only the keys it homes. A first lock on a cold key is one unicast round trip; a repeat lock under a live delegation is the Phase 0 key lock and zero cluster messages.

lock() guarantees that at most one node admits a critical section for a key at a time, and that a node admitted after another released cleanly has applied that node's committed writes — while the key's home still holds that handoff's dependency set.
It adds no fencing generation to conflict resolution and does not confirm locked writes to a quorum: two conflicting writes resolve exactly as they would without a lock, so a predecessor's write can still outrank its successor's, and successor freshness is not promised once the dependency set is gone. Neither limitation is crash-only; both are reachable on a clean handoff.
There is no caller-side mitigation. X-Replicate-To / confirm= is super-user-gated and a residency directive first, and it closes none of the freshness routes anyway. The normative wording is §10 of docs/record-lock-ownership.md; the API docs may not soften it (#2547).

The alternatives — fenced (generation, timestamp, origin) conflict ordering, and quorum-confirmed locked writes — are deferred to #2540 with their costs recorded. Fenced mode would change conflict resolution for every write in the database, including in deployments that never call lock(), and invert a rule Phase 0 documented and shipped, in exchange for one crash-shaped case. Both limitations ship silent: a caller whose locked write loses gets a 200, no log line and no counter. That is recorded in §10 as a choice, with the cheap lock-path detection routed to #2541.

Three levels at three rates

Where does agreement come from if no consensus runs, and what does harper-pro still owe?

Where each level lives

The only agreed state is published by an operator; everything per key is derived or volatile.

flowchart TB
    subgraph harperpro["harper-pro (owed: harper-pro#825)"]
        E["Home map<br>(generation, homes, homeIncarnation)<br>operator-published, immutable per generation"]
    end
    subgraph core["core (this PR)"]
        H["Home node per key<br>rendezvous hash over homes"]
        D["Delegation<br>exclusive right to admit K, bounded time"]
        K["Phase 0 key lock<br>rocksdb-js, process-wide"]
    end
    E -->|"transport.homeMap()"| H
    H -->|"one grant per key"| D
    D -->|"repeat lock: no message"| K

Loading

A cold lock, a repeat lock, and a handoff

1. Cold key — Node B computes homeFor(K, members), sends one unicast requestDelegation to that home, and installs the grant with its deadline anchored at the moment it sent the request.
2. Repeat lock — B holds a live delegation with room for the lease, so lock() takes the local key lock and sends nothing. Releasing the application lock does not release the delegation.
3. Contention — Node C asks the same home. The home records a recall against B, replies contended, and C retries inside its own timeout.
4. Drain — B stops admitting, waits for live admissions, then revokes every handle it admitted — including ones already unlocked whose writes are still staged — and writes a lockRelease entry carrying the full fencing token.
5. Re-grant — the release, ordered behind B's own data writes on B's replication stream, clears the grant; the home now grants C.

homeFor is a rendezvous hash, chosen over a modulo because a membership change then re-homes only the departing member's keys — every re-homed key pays the recovery path on its next lock. Delegations run for DELEGATION_LEASE_MS (six minutes), deliberately longer than any lock lease they admit: a delegation sized to one lock has no room for the next, so every repeat lock would renew and pay a round trip. The first implementation had exactly that defect and its amortization test passed only because the fake clock never advanced inside the loop.

The exclusion argument

Which rules stop two nodes from admitting the same key, and where does core rely on the operator instead of enforcing?

One delegate per key per home. A home never holds two live grants for a key, and re-grants only after the previous delegation was recalled-and-drained or provably expired on the home's own clock plus LOCK_LEASE_SKEW_MS.
The home outwaits the delegate. Both sides measure on their own monotonic clock; no remote timestamp is ever compared against a local one. A delegate may drop a delegation early; a home may never forget a grant before its expiry.
Fencing tokens are ordered. (generation, homeIncarnation, counter), compared lexicographically. A release carries the whole token and is matched on all three, because counters restart when a home restarts.
A delegation is authority within one generation. A delegate drops a delegation whose token belongs to a superseded generation on next use, and a node refuses a map whose generation went backwards — a rollback re-mints tokens ordering below ones already issued.

Example: a delayed grant reply

B asks the home for K with a 100 ms lease. The home grants and starts its clock. The reply is delayed six seconds; meanwhile the home's grant expires and it grants K to C.

Outcome: B's deadline was anchored at send, so the reply is discarded as having outlived its own delegation and B asks again. Anchored at arrival — the first implementation — B would have installed a fresh delegation alongside C's.

Two hazards are handled by different mechanisms because they are different. An in-process transport swap (a component reload re-registering a transport) is handled by adoption: the successor coordinator takes the predecessor's delegations and grants in its constructor before the predecessor is closed, so nothing is dropped and no wait is paid. A process restart cannot be: core has no record of what its previous incarnation issued. Under the epoch design that interval was the transport's to hold — do not name this node in an epoch until a previous incarnation's delegations could have expired, or advance the epoch number — and neither half survives an immutable map, because a generation does not advance merely because a process restarted. So core enforces it: #grantableAfterMono now refuses to grant as a home until DELEGATION_LEASE_MS + skew on the monotonic clock, which counts from process start, so every lazily created table coordinator computes the same horizon. The earlier objection to doing this in core — it made the real-table suite hang for six minutes — is answered by ClusterLockTransport.grantableAfterMono, the explicit override for a caller that can prove a previous incarnation issued nothing.

Recall revokes capability, not admission

Is fencing at commit time the right place to stop a recalled delegate, rather than at unlock()?

Example: a write staged, then unlocked, then recalled

Inside an explicit transaction, B locks K, stages a write through the handle, and calls unlock(). The admission count is now zero, but the write is still uncommitted in B's transaction. A recall arrives; the home re-grants K to C.

Outcome: the delegation retained a revoker for B's handle and calls it on surrender. handle.revokeLease() expires the handle, and the commit-time fence in DatabaseTransaction rejects the staged write with 409 immediately before the native commit submits. A drain that waited only on the admission count — the first implementation — would have let B's write land after C was admitted.

The commit-time fence is the mechanism §6 depends on, and it now runs only when the transaction actually holds a lease-protected write (hasLeaseProtectedWrite, reset by clearWrites()), so a bulk transaction of plain writes in a core-only deployment pays nothing for it. Admissions carry their delegation's token, and release() and registerAdmission() take it back: a key's delegation can be replaced while a handle is open, and an untokened late unlock() would otherwise decrement the successor's count and let it be surrendered while its own callers were still inside.

Boundaries and non-goals

What is deliberately not here, and why can this not be enabled even on purpose?

Before	After
LOCK_REQUEST = 9, LOCK_GRANT = 10, LOCK_RELEASE = 12; payload [key, requester, tsR].	Only LOCK_RELEASE = 12; payload [key, requester, generation, homeIncarnation, counter], validated on exact tuple length. Nibbles 9 and 10 are retired, not migrated — main has since taken 9 for eviction, which the rebase surfaced as a duplicate-key type error.

Not implemented, deliberately: the successor-freshness fence of the design note's §7 — the inherited (origin → position) dependency set on the release and the recovery barrier — is #2542, and harper-pro's operator-agreed home map is harper-pro#825. Until both land a handoff carries exclusion but not the clean-handoff freshness §2 states. The feature cannot be enabled meanwhile: core fails closed without an agreed map, and no core build supplies one. Every fail-closed path — no map, a rolled-back generation, no named homes, an unreachable home, a closed coordinator, a non-coordinating thread — rejects with a retryable 503 rather than downgrading to a node-local lock. harper-pro#822 implements the previous transport interface (participants, ownsCoordination) and pins core to a pre-rebase commit; against this head its transport does not register, so it needs the interface, the home map and the delegation server before the two can be integration-tested together.

• DELEGATION_LEASE_MS is six minutes (MAX_LOCK_LEASE_MS + 60 s). That is the longest a key stays pinned to a crashed delegate before its home re-grants. Is that the right availability-versus-amortization point, or should it be configurable per table?
• The restart quarantine is now on by default in core (six minutes on this node's own share of the ring), because an immutable generation leaves nothing external to hang it on. Is that the right default, or should harper-pro be expected to set grantableAfterMono on every ordinary start and leave core's bound as the failsafe?
• §4.3 requires an operator to stop a node before publishing a generation that removes it — declaring it removed is not a fence. Is that acceptable as a runbook requirement, or does it need the leased generation capability §9 records and overrules?
• Exclusion-only ships two silent limitations. Is a 200 with no log or counter acceptable for a lost locked write, or should the lock-path detection routed to Record locks Phase 1: rendezvous home ring, per-record delegations, and drain/recall #2541 land before enablement?
• A single arbiter per key deletes the contention protocol, but an unreachable home blocks its share of the ring until an operator publishes a new generation — an outage bounded by response time, not by a protocol. Is that preferable to the per-key quorum voting the note rejected?

• Fencing token shape — ordered on (generation, incarnation, counter); the reason counters alone cannot identify a delegation
• Transport contract, and the one obligation core cannot check — the operator-sequenced generation change, and the grantableAfterMono override that is the only way past core's own restart quarantine
• The cold-start quarantine, anchored on when coordination started in THIS thread — performance.now() is process-wide inside a worker, so a process-anchored horizon lets a replacement coordinating worker grant over delegations the worker it replaced issued
• The generation monotonicity floor — why a rolled-back generation is refused rather than assumed away, and why it is keyed per table
• Rendezvous home — why a generation change re-homes only the departing node's keys
• Adoption across a transport swap, and the retirement record a replacement inherits — live authority moves rather than being discarded, and a coordinator rebuilt after a close starts from the bound and counter its predecessor had reached
• acquire: the amortized path, the send-anchored request, and the generation re-read after the await — zero messages under a live delegation; a dead reply is discarded; a grant minted under a generation that has since advanced is handed back rather than installed
• Membership is checked before any state is allocated — a node the current generation does not name gets no delegation, so an authenticated identity that outlived its membership cannot recall the legitimate delegate to take the key
• #installDelegation: renew in place, never move a delegation backwards — a renewal keeps the admissions the node is still answerable for; a losing reply cannot displace a newer token
• The late-reply handback runs on the authority, not on the coordinator that sent the request — look hardest here: handOffTo empties the sender's delegation map, so the existence guard below reads empty after a swap unless the hop is made
• #releaseUnclaimedGrant: any live delegation blocks the handback — the guard is the delegation's existence, not a token match — the home renews in place, so it can hold a newer token than the one this node installed
• Surrender revokes every admitted handle — capability, not just admission
• revokeLease on the handle — what the revoker calls, and why an already-fired handle cannot join a round
• Commit-time fence, gated on a lease-protected write — the mechanism §6 depends on, at no cost to plain transactions
• Two resolvers, because applying a release needs no transport — control entries reach the coordinator that holds the grant; delegation requests and recalls keep the transport-gated getter
• The replicated-event sink applies releases through admittingCoordinator — this sink runs off the replication stream, not off the unicast transport, so a reconnect must not swallow a clean handoff
• Tokened release and admission registration in Table.lock() — the handle is bound to the delegation that admitted it
• The one surviving control entry — nibbles 9/10 retired; 9 now belongs to eviction
• Independent per-node clocks in the unit harness — a shared fake clock would pass a coordinator that compared remote and local readings
• Regression: a handback that spans a transport swap — stalls a request, acquires under the retry, swaps the transport, then delivers the stalled reply and asserts the home never lets go of the grant
• Regression: a transport-pushed release during a reconnect — grants to a peer, unregisters the transport without a standalone claim, and asserts the release still clears the grant

For the human reviewer

• This push is test-only, and it is two findings wide. The single open review thread asked for the regression the follower-reports-leader-failure commit had said it did not have, and the pre-push round that reviewed that test found the previous commit's failing-writer test did not actually test containment. No production code changed.

  finding	ruling
  @claude, Table.ts:2880 — the follower-reports-leader-failure contract has no regression test	accepted, fixed — the follower now reports the leader's 503. The suggested shape needed one extra constraint: leaderFailure is read only on the remaining <= 0 line, and the follower's own deadline timer is armed for exactly that budget. The leader's rejection travels on microtasks and the deadline is a timer, so holding the event loop past a 1 ms budget settles the race on the leader's error with remaining already negative
  pre-push round 36, graded leg — 77ac2ddc5's failing-writer test passes whether or not the throw is contained	accepted, fixed — #surrender deletes the delegation before it calls the writer and the contender's rejection is caught, so both original assertions hold either way; removing the try from #writeControlSafely and rebuilding dist left it green. The home can tell: a recall that resolved is confirmed and never re-sent, a rejected one is re-sent past RECALL_RETRY_MS. 2 !== 1 without the containment
  Gemini, Table.ts:2855-2866 — implicit undefined return drops exclusion for a coalesced follower whose handle expired	refuted at the anchor — the if block is followed by return retryOnRemainingBudget(); at Table.ts:2871. There is no fall-through
  Gemini, recordLock.ts:226 — release() sets expired, so an early unlock aborts the commit	refuted, again — recordLock.ts:324-326 sets released and clears the timer; only revokeLease and #onLeaseExpire touch expired
  Gemini, Table.ts:2933 noop and Table.ts:972 logger are possibly undefined	refuted — noop is a hoisted function declaration at Table.ts:8075, logger is imported at Table.ts:83
  Gemini + graded leg, Table.ts:5851 — table/database retirement can orphan live coordinator authority	real, deferred on reachability — filed as #2591 (P2, epic Add ability to exclusively lock a record for performing a safe operation #483). The earlier "pre-existing, with no line in this delta" reading was wrong: recordLockCoordinator.ts does not exist on main and main's Table.ts contains no lockCoordinator. What holds instead is reachability — lockCoordinator is only constructed behind a registered transport (Table.ts:5857-5866) and registerClusterLockTransport has no non-test caller in core, so nothing can be orphaned in any current build. The gap is that the drop path never calls close(), so the retiredCoordinators counter floor is never written and a same-name re-create restarts #counter below tokens peers still hold. It is testable only once harper-pro registers a transport, which is where the fix belongs

• This push is the PR-review adjudication round: five open threads from @cb1kenobi, four fixed, one refuted. Four of them had already landed in earlier commits of this session; the fifth was still live and is fixed here, and following it turned up three more in the same surface.

  thread	ruling
  commit-time fence dereferences a detached write	declined, refuted at the anchor — DatabaseTransaction.ts:1399 filters the holes out of this.writes on the line above the loop, inside the same if (transaction) block, and that filter is pre-existing on main (8d69f1bd1)
  delegation gate rejects any requester homes[] does not list	fixed in 009cde857, by the second of the two routes offered — the contract now says homes[] names every locking node. §9's Adjacent row still read the other way and is corrected here
  generation denials spin to a 423	fixed in 009cde857 — generation and unknown-node both throw 503, and the terminal 423 is gated on contended alone
  a renewal does not re-arm the recall latch	fixed in f20d98457, by the opposite move: a renewal is refused while the latch is set rather than clearing it, so the release already on the wire for token n stays the thing that completes the handoff
  an unrouted recall resolves as success	fixed here — still live at 71d32bf6d, whose "stop swallowing the resolver error" was a different site

• A recall that did not apply is not a confirmation, in three places. deliverDelegationRecall is core's receiving end of recallDelegation, whose contract is "resolves once the delegate has drained and stopped admitting", and #beginRecall latches recallConfirmed on any resolution and never re-sends. It returned normally when the transport-gated resolver yielded nothing — the reconnect window Table.lockCoordinator deliberately keeps the coordinator alive for — so the home denied the key to every other node for DELEGATION_LEASE_MS + skew. It rejects now. The same shape was in #beginRecall's own local branch, which assigned grant.recalling and never cleared it on a rejection; and transport-gating turned out not to be enough, because acquire refuses off the owner thread, so a delegation only ever lives on the coordinating one and a recall routed anywhere else found nothing and resolved — onDelegationRecall is ownership-gated now, like onDelegationRequest.

• The takeover quarantine is anchored on homeIncarnation now, not on a sampled boolean — and this is the part worth your eye. Three pre-push rounds walked it down, each finding a defect in the previous round's fix, which is the honest reason to look at it rather than take the last one on trust:

  1. #ownershipHorizon was read only from #grant, and onDelegationRequest answers not-home before reaching it while this thread does not own coordination — so a non-owning interval was never observed at all, and ownership moving A→B→A left A's anchor dated from before the gap.
  2. Polling ownership on the tick fixed that but not the waiver: clearing #quarantineWaived on an observed gap never fires on a coordinator that never owned, and Table.lockCoordinator constructs one on every non-owning worker. It also kept every once-owned table in the 100 ms tick set for the life of the process.
  3. A sampled boolean cannot prove continuity at all. §5.1 already advances homeIncarnation once per coordination incarnation — that is what keeps the fencing token orderable — so a value this coordinator has not granted under is the transport stating that something else coordinated for this node, whatever the boolean said in between. ownsCoordination() stays as a weaker second signal — it catches a thread that stopped coordinating without anything else starting — so the tick lifecycle went back to what it was.

  The waiver now ends wherever ownership begins after construction, which is the one thing grantableAfterMono's attestation ("no previous incarnation of this process issued anything") cannot cover, and handOffTo carries it rather than letting a transport reload re-latch it off the new transport. handOffTo also stopped copying blanks over the successor's own reading: a predecessor with no ownership instant and no incarnation re-armed a quarantine on a node that had never stopped coordinating, and it cascaded through every later reload on the thread.

• getNodeNameForId(…, rebuildOnMiss) reads the mapping once per window now, not once per entry. Raised in two rounds by two models, and the refutation recorded in an earlier revision of this description — "a burst costs one read, not one per entry" — holds only for ids the translation path actually minted, which it did not say. For an id this database will never resolve (a node whose mapping was purged, an origin relayed from elsewhere) the bypass cost an exportIdMapping read and unpack each, on the replicated apply thread, which is the burst the interval exists to keep off it. The bypass is kept and bounded: a rebuild inside the window is not stale, because writing the mapping calls invalidateNodeNames and drops the cache entry outright.

• Declined this round, with reasons, so they are not re-filed.

  • A failed leader acquisition wakes every coalesced follower into its own retry — O(N) failing RPCs for one outage. Real, and deliberate: each waiter carries its own remaining budget, and propagating the leader's terminal error would hand a follower with a longer timeout a failure it might have waited past. Changing the coalescing contract under bot feedback is not a call to make in an adjudication round; it belongs to whoever settles the 423/503 boundary above.
  • The local-recall regression stubs onDelegationRecall instead of inducing a failure through it. The subject under test is #beginRecall's bookkeeping when a recall fails; the mechanism of the failure is not part of it, and the alternatives available (a throwing keyIdOf) would test a different thing while reading as if they tested this one.
  • Comment density. Trimmed where it narrated review history — the two verify: markers in §7.2 were an agent workflow instruction shipped in a committed document and are now stated as open questions against harper-pro, and the duplicated block above admittingCoordinator belonged on lockCoordinator. The rest is the orientation this repo keeps next to the code, declined for the fourth round running.
  • Refuted again at their anchors, each previously recorded: release() sets released, never expired (recordLock.ts:324-337); noop is hoisted at Table.ts:8075; writeLockControlEntry's synchronous encode cannot escape, because its only caller reaches it through #writeControlSafely's try/catch; stageWrite is not a synchronous function leaking a promise — all three callers collect its result into writePromises; decodeLockControlPayload has its own try/catch around both the unpack and decodeTuple; and Table.ts:2831's timer calls reject, it does not throw inside setTimeout.

• The staged transition §4.3 now specifies — stage, acknowledge-or-fence, drain, activate, with every acknowledgement bound to the acknowledger's homeIncarnation and one durable activation record. That section is where a reviewer who wants to disagree with this design should start.

• A blocking review finding on the key predicate is fixed, and its reachable set is wider than the finding said. isEncodableKey had drifted back to being tighter than ordered-binary. Too strict is not a safe direction here: the predicate gates onDelegationRequest on a key's home, so a shape it refuses gets not-home — which acquire retries every 25 ms for the caller's whole timeout and then converts to a 423 on a key nobody holds. The asymmetry is what hides it: #grantLocally never goes through onDelegationRequest, so the same table locks fine on its self-homed keys and fails only on the peer's share of the ring. decodeTuple uses the same predicate, so any lockRelease such a key does produce is dropped at every receiver.

  f2ef9375 on this branch set the rule — match the encoder — and the 50b966d1 rewrite lost it, restoring bigint but not binary, boolean or null, and adding a Number.isFinite that refuses two more shapes the encoder takes. The reviewer named three shapes; the reachable set is not the same three. checkValidId (resources/Table.ts:6694) rejects a scalar boolean, null and NaN before lock() can reach the coordinator at all — but it passes all three inside a composite key, and Id itself declares (number | string | null)[]. It also passes Infinity, which the reviewer did not name and Number.isFinite refused. So the predicate now states the one rule that covers the set without enumerating call sites, and the restored agreement test is what keeps the next rewrite from drifting off it a third time.

  Two regressions, each failing on the parent commit after a dist rebuild: the fake-cluster acquire grants a Uint8Array key homed on a peer instead of throwing "Record is locked and was not released in time", and the real-table test fails at "the home granted a Bytes primary key". The second one exists because the coordinator suite's keyIdOf is String(key) and never touches ordered-binary; it is the first test on the branch that pins binary-key identity across the wire against the production encoder.

• One pre-push blocker this round was refuted, and it is the third time the same claim has come back. The claim: handle.release() sets expired = true, so lock(); save(); unlock(); commit() inside the lease hits the pre-submit fence and aborts. release() sets released and clears the timer; only #onLeaseExpire() and revokeLease() set expired, and isLeaseExpired() reads expired || #leaseLapsed(). The distinction is the point of a1a79a7c, and a staged write survives unlock() even when a replay re-saves it in recordLock.test.js executes exactly that sequence and asserts the write lands. Recorded here rather than only in the review log because the same finding was dropped as factually wrong in two earlier rounds.

• And one from the same round that was factually wrong at its anchor. The claim was that a Uint8Array record id breaks the delegation and grant Maps, because msgpackr hands the receiver a new object and a Map compares objects by identity. keyIdOf is writeKeyId (resources/DatabaseTransaction.ts:513), which returns toBufferKey(key).toString('latin1') — a string, so every shape is compared by value. The finding's own premise ("the tighter predicate restricted keys to primitives") never described the old code either, since arrays were already accepted and arrays are objects. The real-table test above asserts writeKeyId(decoded.key) === writeKeyId(sent key) across the Uint8Array→Buffer change and that the delivered release clears the grant, so this is now settled by execution rather than by reading.

• Two review findings were declared "addressed at this head" and were not, so this push fixes them. The reviewer that filed them at 729aefd2 cleared every prior blocking thread at 14c1b551; git show 729aefd2:resources/Table.ts and :resources/recordLockCoordinator.ts show scopeViolation and #pruneAdmissions byte-identical to 14c1b551, and neither commit in between touches either function. Both original claims hold on the code:

  1. scopeViolation gated its fall-through on transport presence alone, while
     lock()'s entry guard fails closed on scopeRequested || isClusterLockRequired. Those two
     predicates disagree across an await: a cluster-scoped caller that coalesces onto an in-flight node-scoped acquisition re-checks its scope only after that wait, and a transport unregistered during it — a swap, which leaves isClusterLockRequired true — made the check answer "no violation" and handed back the leader's node-scoped handle for an explicitly cluster-scoped request. Both guards now read the same predicate, so only the implicit Phase 0 case falls through. The answer is the same 409 the identical call sequence gets without the race, not a 503, because the transaction really does hold a node-scoped lock on the record.
  2. #pruneAdmissions returned at the first admission still inside its lease. That is exact only when every admission shares one lease length; a longer-lease admission at the head hid every shorter one behind it, and — contrary to what its own comment claimed — no later call collected them either, so retention grew at the lock rate rather than with the live set. The leading-run fast path stays; an expiry sweep now runs once the map has outgrown the live set the previous sweep measured. The tradeoff is explicit and worth your eye: that sweep is O(size) synchronous work on one acquisition, in exchange for amortized O(1) per admission and a map bounded at twice the live set instead of unbounded. The sweep can only drop an admission whose own expiresMono has passed, which is the §6 property — asserted directly by the second regression. stats.revocable makes the retention observable, which the finding noted it was not.

  Only the retention half of finding 2 is fixed. The per-lock() allocation half stays deferred below, unchanged: bounding what admission records retain does not make a cached hit allocation-free, and §8 now says both things rather than only the aspiration.

• One finding from the pre-push round on this push is fixed and is worth naming, because it is a rule this branch wrote down and then broke one surface of: warnClusterReleaseFailure latched for the life of the process, while warnOnce two files over states the rule and the reason — a latch shows an operator the first occurrence and hides a fault that persists. A transaction log that stops accepting control entries would have warned once and then gone quiet while every peer fell back to waiting out leases. The reviewer's follow-on suggestion to export warnOnce and reuse it is declined with a concrete disqualifier: recordLockCoordinator.ts computes DELEGATION_LEASE_MS from recordLock.ts's MAX_LOCK_LEASE_MS at module scope, so importing back the other way puts that read in the temporal dead zone.

• Three findings from that round were checked and refuted, recorded here so they are not re-filed: a claimed blocker that writeLockControlEntry never commits its audit write (refuted by recordLockCluster.test.js, which waits for a lockRelease entry to appear in the real audit store and passes); a claim that the coordinator suite shares one fake clock (its header and FakeCluster.advance(name, ms) do the opposite — one node's clock moves at a time, which is why §12 asked for it); and a claim that getNodeNameForId(…, rebuildOnMiss) lets a peer force a store read per entry, which needs a node id that is permanently absent from the mapping — the first rebuild resolves any id the translation path actually minted, so a burst costs one read, not one per entry. A permanently-unresolvable id would be a replication-layer invariant break, and every such entry is already discarded and logged.

• The guarantee decision is made (2026-09-09): exclusion-only. lock() guarantees that at most one node admits a critical section for a key at a time, and that a node admitted after another node released cleanly has already applied that node's committed writes — but only while the key's home still holds that handoff's dependency set. It adds no fencing generation to conflict resolution and does not confirm locked writes to a quorum, so two things are part of the documented contract. Neither is crash-only, and both are reachable on a completely clean handoff:

  1. a predecessor's write can outrank its successor's under LWW, because LWW compares the timestamp assigned when the write was staged and lease expiry orders admissions, not timestamps. lock() changes nothing about conflict resolution — whatever two conflicting writes would do to the record without a lock is what they do with one, silently. (1a) the predecessor's clock ran ahead and its write is still in flight; (1b) it stamped a future context.timestamp — deliberate Phase 0 behavior — then committed, replicated, drained and released cleanly, and the successor's write is the one that loses. No crash, no skew, nothing in flight;
  2. successor freshness is not promised once the dependency set is gone, because the recovery barrier only drains streams from reachable members. Three routes: the predecessor crashed, it is unreachable, or its native commit settled after the barrier was measured. Two damaged effects on the one record — the predecessor's transaction not reflected, and the successor's write computed from the stale value it read.

  There is no caller-side mitigation for (2) at all. X-Replicate-To / confirm= is super-user-gated (checkContextPermissions, resources/Table.ts:7258 — a 403 for an app caller), and for a super-user it is a residency directive first: a numeric value truncates the record's residency (getResidency, :1574) and * falls back to the database's configured replication.replicateTo. Either way a successor's barrier can satisfy over members that never held the locked write. Even with cluster-wide residency, confirmation closes none of (2)'s three routes — the late-settling commit needs fencing, so §2's invariant as written needs both deferred arms. Both limitations also ship silent — a 200, no log line, no counter — which §10 records as a deliberate choice with the cheap lock-path-only detection routed to Record locks Phase 1: rendezvous home ring, per-record delegations, and drain/recall #2541.

  The normative text is §10 of the note, which DESIGN.md points at rather than restating.

  Ten pre-push rounds ran on this push, and the first nine each found the contract text promising something the code does not. In order: crash-only; the late-settling commit filed under the wrong limitation; clean-handoff safety; the LWW mechanism sentence stated backwards; the drop-vs-fold rule splitting on the wrong operand; a whole-write loser guarantee false for a patch on disjoint fields; a per-field one false for CRDT ops; a recommendation to use X-Replicate-To: N;confirm=M that would have narrowed the record's residency and made the limitation more reachable; that the same header is super-user-gated, so it was never a caller-side mitigation in the first place; and — round 10, in code rather than in the note — that the super-user gate itself is bypassable. It converged only when the contract stopped restating engine conflict-resolution and said the thing that is actually true — lock() does not change how conflicting writes resolve, and there is no lever a caller can pull. That history is the argument for §12's schedules asserting the limitations as executable expectations rather than leaving them in
  prose: a paragraph about a conflict-resolution edge does not stay true on its own, and this one is
  going into user-facing API documentation.

  The choice also removes an exception §8 was carrying: with no fencing generation, ordinary writes keep their ungated path with nothing added, which is the property the fenced arm would have given up.

• One earlier claim in this PR's body is withdrawn. It said a crashed holder's in-flight write is stamped older than any successor's and so resolves under LWW exactly as if the release had not overtaken it. That holds for a clean release. It does not hold for a crash: if the crashed holder's clock runs ahead, its delayed write can carry the greater timestamp and overwrite the successor. Monotonic lease expiry orders admissions, not timestamps — §7.3 is that hole, and the table above is the choice about it.

• The freshness property this branch gets for free is the one the new transport must pay for. A grant here rides the grantor's own replication stream behind the grantor's data writes, so applying a grant implies having applied that grantor's earlier writes to the key. Unicast delegation messages lose that, and a scalar record version does not restore it — core breaks equal-version conflicts by node name, so a replica can hold a losing value at the same timestamp and pass a version ≥ V test. §7.1 replaces it with an inherited (origin → position) dependency set on the LOCK_RELEASE entry.

• The work is decomposed and the remaining blocker is measurement. Record locks Phase 1: rendezvous home ring, per-record delegations, and drain/recall #2541 (home ring, delegations, drain and caps, plus the three inherited substrate defects below), Record locks Phase 1: successor freshness — inherited dependency sets and the recovery barrier #2542 (successor freshness), Record locks Phase 1: durable membership epoch protocol (single-decree agreement per database) harper-pro#825 (the membership epoch protocol), and Record locks: measure the Phase 1 cost baseline before the protocol change harper-pro#824 (the measurement gate, the only piece unblocked today).

• The measurement gate comes before the protocol change, not after it. No number in the note is a benchmark; they are message counts. Acquisition latency on a real cluster, audit growth per lock, and throughput with the feature disabled are the first deliverable, and they also set the baseline the new design has to beat.

• The pre-push review ran fifteen rounds on the code that is here. Roughly half the defects it found were introduced by an earlier round's own fix, which is a property of this state machine rather than of the review — and it is the strongest single argument for a design whose arbiter is one node instead of a quorum. That history is preserved in the commit log; the note's §9 records why quorum voting, a replicated Raft coordinator, and a server.shards-based owner map were each considered and not chosen.

• Three defects the cross-model review found in the substrate, not in the rule being deleted. They are recorded in §11 of the note as obligations on the replacement, and are not fixed in this push — the branch does not merge with main yet and two of the three are touched by the replacement anyway. Named so they are not lost: re-registering a transport closes the current coordinator and installs an empty one without invalidating the handles the old one issued, so a successor can grant the same key with no lease time elapsed (resources/Table.ts:5474); a LockUnavailableError from coordinator construction escapes the direct receive callback, which the subscription sink already contains (resources/recordLockCoordinator.ts:908); and the commit-time lease fence scans the whole write set on every commit, including in core-only deployments that never register a transport, which contradicts the ungated-path goal (resources/DatabaseTransaction.ts:1213). The review's other four majors are inside the Ricart–Agrawala state machine and are deliberately left alone.

• Two substrate decisions from the original design still stand and are still worth a check. Control entries carry recordId: null and the locked key in the payload, because a control entry carrying the key answers _writeUpdate's keyed dedup lookup at exactly ts_R and silently drops the holder's own first write. And node identity is the node name, never the audit nodeId: short ids are per-node, so a (ts, nodeId) order would order the same pair differently on two nodes and both would grant.

• THE DESIGN CHANGED IN THIS PUSH, and this is the first thing to read. Round 7's planning gate returned Framing-Verdict: better-alternative-exists against the membership-epoch protocol, and on 2026-09-13 the repo owner adopted it. The epoch protocol is deleted, not deferred: §4 is now an immutable, operator-agreed home map — a recordLockHomes generation published by an administrator, agreed across peers by digest before the feature is enabled, and rendezvous-hashed over exactly as before.

  The argument in one sentence: the epoch protocol automated a decision that has a human authority. Whether an unreachable node is briefly down or permanently gone is knowledge an operator holds, and inferring it safely from timeouts costs a durable consensus subsystem — persisted promises and accepted values, ballots, renewal leases as the liveness signal, acceptor-side retirement reservations, transitive activation protection, and a restart quarantine to rebuild reservations a crash lost. All of that is gone. What it bought was unattended rehoming, and the price of not having it is stated plainly in §4.2 rather than hidden: an unavailable home's keys stay unavailable until an operator acts.

  	before	now
  membership change	single-decree agreement over a majority	operator publishes a generation; fail-closed during the change
  harper-pro owes	harper-pro#825: the epoch protocol, acceptor promises persisted before ack, retirement reservations, transitive activation	a config generation, a peer digest check, and the §4.3 change runbook
  core carries	epoch numbers, ringVersion, rehoming, a recovery barrier for re-homed keys	a generation, a monotonicity floor, and its own restart quarantine
  a dead node's keys	recover when the epoch advances on its own	recover when an operator publishes a new generation

  Cold acquisition is still two unicast messages and one RTT; a repeat lock under a live delegation is still zero cluster messages; an unreachable home still blocks only the keys it homes. §§5–8 are untouched.

• The round-8 planning review found the hole the recut left, and §4.3 is the fix — look here hardest. Its blocker: publishing g+1 does not revoke a live g. A node that never receives the new generation keeps serving the old one from a copy that is internally consistent, and a one-time digest check cannot detect staleness. A and B hold g={A,B}; A partitions; the operator declares A removed and publishes g+1={B}; B drains and re-homes A's keys; A — still running — grants one of them to whoever it can still reach. Two holders under one key, and a drain measured from publication does not bound it because A never stopped.

  §4.3 is now stage → acknowledge-or-externally-fence → drain → activate, and it says the thing the first draft left implicit: declaring a node removed is not a fence; stopping it is. That is a runbook requirement on harper-pro, and it is the step the whole safety argument rests on — it is the most reasonable place to disagree with this design, so it is called out rather than buried.

  Two more of that round's findings are adopted in code: a generation below one already acted on is refused (a rollback arrives by config restore or partial publish, and re-minting under an older generation issues fencing tokens ordering below ones already handed out), keyed per (database, table) because that is the granularity tokens are actually compared at; and the routing hash is versioned by the protocol capability rather than by the generation, because a generation is operator-published and says nothing about which hash a binary implements — two binaries could accept one map and still derive different homes for a key.

  Its own alternative — leased generation capabilities that nodes renew from the control plane, so a partitioned old generation is fenced mechanically — is recorded in §9 and overruled on one fact: it makes every node's ability to lock depend on continuously reaching the control plane, so a control-plane outage longer than the capability lease stops record locking on every node, including healthy ones that agree with each other. It also reintroduces the renewal lease and clock-rate bound this revision exists to delete. What it genuinely buys is recorded rather than dismissed, and §9 names the condition under which it should be taken instead.

• Round 9 found two blockers in that quarantine, both fixed, and the first one is the most interesting bug on this branch. Turning the quarantine on by default made two paths reachable that were dead while it defaulted to -Infinity:

  1. It bounded the wrong lifecycle. The horizon was performance.now() read as process uptime — but performance.now() and performance.timeOrigin are process-wide inside a worker thread too. A probe spawning a worker 1.5 s into a process reads now() = 1554.6, not ~0. Coordinator state is per-thread, so a replacement coordinating worker in a long-lived process would read an uptime far past any process-based horizon and grant immediately, over delegations the worker it replaced had issued. Two holders. The horizon is now anchored on COORDINATION_STARTED_MONO, taken at module scope in each thread.
  2. It answered 423 for a key nobody holds. The denial was contended, which acquire retries; the quarantine runs a full DELEGATION_LEASE_MS + skew (365 s) while MAX_LOCK_TIMEOUT_MS is 300 s, so no legal caller could wait it out — it burned the whole timeout holding the native key, then reported "Record is locked and was not released in time". It is now its own quarantine reason that converts to a retryable 503 immediately.

  Round 11 then found the fix was still anchored on the wrong instant. Module scope closed the process-vs-worker half, but a thread can take coordination ownership long after it booted — so a thread-anchored horizon reads as long elapsed in a thread that has just become the coordinator over a dead one's live delegations. The horizon now runs from construction, the earliest instant core can prove nothing else was granting under. Nothing is lost where a predecessor's authority is actually known: adopt waives it and the retirement record carries it. It also moved the generation rollback floor from per-(database, table) to per-database — a generation is published per database, so refusing a rollback database-wide is strictly stronger and costs nothing. Core's floor still lasts only as long as the coordinating thread; the durable, cross-thread half is harper-pro's.

  Two smaller ones from round 9: a transport swap landing while Table.lock() was parked on the native key lock 503'd the caller, because acquire ran on the coordinator it had captured and the swap closed it — it now hops through #authority() at entry and after the backoff, carrying the remaining wait; and the drain and remote-request races left their losing timers running for a full lease.

  One adjudicator ruling was overridden, with executable evidence. Gemini asked whether handOffTo copies the quarantine to the successor; the adjudicator dropped it as factually wrong by citing successor.#grantableAfterMono = -Infinity — which is the defect, not the refutation. Adopting live authority is evidence about what the predecessor coordinator granted, not about a previous incarnation of the thread, so a swap inside the window silently cleared the quarantine. The two horizons are now merged with max, and the regression fails on the parent with "a transport swap cleared the cold-start quarantine".

• The restart quarantine moved into core, and that is a behaviour change worth your eye. Under the epoch design the interval was the transport's: do not name this node in an epoch until a previous incarnation's delegations could have expired, or advance the epoch number. Neither half survives an immutable map — a generation does not advance because a process restarted — so #grantableAfterMono now defaults to DELEGATION_LEASE_MS + skew on the monotonic clock (which counts from process start, so every lazily created table coordinator computes the same horizon) instead of -Infinity. The first cluster lock on a key this node homes waits out that interval after a restart; keys homed elsewhere are unaffected. The reason it was off before was that turning it on hung the real-table suite for six minutes; the answer is ClusterLockTransport.grantableAfterMono, an explicit override for a caller that can prove a previous incarnation issued nothing.

• Round 12 caught a regression my own round-11 fix introduced, and one contract error under it. Anchoring the horizon at construction was right, but an adopting successor recomputed its own and took the max — so every transport re-registration, which a component reload triggers, rejected this node's whole home share for a full delegation lease. Adoption means the successor knows everything the predecessor knew, so it now inherits the predecessor's horizon exactly; recomputing over-quarantines and clearing it would let a swap inside the window grant over an unseen predecessor incarnation. Both directions have a regression.

  Under that sat a contract error worth a reviewer's attention because core cannot enforce it: homeIncarnation was specified as advancing once per process, while coordinator state — the delegation counter included — is per thread. A replacement coordinating worker restarts that counter at zero, so a per-process incarnation would let it re-mint tokens its predecessor issued, and a delayed release carrying one of those tokens clears a live grant. The quarantine does not cover this: it bounds overlapping delegations, not a stale release replayed later. §5.1 and the LockHomeMap contract now require the counter to advance per coordination incarnation, and §11 lists it beside §4.3's activation record as an obligation core relies on without being able to check.

• Rebased onto main after it advanced mid-review, which is why the history moved. One conflict, in Table.ts's import block: main's #2524 (dedicated application worker threads) replaced the manageThreads.js import list while this branch had added isLockControlType to the auditStore.ts one. Resolved as the union minus getWorkerCount, which main dropped because nothing uses it. The interaction worth your eye is not textual: feat(threads): run an isolated application in a dedicated worker thread #2524 adds a new thread kind, and this branch's coordinator turns on ownsCoordination() and per-thread module state — so the registration obligation now names the dedicated application worker explicitly.

• Rebased a second time, onto main after it advanced roughly 30 more commits. One conflict, again in Table.ts's import block, but a different one from above: main's audit-record refactor had added raiseAuditFloor and boundedAuditPruneEnd to the auditStore.ts import list, while this branch's own commit still only added isLockControlType to that same list. Resolved as the union of all three names — no logic changed, only which names the import statement pulls in. tsc --noEmit, prettier --check and oxlint are clean at the rebased head, the local recordLock* suite is 181 passing / 1 pending, and all four Unit Test CI legs (Node 22/24/26, Windows) plus the full integration and adapter matrix passed on it.

• A control entry no longer claims the table's structure version, and that one is worth a look. writeLockControlEntry declared primaryStore.encoder.structures.length + typedStructs.length on an entry whose bytes were packed by the private control Packr, which shares none of those structures. RocksTransactionLogStore raises the per-(log, table) structure watermark from that field and flags the entry that raises it — so a surrender release landing between a structure mint and the next data write would take HAS_STRUCTURE_UPDATE and leave that write unflagged, and a receiver learning structures only from flagged entries then decodes later records against a stale set. That is harper#1348's failure class reached through the lock path. It is now zero, on a short argument: a payload with no table structures cannot advance them. The flag is consumed in harper-pro's replication, so this is not end-to-end verifiable here; the round-trip test asserts the field.

• The release payload is now pinned against the TABLE decoder, not msgpackr's default. An outside lens argued that an integer record id in 64..127 reaches auditRecord.getValue's decoder as a bare fixint, is read as a structure header, and fails — silently dropping the release and leaving the home holding its grant for a full delegation lease. The existing wire test could not have caught it: it unpacks with the default msgpackr, which is not the production reader. The new regression writes releases for 64, 100, 127, 63, 128, [64] and 'record-64' through writeLockControlEntry and reads each back through getValue — all decode. The claim is refuted, and the gap in coverage it correctly identified is closed.

• Two findings in the last round were claims about code the reviewer could not fully see, and both held. A 409 from the lease fence nulled the refused write out of this.writes but never called detachWrite, which also repairs the per-key chain — so a caller that catches the 409 and stages the same key again in the same transaction would take the rejected operation as its merge basis. That is harper#1968's failure class reached through a different door, and it now goes through detachWrite. Separately, applyLockControlEvent resolved the entry's author before the try/catch that exists so one entry cannot stall the apply loop; getNodeNameForId(..., rebuildOnMiss) reads the audit store, so a store error escaped the sink. The lookup is now inside the guard — §8's rule that a receive boundary settles its callers and keeps admission closed.

• One more from the last round, recorded rather than fixed, because closing it is a wire change. A renewal reply that loses its race strands the home's grant for a full delegation lease. The home renews the token in place to counter n+1 while the raced-out delegate stays on n; the handback is refused because a delegation for the key is still held — which is the guard that closed the two-holder path — and the later surrender writes counter n, which the home's exact three-component match ignores. The grant then survives to DELEGATION_LEASE_MS + skew, every contender gets contended, and the recall carries n+1, which the delegate treats as a no-op. It is the availability half of the requestId residue already tracked as harper#2582; that issue should be read as covering both halves.

• One more minor, recorded rather than fixed. getNodeNameForId(..., rebuildOnMiss) deliberately bypasses the 50 ms negative-cache window, because a dropped release leaves a home holding its grant for a whole delegation lease and control entries are far too rare to drive the store. The residue the last round sharpened is a node id that is permanently unresolvable — a translation gap, or a partially written mapping record — which then costs a store read and a map rebuild per entry carrying it, indefinitely. That is a replication-layer invariant break rather than a lock-path one, and adding a second throttle would re-open the case the bypass exists for; it is called out so the next reader does not have to re-derive the tradeoff.

• One last-round blocker was a misread, and it is worth naming so it does not alarm the next reader. The claim was that the subscriber's lock-control skip uses return inside an async generator, so the first LOCK_RELEASE would terminate a subscriber's whole event stream. It is not a generator — it is the per-event listener callback passed to addSubscription, where return skips one event, which is what the two neighbouring lines (if (dropDuringReplay) return; and the reload branch's return scheduleReloadResnapshot()) do for the same reason.

• And one open major from that round, left as an obligation rather than fixed, because the fix is routing. applyLockControlEvent dispatches a received release through admittingCoordinator, which is per-thread: on a thread that never acquired, it is undefined and the release is dropped — and dropped before applyEntry's own off-owner counter can see it, so the home holds its grant to the delegation deadline with nothing recorded. This is the same "route it to the coordinating thread" obligation already written down for the delegation RPC path (§11 and registerClusterLockTransport), applied to the replication sink; whether it can arise depends on which thread harper-pro runs that sink on, which is not decidable here.

• One finding was refuted three rounds running, so it is recorded here to stop a fourth. The claim is that ringKeyFor's String(keyId) destroys binary-key entropy and collapses distinct keys onto one home. It cannot: keyIdOf is writeKeyId, which is toBufferKey(key).toString('latin1') (DatabaseTransaction.ts:513) — ringKeyFor is handed a latin1 string, never a Buffer, and String() on it is identity. The real-table test that grants a Bytes primary key across the wire pins it by execution.

• Two more are open decisions rather than defects, and both are yours. lock()'s scope defaults to 'cluster', so an unqualified call silently means node-local wherever no transport is registered. An outside lens read that as a two-holder blocker — a non-owner worker taking the Phase 0 path while the owner thread runs the protocol — and the adjudicator disputed the premise on the right grounds: ownsCoordination() exists and fails closed, which only makes sense if the transport is registered on every worker. What survives is the startup window before the first registration on a worker, and since the "this database is clustered" latch is per-thread module state, core cannot close it. This push writes that down as an obligation on registerClusterLockTransport and in §11, beside the activation record and the incarnation rule. The alternative — defaulting to 'node' and making cluster opt-in — is a public API change for every existing caller, cheap now and breaking after release, which is why it is raised rather than decided. Separately, expiry work and cap budgets are per table, so process-wide grant retention is caps × tables and the tick loop scales with table count (harper#2581).

• Findings from the round-6/7 reviews that are still deliberately not fixed, all of them Phase-1 scope rather than defects with a settled answer:

  1. Table cleanup does not close() the coordinator, so no retirement record is written, the coordinator stays in the process-wide tick set, and a same-name re-create restarts the token counter at 0. Reaching a two-holder consequence needs a drop plus a same-name re-create while remote delegations are live, but the retirement record is exactly the mechanism that exists to prevent it — this is table/database lifecycle ownership, and the fix belongs with whoever owns that teardown.
  2. A cached-delegation hit allocates an Admission plus two map entries per lock(), so §8's "zero additional protocol allocation on a cached hit" does not hold as written. Round 8 sharpened the retention arithmetic and §8 is corrected accordingly: an admission deliberately survives its own unlock(), because the write it staged can still commit and the revoker must stay reachable until the handle's lease runs out — so retention is lock rate × lease, the floor any correct implementation pays, and not a bound independent of lock rate. What #pruneAdmissions fixed is a different failure (a longer-lease admission hiding shorter ones behind it), and getting below rate × lease means carrying the revoker on the Phase 0 handle that already exists. That is the redesign harper-pro#824's numbers are for.
  3. admittingCoordinator?.registerAdmission(...) is optional-chained, so a standalone claim landing in the microtask gap after acquire() resolves returns a handle nothing can fence. Binding registration to the authority that minted the round is the right shape and changes the admission contract.
  4. ringVersion reaches no comparison, because DelegationRequest does not carry it. Resolved by the recut, not deferred. ringVersion existed to detect members[] changing within an epoch; an immutable generation cannot disagree with itself, so the field is gone and there is nothing left to compare. The related hazard the recut does not dissolve — two binaries hashing the same map differently — is answered separately, by versioning the routing hash on the protocol capability (above).

  Items 1–3 have now survived four rounds unchanged, which is why they are listed rather than iterated on: each is a scope call, not a defect with a right answer reachable from the code.

• One new hazard the planning round raised that is not in the note: lease safety assumes a suspended process still advances performance.now(). A VM suspend or snapshot restore can pause a guest's monotonic clock, which would let a delegate resume with a deadline the home has already outwaited. §4 should either state the platform assumption explicitly or invalidate delegations when wall-clock and monotonic progress diverge.

One bug this push found in main, filed rather than fixed here

Tracing X-Replicate-To to decide whether it was a usable mitigation turned up #2546 (P1): checkContextPermissions (resources/Table.ts:7258) gates on truthiness, so X-Replicate-To: 0 skips the super-user 403 and reaches getResidency with count = 0, pinning the record to the receiving node. Any authenticated user with write permission can take a record out of replication with one header — 200 response, nothing logged, and the residency persists across updates. Pre-existing on main, unrelated to this branch, and out of scope here; the fix is != undefined on both guards.

Changes

• docs/record-lock-ownership.md is where this push mostly lives. §4 is replaced: the epoch protocol's §§4.0–4.4 — durable acceptor state, ballots, promises, renewal leases, retirement reservations, transitive activation — are deleted, and the section is now the home map, why the operator is the authority, §4.3's staged transition, and the routing algorithm carried over unchanged. §9 moves the operator-agreed map from the rejected list to Chosen, records the epoch protocol as the rejected alternative it now is, and adds the leased-capability option with the fact that overrules it. §14 records both verdicts and rewrites the decomposition so harper-pro#825 is a config generation, a digest check and a change runbook. §8's admission-retention claim is corrected to rate × lease.
• DESIGN.md gains the Phase 1 section — the exclusion-only contract, the three rates (home map, home, delegation), and the pointer to §10 of the note as the normative wording for anything user-facing about lock(). It states, rather than restates, what the note owns; the cold-start paragraph now says core enforces the restart quarantine rather than delegating it to the transport.
• utility/errors/hdbError.ts adds LockUnavailableError (503, LOCK_UNAVAILABLE, retryable). It is the fail-closed answer whenever the cluster guarantee cannot be established — no agreed epoch, an unreachable home, a closed coordinator — and it exists as its own class so a caller can tell "could not promise" from the 423 "held by someone else".
• resources/nodeIdMapping.ts adds getNodeNameForId, the inverse of the existing name→id map, so a received control entry can be attributed to the node that authored it rather than to whatever the payload claims. The inversion is cached per audit store — short ids are minted per database, so id 1 names a different node in each — invalidated wherever nameToId is written, and a cache miss re-reads at most once per 50 ms so a burst of unmapped ids cannot drive the store.
• unitTests/resources/recordLock.test.js adds the Phase 0 regression this branch's fence rewrite needed: a write staged under a lock, then unlock()ed, must survive the commit-time write replay an open read iterator forces. Judging the re-save by isExpired() — which counts a deliberate release — made the same caller code succeed normally and throw 409 only when an iterator happened to be open; the lease alone is the right test.

Verification

• Both tests added this round were verified by rebuilding dist with the behavior removed, not by reading. The follower regression reports ClientError: Record is locked and was not released in time (423) on the parent once the leaderFailure capture at Table.ts:2880 is deleted. The failing-writer test reports 2 !== 1 — the home re-sent the recall — once the try/catch is removed from #writeControlSafely. Both sources were restored and dist rebuilt before the suite runs below.
• The follower regression is deterministic rather than timing-dependent: a mapless transport makes LockCoordinator.acquire throw on its first loop pass with no await before it, so every step from acquireRecordKey's uncontended tryLock to the rejection is a microtask, and V8 drains those before the timer phase. Run 12 times, 12 passes.
• Eight regressions for this adjudication round, each verified failing on the parent by reverting only the behavioral hunks and rebuilding dist — not by editing the .ts and re-running, which proves nothing, because the unit tests load dist/. fails a recall it cannot route drives the registered transport's own onDelegationRecall after the reconnect window closes the getter, and reports "Missing expected rejection" without the fix; leaves a grant recallable when a local recall fails reports "a failed local recall was never retried (1 sent)"; re-arms the quarantine on an incarnation change the ownership poll never sampled holds ownsCoordination() at true throughout, so it fails on any poll-based anchor; and reads the store once per window for an id it will never resolve counts 26 store reads on the parent against 1. The other four are the owner→non-owner→owner sequence, the waiver on a coordinator built off the owner thread, the transport reload re-waiving it, and a recall routed to a non-owner.
• Six regressions for the recut and for round 9's findings, each asserted against the code without its fix. The default restart quarantine is pinned at both ends of the interval against COORDINATION_STARTED_MONO, so it does not depend on how long the suite itself has been running; a generation that went backwards is rejected on acquire and on the home's inbound request path (reverting only the two call sites in the built dist flips it to "Missing expected rejection"); a transport swap must not clear the quarantine (reverting the one line flips it to "a transport swap cleared the cold-start quarantine"); a quarantined home answers 503 rather than spinning to a 423; an acquire whose captured coordinator was closed by a swap lands on the successor; and the existing generation-change and superseded-generation-across-the-await cases are retargeted onto the new contract.
• The worker-clock premise was checked by execution, not by reading. A probe spawning a worker_threads.Worker 1.5 s into a process reports timeOrigin identical to the main thread's and performance.now() = 1554.6 in the worker — which is what makes a process-anchored quarantine the wrong bound for per-thread coordinator state.
• Coordinator unit tests rewritten around independent per-node clocks. A single shared fake clock cannot express a home outwaiting its delegate by skew measured on two different clocks, so it would pass a coordinator that compared a remote reading against a local one. Eight of them are regressions for defects the pre-push review found, and each fails without its fix — including the two added in the last commit, for the handback that spans a transport swap and for a home granting to a node the epoch does not name.
• Real-table cluster tests — the control entry encodes, commits, decodes and stays out of every surface that reports record activity, and does not collide with the holder's own audit entry.
• Three regressions for the two findings this push fixes, each of which fails on the parent commit with the exact defect it names (verified by reverting only the behavioral change and rebuilding dist): the coalesced cluster follower is handed the leader's node-scoped handle instead of a 409; the mixed-lease sweep reports 201 retained admissions instead of bounded; and the surviving long admission must still be revoked at handoff, which is the property the sweep must not break.
• npx mocha "unitTests/resources/recordLock*.test.js": 181 passing, 0 failing, 1 pending, at 68bfa46f4.
• npm run test:unit:resources: 2608 passing, 33 pending, at 68bfa46f4, with one failure (Caching > Source throw error) that passes on its own and is the known local timing flake, not this branch. npm run test:unit:main was not re-run at this head — this round changes only unitTests/resources/, which that gate does not load; at 49ae532ba it reported 5519 passing, 2 failing, both pre-existing environment failures unrelated to this branch (gitCredentials.test.js tmpdir leak, configValidator.test.js cwd-length case). npm run test:integration -- integrationTests/resources/record-lock-concurrency.test.ts: 3/3. The rangeReadActivity read-your-writes failure recorded in an earlier revision of this description is no longer reproducing here; it was search()/query() iterators run outside the request transaction: a handler cannot see its own writes, and its scans and point reads disagree about the snapshot #2506, pre-existing on main.
• tsc --noEmit and prettier --check clean. Rebased onto main twice since (see below); CI ran green on the latest rebase's head.

Review coverage, stated honestly

Three pre-push rounds ran on the implementation. Round 1 returned BLOCK with five paths that admit two nodes on one key, and it was right on every one; round 2 found two more. They are listed in the commit messages, but the ones worth a reviewer's attention here:

• A delegate anchored its deadline when the grant reply arrived rather than when it sent the request, so a delayed reply outlived the home's own grant.
• A delegation survived a generation change, so a re-homed key kept its old delegate admitting.
• Recall waited on the admission count, which unlock() drops to zero while the caller's write is still staged — that write then landed after the successor was admitted. Recall now revokes handle capability, which is what §6 always said and the first implementation did not do.
• A release was matched on its counter alone, and counters restart when a home restarts.
• The amortization did not actually hold. A delegation ran for exactly the caller's lock lease, so it never had room for the next lock and every repeat lock renewed — a round trip per lock where the design promises zero. The test passed only because its fake clock never advanced inside the loop.

Round 3 was degraded and should not be read as a clean bill of health: the Codex graded leg and the Harper domain leg both timed out on this diff and the Claude fallback ran out of budget, so only Gemini returned findings. Its two blockers are fixed; its other two were wrong (release() sets released, not expired, so the commit fence still distinguishes a clean unlock from a revoked lease, and noop is declared at Table.ts:8075).

Round 6 ran the full set — Codex graded, Gemini, Cursor/Grok and the Harper domain adjudicator — and returned one blocker, which is fixed: the late-reply handback ran on the coordinator that sent the request, and handOffTo had already emptied that object's delegation map, so the "never hand back while a delegation for the key is held" guard added the commit before read an empty map after a transport swap and gave back a grant that still backed this node's live delegation. Two of its majors are fixed and four are recorded above as open. Both of Gemini's substantive findings were dropped as factually wrong by the adjudicator, and independently: release() sets released, not expired, and ringKeyFor's String(keyId) is identity because keyIdOf is writeKeyId, which already returns the ordered-binary bytes.

Round 8 was a planning round on the recut, and it did not clear — Framing-Verdict: better-alternative-exists. Its blocker was right and is fixed in §4.3 (publishing a generation does not revoke a live one); two of its findings are adopted in code (the generation monotonicity floor, the capability-versioned routing hash); its own alternative is overruled in §9 with the disqualifier.

Round 9 was the full code round on the recut — graded (Codex), Gemini, Cursor/Grok and the Harper domain adjudicator, Adjudicated-Severity: blocker. Both blockers were in the quarantine this push turned on, and both are fixed above along with two minors. Three Gemini findings were dropped as factually wrong by the adjudicator — writeLockControlEntry does commit (DatabaseTransaction.ts:1099→:1242, and recordLockCluster.test.js reads the landed entry back out of the real audit store), and applyEntry does bind the payload's requester to the authenticated author (recordLockCoordinator.ts:796) — and one of its drops was itself wrong, which is written up above.

Nit declined, for the third round running: the file-header comment block in recordLockCoordinator.ts is a design overview, and this repo keeps design orientation next to the code it explains (DESIGN.md is built on the same convention). The comments that narrate review history rather than an invariant are fair game and were trimmed where they added nothing; the orientation stays.

Rounds 27–31 were this adjudication round's own pre-push passes, and they are the reason the quarantine section above is written the way it is. Round 27 (codex + Gemini + Cursor/Composer + domain) found the ownership-horizon hole; round 28 (full: codex + Gemini + Cursor/Grok + domain, Adjudicated-Severity: major) found that round 27's fix left the waiver uncleared on a coordinator that never owned, and that handOffTo re-latched it; round 29 found that round 28's poll could not prove continuity at all and cost 10 ticks/s per table; round 30 found the rebuildOnMiss bypass. Round 31 found nothing new — its graded leg states "no new correctness or hot-path regression was found in this delta" — which is what stopped the loop. Rounds 29–31 ran without the domain adjudicator (pruned as narrow low-risk deltas), so the outside findings in those rounds were adjudicated here, against source, and each ruling is recorded above with the line it rests on.

Rounds 36–37 cover this test-only push (codex graded + Gemini; Cursor and the domain adjudicator were pruned as a narrow low-risk delta). Round 36 is where the failing-writer finding came from, and round 37 — run on the fix — returned no new actionable finding on the delta, its graded leg confirming the new assertion discriminates. Both rounds' outside findings were adjudicated here against source, in the table at the top. Round 37's first attempt was killed by SIGTERM at 188 s and was re-run to completion.

Round 38 is the full round on the second rebase above (codex graded + Gemini + Cursor/Composer + the Harper domain adjudicator, --timeout 3600 to give the graded leg room on a 12-file/6.4k-churn full diff). It found no new regression: the import-conflict resolution retains both mainline audit-floor helpers and the lock classifier, and code trace independently confirmed the failing-writer test still discriminates. Three findings recurred, all classified previously-adjudicated and left as-is — the table/database-teardown coordinator-orphan minor (already filed as #2591 above), the double isLockControlType classification nit on the replicated hot path, and the comment-narration nit declined since round 6. Receipt is on the pushed head, 6dd59d6b5.

That history is the argument for the §12 suite the note asks for, and for not enabling this until the measurement gate (harper-pro#824) has run against the replacement. Its before-figures half has
run: harper-pro#822 carries RECORD_LOCK_COST_BASELINE.md for Ricart–Agrawala on a 3-node loopback
mesh — repeat-lock p50 0.68 ms, 4 durable entries and ~310 B per acquisition, off-vs-absent inside noise. The repeat-lock figure is the one a live delegation has to turn into a local key lock.

Refs #483

Complexity: complicated

_{Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex; adjudicated=domain; declined=cursor-grok; rounds=38; full=8 @ 6dd59d6}

_{Human-Review-Need: 4 (decisions: exclusion-only-without-freshness-fence, cluster-is-the-default-scope, unregister-does-not-close-the-coordinator, staged-then-unlocked-writes-are-revoked-not-drained, per-table-delegation-caps, implementing-ahead-of-the-measurement-gate, design-note-in-repo) @ 6dd59d6}

[gemini-code-assist]

Code Review

This pull request implements Phase 1 of cluster-wide record locks using a Ricart-Agrawala consensus algorithm over replicated control entries. It introduces a LockCoordinator to manage lock requests, grants, and releases, integrates these control entries into the transaction log and replication flow, and updates transaction committing to enforce monotonic lease deadlines. Feedback on the changes is minimal but highlights a critical issue in resources/Table.ts where an undefined noop reference in a promise catch block could cause a runtime ReferenceError.

[cb1kenobi]

Four places hand a key onward without revoking the handles the previous delegation admitted, so a node can still commit to a record after another node has been admitted to it. The worst is a renewal reply that arrives after the caller gave up: it writes a release for the token the home has live, clearing the grant while this node's older delegation is still admitting. A renewal and an epoch-change drop both replace or delete a delegation without calling revokeAll, so a write that was staged and then unlocked still commits past the commit fence. Separately, admission revokers are never removed on release, so they pile up for the full six-minute delegation on the repeat-lock path the design is built around.

—
Reviewed 1deac50

[cb1kenobi]

Three blocking issues remain. Transport transitions can silently weaken cluster exclusion or release a grant still in use. Admission pruning also permits memory exhaustion; preserve current authority, reject scope downgrades, and prune by expiry.

—
Reviewed 729aefd

[cb1kenobi]

Prior blocking threads on unbounded revokers, renewal dropping admissions, superseded-epoch unfencing, raced-out renewal releases, transport-swap cleanup, cluster-to-node scope downgrade, and prune-early-exit are addressed at this head. I re-traced acquire, release, applyEntry, lease fencing, and control-entry isolation against the current diff and existing discussion. No new confirmed blocking defect remains on the changed lines.

—
Reviewed 14c1b55

[cb1kenobi]

The newest commit is test-only and makes the failing-writer containment test actually discriminate, by advancing the home node's clock past the recall retry interval and asserting the recall count is unchanged. Without that change the assertion passed whether or not the writer's throw was contained. No new blocking problem was found on the changed lines, and everything traced is already fixed in this PR, previously refuted, or tracked in an existing ticket. Unit and integration CI were still running at capture time, so those results are unconfirmed.

—
Reviewed 6dd59d6