Repository navigation
TrieDbMut transaction support: snapshot and rollback. - #240
Open
michalkucharczyk wants to merge 47 commits into
Open
michalkucharczyk wants to merge 47 commits into
michalkucharczyk wants to merge 47 commits into
Conversation
Keeps one arena alive across sessions so PoV estimation records joint collapses of deletes from different extrinsics, and lets runtime transactions roll it back.
skunert
reviewed
Oct 16, 2026
Lets an owner that is itself cloneable, such as polkadot-sdk's overlay, keep the estimation trie in its clone.
An owner can now hold a default DetachedTrieDBMut before it knows the root, and roll it back to a snapshot taken before the first session, without tracking the root next to it.
polkadot-sdk no longer clones its overlay, so nothing needs to clone a detached trie. The changelog now also mentions DetachedTrieDBMut::default and how from_detached continues it.
Emptying the trie wrote the null root immediately, so an uncommitted or rolled back removal of the last key leaked into the caller's root.
Snapshot, rollback and detach exist only on the append-only type, commit only on TrieDBMut, so neither needs a runtime mode check. The append-only builder takes the root by value: that trie never writes it.
The root handle belongs to the trie, so the snapshot is assembled where the root handle lives, not passed through the storage.
The append-only storage logs every rewind; a snapshot is dead once a later rewind cut the arena below its mark, and using it would install a root handle that points at an unrelated node.
…ference The core only reads; it is generic over the database reference a front holds, mutable for TrieDBMut, shared and read-only for the append-only trie.
The root enters a detached trie exactly once, so attaching cannot pass a different one; the empty-arena rule and Default are gone with it.
skunert
reviewed
Oct 20, 2026
lexnv
reviewed
Oct 26, 2026
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
skunert
reviewed
Nov 2, 2026
lexnv
reviewed
Nov 2, 2026
The core holds its nodes and root handle in a RootedNodeStorage, owned by TrieDBMut and borrowed from the DetachedTrieDBMut by AppendOnlyTrieDBMut, so changes land in the caller's state as they happen and survive a panic. Replaces from_existing/detach. The core reaches its storage through explicit accessors.
Removing the last key allocates nothing, so two snapshots could share a mark and a stale one was accepted (PR 240 review r4443471967). A mark now carries a sequence, and the storage keeps the sequences that can still be rolled back to; a rollback drops every later one. Replaces the allocation IDs.
A snapshot froze the arena until a rollback, so committed transactions left their dead path copies behind for the whole block (polkadot-sdk PR 10215 review r4446162753). discard_from(snapshot) drops the snapshot and every later one without changing the trie. The freeze line falls back to the newest remaining snapshot and the slots only the dropped ones kept alive become free. Free slots stay reusable across a snapshot; a reuse below the freeze line is logged so that a rollback can free the slot again.
skunert
approved these changes
Nov 3, 2026
lexnv
reviewed
Nov 3, 2026
lexnv
reviewed
Nov 3, 2026
lexnv
reviewed
Nov 3, 2026
lexnv
approved these changes
Nov 3, 2026
destroy scanned the reuse log after the newest mark for every slot below the freeze line, so a block of deletes after a large committed transaction was quadratic (PR 240 review r4459279927). Each slot now keeps its position in the log.
lexnv
approved these changes
Nov 4, 2026
A nested commit rescanned every dead slot and re-logged every reuse since the surviving mark, so its cost grew with the outer transaction. Entries from before the dropped mark are already below the new freeze line, so discard_from now starts at the dropped mark's own watermarks.
A rollback filtered the whole free list, so even an empty transaction cost every free slot. Pops take the newest entry, so the free slots that survive from the mark form a prefix. The storage now tracks its lowest length per mark, and a rewind only filters what came after it.
A slot free at the outer mark, written and then emptied under a nested mark, stayed dead after the nested mark was discarded, so the arena grew by up to the outer free list. The discard now asks the question destroy asks, after it settles the reuse log.
AppendOnlyTrieDBMut -> TransactionalTrieDBMut AppendOnlyTrieDBMutBuilder -> TransactionalTrieDBMutBuilder AppendOnlyNodeStorage -> TransactionalNodeStorage triedbmut_append_only (tests) -> triedbmut_transactional "Append-only" named the mechanism and is no longer accurate; the type exists to support transactions.
The new module triedbmut/transactional.rs holds TransactionalTrieDBMut, its builder, its node storage, DetachedTrieDBMut and the snapshot types. It is a move without changes, and re-exports keep the public paths.
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds
TransactionalTrieDBMut, an in-memory trie with snapshots. Taking a snapshot is cheap, the trie can roll back to it, and it can be discarded once nothing will roll back to it. That is enough to mirror storage transactions: snapshot when one starts, roll back if it fails, discard when it commits. One instance can then live for a whole block.Motivation: in paritytech/polkadot-sdk#10215 the PoV size estimator replays each extrinsic's storage deletes against the pristine root in a separate
TrieDBMut. Node collapses that only happen jointly across extrinsics are therefore never recorded, and the estimate is too low. With a long-lived instance the collapses are observed, and snapshots let the estimator discard the changes of a host transaction that was rolled back.Review notes:
TransactionalTrieDBMutis a second front over the same core, with a node storage in which the nodes a snapshot uses are never overwritten. It lives intriedbmut/transactional.rs.TransactionalTrieDBMut::snapshottakes aTrieDBMutSnapshotin constant time andTransactionalTrieDBMut::rollback_torestores it without copying the trie.DetachedTrieDBMut, which also supports snapshots and rollbacks.TransactionalTrieDBMutBuilder::from_detachedborrows it together with a database, cache and recorder, and the trie writes every change into it directly. So the state survives a session that fails halfway, and a rollback still restores it. The root enters once, inDetachedTrieDBMut::new. The database is only read and is taken as&dyn HashDBRef.discard_fromdrops a snapshot and every later one without changing the trie. The nodes only those snapshots kept alive are reused. Without it, every change made after a snapshot keeps a copy of its path until a rollback, so committed transactions would leave dead copies behind for the whole block.commit, does not implementTrieMutand holds no reference to a root.TrieDBMuthas no snapshots. Neither needs a runtime mode check.InvalidSnapshotand leaves the trie as it is.hash-dbgainsimpl HashDBRef for &dyn HashDBRef, so it needs a point release too.