Repository navigation
backport: bitcoin#29215, #29370, #29720, #29726, #30053, #30267, #29996, #30320, #29519, #30807, #30962, #30909, #34286, #34728, #34852 - #8079
Draft
PastaPastaPasta wants to merge 70 commits into
Conversation
Collaborator
|
🕓 Review not started yet because this PR is a draft.
Commit 91294b1. Normal review starts when eligible; priority review starts as soon as a slot is available. |
|
This pull request has conflicts, please rebase. |
…ncomplete While the snapshot chainstate is active and unvalidated, every DKG phase handler thread spun with no delay: WaitForNextPhase(nullopt, Initialized), the first call of each round, threw AbortPhaseException on its first check, and PhaseHandlerThread caught it and restarted the round at once. Each iteration took cs_main, threw and caught an exception, called MarkAborted and, under -debug=llmq-dkg, logged four lines. No round can start in that state (InitNewQuorum refuses it), so the round-start wait now sleeps and re-checks instead of aborting. A round in progress is still abandoned, a stop request still ends the wait, and nodes not running on an unvalidated snapshot execute the same code as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BLS operations that are not given a scheme read the process-wide bls::bls_legacy_scheme. Add bls::EffectiveLegacyScheme(), which returns a thread-local override when one is set (bls::ScopedLegacySchemeOverride) and the process-wide value otherwise, and route every implicit read through it: the CBLSWrapper and CBLSLazyWrapper serialization defaults, the implicit aggregate and verify overloads, sig share signing, recovered sigs, DKG contributions, the coinbase ChainLock decode cache, the miner and `verifyrecsig`. Work that runs on another thread captures the scheme where it is created: utils::BlsCheck (run by the commitment CCheckQueue), CBLSBatchVerifier, and CBLSWorker's AsyncSign and AsyncVerifySig jobs, using new explicit-scheme VerifyInsecureAggregated/VerifySecureAggregated overloads. AsyncVerifySig no longer batches jobs of different schemes. Nothing sets an override yet, so behaviour is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ConnectBlock establishes the BLS scheme of the block it connects by writing the process-wide bls::bls_legacy_scheme, and ConnectTip and DisconnectTip restore it only on return. A background chainstate validating pre-V19 history under a post-V19 snapshot therefore leaves the whole process on the legacy scheme for most of each background block connection, while other threads verify ChainLocks, recovered signatures and InstantSend locks for the active chain outside cs_main. Valid ChainLocks then fail (Misbehaving(10) and a poisoned seen set) and batched recovered-signature verification bans honest peers. ScopedBLSLegacyScheme now writes the process-wide scheme only for a scope that may commit the active chainstate's tip. Every other scope (a background chainstate, a TestBlockValidity dry run, VerifyDB, RollforwardBlock), and any scope nested in one, sets a thread-local override instead. The V19 switch in Process/UndoSpecialTxsInBlock goes through bls::SetEffectiveLegacyScheme(), and the EHF signal rebuild at startup uses a scoped override instead of flipping the global. ConnectTip and DisconnectTip Assume() that no thread-local scope encloses a move of the active chainstate's tip. With a single chainstate the active chain's path is unchanged. The thread-local scopes give their own thread the values the process-wide flag used to take, but other threads no longer see them; for example a `verifychain 4` reaching below V19 no longer switches the running node to legacy while it runs. VerifyDB no longer touches the process-wide scheme at all, so VerifyLoadedChainstate's re-store of it after VerifyDB (already unreachable, because VerifyDB restored the scheme on return) is removed. The existing background-connect test never connected a block (the snapshot setup drops the data below the base). It now restores one block's data, connects it, and checks from inside ConnectTip that another thread still verifies a basic signature and batch. The V19 boundary test also runs a level-4 VerifyDB across V19 and checks that the process-wide scheme stays basic throughout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…changes The process-wide bls::bls_legacy_scheme describes the active chainstate's tip, and while the node runs only the active chainstate's ConnectTip and DisconnectTip update it. ActivateSnapshot and the invalid-snapshot fallback in MaybeCompleteSnapshotValidation switch the active chainstate without doing so. After a node whose tip is pre-V19, or that has no coins yet, activates a post-V19 snapshot, every thread that verifies for the active chain stays on the legacy scheme until the snapshot chainstate connects a block above its base: valid ChainLocks fail and recovered-signature batches mark honest sources bad. The fallback leaves the reverse mismatch. Set the process-wide scheme from the new active tip right after both switches. The fallback runs inside the background chainstate's ConnectTip, under that thread's override, so the helper writes the global directly instead of going through SetEffectiveLegacyScheme(). ActivateExistingSnapshot and InitializeChainstate need no call: they run at init, before the tip is loaded and before any consumer thread starts, and VerifyLoadedChainstate establishes a post-V19 tip's scheme. On develop only tests activate a snapshot, so this is latent until a snapshot loader lands. The new tests activate a post-V19 snapshot over a legacy process and a pre-V19 snapshot over a basic one, and fall back from an invalid snapshot under a thread-local override; each checks the process-wide scheme, and the first and last also check that a fresh thread accepts a signature in the new active tip's scheme. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GetMinedCommitmentsUntilBlock stops silently when an index entry cannot be read, so a short result cannot tell a young chain from a damaged index. A caller that must not mistake a damaged index for a short one needs that difference. ScanMinedCommitmentsUntilBlock and its indexed variant return the same quorum bases plus `complete`: true when the requested number of entries was read or the walk left the type's index range, false when an entry inside the range could not be read or cannot be trusted: a value that does not deserialize, an impossible base height, quorum bases that do not strictly decrease along an index (a commitment is mined within its own quorum's DKG interval) or repeat across quorum indexes, a LevelDB read error, or a table block that the iterator stepped over because it failed its checksum. The bases read before that point are kept, except those a contradiction implicates: either entry of a pair that does not decrease, and both entries naming a repeated base, may be the wrong one, so neither is returned. Once the window is full no further entry or iterator status decides coverage. CDBIterator::HasError() exposes LevelDB's iterator status for this, since Valid() alone reads a skipped corrupt block as the end of the data. The range end is detected by comparing the raw key prefix before deserializing, and malformed values are reported instead of asserted on. Existing GetMinedCommitments* callers are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
getPlatformQuorums exported every retained Platform quorum key (1440 on mainnet) with no notion of when each quorum could sign. A verifier that accepts any of them accepts state signed by a quorum that was never in the signing set at the state's Core height, and an empty or short list reads the same as "this quorum does not exist", which would ban an honest endpoint on a node that is merely behind or has a damaged index. It has no production caller. getPlatformQuorum(llmq_type, quorum_hash, core_chain_locked_height) replaces it and answers FOUND (with the 48-byte basic-scheme key), INVALID or UNAVAILABLE: - FOUND: the quorum is in the window at that height, the newest commitments mined at or below it: signingActiveQuorumCount + 2 for a non-rotating type, and 3 * signingActiveQuorumCount for a rotating type, read cycle by cycle across its quorum indexes. Each is a quorum's signing-active period plus two DKG intervals of slack. - INVALID: not the Platform type; a height above INT32_MAX; a height more than those DKG intervals ((signingActiveQuorumCount + 2), or 3 for a rotating type) below a ChainLock this node knows, clamped to the tip; or a quorum hash that is no block of the active chain below that height, when the window was read in full and the block at that height is covered by the best ChainLock whose block this node knows. - UNAVAILABLE: everything else. That includes missing node services, a height above the tip, a height that only the tip shows stale because no ChainLock is known (after a restart during a ChainLock stall, Platform may still carry the stalled one), and a selected commitment record that cannot be read, is missing, is inconsistent with its index entry or carries an invalid key. Mined commitments passed consensus validation, so those record states are local faults, never evidence against the quorum, and a database read error is answered rather than thrown. A quorum outside the window is UNAVAILABLE, not INVALID, when it was mined at or below the height. Platform verifies a block against its last committed validator set, and Tenderdash installs a new set only when it commits, so after a Core-height jump the first Platform block carrying the new height is signed by the outgoing quorum however far it has fallen behind the window. The lookup counts a quorum as never mined by then only when its hash is no block of the active chain below the height. That proof covers this node's active chain only. Above a ChainLock that chain may be a competing branch, and a ChainLock's height alone does not rule that out: a CLSIG can name a block this node does not know or a branch it has not activated yet. A quorum found in the window is answered regardless of ChainLock coverage. Both ChainLock reads happen before taking cs_main: the best lock whose block is known, which alone can lock part of the active chain, and the best height, which bounds staleness only and is clamped to the tip, since a ChainLock can arrive before its block. The staleness bound is the pure llmq::PlatformQuorumMinSignedHeight, the window size is llmq::PlatformEligibleQuorumCount, and both follow llmq::PlatformEligibleIntervals. For non-rotating types a lookup reads at most signingActiveQuorumCount + 2 index entries and one commitment record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
During dual-chainstate operation a snapshot chainstate block whose Evo state depends on data this node does not have yet (a pre-base MN-list diff on a cold EvoDB) raised BlockDataUnavailableError. The generic std::exception catch in ProcessSpecialTxsInBlock turned it into failed-procspectxsinblock, so the block was marked BLOCK_FAILED_VALID and its relayer could be punished, although nothing is wrong with it. Add evo/snapshot_types.h with a LocalStateError base for errors about this node's own Evo state: BlockDataUnavailableError (moved here), SnapshotStateMismatchError and EvoViewBindingError. Rebuilding the credit pool or the EHF signals from a block body this node does not have (pruned, or below a snapshot base) raises BlockDataUnavailableError; a body the node has (BLOCK_HAVE_DATA) but cannot read raises BlockDataReadError. The readers take the data flag and the block position under one cs_main lock. For a pruned node this changes NORMAL too: such a block now stalls the chainstate instead of being marked invalid. Every generic catch on the block-connection path (CheckSpecialTxInner, ProcessSpecialTxsInBlock, CheckCreditPoolDiffForBlock, UndoSpecialTxsInBlock, GetCreditPoolDiffForBlock, CMNHFManager::ProcessBlock, CDeterministicMNManager::ProcessBlock) rethrows them. ConnectTip, DisconnectTip and TestBlockValidity classify them: a mismatch rejects the snapshot through the new ChainstateManager::HandleSnapshotStateMismatch, a read error stops the node like a block ConnectTip cannot read (AbortNode), and anything else is state.Error, so the chainstate stalls instead of invalidating the block. ConnectTip announces that error through BlockChecked like any other connection result, so submitblock reports it and the relaying peer is neither punished nor credited. Mempool and mining callers of CheckSpecialTx fail locally, the asset-unlock credit-pool precheck passes the error on, and mempool acceptance (single, package and test-accept) turns any local Evo error into an error result, dropping the coins it loaded like any failure. A package is refused with the mempool error evo-local-unavailable, which testmempoolaccept reports as its package error and submitpackage as its failure. HandleSnapshotStateMismatch is extracted from the MaybeCompleteSnapshotValidation rejection lambda, and flushes both chainstates first so it can also run mid-connection. When it does, ConnectTip and DisconnectTip keep the BLS scheme it set for the again active background chainstate instead of restoring the rejected one's. A stall is typed: BlockValidationState carries a LocalEvoStateResult (unavailable data, a binding fault, or a mismatch that could not reject the snapshot), queried with IsLocalEvoStateError(). It is an error, never invalidity, and is distinct from storage errors and from a rejected snapshot. The chainstate remembers the blocks it stalled on (every one: they are all in the block index) and refuses them at once on later attempts instead of revalidating them, until the node restarts; the first stall logs an error and sets the misc warning. Without a snapshot, Evo state is missing only because the block data it is derived from was pruned, and the warning says so and suggests -reindex. Completing snapshot validation does not make the background chainstate's state readable to the snapshot chainstate before the next startup promotes it, so a chainstate still stalled then is told to restart. Candidate selection passes over the branches that need the stalled block connected (or disconnected), so a usable branch with less work still advances the chain; ActivateBestChain reports the stall when only it keeps a chain with more work out of reach. A stalled block found invalid or conflicting with a ChainLock since obstructs nothing: it is forgotten, and a branch through it is removed like any unusable one. Callers that treated any failure as fatal no longer abort on a stall: ActivateBestChain announces the blocks it connected before stopping, ActivateBestChainStep returns the disconnected transactions to the mempool without AbortNode, block import continues with the other chainstates and the rest of startup, LoadExternalBlockFile keeps importing for the other chainstates, ChainLock enforcement retries on its next run, and InstantSend keeps the conflicting blocks a lock could not invalidate and retries them after the next tip or ChainLock change instead of asserting; an invalidation that is only temporary, for a lock without its transaction, clears the failure flags it set however it ends. A stall partway through a rewind keeps the blocks already disconnected and announces the tip reached: InvalidateBlock and ChainLock conflict marking finish and announce what they disconnected, and ActivateBestChain never releases cs_main with the chain rewound below the candidates it passed: it lists them again, as InvalidateBlock does, and selects again with the stall remembered, so it normally reconnects the chain it started from before it reports the stall. A block whose disconnect stalls stays in the chain, so the best header (and best invalid block) adjusted for it are restored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An EvoViewToken names the Evo state one operation works on: an EvoDB, an identity, a generation and the snapshot base. EvoViewScope binds the calling thread's accesses on that EvoDB to a token until it goes out of scope; scopes nest and restore the enclosing binding on exit. A transaction the same thread holds open on the EvoDB also binds it, and a scope and a transaction must agree: a conflict in either order raises EvoViewBindingError. CEvoDB::CurrentView() resolves the open transaction, then the innermost scope, then the default identity. Nothing binds scopes yet, so behavior is unchanged: unbound access still resolves to the default identity. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…th journaled promotion
The snapshot chainstate shared the NORMAL keyspace. Its records above the
snapshot base (MN lists, mined commitments, credit pools) stayed readable
by NORMAL after the snapshot was rejected, so the background chainstate
could consume them instead of deriving its own state, or abort on a
WriteDerived mismatch.
Every key the SNAPSHOT identity writes is now stored as ("S",
generation, key), and its reads and iterators see only that namespace:
there is no NORMAL fallback, and NORMAL iterators step over all
generations. Like the database iterator it wraps, a namespace iterator
reports read errors (HasError). WriteDerived therefore compares only
within its own view. CEvoDB gains WriteImmediate for view-aware writes
outside a transaction, unsynced like the raw write it replaces;
StoreSnapshotForBlock uses it, and under the snapshot view at or below
the base it only compares a carried quarter and neither persists nor
caches an uncarried one. The remaining raw-DB users (DMN migration and
repair, quorum cleanup) are marked NORMAL-only lifecycle operations.
Generations carry an unprefixed lifecycle record (unpublished,
published, rejected, promoting; base; tip; local schema version) and
the EvoSnapshotConsensusConfig fingerprint of the consensus and
deployment configuration they were published under. Startup collects
unpublished and rejected generations, refuses a published one whose
fingerprint no longer matches, and refuses dual-chainstate markers that
predate isolation.
Rejection marks the generation rejected in the same synced batch that
discards the markers. The rejected record stays until the snapshot's coins
directory has been invalidated, and only then is the generation collected,
so a crash in between still finishes the invalidation at startup. Promotion writes a journal
before the first coins-directory rename, merges the generation into
NORMAL keys with a resumable cursor, collects it, and publishes the final markers
atomically with removing the journal. Startup finishes a journaled
promotion; without a journal, a completed coins swap can only be an
interrupted invalid-snapshot cleanup and is discarded.
LevelDB skips a table block that fails its checksum and its iterator
stays valid at the next one, reporting the error only through its
status. Promotion, collection and generation loading check it before
each key and at the end, and stop with their journal and records in
place: otherwise a promotion could publish its tip without the records
it never copied, and a collection erase the record of a generation it
did not finish.
Every key a generation is seeded with is recorded in its seed manifest
with its family's merge policy (CEvoDB::WriteSeed). Promotion never
copies a seeded key into NORMAL, which derives that state itself. Where
NORMAL holds the same key, a masternode list must be canonically equal
(the same masternodes, internal ids and registration counter, however
the maps are laid out) and a byte-deterministic record byte-equal;
snapshot-only scaffolding is never compared, and a policy this version
does not know fails the promotion. An unseeded key comes from a block
above the base, which NORMAL never connects while the generation is
live, so one NORMAL also holds fails the promotion as well; only the
LLMQ best-block singleton is superseded.
A refusal for a changed consensus configuration names the generation,
its base block and height, its phase, that the changed component is
unknown because only the aggregate fingerprint is recorded, and how to
recover. It is made before startup changes anything on disk.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Isolating the snapshot generation's records is not enough while the managers keep one cache for both chainstates: a list, quorum member set, mined-commitment location or deployment state derived under the snapshot view at a block the background chainstate also reaches was served to NORMAL lookups, which then never derived their own. EvoViewCache keeps one partition per Evo view (EvoDB, identity, generation). Every cache keyed by a block or quorum hash now resolves the calling operation's view and uses only that partition: the MN-list lists, diffs, tip and initial-snapshot anchor; quorum snapshots; credit pools; MNHF signals; mined-commitment locations and the qc-hash caches; the quorum member caches; the quorum manager's quorum, scan and commitment-scan caches; the versionbits and warning caches; and the previous-coinbase ChainLock statics. The versionbits partition and the EHF signals SignalHeight reads both come from the calling thread's view. Caches of data parsed from block bytes and the vvec/skshare stores stay shared. Retiring a generation drops its partitions at the next lookup and leaves NORMAL's, so a failed snapshot load no longer empties the active masternode list until the next block. Promoting a generation, like opening a CEvoDB, bumps an epoch that drops every partition. DeploymentActiveAfter/At through a ChainstateManager decide a buried deployment by height alone, without resolving a view. The MN-list cache of a view is pruned against the higher of its tip and the highest block it processed: a view stops receiving tip notifications once its chainstate is no longer the active one, so a stale tip would keep its cache from ever being pruned. A snapshot view has no quorum history at or below its base once DIP3 is active there (a base at or above the DIP3 height), so lookups that would need it raise BlockDataUnavailableError instead of answering from partial data: a mined-commitment scan that comes up short (GetMinedCommitmentsUntilBlock, and the rotating per-index scan on a miss), a HasMinedCommitment miss for a quorum at or below the base, and the coinbase ChainLock read of a block at or below the base whose body is not stored. EHF signals derived from block bodies are written only inside a block-scoped Evo transaction, on a cache hit too, so lookups from mempool acceptance, mining or RPC leave the view clean. Adds CDeterministicMNManager::SetListSnapshotMissHookForTesting, which reports lookups that miss both the cache and the stored lists, and CachedListHeightsForTesting, for the partition and pruning tests and the stall tests that need one lookup to fail. With the hook, a test also rejects a snapshot in the middle of a block connection and checks that the process-wide BLS scheme stays the background chainstate's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Evo accesses resolved their view from an open transaction or, failing that, the active chainstate. Work for the background chainstate outside a transaction, and work handed to another thread, silently used the active (snapshot) view instead. ForceSignalDBUpdate, for one, opened a NORMAL transaction at the active snapshot tip and flushed the NORMAL root. Every Chainstate method that reads Evo state (ActivateBestChain, ConnectTip, DisconnectTip, ReplayBlocks, LoadChainTip, RollforwardBlock, FlushStateToDisk, InvalidateBlock, EnforceBlock), TestBlockValidity, VerifyDB and the per-chainstate startup loops now bind that chainstate's view. ForceSignalDBUpdate takes the intended chainstate's view and uses its identity for the transaction and the root commit. Signals that view cannot rebuild are left to be rebuilt on demand when it is a snapshot view, so startup can still reach the promotion of a validated snapshot; NORMAL, which lacks history only when it was pruned, still fails startup, as does a body that cannot be read. DMN repair binds NORMAL. Active-chain dispatchers bind the active chainstate's view per operation unless the thread is already bound, with a single view too, so a turn that a snapshot publication overtakes finishes in the view it started in. While a snapshot generation is live, the view is read under cs_main, together with the active tip where the turn needs one (ActiveEvoViewScope::Tip()), so a turn never pairs one chainstate's view with another's tip. With a single view there is nothing to pair, and a turn that needs no tip binds without waiting for cs_main. The dispatchers are: RPC execution, each peer's message-handler turn (its queued GETDATA requests, its next message and its sends), the miner, mempool admission, Evo-reading interfaces calls, each node-scheduler task and each ChainLock, DKG and InstantSend work round. A failing work round, the LLMQ signing loops' and each DKG phase handler's included, is logged once per operation, view generation and reason, and its recovery once the operation has gone a minute without failing. A local Evo state error in a message handler drops that message, like any exception there, without punishing the peer; one from the peer's queued GETDATA requests ends receiving for that turn. Neither ends the message-handler thread, and the peer's sends (including its discouragement and ping timeout) still run. GETDATA requests leave the queue before they are served, so a failed lookup neither serves earlier requests again nor keeps the peer's later messages waiting; it is answered with NOTFOUND together with the requests found missing before it, and the requests after it are served on the next turn. The node scheduler and REST start before the block databases load, and a load retry replaces the Evo database, so they bind only to the one startup publishes once loading is final (and withdraws before destroying it, after they have stopped); until then their work runs unbound. RPC is held off by its warm-up until then. A chainstate load retry first lets the validation events the failed attempt queued run, since they are bound to the databases it then releases. Queued validation events and LLMQ worker-pool tasks capture the enqueuing thread's binding and reinstall it when they run; a task bound to a retired generation, or whose binding cannot be installed, is dropped, and what its enqueuer claimed for it is released. A local Evo state error in one subscriber to a queued event stays with that subscriber, and the others still receive the event; synchronous signals still pass it on to their caller. Either way the subscriber's registration count stays right, so it can still be unregistered. Every GUI- and wallet-facing interface operation that reads Evo state (the EVO provider operations, Evo, governance and LLMQ reads, isV24Active and listMNCollaterials) binds the active view and contains evo::LocalStateError at the interface boundary: it logs once per operation and generation, sets the misc warning, and reports failure through the operation's own channel. Operations without one now return std::optional (or, for GovernanceInfo, an optional fundingthreshold), and their Qt and wallet consumers keep their current data, show that the masternode state is unavailable, or refuse the action, instead of treating it as data. getPlatformQuorum runs Q's lookup in the bound view as well, and answers UNAVAILABLE when it cannot, or when a snapshot was published after it bound its view. While the masternode list is unavailable, the wallet keeps outputs it cannot classify out of automatic coin selection, CoinJoin's included, and classifies them again on the next block. A block builds its wallet batch only for that work or for a block it scans. A chainstate's remembered Evo stall lapses once its view changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With two chainstates, an Evo access that no transaction or scope binds says nothing about which chainstate it serves. Resolving it to the default identity silently computed one chainstate's state from the other's data. While a snapshot generation is live, an unbound read, write, iterator or cache lookup, and one bound to a retired generation, now raises EvoViewBindingError before touching any state. The check is a plain throw, so it holds in release builds. Manager entry points that take an explicit chainstate (ProcessSpecialTxsInBlock, UndoSpecialTxsInBlock and the quorum block processor's ProcessBlock and UndoBlock) refuse to run unless the calling thread is bound to that chainstate's view, raising EvoViewBindingError on connect and undo alike, so the chainstate boundaries classify it as a local stall. With a single chainstate the NORMAL default is unchanged unless -checkevobinding is set. A binding fault is a local error everywhere: state.Error during validation, RPC_INTERNAL_ERROR over RPC, an aborted operation for P2P and the miner, a logged and ended task for queued and scheduled work, and an initialization failure at startup. It never becomes an empty success, consensus invalidity, a peer penalty or a process abort. A new test extends the snapshot chain past DIP3 and V20 activation and the first disk-snapshot period with a registered masternode, and checks that the real MN-list, credit-pool and EHF state it writes stays in its generation and its cache partitions, and that NORMAL rebuilds all of it after the snapshot is rejected and the node restarts. -checkevobinding (default: on for regtest, and forced on in the unit test fixtures) applies the same rule with a single chainstate: an unbound access raises EvoViewBindingError before any snapshot exists, so tests and regtest exercise the bindings everywhere. Test code that drives the node the way a dispatcher does binds like one: NotifyMNListTip delivers a tip notification in the active view, tests and fuzz targets that drive a peer through the test connman bind its turn as ServicePeer does, and test bodies that call into the node directly open an active-chain turn. The interfaces sweep covers getPlatformQuorum. The release note records the externally visible changes: RPC_INTERNAL_ERROR for local Evo errors, the package error, the read-error shutdown, the pruned-data wording, the stall that replaces invalidity on a pruned node, and strict binding on regtest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pshot base During dual operation a quorum based at or below the snapshot base is in history the snapshot view does not have, and selecting it throws a local Evo state error. That error escaped BuildVerificationBatch, ended the work round, and dropped every lock FetchPendingLocks had already taken off the queue, so one such lock cost up to 31 others. Catch it per lock instead. The lock is left out of the batch and of the results: it cannot be verified in this run, and its peer is not to blame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…base as complete RequireSatisfiedScan treated any short height-index scan under a snapshot view based at or above DIP3 as needing the missing history. A type that was not enabled up to the base has no commitments there, so its scan is complete, but it still threw: a CbTx quorum-root check over such a type stalled the snapshot chainstate until promotion. Require the type to be enabled at the end of the missing range or at DIP3 activation. Deployment-gated types stay enabled once enabled, and the types DIP0024 retires are enabled from DIP3 on. The check runs before the EvoDB lock is taken, because deployment state can read EvoDB. The non-throwing mined-commitment scans that getPlatformQuorum uses (ScanMinedCommitments[Indexed]UntilBlock) report a walk that ends short in history the view lacks as incomplete by the same rule, so a quorum that may have been mined at or below the base is UNAVAILABLE there, not INVALID. A contradiction still leaves out the entries it implicates, as in a full view. A quorum mined below the base is a block of the active chain below any later height, so it stays UNAVAILABLE once a window above the base has left it behind, too. The base is checked first, and the DIP3 ancestor is only looked up when that check fails: a chain that does not reach genesis, such as the testnet-height chain quorum_proofs_tests builds, has no such ancestor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and view GUI models poll the wrapped operations, and every failed poll set the misc warning again and notified the GUI. Raise it together with the log line, once per operation and active Evo view. On a node without a snapshot it says that block data was pruned and suggests -reindex. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ction Startup recovery of a persisted snapshot rejection renamed chainstate_snapshot to chainstate_snapshot_INVALID. When an earlier rejection had left that directory behind, the rename failed and startup aborted until the user removed one by hand. Keep the earlier directory, which may be the one the user was asked to report, and durably delete the interrupted one instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…is not saved When HandleSnapshotStateMismatch could not sync EvoDB or persist the rejection, the snapshot generation stayed bound and the default Evo identity stayed SNAPSHOT, although the IBD chainstate was active again. For the rest of the run, dispatches bound to the active view read the rejected generation, and so did every operation that had bound it before the rejection: a DKG round waiting at its start for background validation, a message-handler turn waiting for cs_main, a queued event or task. In that case, revoke the generation for the rest of the run (CEvoDB::RevokeSnapshotGeneration). In memory it is retired as a saved rejection retires it: views bound to it no longer resolve, the default identity is NORMAL, the node no longer counts as running two chainstates, and what the generation has not written is dropped. Its records stay on disk, so startup recovery still rejects it, and until then no new generation can be created. A test makes the rejection fail to persist through a new CEvoDB test failpoint while a DKG-style round bound to the snapshot view waits at its start. The round resumes and fails instead of reading the rejected generation, active-view reads see NORMAL, and after a restart startup recovery rejects the generation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ap work The v3 Evo section carried a fixed rotation horizon (h-C..h-4C) and treated parameter-derived counts as maxima. A carried safety quorum from h-2C reads quarters down to h-5C, the next cycle reads the quarter of h itself, and a quorum index whose DKG keeps failing has carried commitments many cycles back, so the snapshot chainstate either computed different members from too few quarters or refused a base the exporter accepted. Bump EVO_SNAPSHOT_VERSION to 4 and reject v3. PlanEvoSnapshotDependencies() derives, for one type at base B from the carried commitments, the exact set of rotation quarters (S_T), work blocks (W_T) and bootstrap entries: the work blocks of h and N that nothing carried references. The builder, the chain validator and later the completion rebuild all use it. The builder fails when an applicable quarter is missing (the no-store paths of the ordinary computation), and the validator requires exact set equality instead of the old tolerance, plus a quarter only where the ordinary computation stores one. Both refuse a base whose next cycle N reads work data at or below B while some activation input (a buried deployment, the DIP0024 quorum height, or a versionbits period boundary) can still change before N: a type enabled, or switching rotation, only after B would need inputs nothing carries. QuorumSnapshotData gains bootstrap_work (at most two entries, ascending by cycle height); rotation quarters are bounded by 3K+4, historical lists by the per-type 4K+6 / K+2 work-block bounds, and modifiers by their sum. Seeding APIs take the target view token and write inside its open transaction through CEvoDB::WriteSeed, which records each key in the generation's seed manifest: masternode lists as MN_LIST; mined commitments, their height index, rotation quarters, the credit pool and the MNHF signals at the base as BYTE_DETERMINISTIC; score modifiers as SCAFFOLDING. SeedEvoSnapshotState() seeds a decoded section. Under the snapshot view a list lookup at or below the base that misses the carried full lists raises BlockDataUnavailableError instead of walking absent diffs or guessing an empty list. Score modifiers prefer a seeded exact value when the work block's data is unavailable and check it against the block otherwise. Also carried over from the earlier dashpay#7593 head: GetDeterministicMNListHash() and the background base-list marker use the canonical codec hash. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pool after block X reads the {credit_pool, unlocked} pair of block X - P
(CreditPoolPeriodBlocks()) from that block's body. A snapshot chainstate has no
bodies at or below its base, so constructing the pool of B + 2 already failed
with failed-getcbforblock-read.
v4 gains credit_history: the window entries of [max(0, B - P + 1), B], oldest
first, built from the block bodies. A pre-DIP3 block contributes {0, 0}; an
unreadable or malformed DIP3-active block is an export error. Context-free
validation requires money-range amounts, a checked sum equal to the pool's
latelyUnlocked, and a last entry equal to the pool's balance. The chain
validator requires exactly min(P, B + 1) entries whenever DIP3 is active at B,
zero entries before v20, and an empty base pool while v20 is inactive.
CCreditPoolManager::SeedHistory() stores the entries in the snapshot view, as
snapshot-only scaffolding that promotion never copies into NORMAL.
ConstructCreditPool() resolves the distant block through a view-bound reader:
at or below the base in the snapshot view it uses the carried entry (missing:
unavailable; compared against the body when that is readable: mismatch), never
inserting it into the cache of entries parsed from bodies. Above the base and
in NORMAL it reads the body as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hainLock checks CheckCbTxBestChainlock() compares a block's coinbase ChainLock with the one its parent's coinbase carries. For the first block above a snapshot base the parent is the base itself, whose body the snapshot chainstate does not have: the ordinary reader then answers "no ChainLock" and B+1 may drop or regress the certified height. The section also had no anchor to the base header of its own. v4 gains the base coinbase (length-delimited, at most 1,000,000 bytes checked before allocating, exactly one canonical transaction and nothing after it) and its index-0 merkle branch (at most 32 nodes). Context-free validation requires a coinbase special transaction larger than 64 bytes whose CbTx height is the list's, and no MN or quorum state without one. The chain validator requires the coinbase exactly when DIP3 is active at the base and proves it through the branch against the base header. VerifyEvoSnapshotBaseBlock() is the load helper: the same proof, then the CbTx checks over a payload decoded with the BLS scheme in force at the base (DecodeCbTxPayload()), independent of the scheme the loading node happens to be in. CSpecialTxProcessor::GetCoinbaseChainlock() returns certified, none or unavailable. In the snapshot view it serves the base from the seeded coinbase and answers unavailable for a v20 block without data; NORMAL keeps the legacy "none". CheckCbTxBestChainlock() uses it in place of the GetNonNullCoinbaseChainlock() fallback and raises BlockDataUnavailableError on unavailable. The miner's CbTx ChainLock choice uses the same reader, so a template built on the base carries the base's ChainLock forward instead of a null one that validation would reject. The asset-unlock status RPC uses it too. The mnlistdiff/QRINFO quorum ChainLock info, which cannot reach the processor without an include cycle, reports a work block without a body in the snapshot view as unavailable (no peer penalty) instead of serving a null signature. GetNonNullCoinbaseChainlock() decodes with the scheme in force at the block it reads, and Validate() applies the decoder's coinbase size cap. SeedEvoSnapshotState() now takes the special-tx processor, which owns every manager it seeds. Both coinbase readers take a block's status and position in one cs_main scope, and a block this node has but cannot read raises BlockDataReadError instead of reading as "no ChainLock". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e stage below it The builder exported GetSignalsStage(B), the record already filtered for the block after B. Seeding it as DB_SIGNALS_v2(B) made the snapshot chainstate filter it a second time, and a signal that the deployment table filters at B + 1 (a bit reused by a deployment whose window has not started) vanished from the carried record while NORMAL keeps it, so completion could never agree. Export GetForBlock(B), the raw record, which SeedSignals() writes as DB_SIGNALS_v2(B); GetSignalsStage(B) then filters it for B + 1 exactly as NORMAL does. GetForBlock() becomes public for the builder. Below the base the snapshot view holds no record, and a node that loaded the snapshot without the bodies could not rebuild one, so a versionbits query of an EHF deployment there (the next cycle's context check at the base, StateSinceHeight, getdeploymentinfo) failed. A record is the parent's stage plus the block's new signals, and the stage drops a signal only when a deployment on its bit starts after the signal's block: a rule on the signal alone. So for a block X on the base's chain, the base record's signals mined at or below X, filtered, are X's own stage. The snapshot view answers GetSignalsStage(X) that way, never caching or writing the derived record, and never returns it from GetForBlock(), whose raw record still needs X's body. A block off the base's chain (a side branch forking below the base) does not inherit the base's signals: it is rebuilt from its own branch on top of the derived record of its fork point, or is unavailable. A view that carries no base record (a snapshot without an evo section) rebuilds from the blocks as before. The snapshot view also never writes any signal record at or below its base other than the seeded one, which promotion would find overlapping NORMAL's, and SeedSignals() refuses a signal mined above the base. Versionbits evaluates the signal height after the always- and never-active returns, so a deployment that cannot change state never reads it. The exporter runs outside any block transaction, where a rebuilt record is held only in the cache; it exports that record all the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…napshot view A v20 score modifier commits to the work block's coinbase ChainLock. When the block cannot be read, the ordinary calculation silently uses the block-hash modifier instead, and so did the prediction path (ComputeQuorumMembersFromWorkBlock), even where the snapshot carries the exact value. In the snapshot view that computes different members than the network without any error. One resolver, keyed by (type, work block), now serves GetHashModifier(), quarter decoding and the prediction path: it returns the seeded value or the block's own, checks them against each other when both exist (SnapshotStateMismatchError), and raises BlockDataUnavailableError when it has neither, in every view: NORMAL no longer guesses the block-hash modifier for a work block it pruned. A work block the node has but cannot read raises BlockDataReadError in either view; the resolver and the exporter's ExactWorkModifier() read its status and position in one cs_main scope. llmq::utils::GetQuorumModifierForWorkBlock() exposes the resolver. A quorum object computed its members eagerly, so without that fallback a pruned node could no longer build one for an old quorum, and would stall on an MNHF signal it signed: LLMQ_400_85's active quorums reach further back than a pruned node keeps. Where NORMAL cannot resolve the work block's modifier (WorkBlockModifierUnavailableError), a quorum is now built from its mined commitment alone. It verifies signatures with the commitment's key, and CQuorum::Members(), which replaces the public member vector, raises BlockDataUnavailableError for it, as do IsMember(), IsValidMember() and GetMemberIndex(), so nothing reads a guessed or empty list. Such a quorum takes no verification vector or contributions from QDATA, is not queued for key-share warming, and has no contributions for the old-data cleanup to keep. Commitment verification and quorum rotation still need the members and stay unavailable without them. The quorum health figures of the GUI and quorum listextended count such a quorum's commitment member slots. A release note describes the change for pruned nodes. The snapshot view raised unavailable for any quarter missing at or below the base. Only an applicable one (rotation and the type enabled at that cycle) is an input the ordinary computation stored; a missing inapplicable quarter keeps the legacy break there, as NORMAL does, so a cycle near rotation's start still reconstructs. Both the resolver and the exporter decode the work block's CbTx ChainLock in the block's own BLS scheme. A background chainstate connects pre-v19 blocks with the process-wide scheme set to legacy, and a member computation on another thread then read a v19 ChainLock as absent and fell back to the block-hash modifier. The member-computation fixtures live in evo_snapshot_tests.cpp, so the tests do too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScanQuorums() and ScanCommitments() read the mined-commitment index. In the snapshot view that index holds only the carried commitments at or below the base, so a scan reaching past them returned a short or (with rotation) a different vector, and quorum selection for a ChainLock or an asset unlock silently disagreed with the network. A carried scan sequence (the type's history without rotation, one quorum index with it) ends either in END, when the canonical selection carries fewer records than it takes (K, or two per index), or in UNKNOWN. A missing index is END; a rotating type without rotation at the base has an empty indexed family. TraverseScanSequences() replays the ordinary selection over such sequences (newest first, or rank by rank in ascending index order) and proves the identical vector, including a legitimately short history, or reports that an UNKNOWN end was reached first. Export and load apply one gate: every ChainLock verification scan for the heights B + 1 may certify (from the base's certified height, or from genesis when it certifies none; none at all without v20 at B + 1) with n = a and SIGN_HEIGHT_OFFSET, and the Platform asset-unlock scan at B with n = a + 1, must be proven from the section; the exporter also compares the proven vectors with its full index. Coverage only grows with the cutoff for a fixed type and count, so only the first start of each enablement context is proven. Bases that fail are refused. At runtime, the snapshot view's scans call CQuorumManager::ProveScan() over the view's commitment index, whose ends are recorded locally at seeding (ScanCoverageRecord, never on the wire). Every request proves its own prefix: cached prefixes are proven ones, a shorter cache entry is re-proven from the original cutoff instead of continued, there is no keepOldConnections warm-up to fail a covered smaller request, and an uncovered prefix raises BlockDataUnavailableError, as does a proven quorum this node fails to build. Records above the base extend the sequences. CQuorumBlockProcessor::SequenceEnd() is the one answer to how a view's history of a scan sequence ends, and both ProveScan() and the height-index primitives' short-scan gate (RequireSatisfiedScan) read it. It is END: - in NORMAL, and in a snapshot view whose base precedes DIP3, which hold every commitment mined on their chain; - for a family the type never writes. A type is indexed exactly when it rotates, and a rotating type is enabled exactly when rotation is at its cycle base, so a rotating type read non-indexed before DIP0024 (a base before activation) is empty; - where the seeded coverage says so: a type the payload omits (disabled at B with no history, e.g. one enabled only later), an under-filled type, or a quorum index that never formed one. A short scan ending there is an answer, so B + 1 reads what NORMAL reads. Anywhere else it stalls, as a need deeper than the payload carries does. END rests on the carried counts, which only the mandatory evo-hash pin binds. RequireSatisfiedScan keeps the header check that a type enabled at neither end of the missing range has nothing in it; both checks now run with the EvoDB lock released. The shared quorum and commitment scan (ScanMined) and ProveScan() normalize the start with one NormalizeScanStart(), and in the snapshot view the shared scan takes the proven path for both result types. Lookups by quorum hash follow the same rule. An MNHF signal may be signed by any historical quorum of its type. In the snapshot view HasMinedCommitment raises BlockDataUnavailableError (the chainstate stalls) for a quorum at or below the base without a commitment, unless headers and the carried sequences prove it was never mined: not on the base's chain, not a quorum height of an enabled type, or no older than the oldest carried record of an ending or newest-first sequence. Such a miss stays a plain miss, so a block relying on that quorum is still invalid. RequireSatisfiedScan's rule is ShortScanIsComplete(), which the non-throwing scans getPlatformQuorum reads (ScanMinedCommitments[Indexed]UntilBlock) use too: a walk that ends short is complete exactly when the throwing scans would not throw, so a sequence whose recorded coverage ends lets a hash that is no block of the active chain below the height be INVALID, and one whose end is unknown keeps it UNAVAILABLE. getPlatformQuorum over a seeded generation: carried commitments are found at the base; a window reaching past the oldest carried one stays unavailable for quorums NORMAL would find there and for unmined ones alike; a window full again from commitments above the base, or a history whose seeded coverage ends, makes an unmined hash invalid once a ChainLock anchors it, while a carried quorum that has left the window stays unavailable; after promotion NORMAL decides. Promotion, also interrupted and resumed, copies no seed-only record (modifiers, credit window, base coinbase, scan coverage) into NORMAL. With the quorum keys, the key-only quorum test also checks that a pruned node accepts an MNHF signal its quorum signed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dumptxoutset builds the evo section from the dumped chainstate's own view at the same cs_main-pinned point as the coins cursor, appends it after the last coin as [marker][EvoSnapshot], and reports a summary of it. A base the section cannot represent (for example a needed rotation quarter is missing, or the ChainLock scans cannot be proven), or a section failing the chain checks a loading node applies, is an RPC error. PreflightEvoSnapshot() is the one chain check export and load share: ValidateEvoSnapshotAgainstChain() and, once DIP3 is active at the base, VerifyEvoSnapshotBaseBlock(). It reads the calling thread's Evo view, so the exporter runs it on NORMAL and the loader in the generation it seeds. Snapshot load decodes a trailing section and requires it to end the file; a UTXO-only file still loads. The section is not used yet. Until a later commit compares the whole section at completion, a DIP3-active snapshot is never validated or promoted: completion returns the new EVO_DATA_UNAVAILABLE, which startup treats like a skip. feature_assumeutxo_dash.py dumps a rotating-quorum chain at a base before its own cycle's mining window and at the block before the next cycle, and checks the carried rotation cycles and bootstrap entries against the dependency rule, the credit-pool window length, the base coinbase, and that the reported hash is the SHA256 of the section bytes in the file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…data A DIP3-active snapshot base now requires the evo section and a non-null pinned evo hash that the decoded section matches, and a non-null pin requires a section. The pin comes only from chainparams or, on regtest, from the new -assumeutxodata=<height>:<hash_serialized>:<evo_hash>:<nchaintx>:<blockhash> option, never from the file. Each such entry is bound to its base block, and the option authorizes nothing on other networks. A pre-DIP3 UTXO-only snapshot loads as before. EvoSnapshotTestSetup is a deterministic DIP3/v20 chain with four fixed-key masternodes whose base at 219 is pinned through -assumeutxodata. The pin is the SHA256 of the section as a test-side assembler writes the format table field by field, and the section holds what the fixture implies (its masternodes, quarters and credit window). A null pin, a UTXO-only file, and a section with a substituted field that every later check accepts are each refused before any generation is created. The honest snapshot activates, and with background validation already at its base it is still not validated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…publish After the pin check, snapshot load seeds the section into the new unpublished generation and, in that generation's view, runs the chain checks export runs (PreflightEvoSnapshot(): exact equality with the planned dependency sets, provable ChainLock and asset-unlock scans, the credit-pool window, and the anchor to the base header through the coinbase merkle proof and CbTx). Deployment state at or below the base then comes from the carried signal record, never from history a node with only the headers lacks: such a node could not load a base whose next cycle reads work data at or below it. Load then dry-runs the quorum member computations the snapshot chainstate may need at or right after the base from the seeded state alone: every carried quorum, the base's cycle, and the next cycle when its work block is at or below the base (refused if a v20 work block's members cannot be computed). Once DIP3 is active the snapshot-side base MN-list marker is the carried list's. The active chainstate keeps running whenever activation releases cs_main. Preparation therefore first flushes it, coins before Evo state, so the NORMAL records it then commits never reach disk ahead of the coins, and registers the base before releasing cs_main: until the switch the active chainstate records the base's MN list as the background chainstate would. Activation then waits, at most a few seconds, for the dispatch turns of other threads to end (so its caller must not hold cs_main), and switches in one cs_main hold: the mempool must still be empty, the active tip must be the base or have less work than it (an equal-work sibling at the base height would be loaded as the background tip at the next startup, whose completion would then reject the snapshot), and no block of the base's chain may be invalid or conflict with a ChainLock; only then are the generation published (with the markers, the configuration fingerprint and the encoded section, retained in the lifecycle metadata), the base recorded in the snapshot directory and the block index faked, and the snapshot view told its tip before it becomes the default. A turn still running in NORMAL finishes there. A turn counts itself without a lock; one that ends takes the waiter's mutex and notifies only while activation waits. Every failure before the switch, from opening the snapshot's coins database on, undoes the generation, the caches and the directory; none leaves faked block-index state behind. Rejection, completion and cleanup flush each chainstate's coins, checked, before syncing its Evo state. A local Evo state error while populating (state this node cannot derive or read) refuses the snapshot as bad input does, instead of escaping activation. On restart, an unpublished generation is collected as before; a published one is used only while its retained section still decodes and matches the pin of its base. evo/snapshot_chain.cpp joins the kernel sources, since validation now calls it. A DIP3-active snapshot is still never validated or promoted. The unknown-version warning check, where it needs Evo state the chainstate lacks, is skipped instead of escaping block connection. The "snapshot state unavailable" warning no longer takes the single misc warning slot. Each stalled chainstate and each failing interfaces:: operation has its own; a stall's stays with its chainstate (at completion it says to restart), an operation's goes on its next success. Setting or clearing one notifies the GUI only when the warnings change, so an operation the GUI keeps polling while it fails notifies once. A simulated restart in the tests drops the Evo view caches, as a real one does: their tips point into the block index it unloads. The failed-open case meets a leak in CDBWrapper's constructor: it throws before its destructor could free the LevelDB cache, filter policy and logger GetOptions allocated. LeakSanitizer ignores those allocations. Tests, on the deterministic DIP3 fixture, and on a variant whose quorums really form (commitments signed with the fixture's operator keys and a quorum key it keeps, ChainLocks and asset unlocks signed with that key); a node "with only the headers" also starts with an empty NORMAL EvoDB: - a node with only the headers loads the snapshot and connects, across a restart, the blocks a synced node mined up to B+P+2, through the next cycle; members of the base and next cycle match the synced node's; - such a node loads the section a synced node dumps at every base of a 24-block cycle, including those inside the next cycle's work window; - a work-block body the node has but cannot read refuses the load cleanly; - at the block before the next cycle, the bootstrapped next cycle's members are predicted from the seeded state and match the synced node's; - a crash after seeding, before or after publication, leaves no live generation after restart, and NORMAL carries on; - right after activation the snapshot view serves the base's MN list as its tip list; - the switch is refused when the active chainstate meanwhile took a mempool transaction, the base's chain became invalid, the chainstate moved to a sibling of the base with the same work, or it went past the base, and a snapshot coins database that fails to open fails the load; each time the node is left as before, and the load then succeeds at the base; - the switch waits for a dispatch turn that ends within the bound; one that outlives it carries on in NORMAL without failing or stopping the node; - a load refused by its dry run on a node with only the headers marks no block assumed-valid and fakes no transaction count; - a crash right after preparing restarts without an Evo/coins mismatch, although the chainstate had connected blocks since its last flush; - a carried credit-window record that has gone missing stalls the chainstate with a local error, without marking the block invalid, and raises a warning that leaves others in place; - a retained section that no longer matches the pin is refused at startup; - with real quorums, the next cycle's signed commitments, verified against members from the carried quarters (including the base cycle's), and the ChainLocks in every coinbase are accepted, and stay in the snapshot view, for a base right before the next cycle and for one five blocks after its own cycle, before the next cycle's work block; - each quorum cache the snapshot chainstate warms at or below the base (quarters, members, quorums, quorum and commitment scans, mined-commitment blocks, commitment indexes, CbTx commitment hashes, the remembered coinbase ChainLock) stays in its view: NORMAL finds nothing there or cannot tell until it has connected those blocks, then answers as the synced node did, also when the snapshot view holds a different commitment; - with a rotating index whose DKG fails for ten cycles, the members of its ten- and eleven-cycle-old quorums are computed at activation, with cold caches, as the synced node's; the ten-cycle-old quarter, only an output, is not kept, and without a quarter they need the members are unavailable; - on a node without the base body, the block after the base, building on the real ChainLock the carried base coinbase holds, connects; - withdrawals at exactly the limit, before the base and across a restart and the window edge after it, are accepted; on both sides of that edge anything above the synced node's limit, and a carried index, are rejected; - a rejected snapshot's commitments are rebuilt by NORMAL from the blocks; - NORMAL's MN-list cache stays bounded during a long background sync; - a carried record missing for a reorg or ChainLock enforcement stalls both (a local Evo state error, with the warning) without marking anything invalid, until a restart; - background validation reaching the base validates nothing yet; - an unknown bit active, and warned about, on the synced node: once that node, still holding NORMAL's warning-cache states, loads a snapshot without the bodies below its base, the snapshot chainstate computes the states itself from the headers and the carried signal record, and warns too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Completion now rebuilds the entire evo section from the state background validation derived and requires its hash to equal the retained section's, which must still equal the pin. This replaces the interim refusal to validate a DIP3-active snapshot. The rebuild and the shared preflight report why they failed: missing input, local corruption, an internal error, or a disagreement (at export, where nothing is compared, a base the section cannot be built for). Only a disagreement rejects the snapshot (EVO_STATE_MISMATCH). Missing input (EVO_DATA_UNAVAILABLE) leaves it neither validated nor rejected, with a warning that says the next restart retries. Data this node has but cannot read or finds inconsistent (a retained section that is gone, does not decode or no longer matches its pin, which startup checked; an unreadable body; EvoDB best blocks that differ from their coins tips) stops the node with EVO_LOCAL_FAILURE, and an internal error (including a missing record of the background chainstate's list at the base) with EVO_INTERNAL_ERROR; neither rejects the snapshot, and startup then fails with the same advice instead of retrying at every restart. The advice says that -reindex-chainstate discards the snapshot. A coin of the background chainstate that cannot be read, so that its UTXO set hash cannot be computed, is local corruption too. Upstream rejects the snapshot there, which would leave the node on the coins it cannot read and tell the user the snapshot was invalid; this is a deliberate divergence. A snapshot still in use that cannot be rejected stops the node rather than being kept. The inputs only block bodies provide (the v20 work-block modifiers, the credit-pool window, the base coinbase and its branch, and the base's credit pool, which folds in blocks back to the last pool on disk) are captured as the background chainstate connects those blocks, in its block transaction, or at activation for blocks it has already connected; activation refuses a base whose such body is gone. The capture plan refuses a credit window reaching below genesis, since the chain checks run only after it. The rebuild prefers the captures over the bodies. A local Evo error while capturing the base pool stalls the background chainstate as other local Evo errors do. Capturing follows the registered plan, so a block the still-active chainstate connects between preparation and the switch is captured too. Since promotion repeats completion at the next startup, the captures stay until promotion; they are also dropped on rejection, when activation fails (after flushing that chainstate's coins), and at startup when no generation is live or the one left never activated (after a crash before or after publication). No prune lock is kept for them: it would not cover what either chainstate reads, and pruning is to be refused while a snapshot is in use. The MN-list work captures and the old comparison block that the design removes do not exist on this base; the existing base MN-list marker comparison is kept. Tests, on the deterministic DIP3 fixture, with sections pinned as they are: - a mutated currentLimit, credit-pool index range, signal height, rotation quarter bit or credit-window entry passes every load gate and the snapshot chainstate's own blocks, and is rejected only at completion, with no block marked invalid; a wrong modifier is, on a node without the work-block bodies at load; - a corrupted capture is a mismatch; a missing one whose block is gone is missing input with a warning naming the retry, after which completion succeeds once the body is back; activation refuses a base whose connected input body is gone; with real quorums, completion succeeds from the captures alone once every body at or below the base is gone and the caches are cold, both for those activation wrote on a synced node (a withdrawal in the window) and for those the background chainstate of a node with only the headers wrote as it connected the blocks, across a restart; - the honest run on a node with only the headers completes across a restart and dropped caches; the background chainstate announces none of the MN-list changes it connects and its ForceSignalDBUpdate writes only its own view; the captures last until promotion; NORMAL consumers read the promoted MN list and credit pool; - a published generation is refused after a V24 min_activation_height change, naming the generation, base, phase and recovery and saying the changed part is unknown, with nothing changed on disk; so is a promotion interrupted mid-merge, whose failing recovery names the way out; both resume under the original configuration; - blocks the still-active chainstate connects between preparation and the switch, through the base, are captured and its base list recorded, so completion succeeds; - a crash right after a refused switch restarts without an Evo/coins mismatch, although the chainstate connected blocks meanwhile; - an unreadable body, a changed retained section and mismatched best blocks stop the node as local corruption, an internal error and a missing base list record as internal errors, none rejecting the snapshot; startup then fails the same way, fails as local corruption on an unreadable background coin, and promotes once the data is whole again; - with real quorums: a carried safety commitment with another public key, signature or mined block in the same window is rejected only at completion; the honest section completes, the next cycle's mined commitment is NORMAL's after promotion, and a quorum-signed signal for an unused bit mined at the base is carried raw and kept by promotion; - a crash before or after publication leaves no captures after restart; - a chainstate stalled on a missing carried record keeps its warning past completion, and after the restart that promotes the snapshot the warning is gone and the stalled block connects. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hainstate While a snapshot is validated in the background, the governance collateral index, like every index, follows the background chainstate, and reports itself synced once it reaches that chainstate's tip. Governance read that as "every block up to the active tip has been read": it stopped holding governance announcements back and processed proposals whose collateral lies above the background tip, which the index cannot have yet. The lookup failed as "Can't find collateral tx", and the honest peer that relayed each such proposal got Misbehaving(20). This happened both after loading a snapshot and after a restart while it was not yet validated. GovCollateralIndex now records the chainstate it was initialized on, in CustomInit(), within the same cs_main hold in which BaseIndex::Init() sets m_synced, and IsCollateralSynced() also requires that chainstate to be the active one. Governance therefore waits until background validation has completed and the index, restarted on the active chainstate, has caught up. With a single chainstate, and after a rejection, nothing changes. A cached flag would not do, as the active chainstate changes at runtime activation, which the index does not observe. CanSync() still decides at startup on the indexing chainstate. IsCollateralSynced() takes cs_main itself. Each caller either holds cs_main already, so the acquisition is recursive and records no lock-order pair (the collateral check, AlreadyHave(), the INV loop, CheckPostponedObjects(), governance message processing), or holds no lock that is taken after cs_main elsewhere (the sync manager's tick, the release of held announcements in SendMessages(), the RPCs). While the index is behind, CheckPostponedObjects() leaves postponed proposals alone, since a failed lookup would drop them. The new functional test cannot reach this, as mnsync stays at its governance step while the index is behind; it was verified from the source. gobject prepare and gobject submit refuse with RPC_MISC_ERROR instead of failing on the lookup. Snapshot activation can leave the index behind after ProcessMessage() checked it. The object handler checks again in the cs_main hold in which it consumes the request authorization and processes the object, and when the index is behind it drops the object without consuming the authorization or penalizing the sender. Votes need no such check: processing one never looks up proposal collateral. feature_assumeutxo_refusals.py has a peer that serves no blocks announce a valid proposal whose collateral is above the background tip: the node holds the announcement without penalty, and gobject submit refuses. Once background validation completes and the index catches up, the node requests the proposal from that peer and accepts it. The test framework's CGovernanceObject.serialize() is fixed, and msg_govobj added. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two activations could run at once: the snapshot chainstate is only installed at the end, so the check for an existing one let a second loadtxoutset through. The second one failed, at the latest on the first one's database lock, and its cleanup then rebalanced the caches, deleted the snapshot directory and abandoned the generation of the activation still running. ActivateSnapshot now refuses, before it changes anything, while another activation is running. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Promotion renames the background chainstate directory to chainstate_todelete and deletes it at the end. When that deletion failed, or the node stopped before it, the directory stays; the next startup removes it, unless it reindexes. A snapshot loaded after such a reindex then failed its promotion on the rename, and startup aborted until the user removed the directory. The leftover is of no use, so promotion removes it durably before the rename. A failure to remove it aborts the node with its own message, as a failed rename does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n a snapshot is rejected The rejection makes the background chainstate active, but its candidates held only ancestors of the base. Blocks the snapshot chain had connected above the base were missing, so the next CheckBlockIndex aborted (with -checkblockindex, the regtest default). Re-list them with RestoreBlockIndexCandidates, the rule startup applies. Inherited from bitcoin#25740; bitcoin/master still has it, so there is no upstream fix to backport. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…check in ActivateSnapshot 33c48c1 validation: Check if mempool exists before asserting in ActivateSnapshot (TheCharlatan) Pull request description: The mempool is an optional component of the chainstate manager, so don't assume its presence and instead check if it is there first. Dash: pulled forward from the later assumeutxo backports. After a snapshot rejection the active chainstate has no mempool (next commit), so a second activation in the same run must reach the "a snapshot chainstate already exists" refusal instead of this assert. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A rejected snapshot handed its mempool to the validated chainstate. The mempool's transactions were accepted against the snapshot tip, so with -checkmempool (the regtest default) the check that follows completion aborted on a child of a transaction mined above the base, instead of the node stopping with its error. After a rejection raised while the snapshot chainstate connects a block, the validated chainstate can be far below the base, and it met that mempool with historical blocks. Leaving the mempool with the rejected chainstate is not enough: a rejection raised mid-step returns through that chainstate's mempool update after its coins views were reset. Detach the mempool from both chainstates for the rest of the run. Transactions are refused with no-mempool. Nothing is removed, so no InstantSend lock is dropped through the removal notification, and DumpMempool still saves the mempool for the next startup to re-validate. This drops the handback from "validation: move the mempool to the snapshot chainstate on activation" (dashpay#7553); develop cannot reach it without the snapshot loader. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…onger holds the mempool A snapshot rejection detaches the mempool from both chainstates, and a snapshot activation moves it from the background chainstate to the snapshot one. A mempool load (importmempool, or the load at startup) that started on the other chainstate, or after the rejection, then reached AcceptToMemoryPool's assertion that the chainstate has a mempool. Stop the load instead. bitcoin/master has the same gap after a rejection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chainstatemanager_activate_snapshot deferred an InstantSend conflict on a stalled snapshot chainstate but never retried it. Keep the NetInstantSend alive and retry the deferred conflict through RetryUnresolvedConflicts: - on the next tip, while the stall persists, the retry defers it again and leaves no block invalid and the lock in place; - once a ChainLock covers the conflicting block, the retry drops the lock and leaves the chain where it is. A retry that succeeds once the unavailability clears is not reachable in a run: stalls stick until restart, and the queue is in memory. The work thread's call to RetryUnresolvedConflicts stays covered by review only: catching its removal would mean running the thread on a timer, which makes the test timing-dependent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feature_assumeutxo_dash.py checked only the evo section dumptxoutset writes. Extend it so that three cold nodes, which know only the headers, load node 0's dumps at bases 3, 23 and 18 blocks into a rotation cycle (the last inside the mining window), with the regtest default -checkblockindex: - node X refuses a mutated coin, a mutated evo section, and a pin with either hash of another dump. -reindex and then -reindex-chainstate drop snapshots whose chains connected blocks above the base, and plain IBD follows. - node R loads a section that adds an EHF signal, under a pin taken from that section. Background validation over P2P rejects it at completion: the node stops, keeps chainstate_snapshot_INVALID, misbehaves no peer and marks no block invalid. After the restart it refuses another snapshot while the rejected one is kept, then syncs from blocks. - node A takes a transaction spending snapshot coins into its mempool with an InstantSend lock from a carried quorum, and connects InstantSend, withdrawal and EHF traffic above the base. A carried platform quorum verifies the withdrawal's signature. An InstantSend lock disconnects a block A mined with a conflicting spend. With the background chainstate just past blocks before v19, a basic-scheme ChainLock is accepted with no misbehaviour. An EHF signal of a quorum the snapshot does not carry stalls the snapshot chain, also across a restart; background validation still completes in the same run, and an InstantSend lock two cycles old verifies after completion (bitcoin#31556). A masternode key is refused while the snapshot is unpromoted, and the restart that promotes it connects the stalled block. The network adds two EvoNodes for platform quorums and activates v19 at height 60, so a background chainstate can be before v19 while the snapshot chain is after it. The test now takes about 100 s, so test_runner.py lists it first among the base tests under five minutes.
… readable registeredCollateralExclusion checks that the wizard keeps its collateral candidates and shows the "state unavailable" label while the masternode list cannot be read, as it cannot under a snapshot view below the base. Check the other half: once the list is readable again, a refresh shows the candidates and clears the label.
…state after loading 9315754 test: assumeutxo: spend coin from snapshot chainstate after loading Pull request description: Submit a transaction that spends a coin only the snapshot chainstate knows (the coinbase of the first block the loading node lacks) and check that it enters the mempool and spends the coin. Dash adaptations: - The output pays the coinbase value less a 0.001 DASH fee, instead of upstream's fixed 24.99, which is specific to Bitcoin's subsidy. - The coin is the coinbase of block START_HEIGHT + 1, which n0 mines to its deterministic key; the MiniWallet transactions at EXTRA_TX_HEIGHTS spend other coins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lues 9d9a745 assumeutxo: Remove BLOCK_ASSUMED_VALID flag ef174e9 test: assumeutxo snapshot block CheckBlockIndex crash test 0391458 test: assumeutxo stale block CheckBlockIndex crash test ef29c8b assumeutxo: Get rid of faked nTx and nChainTx values 9b97d5b doc: Improve comments describing setBlockIndexCandidates checks 0fd915e validation: Check GuessVerificationProgress is not called with disconnected block 63e8fc9 ci: add getchaintxstats ubsan suppressions f252e68 assumeutxo test: Add RPC test for fake nTx and nChainTx values Pull request description: Snapshot activation no longer fakes nTx and nChainTx for the blocks below the base, and BLOCK_ASSUMED_VALID goes. nTx and nChainTx stay 0 until a block's transactions are known; only the base gets the pinned nChainTx. CheckBlockIndex() checks the exact nChainTx sum and restarts its "first missing/invalid" tracking at the snapshot base. Dash adaptations: - This removes the interim the bitcoin#27596 commit added while counts were faked: CheckBlockIndex()'s two exceptions to the monotonic nChainTx check and BlockManager::m_faked_tx_height. The monotonic check gives way to bitcoin#28562's exact sum check (73700fb), which the bitcoin#28562 commit left out, in its bitcoin#29370 form. - BLOCK_ASSUMED_VALID also covered the block index a dropped snapshot leaves behind (-reindex-chainstate, or a restart after a rejection): blocks it connected above a base whose ancestors were never validated keep CHAIN and SCRIPTS validity, which CheckBlockIndex now rejects. LoadBlockIndex lowers such blocks to the level their ancestors back up, and never below TRANSACTIONS. bitcoin/master has the same gap after -reindex-chainstate. - CheckBlockIndex() keeps tracking ChainLock-conflicting ancestors beside the invalid ones, and the candidate checks skip both. - BLOCK_STATUS_RESERVED keeps the value 256. No released version of this tree loads a snapshot, so no block index on disk has faked counts and no compatibility code is needed (upstream's compatibility note). - The loop at the snapshot switch keeps marking the chain's entries dirty but changes none of them: the BLOCK_OPT_WITNESS faking that upstream keeps there is SegWit-only. - clientversion.h is included for FormatFullVersion() in the new warnings. - Dash unit tests: the cold-importer fixture (ForgetBlocks) resets the forgotten entries as CreateAndActivateUTXOSnapshot() now does; the switch-recheck case no longer looks for the removed flag; the failed- preparation case checks that only the switch sets a count, the base's pinned nChainTx; the BLS-scheme case gives the background chainstate its next block through AcceptBlock() from the data left on disk, as a re-download would, instead of setting BLOCK_HAVE_DATA on an entry that now has no transaction count. - feature_assumeutxo.py counts the transactions of each block from its EXTRA_TX_HEIGHTS rows rather than upstream's every third block, checks the dump's nChainTx against that count, and adds the stale block as upstream. - The ubsan suppressions use this tree's file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A snapshot rejected before background validation reached its base left the base with the transaction count from the snapshot metadata, and the blocks above it counted from that. Restoring the validated chainstate's candidates then offered it blocks above an unlinked gap: the next block in the shutdown window failed FindMostWorkChain's assert, and -checkblockindex failed on the counts and on validity. Give the base and the blocks above it the state the next startup derives without the snapshot: counts only through linked parents, unlinked blocks with data back in m_blocks_unlinked, and CHAIN/SCRIPTS validity only where validated ancestors back it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2342b46 test: Add coverage for getchaintxstats in assumeutxo context faf2a67 rpc: Reorder getchaintxstats output fa2dada rpc: Avoid getchaintxstats invalid results Pull request description: getchaintxstats computed its window from nChainTx values that are unset while an assumeutxo snapshot is being validated, and reported wrong or wrapped counts. Omit txcount, window_tx_count and txrate when they are unknown, and drop the ubsan suppressions bitcoin#29370 added for this RPC. Dash adaptations: - The window count is a plain value with a flag instead of std::optional: gcc-11 reports -Wmaybe-uninitialized on upstream's form, which --enable-werror turns into an error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…idated b7ba60f test: add coverage for -reindex and assumeutxo e57f951 init, validation: Fix -reindex option with an existing snapshot Pull request description: Restarting with -reindex or -reindex-chainstate while a snapshot chainstate exists must delete it, and a snapshot can be loaded again afterwards. Dash adaptations: - e57f951 is equivalent in this tree and not applied: node::LoadChainstate() deletes the snapshot chainstate directory with DeleteSnapshotChainstateFromDisk() before it detects a snapshot, when reindexing, so no snapshot chainstate is created that would need deleting, and the validated chainstate is initialized with the node's mempool from the start. Upstream's DeleteSnapshotChainstate() and its mempool handover therefore have no counterpart. - The test's coin count comes from the dump, as in the rest of the file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…not open file ee67bba test: added test coverage to loadtxoutset Pull request description: Check that loadtxoutset fails with a clear error for a path that does not exist. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…invalid chain 2f9bde6 test: Remove unnecessary restart in assumeutxo test 19ce3d4 assumeutxo: Check snapshot base block is not marked invalid 80315c0 refactor: Move early loadtxoutset checks into ActiveSnapshot Pull request description: ActivateSnapshot() returns a util::Result, so loadtxoutset reports why a load failed, and the checks that loadtxoutset made before calling it move into it. A base block in an invalid chain is refused. Dash adaptations: - 19ce3d4 is equivalent here: SnapshotActivationRefusal() already refuses a base whose chain has a block marked failed or ChainLock- conflicting, at load and again at the switch, so no second check is added. test_snapshot_block_invalidated runs on n0, the node with the downloaded chain, restarted with the pin for this check only (it never loads a snapshot), and matches the refusal's message. - The snapshot metadata has no base height in this tree (bitcoin#29612 is not ported), so the unrecognized-hash error names the hash only, without the available heights. - loadtxoutset still waits for the base header (bitcoin#29345 is not ported), so it keeps a hash check before that wait, with the same message, rather than wait for a header that cannot be authorized. - The refusals that this tree logs keep their log lines, which its tests and operators read; each now also returns its reason. Dash's own failures (another activation running, an earlier rejected snapshot, the Evo publication) return theirs too. - loadtxoutset keeps reporting the tip it activated. - test_invalid_mempool_state comes from bitcoin#29394, which this file lacked, in its bitcoin#30267 form, with n2 started with -persistmempool=0 as there. It overlaps a unit case of the refusals commit; both are kept. - 2f9bde6 and the hunks for bitcoin#29428's test_snapshot_with_less_work, bitcoin#29478's test_headers_not_synced, bitcoin#29612's metadata rows and bitcoin#29973's double load are not applicable: those tests are not in this file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ent chain + test 5b7f70b test: loadtxoutset in divergent chain with less work d35efe1 p2p: Start downloading historical blocks from common ancestor Pull request description: When the active chain tip is not an ancestor of the snapshot base, the background chainstate must download the snapshot's history from the last common ancestor of its tip and the base, not from its own tip. Dash adaptations: - The divergent-chain node is a fifth node (n4): n3 is the node that drops its snapshot with -reindex-chainstate. It is authorized with the same -assumeutxodata pin as the other loading nodes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the best header chain 55b6d7b validation: Don't load a snapshot if it's not in the best header chain. Pull request description: Refuse a snapshot whose base is not an ancestor of the best known header: the node could not reorg away from it before background validation ends. Dash adaptations: - The snapshot metadata carries no base height in this tree (bitcoin#29612 is not ported), so the check takes the height from the base's block index entry. - As upstream, the check runs once at load; it is not part of SnapshotActivationRefusal(), which is repeated at the switch. It follows that function's checks, so an invalid base is reported as such first, as upstream orders its invalid-chain check before this one. - Invalidating a block moves the best header back and reconsidering it does not restore it (bitcoin#30666 is not ported). The unit cases that reconsider the base before loading again set the best header back to the base. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ore background chain 49d569c p2p: For assumeutxo, download snapshot chain before background chain 7a88551 p2p: Restrict downloading of blocks for snapshot chain Pull request description: Download the snapshot chain's blocks first by resetting the last common block of each peer to at least the snapshot base, and do not download from peers whose best chain lacks the snapshot base, since the node could not reorg to their chain before background validation ends. Upstream adds no test; the functional rows that sync a snapshot node from n0 cover the download order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… nodes during IBD 992f83b test: add coverage for assumeUTXO honest peers disconnection 6d5812e assumeUTXO: fix peers disconnection during sync Pull request description: A node with a snapshot chainstate advertised NODE_NETWORK although it cannot serve the blocks below the base, so IBD peers requested them, got no answer and disconnected it as stalling. loadtxoutset now replaces NODE_NETWORK with NODE_NETWORK_LIMITED, startup does not set NODE_NETWORK while background sync is in progress, and NODE_NETWORK returns when background validation completes (restart_indexes becomes snapshot_download_completed). Dash adaptations: - Snapshots are refused on pruned nodes here, so feature_assumeutxo.py's n1 is a full node: its checks expect NODE_NETWORK_LIMITED only while it is in dual mode and NODE_NETWORK again after completion, as for n2, instead of upstream's pruned-node checks. - test_sync_from_assumeutxo_node waits for NODE_NETWORK after the base block is stored, since background completion can follow the block's storage. The n1 and n2 rows likewise wait for NODE_NETWORK after completion: the callback that restores it runs after the step that disabled the background chainstate, which getchainstates can already reflect. - A snapshot rejected at completion keeps NODE_NETWORK_LIMITED until the next restart: the node stays up after a rejection in this tree, and the advertisement is conservative, never dishonest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…or background chainstate c0a0c72 validation: Disable CheckForkWarningConditions for background chainstate Pull request description: The background chainstate's tip lags far behind, so comparing it with the best invalid block raised a fork warning for a fork that the active chain does not have. Dash adaptations: - feature_assumeutxo_dash.py's dual-mode scenario checks that the warnings lack "We do not appear to fully agree" after the background chainstate connected blocks below a block invalidated by an InstantSend lock. - As upstream until bitcoin#30214, the step that completes background validation reports NORMAL, so that one step still evaluates the condition; the next active step resets it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_tx_count, Improve wallet RPC errors 9d2d9f7 rpc: Include assumeutxo as a failure reason of rescanblockchain 595edee test, assumeutxo: import descriptors during background sync d73ae60 rpc: Improve importdescriptor RPC error messages 27f99b6 validation: Don't assume m_chain_tx_count in GuessVerificationProgress 42d5d53 interfaces: Add helper function for wallet on pruning Pull request description: GuessVerificationProgress() can legitimately see a block without a transaction count, e.g. a wallet rescan below a snapshot base, so it returns 0 with a debug log instead of an Assume. importdescriptors and rescanblockchain name the assumeutxo background sync as the likely cause when blocks are unavailable. Dash adaptations: - 42d5d53 (Chain::getPruneHeight) is not ported: it needs GetPruneHeight() from bitcoin#29668, which this tree lacks. rescanblockchain therefore reports pruning whenever the node has pruned, as before; a node with a snapshot chainstate never prunes here, so the assumeutxo message is unaffected. - m_chain_tx_count is nChainTx in this tree; the log names dashd. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… promoted bitcoin#33604 is not adopted. Before the restart that promotes a validated snapshot, Dash cannot disconnect below the base's carried Evo state, so downloading from such peers only stalls and churns the chain. bitcoin#29519's refusal, which keys on the snapshot base, already lasts until promotion here, because no snapshot base exists after the promoting restart. Its test is replaced by this one. The snapshot node, validated but not yet restarted, still reports its snapshot base in getchainstates. It refuses a peer whose more-work chain forks below the base, without an Evo state error, and keeps its tip. The restart promotes it, so getchainstates no longer reports a snapshot base, and it then reorgs to that chain below the old base, through the validated chainstate's Evo state. The fork is mined to a fresh address: with the mock time fixed, the usual address would rebuild the block just invalidated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5cd5794 test: verify node state after restart in assumeutxo Pull request description: Replace the TODO after the restart during background sync with checks of the chainstates and of wallet loading, which still needs the blocks below the snapshot. Dash adaptations: - The pruned-node rows that use the same error helper are absent: a node does not load snapshots with -prune in this tree. - A blank line precedes the nested helper, for this tree's flake8 (E306). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o.py faa68ed test: Fix intermittent issue in wallet_assumeutxo.py Pull request description: -stopatheight may let a few more blocks through during shutdown. Sync the node to the pause height with submitblock instead of restarting it, and unload the wallets so the loading checks still apply. Dash adaptations: - setup_network() keeps its form; the pruned-node rows are absent, as for bitcoin#34286. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…xo.py faf71d6 test: [refactor] Use verbosity=0 named arg 99996f6 test: Fix intermittent issue in feature_assumeutxo.py Pull request description: Sync n1 to the pause height with submitblock instead of a -stopatheight restart, which could stop a few blocks later than expected. Dash adaptations: - The blockfile segmentation checks between the two hunks are absent, as the segmentation is not part of this tree (bitcoin#27596's commit). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PastaPastaPasta
force-pushed
the
feat/assumeutxo-hardening
branch
from
November 9, 2026 19:14
dc81f37 to
91294b1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue being fixed or feature implemented
The snapshot loader in #8078 (bitcoin#27596 and its follow-ups) works, but it carries three kinds of debt that upstream fixed after bitcoin#27596 and that Dash has not taken yet:
nTxandnChainTxfor every block below the base and marks themBLOCK_ASSUMED_VALID. feat: load AssumeUTXO snapshots with loadtxoutset and validate them in the background (regtest only) #8078 has to carry two exceptions inCheckBlockIndexand aBlockManager::m_faked_tx_heightto keep its checks from aborting on those values. assumeutxo: Get rid of faked nTx and nChainTx values bitcoin/bitcoin#29370 removes the faking, which is the root fix.loadtxoutsetaccepts a base that is not in the best header chain, and it reports no reason when it refuses. Historical block download starts from the wrong block when the snapshot chain diverges from the node's chain, and can run before the snapshot chain. AssumeUTXO nodes disconnect peers during IBD. The background chainstate raises fork warnings.getchaintxstatsreturns wrong numbers while counts are unset. The wallet RPCs assumem_chain_tx_count, and-reindexbefore validation was untested.feature_assumeutxo.pyandwallet_assumeutxo.py.Taking the root fix also exposed a gap that upstream has too. A snapshot chainstate raises the blocks it connects above its base to CHAIN and SCRIPTS validity. If the snapshot is then dropped before background validation reached the base (
-reindex-chainstate, or a restart after a rejection), those blocks keep a validity claim that no validated ancestor backs. WithBLOCK_ASSUMED_VALIDgone,CheckBlockIndexrejects that state, and in the same run a rejection raised while the snapshot chainstate connects a block can reachFindMostWorkChain's release assert. bitcoin/master has the same gap after-reindex-chainstateand no upstream fix. It is repaired here, in two places.Stack position. This is the fifth layer. It depends on #8078 (L4a), #7601, #7593, #7887, and below those #8049, #8067, #8066 and #8069; none of them has merged. Against
developthe PR shows 71 commits; only the top 17 belong to this PR. It stays a draft until the layers below it merge. L4a and this PR are meant to merge together.Two decisions a reviewer should check:
Quorum snapshot ... is not available, records a stall, and reconnects, with notifications both ways. Refusing (p2p: For assumeutxo, download snapshot chain before background chain bitcoin/bitcoin#29519's behaviour) has the same liveness, since neither can follow the fork before the restart, and none of that churn. p2p: For assumeutxo, download snapshot chain before background chain bitcoin/bitcoin#29519's condition keys on the snapshot base, which is gone after the promoting restart, so the refusal already lasts exactly until promotion. A Dash test replaces upstream's test.What was done?
17 commits: 15 upstream merges and 2 Dash commits. Each commit message lists the upstream commits, the Dash adaptations and the omitted hunks.
START_HEIGHT+1, spent with a 0.001 DASH fee instead of upstream's fixed amount.BLOCK_ASSUMED_VALIDbecomes the reserved status bit 256, nothing fakes counts at the switch,CheckBlockIndexchecks the exactnChainTxsum and restarts its tracking at the snapshot base. It removes feat: load AssumeUTXO snapshots with loadtxoutset and validate them in the background (regtest only) #8078's interim (bothCheckBlockIndexexceptions andm_faked_tx_height). Dash adaptations: ChainLock-conflicting ancestors are tracked beside invalid ones; the SegWit-onlyBLOCK_OPT_WITNESSfaking is not ported; no compatibility code, because no released Dash loads snapshots. Dash addition in this commit:LoadBlockIndexlowers CHAIN/SCRIPTS validity for blocks a dropped snapshot connected above a base that was never validated, to the level their ancestors back and never below TRANSACTIONS.fix(validation): forget a rejected snapshot's base in the block index(Dash). The in-run half of the same rule: a snapshot rejected before background validation reached its base left the base with the transaction count from the snapshot metadata, and restoring the validated chainstate's candidates offered it blocks above an unlinked gap. The base and the blocks above it now get the state the next startup would derive.getchaintxstatsno longer returns invalid results when counts are unset. Adaptation: a flag plus a plain value instead ofstd::optional, because gcc-11-Wmaybe-uninitializedrejects upstream's form under--enable-werror.LoadChainstatealready deletes the snapshot directory before detection when reindexing.ActivateSnapshotreturnsutil::Result<void>andloadtxoutsetreports the reason; the base block must not be in an invalid chain. One of its upstream commits (19ce3d407ef) is Dash-equivalent and not ported again, because the refusal in feat: load AssumeUTXO snapshots with loadtxoutset and validate them in the background (regtest only) #8078 already covers a failed or ChainLock-conflicting block anywhere in the base's chain. Omitted: tests of PRs that are not ported (test: Assumeutxo: snapshots with less work should not be loaded bitcoin/bitcoin#29428's less-work test, test: Test new header sync behavior in loadtxoutset bitcoin/bitcoin#29478's headers-not-synced test, rpc: Optimize serialization and enhance metadata of dumptxoutset output bitcoin/bitcoin#29612's metadata rows, test: Assumeutxo: ensure failure when importing a snapshot twice bitcoin/bitcoin#29973's double load).reconsiderblockcan leavem_best_headerstale until a new header arrives (validation: fix m_best_header tracking and BLOCK_FAILED_CHILD assignment bitcoin/bitcoin#30666 is absent); the node then refusesloadtxoutsetconservatively, as upstream did before validation: fix m_best_header tracking and BLOCK_FAILED_CHILD assignment bitcoin/bitcoin#30666.nLocalServicesbecomes atomic, the node is NODE_NETWORK_LIMITED while in dual mode, andloadtxoutsetswitches to it. The node keeps NODE_NETWORK_LIMITED after an in-run rejection until restart.CheckForkWarningConditionsis disabled for the background chainstate. Added: scenario A infeature_assumeutxo_dash.pyasserts that the warnings do not contain "We do not appear to fully agree" after the background connects blocks below the InstantSend-invalidated block. The step that completes validation still reports NORMAL until refactor: Improve assumeutxo state representation bitcoin/bitcoin#30214.Assumeon an unsetnChainTx, better wallet RPC errors, and the wallet test rows. Omitted: thegetPruneHeighthunk (42d5d533631), which needs prune, rpc: Check undo data when finding pruneheight bitcoin/bitcoin#29668. Until thenrescanblockchainreports pruning whenever the node has pruned; snapshot nodes never prune here.test: use peers without the snapshot block only after the snapshot is promoted(Dash), replacing p2p: Allow block downloads from peers without snapshot block after assumeutxo validation bitcoin/bitcoin#33604's port and test. The snapshot node, validated but not yet restarted, refuses a peer whose more-work chain forks below the base, without an Evo state error, and keeps its tip. After the promoting restart it reorgs to that chain through the validated chainstate's Evo state.dumb_sync_blocksinstead of the-stopatheightrestart that madewallet_assumeutxo.pyandfeature_assumeutxo.pyintermittent. Pruned-node rows and the blockfile segmentation checks are absent here.Upstream test-only PRs not ported (not required for correctness): bitcoin#29478 (needs bitcoin#29345), bitcoin#29617 (its fixed byte offsets do not fit this file's dynamic pin; the code paths it checks exist), bitcoin#30817 and bitcoin#30892 (need bitcoin#29553), bitcoin#34377 (renames a wallet in a pruned-node row that does not exist here). bitcoin#30214, bitcoin#29553 and bitcoin#30598 are deferred; no code in this stack depends on them.
How Has This Been Tested?
Environment: gcc-11
--enable-werrorrelease configuration with Qt, sqlite and no BDB, and clang-19 with-Wthread-safety -Werror, in containers on Linux x86_64; functional tests withdash_hash. Head:dc81f37b7c0b.Pending, not passed:
CIworkflow on PastaPastaPasta/dash for this head (run 42665595634) was still running when this description was written.--extendedfunctional tests, the sanitizer builds (TSan for the atomicnLocalServicesandsnapshot_download_completedpath, ASan/UBSan forgetchaintxstats),test_dash-qt, per-commit unit suites for the commits that did not change, and the code-review-validator and code-simplifier passes. They are left to GitHub CI and the reviews.Run locally at this head:
--enable-werror) from the assumeutxo: Get rid of faked nTx and nChainTx values bitcoin/bitcoin#29370 commit to the tip: 17 of 17 pass (test: assumeutxo: spend coin from snapshot chainstate after loading bitcoin/bitcoin#29215 is unchanged from an earlier checked build). The fullvalidation_chainstatemanager_testswith no excluded case passes at the assumeutxo: Get rid of faked nTx and nChainTx values bitcoin/bitcoin#29370 commit and at the forget-base fix, together with the other suites those commits touch. At the tip,validation_chainstatemanager_tests,validation_chainstate_tests,validation_testsandblockmanager_testspass.feature_assumeutxo.py,feature_assumeutxo_dash.py,wallet_assumeutxo.py --descriptorsandfeature_assumeutxo_refusals.pyall pass.rpc_blockchain.pystands in forrpc_getchaintxstats.py, which this tree does not have; it passed on the earlier head, before the repair commits, and was not rerun on this one.-Wthread-safety -Werror: the full build passes at the assumeutxo: Get rid of faked nTx and nChainTx values bitcoin/bitcoin#29370 commit, at the forget-base fix and at the tip, with 4 suites passing at each.ci/dash/lint.shover4c5c0153500f..HEADandcppcheck-dash.pyexit 0.Fail-without-fix. Each mutant ran in its own copy with only the stated source change, next to an unmutated control that passed:
LowerUnbackedValiditycall inLoadBlockIndexat the assumeutxo: Get rid of faked nTx and nChainTx values bitcoin/bitcoin#29370 commit: node 3 offeature_assumeutxo.pyand the unit casechainstatemanager_evo_snapshot_real_quorums_rejectedabort inCheckBlockIndexwithpindexFirstNotChainValid == nullptr. At the tip, node 3 and node 7 (the-reindex-chainstaterow offeature_assumeutxo_dash.py) abort the same way; the unit case passes there, because the forget-base fix has already persisted the repair before its restart.ForgetSnapshotBasecall at the tip: the unit case aborts inFindMostWorkChain(pindexTest->HaveNumChainTxs() || pindexTest->nHeight == 0) right after the rejection.!IsSnapshotValidated() &&condition at the tip: step 1 of the replacement test fails withUnexpected message "local Evo state error".feature_assumeutxo.pyfails atassert_only_network_limited_service(n1); without validation: Disable CheckForkWarningConditions for background chainstate bitcoin/bitcoin#30962's early return, scenario A's warnings assertion fails; with Assumeutxo: bugfix on loadtxoutset with a divergent chain + test bitcoin/bitcoin#29996's start reverted to the background sync tip,test_snapshot_in_a_divergent_chaintimes out; with assumeutxo: Don't load a snapshot if it's not in the best header chain bitcoin/bitcoin#30320's check disabled,test_snapshot_not_on_most_work_chainraises no exception; with assumeutxo: Get rid of faked nTx and nChainTx values bitcoin/bitcoin#29370's source reverted (before the repair),feature_assumeutxo.pyfails incheck_tx_countson the fakednTx.The three rows that exercise a dropped snapshot (node 3 in
feature_assumeutxo.py, node X's-reindex-chainstaterow infeature_assumeutxo_dash.py, and the unit case above) aborted inCheckBlockIndexon the plain bitcoin#29370 port. They are left unchanged by the repair and are its tests.Breaking Changes
None for any network except regtest, where the loader is the only way to get a snapshot chainstate.
getchaintxstatsfieldstxcount,window_tx_countandtxratecan now be absent while counts are unset, as upstream.Known limitations, not fixed here:
Checklist:
🤖 Generated with Claude Code