diff --git a/AGENTS.md b/AGENTS.md index 08134d219..98cdb836b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -28,11 +28,10 @@ Open [`CONTRIBUTING.md`](CONTRIBUTING.md) principles 7–11 and - `//` only for an invariant, protocol rule, `SAFETY`, or library quirk. Crate and public rustdoc (`//!` / `///`) is not this rule (principle 7). -- Tests assert shipped behavior, not repo text. No `*_for_test` backdoors. - IO pins use session or table stats or on-disk state (principle 8). - A test that only calls other tests is not a journey. Small units are the - exception when a real session cannot reach the behavior. - [`TESTING.md`](TESTING.md) (True journeys). +- Tests assert a result a peer, client, or operator observes, not repo + text. No `*_for_test` backdoors. IO pins use on-disk state or session + stats (principle 8). A behavior no session can trigger is deleted, not + given a unit. Narrow exception and the 92% floor: [`TESTING.md`](TESTING.md). - A RAM or CPU trade is named (principle 9). - Control flow and composition: principle 10 and `docs/code-shape.md`. - Crate `pub` is the cross-crate graph only. No unused `pub`. No diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5dd4bfb3d..e11ccf151 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -157,11 +157,13 @@ to it. Darwin operator binaries come from the `macos-14` release job, not Nix. and [`docs/crash-recovery.md`](./docs/crash-recovery.md) over inventing new design notes. 2. Prefer **high-level functional/integration tests** over unit tests - ([`TESTING.md`](./TESTING.md)). + ([`TESTING.md`](./TESTING.md)). A branch no peer, client, or operator can + trigger is removed rather than covered by a unit. 3. Every PR must keep production line coverage **≥ 92%** (`LH*100 >= LF*92`). Nightly branch coverage stays ≥90% when measured. Same bar as CI via `./scripts/coverage.sh`. CRAP `--fail-above 30` with the allowlist in - `.cargo-crap.toml`. + `.cargo-crap.toml`. Hold the floor by covering from the surface journey + or by deleting unreachable lines, not by adding a private-helper test. 4. Target is **production server-side** node software (wallet backends, etc.). Tip-mode mempool + tx relay are **in scope**; no pruning/GUI/end-user wallet/ mining without an explicit plan change. @@ -191,9 +193,10 @@ to it. Darwin operator binaries come from the `macos-14` release job, not Nix. call graphs. Fixture JSON/hex and tests that read **datadir** bytes are not this rule. Prefer **one** [true journey](./TESTING.md#true-journeys) over a twin unit for the same reject. A `#[test]` that only calls other - tests is not a journey. A small unit is the exception when a real peer, - client, or operator session cannot reach the behavior. Core functional - is nightly, not a substitute for that journey. + tests is not a journey. A small unit is only for a consensus or schema + result a real peer, client, or operator session cannot reach without an + absurd chain. Any other behavior no session can trigger is deleted, not + unit-tested. Core functional is nightly, not a substitute for that journey. 9. **RAM and CPU are design inputs.** This node indexes chain-scale structures (tens of millions of keys, hundred-MiB arrays, GiB-class heads). Iterating those structures is expensive. Every algorithm should @@ -295,8 +298,9 @@ IO; they do not package zips. GitHub Releases: ## Code review checklist -- [ ] Behavior covered by a true journey (or a justified unit next to a - pure helper a real session cannot reach). No caller-of-tests, no twin +- [ ] Behavior covered by a true journey (or a justified unit whose result + is consensus or schema math a real session cannot reach). A behavior + no session can trigger is deleted. No caller-of-tests, no twin for the same reject. Core functional is not the PR pin ([`TESTING.md`](./TESTING.md#true-journeys)). - [ ] Core-facing RPC / P2P / Electrum / Esplora: [`COMPAT.md`](./COMPAT.md) diff --git a/TESTING.md b/TESTING.md index d3b5d41f5..3f06bff79 100644 --- a/TESTING.md +++ b/TESTING.md @@ -6,7 +6,7 @@ |--------|--------| | **Journey scenarios**: one setup a peer, client, or operator could use, then a **sequence** of asserts on what they would observe | Many skinny scenarios that each remine maturity and re-open the store | | The socket, RPC, HTTP route, operator config, or scripted peer | An in-process helper the product never calls, or a thread-local the client does not share | -| **Pure units** on pure helpers (scriptnum, bits, fuse8, open-hash) with **no store**, when a real session cannot reach them | Units that re-implement confirm and only paint lines a journey already hits | +| **Pure units** whose return **is** the consensus or schema result, when a real session cannot reach that result without an absurd chain | Units whose job is to execute a private branch, or that re-implement confirm and only paint lines a journey already hits | | **One entry** per production path (the journey owns the asserts) | A `#[test]` that only calls other tests, or those tests kept as private bodies | | Core JSON corpora for **script engine** breadth | A second parallel script suite | @@ -18,7 +18,7 @@ A journey is one setup and one story. A peer, a client, or an operator does a se A `#[test]` whose body only calls other tests is not a journey. Each callee still opens its own store, hub, or socket. The suite gains one name and the same N boots. Delete the callees. The journey writes the asserts. Do not keep the old functions as private bodies the new test calls. -Small tests are the exception. Use one when a real session cannot reach the behavior without a setup no peer, client, or operator has: pure arithmetic, a codec with no socket, two networks that cannot be the same chain. Say why next to the test. A tall chain the rest of the story never builds is a named second chapter on a second setup. Two setups only when the objects cannot be the same. +If no peer, client, or operator can cause the behavior, delete the behavior in the same change. A small test stays only when the function's return is the consensus or schema result and a session cannot reach it without an absurd chain: pure arithmetic, a codec with no socket, two networks that cannot be the same chain. Say which of those it is, next to the test. A small unit is not a cheaper substitute for a journey the session can already run. A tall chain the rest of the story never builds is a named second chapter on a second setup. Two setups only when the objects cannot be the same. Push the entry up. Prefer the surface a real session uses over an in-process dispatch the product never calls. In-process is for a fact that surface cannot show. Do not assert a thread-local, a counter on a worker the client does not share, or a helper's name. If the only proof lives there, it is not the contract yet. @@ -42,20 +42,29 @@ When adding or folding a pin: |----|--------| | Extend an existing [catalog](#scenario-catalog) journey (same `/tmp` pad, more asserts) | A new skinny scenario that remine-pads the same chain | | Fold a twin unit once the journey hits the same shipped path | Twin unit + scenario for the same reject string | -| Keep a unit only while no real session can reach that path, then move the assert onto the journey | A new small test for a path a peer, client, or operator can already hit. Handshake **format** needles, `decode_rpc_subset`, and BIP324 encode vectors stay until a journey hits them | +| Delete a lower test only in a commit where a surface journey already hits those lines, so the 92% ratio holds. When no surface can hit the lines, delete the production branch and the test that only painted it in that same commit | A new small test for a path a peer, client, or operator can already hit, or a private-helper test written to turn coverage or CRAP green. Handshake **format** needles, `decode_rpc_subset`, and BIP324 encode vectors stay only until the live v2 journey observes those bytes | | Live P2P/RPC on `cross_surface` / `integration_multinode` catalog tests | Grow `node_cli_and_surface_smoke` into a second live node | | New P2P behavior on `p2p_timeout_*` / compact / feeler / inbound-full | Stuff more asserts onto `two_node` | -Coverage (LCOV `LH`/`LF` never below last green `master`) is a required PR -job. If deleting a guts test drops the ratio, the journey did not cover the -path — keep the guts or hit those lines from the journey first. - -Keep until a **default** journey hits the same lines: store packed / v17 / -fuse / SH machines, unsorted pack/lag, -IBD wave fence / 8×8000, SH writebehind / uring CAS, -handshake format needles, `getaddr_cache_*`, sole-preferred stall KEEP, `stamp_reject_names_*`, -`multi_hop_bad_prev_*`, structure s1–s18, rate-limiter, netgroup, subsidy -table. Optional leftovers (more HTTP methods on `cross_surface`, a tiny +The coverage job stays at `LH*100 >= LF*92`. CRAP stays `--fail-above 30`. +Delete a lower test only in a commit where a surface journey already hits +those lines, so the ratio holds. When no surface can hit the lines, delete +the production branch and the test that only painted it in that commit. The +ratio holds because those lines leave `LF`. + +Do not add a test of a private helper to turn a coverage or CRAP failure +green. Extend the catalog journey, or remove the branch and tighten the +invariant ([`docs/invariants.md`](docs/invariants.md): a missing promised +fact is `StoreError::Corrupt("invariant: …")`, no silent fallback). + +Internal witnesses still exist for store packed / v17 / fuse / scripthash +machines, unsorted pack/lag, the IBD wave fence / 8×8000, scripthash +write-behind / uring CAS, handshake format needles, `getaddr_cache_*`, +sole-preferred stall, `stamp_reject_names_*`, `multi_hop_bad_prev_*`, +structure s1–s18, the rate-limiter, netgroup, and the subsidy table. The +next change that touches one of them applies the two rules above. It does +not add another witness, and the list is not a permanent exception. +Optional leftovers (more HTTP methods on `cross_surface`, a tiny legacy-head `Store::open` fixture, testnet 20-minute min-diff header walk) are not a backlog. @@ -67,7 +76,7 @@ file state, not thread-local hot-path IO probes. Do not: - Put HOLD / wait hooks in a shipped function other tests also call (`confirm_scripts_phase`). - Assert process-global last-writer meters as the contract. Confirm / query / IBD window meters are instance-owned (`Query::confirm_stats`, take-and-reset). Pin two engines, not crate-root atomics. Store head-resolve window meters and `last_union_miss` / leftover probe diag remain process-global; use pin/layout, error strings, or a pure formatter. -- Thread-local `test_take_*` IO probes on store hot paths (example of the rule above; assert session/table instance stats or file state). Class A three-stem append is `UringSession` max-batch pwrite SQEs (`tls_take_max_batch_pwrite_n` after `with_thread_local`), not `test_take_pwrite_waves`. +- Thread-local `test_take_*` IO probes on store hot paths (example of the rule above; assert session/table instance stats or file state). Class A three-stem append is the bytes on disk after the append, not `test_take_pwrite_waves` or an SQE counter (`tls_take_max_batch_pwrite_n`). - `std::env::set_var` without the crate lock (or pass the knob as an argument). - Bind a fixed port (use `:0`) or share a `/tmp` path (use `rbitcoin_store::testutil::TempDir` / `tiny_store`, `rbitcoin_query::testutil::tiny_query`, net `tiny_regtest_hub`, or `rbitcoin_test::TestDatadir`). @@ -259,17 +268,26 @@ counts that line. through a **shipped** config / error / CLI path. Do not add a `pub` or `*_for_test` injector so a unit can see it ([`CONTRIBUTING.md`](./CONTRIBUTING.md) principle 11). -4. A small unit only when a real session cannot reach the behavior without - absurd cost — say why in the test file. Drive the shipped function, not a +4. A small unit only when the function's return is the consensus or schema + result and a real session cannot reach it without an absurd chain — say + why in the test file. Any other behavior no session can trigger is + deleted, not unit-tested. Drive the shipped function, not a `#[cfg(test)]` wrapper around it. ### Closing a red region 1. Open the HTML/LCOV report from `./scripts/coverage.sh`. -2. Identify high-miss production files (largest `LF − LH`). -3. Add or extend a **scenario** in `rbitcoin-test` or a unit test next to the - shipped path that drives the real entry point. -4. Re-run `./scripts/coverage.sh` until the ratio is **≥ 92%**. +2. For each missed region, decide which case it is. +3. A session can cause it: extend the [catalog](#scenario-catalog) journey + that already has that peer, client, or datadir. Re-run until the lines + are hit. +4. A session cannot cause it: delete the branch. Re-run. The ratio rises or + holds because `LF` fell. +5. The function is pure consensus or schema math and the journey would need + an absurd chain: one unit on that shipped function, reason in the test. + Drive that function, not a `#[cfg(test)]` wrapper. +6. Stop when the ratio is **≥ 92%**. Do not add a private-helper test to + get there. ## Structural lints, CRAP, Miri, mutants @@ -284,7 +302,7 @@ matches scalar in default tests). Owner: [`docs/quality.md`](./docs/quality.md). | Tool | How to run | CI | |------|------------|----| | **ast-grep** | `./scripts/ast-grep.sh` (needs `ast-grep` on `PATH`; `nix-shell` / `nix develop` provide it). Fixture self-test: `./scripts/ast-grep.test.sh` | Step in required job `qc` | -| **cargo-crap** | After LCOV, `./scripts/coverage.sh` calls `./scripts/coverage-crap.sh` (skip if `cargo-crap` missing). `--fail-above --threshold 30`; `.cargo-crap.toml` allowlists today's production CRAP>30 functions (remove a name when it scores ≤30). Dry-run: `CRAP_DRY_RUN=1 ./scripts/coverage-crap.sh`. Self-test: `./scripts/coverage-crap.test.sh` | Rides required `coverage`. No `--fail-regression` (llvm-cov coverage % jitters per function) | +| **cargo-crap** | After LCOV, `./scripts/coverage.sh` calls `./scripts/coverage-crap.sh` (skip if `cargo-crap` missing). `--fail-above --threshold 30`; `.cargo-crap.toml` allowlists today's production CRAP>30 functions (remove a name when it scores ≤30). A new failure is answered by deleting branches until one path remains, or by extending the catalog journey that owns the feature so those branches run. Allowlist additions follow [`docs/code-shape.md`](docs/code-shape.md): do not split a Core-faithful opcode loop or an io_uring machine to beat a score. A test whose only effect is to change the CRAP input is not the fix. Dry-run: `CRAP_DRY_RUN=1 ./scripts/coverage-crap.sh`. Self-test: `./scripts/coverage-crap.test.sh` | Rides required `coverage`. No `--fail-regression` (llvm-cov coverage % jitters per function) | | **coverage ignore / badge** | `./scripts/coverage.test.sh` (filename ignore, Tier A IBD not skipped, 92% floor, Shields JSON). Publish dry-run: `BADGE_DRY_RUN=1 ./scripts/publish-coverage-badge.sh` | `test` job self-test; `coverage` job writes `coverage/badge.json` and, on green `master`, pushes `badges/coverage.json` | | **Miri** | `./scripts/miri.sh` → `cargo +nightly miri test -p rbitcoin-primitives`. Dry-run: `MIRI_DRY_RUN=1 ./scripts/miri.sh`. Self-test: `./scripts/miri.test.sh` | Nightly `miri.yml` (not required). Never `--workspace` | | **cargo-mutants** | Nightly, not a PR check. `./scripts/mutants-nightly.sh` lists `--workspace` mutants, runs new diff lines first, then walks a cursor through the rest. `rbitcoin-bench` is excluded (`.cargo/mutants.toml` and `--exclude` on both invocations; CLI replaces the config glob). `#[mutants::skip]` on an expression is the won't-test list; those mutants are omitted, not `MISSED`. Each mutant uses `--test-workspace=true` so a higher journey counts, `--baseline=skip`, `-j 1`, test binaries on tmpfs (runner above; the mutants copy stays on disk), 20 minute mutant timeout, 5 hour budget. Cursor artifact `mutants-cursor`. `MISSED` is uploaded and does not fail the run. Each invocation passes `--file` for one source file and at most `MUTANTS_BATCH` queued names from that file (default 200). cargo-mutants 27.1.0 emits `..` struct-field deletes without applying `--re`; one file keeps that repeat inside the file under test instead of retesting every such delete in the workspace on every batch. The miss list is that night's `mutants-nightly` artifact. | `mutants.yml` daily `47 3 * * *` (20:47 Pacific during PDT) and `workflow_dispatch`. Job timeout 330 minutes. | @@ -312,6 +330,8 @@ let (tip, tip_time) = pad_empty_from(&query, ¶ms, tip, tip_time, 2, maturity Prefer **one high-level scenario** per behavior cluster. Delete lower-level tests when a newer scenario covers the same production paths. +A cell that names a crate-local test describes the witness that exists today. It is not an instruction to keep that witness. On the next change to that behavior, move any unique assert onto the surface journey and delete the lower test once `./scripts/coverage.sh` still passes. If the surface journey cannot reach the lines, delete the production branch in that change. + | ID | Layer | Description | |----|-------|-------------| | `overlay_config` | Node CLI + net | In-process onion / I2P / cjdns matrix: `onlynet`, proxy, SAM, reachable, accept-incoming, one parse or learn check per network, `--only-net` dial candidates, no cjdns dial until reachable, and one `peers` file round trip for all three. Live Tor, i2pd, and cjdns stay in overlay-functional. | @@ -320,42 +340,42 @@ Prefer **one high-level scenario** per behavior cluster. Delete lower-level test | `mempool_under_pressure` | Mempool + RPC (crate) | One entry in `orphanage`, `accept`, `tx_relay`, and `methods_tests`: orphan reserve and expiry, sigops before script, rolling fee floor, cluster cap, parked min-relay orphan, and the package RPC rejects (unsorted, missing inputs, conflict, min-relay parent with maxfeerate child). | | `mempool_accept_life` | Mempool (crate) | One `ActiveMempool` against one chain view that blocks move. Sigop-adjusted vsize (boundary, min relay, full-pool floor, RBF, package and 1p1c), the raw-weight cluster limit, the shared block sigop budget, and sigop cost plus bytes-per-sigop and reserve overlays across reopen and compact. Orphan parks and re-announce, dry run not parked, parent promotes the child; missing vout, invalid parent, and block-spent coin reject without parking. Full RBF and no return, the staged commit failing closed on a conflict that landed after prepare, pure RBFR unpinning a child, replaced txs out of the cluster count; a ~30 kvB single tx under the vsize cap and the ten-way merge over it. Package order, CPFP, child fail restoring the RBF victim. A block evicts double-spent txs with descendants; a reorg readmits the parent and evicts the BIP68 and coinbase-maturity spends. Raised `-minrelaytxfee`: 1p1c needs a paying child, an unrelated tx does not ride the waiver, child fail takes a promoted spender down. Full pool: a protected lone worst chunk evicts nothing and leaves the floor, the next arrival evicts the CPFP pair together. | | `rpc_regtest_chain_ops` | RPC (crate) | One regtest hub from genesis through `dispatch`. At genesis: `size_on_disk` is the store walk, IBD comes from the hub and not the stale atomic, buried deployments, `generateblock submit=false` connects nothing, and the priority, mocktime, mockscheduler, submitheader decode, and not-found refuses. The first block pays a p2wpkh address: display-order hashes and txids, raw `getblock` and header, a headers-only child at progress 0.5. Mocktime stamps `generate` and makes a far block `time-too-new`. Coinbase-only blocks: verbosity 1 without a seqsigwit zip, `getnetworkhashps` over chainwork, the empty template and proposal needles, a `time-too-old` header, an invalid parent body that marks its branch, and the `submitblock` merkle, length, coinbase, duplicate, value, and missing-input rejects. After a 120-block pad: the `nblocks=0` window, GBT fee and sigops (bare, P2SH, P2WSH), sigop-adjusted mempool vsize (`getmempoolentry`, package retry, `blockmintxfee`; weight and the Esplora/Electrum histogram stay raw) and a big-sigops cluster under the block budget, a non-DER spend with Core `reject-details`, deprioritise, `generateblock` reject shapes then parent-first mining, a premature coinbase, proposal spend/value/final needles against the chain, default and explicit `maxfeerate`, `testmempoolaccept` known vs mempool vs archived, invalidate and reconsider, and a parked sibling (held `getblock`, `preciousblock`). Last, a mainnet view of the same hub refuses the regtest-only methods and still takes `submitblock`. | -| `block_cache_and_mempool_hub_surface` | Net | BlockCache locator/eviction + MempoolHub accept/remove/reorg on mature chain. `DEFAULT_BODY_DEPTH == 16` stays a unit. | +| `block_cache_and_mempool_hub_surface` | Net | BlockCache locator/eviction + MempoolHub accept/remove/reorg on mature chain. Eviction uses body depth 16. | | `store_error_and_corrupt_paths` | Store | Error/corrupt surfaces | | `store_table_header_and_idx_corrupt` | Store | Table header/head corrupt open | | `pruned_seqsigwit_life` | Query + RPC (crate) | One `query_tests` entry for the prune watermark, RAM window, reopen, refuse-disable, reorg-through-pruneheight, and corrupt spill. One `methods_tests` entry for pruned `getblock` and `getblockstats` with and without txstat. | | `chain_view_pin_asof_reorg` | Query (crate) | One `query_tests` pad: no view on an empty store, tip and buried pins across extension, as-of scripthash and outpoint answers around a spend, a same-height replace killing the tip pin and the join slot, a stale write-behind job not seeding the replaced branch, the pinned-run retry and its moved-view error, and disconnect to empty. Electrum and Esplora as-of are `electrum_and_esplora_asof_hides_later_spend` | | `resume_most_work_header_path` | Query (crate) | One `query_tests` pad with a confirmed loser tip: a heavier header-only sibling beats the loser's body, `exclude` falls back, the ancestor walk takes the heaviest fork under the grandparent once the nearer sibling is excluded, a 12k-header band ahead of the tip does not overflow the stack, and a `prev_fk` cycle ends | -| `sp_tweaks_confirm_life` | Consensus (crate) | One `silent_payments` chain through the index builder: two P2WPKH→P2TR spends around a fat ineligible spend; the naive walk, the window reader (parent inside or outside), the engine on the rebuilt wire tx, and the served index agree; the thin serve spans the middle txout but not its seqsigwit; two heights sealed in one window; range limits, the hole, singles vs range, and cut-through down to an all-spent tx. BIP352 vectors stay the `silent_payments` units | +| `sp_tweaks_confirm_life` | Consensus (crate) | One `silent_payments` chain through the index builder: two P2WPKH→P2TR spends around a fat ineligible spend; the naive walk, the window reader (parent inside or outside), the engine on the rebuilt wire tx, and the served index agree; the thin serve spans the middle txout but not its seqsigwit; two heights sealed in one window; range limits, the hole, singles vs range, and cut-through down to an all-spent tx. BIP352 vectors are the `silent_payments` consensus results | | `sh_history_caps` | Query (crate) | One `query_tests` scripthash with more creates than `--max-sh-creates`: the create count includes the pending write-behind; the full join and chain stats refuse; ascending, cursor, and newest-first (a spend of the oldest create) pages that close before the cap are still served; a separate unspent script confirms a full page past the cursor stops before loading later creates | | `chain_connect_reorg_and_growth` | Query | Synthetic growth; height index full rebuild; disconnect logs `DisconnectTip` at warn and rewinds IBD marks and write locs; disconnect to empty (then refuses); same-height replace; reconnect and idempotent tip re-confirm. Corrupt merkle in the last 6 confirmed heights: `Query::open` shrinks (`VERIFY_TIP_BLOCKS=6`). A 257-create megakey block unlinks from SH and truncates tweaks on disconnect | | `consensus_mature_chain_spend_reconstruct_and_scripthash` | Consensus+query | **One** mature mine: spend, local prev_fk, double-spend (accept, Class A then accept, and `confirm_wire_run`), reopen reconstruct (`witness_block_bytes` == serialize), SH history windows, newest-first page, net-value summary, chain stats, join slot (dropped on tip change), listunspent without spent-create identity, spend-index-off fallback, touched-at-height; merkle proof, archived span reconstruct (P2TR / P2A sibling), confirm-run refuses, clear archived body. After disconnect, a queued child of the abandoned spend is neither a TipOnly hit nor a leftover fill, `resume_work_path_after_tip` still sees Class A bodies, and reconnecting them extends the height index. Packed create_fk layout stays `input_encode_create_fk_not_prev_txid` | | `confirm_load_ahead_of_write_does_not_badprev` | Consensus+query | Pipelined load 11..=20 while 1..=10 is still unwritten; then load 21..=32 after tip-GC (store MTP for confirmed parents) | | `wire_prep_parent_layout_and_load_ahead` | Consensus+query | One mature pad: load-ahead fill of parent denserels after commit (plan stamps the head parent's fk, spend edges, same-batch overlay; write annotates the spent slot), already-archived plan=None spend annotate, cold Class A denserels for sequential spends of one create | | `resume_tx_head_resolves_external_prev` | Query+consensus | Reopen `tx.head` create_fk spend: the parent TipOnly-heads through the leftover probe and through two BQ waves (skeleton needs no probe); once archived, lookup stamps plan=None with the block's create pairs and refuses a tampered tx list; leftover TipOnly stamp matches the one connected fk; RAM leftover map clobber stays one slot | -| `buried_rules_and_a_lying_header_path` | Consensus (crate) | Regtest and signet BIP30 overwrite, mainnet BIP30 only when the BIP34 ancestor matches, and the anchored milestone, in `structure_rule_tests`. The heavier contiguous header-work path is the same name in `query_tests`. Store-less sigop and tx-count caps stay units. | -| `consensus_rules` (test binary) | Consensus | Focused reject paths for structure/header/connect rules we own — see [`docs/consensus-tests.md`](./docs/consensus-tests.md). Combined `header_and_spending_boundaries` includes H1/H2/H4/H5/H6, BIP68 height + time, and subsidy interval=2 overlay (50 BTC at interval−1, 25 BTC at interval, `subsidy+1` rejects). Same-block double spend, child-before-parent, `in < out`, and immature coinbase stay on that journey. Crate `accept_rejects_connect_spend_rules` is the LCOV pin (same-block coinbase spend, those connect rejects, 4-deep immature). Store-less `rejects_coinbase_excess_value_fast` stays with the subsidy table. H8 exact +2h stays `h8_timestamp_exactly_two_hours_accepts_plus_one_rejects`. Hornet-mapped subset: `./scripts/test-hornet-rules.sh` | +| `buried_rules_and_a_lying_header_path` | Consensus (crate) | Regtest and signet BIP30 overwrite, mainnet BIP30 only when the BIP34 ancestor matches, and the anchored milestone, in `structure_rule_tests`. The heavier contiguous header-work path is the same name in `query_tests`. A block over the sigop cap or the tx-count cap is rejected. | +| `consensus_rules` (test binary) | Consensus | Focused reject paths for structure/header/connect rules we own — see [`docs/consensus-tests.md`](./docs/consensus-tests.md). Combined `header_and_spending_boundaries` includes H1/H2/H4/H5/H6, BIP68 height + time, and subsidy interval=2 overlay (50 BTC at interval−1, 25 BTC at interval, `subsidy+1` rejects). Same-block double spend, same-block coinbase spend, child-before-parent, `in < out`, immature coinbase, and 4-deep immature are beats on `header_and_spending_boundaries`. Coinbase excess is the store-less subsidy-table result. H8 at exact +2h accepts, and one second later rejects. Hornet-mapped subset: `./scripts/test-hornet-rules.sh` | | `core_analogs::analog_milestone_and_mempool_persist` | Consensus | Milestone skip-below/check-above, missing prevout under high milestone, mempool persist (one pad). Restart leftover pool through catch-up then tip-mode relay-on: same-txid confirmed and input-conflict gone, child of a now-confirmed parent kept, DEAD marks durable without `flush`. Leftover `slots.tmp` after mid-compact: `MempoolHub` open finishes the rename and live count matches. Truncated body vs slots refuses (not empty pool), and so does the empty pool's older body behind live slots (`live slot body range`). Fee survives flush and reopen | | `core_analogs::analog_reconstruct_after_lost_head` | Store+query | Wipe `tx.head/`, reopen, reconstruct height 1 and txid probe. Crash-open clamps unsealed `confirmed[]` (Electrum/RPC `chain_tip`). Leftover fuse8 v1 refuses at `Query::open` (Class A kept). Truncated MPHF and empty `tx.head/meta` rebuild from Class A | | `core_analogs::analog_block_filters_from_class_a` | Store+query | Coinbase-only, a spend, and a block with duplicate scripts, OP_RETURN, and a segwit output; `--prune-seqsigwit` passes them (reconstruct refuses). `rbtc-idx-wb` then materializes every height from Class A through `read_index_window`, stopped after its first commit and restarted: each mined block's stored filter equals rust-bitcoin `new_script_filter` and the header chain is unbroken. The completion-session window reader (`read_index_window`, windows of 37 heights across the prune line) builds the same filter at every height. A tip reorg while the index is off: reopening with it on drops the stale-branch slot (`header_fk` ≠ `confirmed[h]`) and the rebuilt filter matches the new block (`rpc_getblockfilter.py`, `feature_blockfilterindex_prune.py`) | | `unified_wire_pipeline_multi_block_to_tip` | Consensus+query | Class A archived ahead of tip (tip stays, a second archive does not re-append, size/weight from txstat with a zero-row rebuild) then `confirm_wire_run` (no re-append + re-entry); empty run, empty or gapped load, and a body-less header all refuse without advancing tip; then heights 2..=4 unified load/scripts/write, and a BQ-queued coinbase-only block through the split stages | -| `direct_indexes_then_sh_bulk_at_tip` | Query | Direct IBD fills `tx.head` and collects no SH (no run worker, no write-behind); SH bulk at tip, idempotent on repeat and on Direct re-entry; Tip mode enqueues write-behind only with shindex on; a tip confirm leaves the SH watermark to the write-behind; `include_hwm` covers the tip without a SEAL file; wipe SH shards, reopen (leftover catch-up artifacts removed), history still works. Keep unsorted pack/lag guts | -| `electrum_server_version_history_balance` | Electrum | One mature pad: version (`CARGO_PKG_VERSION`)/history/balance/headers, ping/features (`protocol_min` / `asof_protocol` / `server_version`)/tx/errors, confirmed history omits `fee`, scripthash subscribe notify, skip restatus when a new block misses the SH. TCP `get_history` `from_height` at create / exclusive `to_height` hides spend / `to_height=-1` open; subscribe status stays full; invalid `from_height` type. `get_merkle` wrong height for a known txid. Verbose `transaction.get` stamps `time`/`blocktime`/`confirmations`/`blockhash` and coinbase `vin`/`vout`. `id_from_pos` string vs `merkle=true` `{tx_hash, merkle}`; pos OOB errors. Line at `max_request_bytes` ignored; one past is JSON-RPC `-32600` `request line too long`. Electrum **1.6**: `mempool.get_info`, `blockchain.outpoint.*` (unsubscribe miss is `false`; tip notify while subscribed; missing vout errors), Frigate `silentpayments.subscribe` / unsubscribe / second unsubscribe / start past tip clamps / bad-params error, `block.headers` as a list, `broadcast_package` without a hub errors. Keep `read_line_capped` helper, dispatch height-window units, and crate TCP no-hub `estimatefee` `-1.0` / empty histogram / Cake `tweaks.subscribe [0,1,false]` | +| `direct_indexes_then_sh_bulk_at_tip` | Query | Direct IBD fills `tx.head` and collects no SH (no run worker, no write-behind); SH bulk at tip, idempotent on repeat and on Direct re-entry; Tip mode enqueues write-behind only with shindex on; a tip confirm leaves the SH watermark to the write-behind; `include_hwm` covers the tip without a SEAL file; wipe SH shards, reopen (leftover catch-up artifacts removed), history still works, including after an unsorted pack and a lagging include | +| `electrum_server_version_history_balance` | Electrum | One mature pad: version (`CARGO_PKG_VERSION`)/history/balance/headers, ping/features (`protocol_min` / `asof_protocol` / `server_version`)/tx/errors, confirmed history omits `fee`, scripthash subscribe notify, skip restatus when a new block misses the SH. TCP `get_history` `from_height` at create / exclusive `to_height` hides spend / `to_height=-1` open; subscribe status stays full; invalid `from_height` type. `get_merkle` wrong height for a known txid. Verbose `transaction.get` stamps `time`/`blocktime`/`confirmations`/`blockhash` and coinbase `vin`/`vout`. `id_from_pos` string vs `merkle=true` `{tx_hash, merkle}`; pos OOB errors. Line at `max_request_bytes` ignored; one past is JSON-RPC `-32600` `request line too long`. Electrum **1.6**: `mempool.get_info`, `blockchain.outpoint.*` (unsubscribe miss is `false`; tip notify while subscribed; missing vout errors), Frigate `silentpayments.subscribe` / unsubscribe / second unsubscribe / start past tip clamps / bad-params error, `block.headers` as a list, `broadcast_package` without a hub errors. With no hub, `estimatefee` is `-1.0`, the histogram is empty, and Cake `tweaks.subscribe [0,1,false]` answers | | `electrum_scripthash_sub_cap_unsubscribe_frees_slot` | Electrum | Per-connection scripthash cap + unsubscribe frees a slot. Outpoint subscriptions use that same numeric cap on their own set (resubscribe does not consume another slot; a third distinct outpoint is `max 2` until unsubscribe). | | `electrum_leftover_mempool_does_not_double_count` | Electrum | Relay-off leftover is confirmed, not a second mempool UTXO. With hub attached, `transaction.broadcast` of non-hex and consensus-invalid rejects (not hang / not admit). Same pad: confirmed `outpoint.get_status` (tip height, no `spending_txid`); `broadcast_package` invalid hex / reject / verbose success / non-verbose `"success"`; mempool-spent outpoint is `height=0` with `spending_txid`; mempool verbose `transaction.get` is `confirmations=0` with no `time`/`blocktime`/`blockhash` | | `electrum_and_esplora_asof_hides_later_spend` | Electrum + Esplora | One pad: TCP `1.4.2-asof` plus HTTP `?asof=` hide a later spend; unknown asof errors; `GET /tx` `v0_p2wpkh` vout. SH-off tip+1: asof at visible SH watermark accepts, asof of the confirmed hash ahead of SH is `asof not on chain` / HTTP 404. Same-height A-B-A restatuses subscribe (confirming blockhash); `asof:` of the loser hash does not retry onto the sibling. Disconnect spend: history/utxo show the create again; live Esplora `/utxo` stamps the create hash (not the fork); loser `?asof=` 404 with no tip header. HTTP `503` `chain view moved` (no tip header) is `http_503_chain_view_moved_omits_tip_header` | | `electrum_empty_chain_headers_subscribe_and_empty_scripthash` | Electrum | Empty store: `headers.subscribe` errors; scripthash history/balance/unspent/mempool empty | -| `electrum_tweaks_subscribe_streams_then_done` | Electrum | Cake `tweaks.subscribe`: one-height result, per-height notifies, `done`, and BIP352 hash-bind on a P2WPKH→P2TR spend. Keep zero-chunk / pre-taproot units | +| `electrum_tweaks_subscribe_streams_then_done` | Electrum | Cake `tweaks.subscribe`: one-height result, per-height notifies, `done`, and BIP352 hash-bind on a P2WPKH→P2TR spend. A zero-chunk subscribe returns wave 0 and then `done` while heights remain, and a resubscribe continues. Pre-taproot empty heights collapse into one notify | | `electrum_max_connections_rejects_extra_client` | Electrum | TCP cap drops the extra client | | `electrum_idle_timeout_disconnects_quiet_client` | Electrum | Idle timeout closes a quiet socket | -| `esplora_broadcast_visible_in_rpc_and_electrum` | Node + Electrum + Esplora + RPC | One `run_p2p` datadir: HTTP `sendrawtransaction` / `testmempoolaccept` (allowed, missing-or-spent, exact 100 sat/kvB min-relay accept + one-sat-under reject, RBF one-sat-short incremental reject + exact incremental accept); Esplora `POST /tx` parent and mempool child appear in `getrawmempool` and Electrum mempool/history (`fee` on unconfirmed, including child `height = -1`); Electrum `listunspent` of that child is `height=-1` and the parent UTXO drops; process `gettxout` / `getchaintips`; Esplora `POST /txs/package` 1p1c (including parent-alone below min-relay + paying child), 25-tx accept, 26-tx and over-weight `package too large`; serving-only `submitpackage` refuses (relay off); live `GET /mempool` / `/mempool/txids` / `/mempool/recent` / `/fee-estimates` answers 503 while flow is cold and the chain holds too little fee history (a thin live pool alone sets no rate); process `getmempoolancestors` / descendants / cluster / `gettxspendingprevout` / feerate diagram / verbose `getrawmempool` on that 1p1c; `waitforblockheight` timeout=0 while behind returns the live tip; GBT stale `longpollid` is immediate; current id / `waitfornewblock` / `waitforblockheight` wake on the pad `generate`; `getblockhash` tip ok / tip+1 `-8`; unknown `getblock` `-5`; verbosity 0 hex and 2 vin/vout; `GET /blocks` 10 newest, `/blocks/0` and `/blocks/:tip` (start past tip clamps); `/block/:hash/txs/:start` last page shorter than 25, one-past last page 404, unknown hash 404; `/block/:hash/txids` + coinbase merkle-proof + unspent `outspend/0`; `/tx/:id/outspends`; `/block` JSON/raw/status/`txid/0` (OOB 404); `/tx/:id/raw` vs hex; merkleblock-proof; `/block-height` (missing 404); `/block/:hash/header` 160 hex; `/tx/:id/status` + full JSON (`unknown` OP_TRUE type, coinbase vin) and missing-tx 404s; OP_TRUE scripthash info/summary/utxo/`txs/chain` cursor and combined `/txs`. Keep crate no-hub mempool/fees/POST 503, reconstruct meters, header wire match, and `tx_status_json`; Esplora `/tx/:id/status` confirms the package parent on generate; `generate` includes those txs (parent before child) then leaves IBD (relay on); `scantxoutset` drops the spent coinbase and still sees a non-coinbase unspent; `submitpackage` maxfeerate reject, 1p1c success, already-in-mempool continue, below-min-relay parent + paying child success, 26-tx / over-weight `package too large`; immature coinbase sendraw rejects. Keep `accept.rs` reject units, package JSON errors, RPC dry-run orphan-count, leftover `gettxout` include_mempool / disconnected / leftover, `generate_selects_chained_mempool_parent_first`, `submitpackage_child_fail_keeps_parent`, maxburn `submitpackage`, and wait-on-stop units. Unix `--rpc-socket` (mode 0660, no datadir `rpc.sock`) `getblockcount` without Authorization; TCP `GET`/`POST /internal/mempool/txs`; `GET /internal/block/:hash/txs` full list vs public 25/page; `POST /internal/txs/outspends/by-txid` same-length unknown `[]` slot; `GET /address-prefix/bc1` **404**; unauthenticated Core REST on the RPC listener (`chaininfo`, block hash/headers/block/tx, mempool info/contents, `getutxos`, `deploymentinfo`, basic `blockfilter` bin/hex/json) and `getblockfilter` with `--block-filter-index` | +| `esplora_broadcast_visible_in_rpc_and_electrum` | Node + Electrum + Esplora + RPC | One `run_p2p` datadir: HTTP `sendrawtransaction` / `testmempoolaccept` (allowed, missing-or-spent, exact 100 sat/kvB min-relay accept + one-sat-under reject, RBF one-sat-short incremental reject + exact incremental accept); Esplora `POST /tx` parent and mempool child appear in `getrawmempool` and Electrum mempool/history (`fee` on unconfirmed, including child `height = -1`); Electrum `listunspent` of that child is `height=-1` and the parent UTXO drops; process `gettxout` / `getchaintips`; Esplora `POST /txs/package` 1p1c (including parent-alone below min-relay + paying child), 25-tx accept, 26-tx and over-weight `package too large`; serving-only `submitpackage` refuses (relay off); live `GET /mempool` / `/mempool/txids` / `/mempool/recent` / `/fee-estimates` answers 503 while flow is cold and the chain holds too little fee history (a thin live pool alone sets no rate); process `getmempoolancestors` / descendants / cluster / `gettxspendingprevout` / feerate diagram / verbose `getrawmempool` on that 1p1c; `waitforblockheight` timeout=0 while behind returns the live tip; GBT stale `longpollid` is immediate; current id / `waitfornewblock` / `waitforblockheight` wake on the pad `generate`; `getblockhash` tip ok / tip+1 `-8`; unknown `getblock` `-5`; verbosity 0 hex and 2 vin/vout; `GET /blocks` 10 newest, `/blocks/0` and `/blocks/:tip` (start past tip clamps); `/block/:hash/txs/:start` last page shorter than 25, one-past last page 404, unknown hash 404; `/block/:hash/txids` + coinbase merkle-proof + unspent `outspend/0`; `/tx/:id/outspends`; `/block` JSON/raw/status/`txid/0` (OOB 404); `/tx/:id/raw` vs hex; merkleblock-proof; `/block-height` (missing 404); `/block/:hash/header` 160 hex; `/tx/:id/status` + full JSON (`unknown` OP_TRUE type, coinbase vin) and missing-tx 404s; OP_TRUE scripthash info/summary/utxo/`txs/chain` cursor and combined `/txs`. No-hub: mempool and fee routes, and `POST /tx` 503. Header bytes match the wire header. `tx_status_json` matches the status route. Esplora `/tx/:id/status` confirms the package parent on generate; `generate` includes those txs (parent before child) then leaves IBD (relay on); `scantxoutset` drops the spent coinbase and still sees a non-coinbase unspent; `submitpackage` maxfeerate reject, 1p1c success, already-in-mempool continue, below-min-relay parent + paying child success, 26-tx / over-weight `package too large`; immature coinbase sendraw rejects. Package JSON errors. `testmempoolaccept` dry-run reports the orphan count. `gettxout` with `include_mempool`, on a disconnected tip, and on a leftover. `generate` selects the chained mempool parent first. A `submitpackage` child failure keeps the parent. Maxburn `submitpackage` rejects. Waiters return when the node stops. Unix `--rpc-socket` (mode 0660, no datadir `rpc.sock`) `getblockcount` without Authorization; TCP `GET`/`POST /internal/mempool/txs`; `GET /internal/block/:hash/txs` full list vs public 25/page; `POST /internal/txs/outspends/by-txid` same-length unknown `[]` slot; `GET /address-prefix/bc1` **404**; unauthenticated Core REST on the RPC listener (`chaininfo`, block hash/headers/block/tx, mempool info/contents, `getutxos`, `deploymentinfo`, basic `blockfilter` bin/hex/json) and `getblockfilter` with `--block-filter-index` | | `fee_history_backfills_from_the_chain_when_relay_starts` | Node + RPC | `run_p2p` on a mature regtest datadir, started twice; `generate` leaves IBD and turns relay on; the preload writes the fee history file and the restart extends it; with flow cold and too little history, `estimatesmartfee` answers Core's insufficient-data shape. Rates from a ready history are the hub's `far_horizon_follows_block_history_not_pool_tail`; the success object is `smart_fee_json` | | `node_listen_and_exit` | Node + Electrum + Esplora + RPC | One `run_p2p` datadir, restarted with its one `--connect` refusing (a pinned connect at genesis still enters tip mode): a taken `--health-listen` port stops the start before the store opens; a junk `peers` file and a missing `--asmap` start an empty book and exit, and the saved book records the refused connect; the next start loads that book and a valid `ip_asn.dat`, and Esplora, Electrum, and RPC answer at genesis until `stop`. On that start `/healthz` is 200 and `/progress` is 200 JSON while genesis `/readyz` is initial block download and `/metrics` reports the same; `generate` makes `/readyz` 200 and the scrape ready at height 1. An Electrum port and an RPC port another process holds warn, `/readyz` names both, and `/metrics` is absent without `--metrics`. Without `--connect` and with seeds on, regtest resolves none and the node exits short of tip mode; after a `--prune-seqsigwit` start, an unpruned start refuses. Live peers are `node_run_p2p_short`. The `/readyz` phase, lag, and tip-age table stays `readiness_reports_the_first_failing_gate`. A timed-out probe keeping its permit, the full-gate 503, and the routes answering again are `health_gate_caps` | | `tor_control_onion_lifecycle` | Node + RPC | One `run_p2p` datadir against a fake Tor control port and SAM bridge (live Tor and i2pd are overlay-functional). A cookie from another Tor (SAFECOOKIE server hash mismatch), a 2-byte cookie, and a Tor that offers only plain COOKIE each refuse the start, and none sends `AUTHENTICATE`. Password auth with `--listen-onion`, `--i2p-accept-incoming`, Electrum, and Esplora: `ADD_ONION NEW` per service with the P2P virtual port on the loopback bind, each key saved `0600` under `onion/`, each SAM destination under `i2p/`, `STREAM FORWARD` to each port, and `getnetworkinfo.localaddresses` lists the three onions and the I2P address. A SAFECOOKIE restart reuses every saved key and destination | | `enter_tip_mode_indexes` | Node + Electrum + RPC | One `run_p2p` datadir restarted with `--sh-index` off, on, off, on. Off with no index yet: RPC, tip follow, and Electrum listen, `blockchain.scripthash.get_history` fails closed, and `generateblock` mines three OP_TRUE coinbases. First start on: the index is collected from Class A before history answers, and the OP_TRUE history has three rows. Off again: Electrum still listens and the leftover watermark keeps those three history rows. On after a crash that left a collect run and a lagging include high-water mark: the durable index resumes under write-behind, so the run is discarded (not merged) and history answers, and the next block lands in history | | `two_node_header_and_block_sync` | P2P (**default**) | Seeder → peer genesis+1 IBD; peer `last_write` meter. Empty `headers` lag keep-sync is `apply_peer_event_body_and_control_surface`; drained-path EOF `headers_done` is `apply_peer_event_block_framed_bq_horizon_and_headers_done`. 8-block dual-seeder stays `ibd_two_peers` | -| `p2p_timeout_getaddr_and_keepalive_ping` | P2P (**default**) | One pad: v1-magic inbound drops at `peertimeout=1`, obsolete VERSION and pre-verack ping close the peer, full-relay GetAddr cache 1000, headers-sync stall replace, self-connect refuses, a completed handshake outlives `peertimeout=1`, AddrFetch `getaddr`/`addrv2` (no `getheaders`) stays for one addr, times out at 300s, and completes on a longer list, one keepalive ping/pong. Handshake **format** needles stay. Sole-preferred stall KEEP stays a PeerHub unit (`noban_headers_timeout_clears_awaiting_so_a_new_getheaders_can_send`: CIDR `noban@127.0.0.1` and hub `--trusted`). | -| `hostile_peer_session` | P2P (**default**, crate) | One lib entry in `peer::tests`, one in `chain::tests`, one in `assign::tests`. Header cap, send budget, one-shot `getaddr`, addr relay to one or two peers, zero-prev not held, witness padding and time-too-new not cached invalid, getdata stops at the byte budget. `tip_script_pres_skips_only_matching_wtxid` stays its own test (mempool graph, not the peer session). | +| `p2p_timeout_getaddr_and_keepalive_ping` | P2P (**default**) | One pad: v1-magic inbound drops at `peertimeout=1`, obsolete VERSION and pre-verack ping close the peer, full-relay GetAddr cache 1000, headers-sync stall replace, self-connect refuses, a completed handshake outlives `peertimeout=1`, AddrFetch `getaddr`/`addrv2` (no `getheaders`) stays for one addr, times out at 300s, and completes on a longer list, one keepalive ping/pong. A sole preferred peer that stalls headers is kept, and a new `getheaders` can send (`noban@127.0.0.1`, hub `--trusted`). | +| `hostile_peer_session` | P2P (**default**, crate) | One lib entry in `peer::tests`, one in `chain::tests`, one in `assign::tests`. Header cap, send budget, one-shot `getaddr`, addr relay to one or two peers, zero-prev not held, witness padding and time-too-new not cached invalid, getdata stops at the byte budget. `tip_script_pres_skips_only_matching_wtxid` is the mempool-graph result, not this peer session. | | `peer_blocksonly_and_orphan_tx` | P2P (**default**, crate) | One hub, relay off and then on. Blocks-only: a tx or wtx inv from an ordinary peer disconnects, a sendraw INVs once it is unbroadcast (never to block-relay) and serves, a `relay@` whitelisted peer's tx is kept and INV'd to the other inbound, and a type-0 getdata is ignored while the tip block still serves. A block confirms those txs and relay-on purges them. Then forced INVs skip block-relay, seen, feefilter, and isolated local-origin peers. Inbound waits 30s even for unbroadcast, never gets a tx older than its connection, and idle ticks do not clone or rescan. GetData serves only an announced or reorged-back tx, and a sendraw after a +300s jump neither INVs nor serves. An orphan parks on a tokio worker without a reject log and asks only the still-missing parent after NONPREF+TXID. | | `stored_header_resends_walk_once` | IBD (crate) | One 240-header genesis chain. A re-sent stored run walks no ancestors, an unmapped stored run walks once, an unknown parent is not stored, a rejected tail keeps the stored prefix under a lowered walk cap, and a run past that cap walks once and is still accepted. | | `two_peers_reserve_the_header_walk` | IBD (crate) | One hub. The lowest time-to-first-byte peer takes headers and the other takes blocks; a short miss moves the reservation without disconnecting; a heavier inv takes it and a lighter inv restores it; refill under the low-water mark stays on the reserved peer. A second chapter, because that lighter challenger is retired: a quiet header peer is skipped, asked again, then disconnected. A third, because that survivor already missed once: the only header peer is disconnected after two misses. | @@ -364,9 +384,9 @@ Prefer **one high-level scenario** per behavior cluster. Delete lower-level test | `analog_selector_keeps_each_boundary_rate` | Mempool (crate) | One selector table: the exact band edge, absolute log distance, the requested quantile, and the two-hundredth nearest window each keep their own rate. | | `fee_history_gaps_cache_and_reorg_share_one_history` | IBD (crate) | One history: heights without a hurdle are not observations, rates stay cached until a new height, only the newest hashes are kept, and a reorg drops those above. The RPC backfill stays `fee_history_backfills_from_the_chain_when_relay_starts`. | | `peer_header_dos_and_self_announce` | P2P (**default**, crate) | One hub and one peer: verack order (wtxidrelay and sendaddrv2 stick, ping is logged, redundant verack is ignored, sendaddrv2 and oversized addrv2 after verack disconnect), unknown-parent block and compact, a full header batch continues from its last header, minchainwork stays silent until the floor, bad proof-of-work disconnects and time-too-new does not, empty-locator serves only a hash that has a body, an ancient weaker header disconnects unless noban, externalip is daily, `--no-discover` suppresses it, and onion / i2p replace that clearnet address. A noban peer stays up on a bad block and gathers no score. A version-3 header at CLTV activation logs `bad-version`, and a far stamp logs `time-too-new`. | -| `p2p_compact_hb_getblocktxn_and_orphan` | P2P (**default**) | One mature pad: HB coinbase `cmpctblock`, 2-tx compact → `getblocktxn` then same-peer compact retry while pending + `blocktxn` connect, unique short-id fill that fails header merkle → `getdata` (not `getblocktxn`, header not `BLOCK_FAILED`) then honest full `block` connects, orphan child GetData then parent accept (INV AlreadyHave), then live `getblocks` → `inv`, inbound `feefilter`, `filterload` disconnect. The pad seals filters through height 1, does not advertise `NODE_COMPACT_FILTERS` while the tip is ahead, stays silent for a stop at the tip, and answers `getcfilters` / `getcfheaders` / `getcfcheckpt` for the sealed height. Oversize locator and MemPool/`filteradd`/`filterclear` stay PeerHub units. Live mutated `block` disconnects in `on_block`. Inbound `getdata` of 20 witness blocks serves `MAX_SERVE_BLOCKS` (16); 17th is not queued (`getdata_skips_reconstruct_when_serve_inflight_at_cap`). Same-peer pending compact skip is `same_peer_pending_cmpct_does_not_getblocktxn_again`. Tokio-worker park and park-not-reject logs are `peer_blocksonly_and_orphan_tx` | +| `p2p_compact_hb_getblocktxn_and_orphan` | P2P (**default**) | One mature pad: HB coinbase `cmpctblock`, 2-tx compact → `getblocktxn` then same-peer compact retry while pending + `blocktxn` connect, unique short-id fill that fails header merkle → `getdata` (not `getblocktxn`, header not `BLOCK_FAILED`) then honest full `block` connects, orphan child GetData then parent accept (INV AlreadyHave), then live `getblocks` → `inv`, inbound `feefilter`, `filterload` disconnect. The pad seals filters through height 1, does not advertise `NODE_COMPACT_FILTERS` while the tip is ahead, stays silent for a stop at the tip, and answers `getcfilters` / `getcfheaders` / `getcfcheckpt` for the sealed height. An oversize locator is rejected. `mempool`, `filteradd`, and `filterclear` disconnect. Live mutated `block` disconnects in `on_block`. Inbound `getdata` of 20 witness blocks serves `MAX_SERVE_BLOCKS` (16); 17th is not queued (`getdata_skips_reconstruct_when_serve_inflight_at_cap`). Same-peer pending compact skip is `same_peer_pending_cmpct_does_not_getblocktxn_again`. Tokio-worker park and park-not-reject logs are `peer_blocksonly_and_orphan_tx` | | `p2p_feeler_completes_and_closes` | P2P (**default**) | Outbound feeler: VERSION then close (`feeler connection completed`). No live follow; dummy has no completed inbound. Same test: `run_feeler_timed` silence is `Timeout`. Inbound/outbound/plain silence stay `handshake_timeout_after_silence` | -| `p2p_inbound_full_rejects_extra` | P2P (**default**) | `max_inbound=1`: second follow is refused; first inbound stays. Same test: `select_inbound_eviction` 21-cand ranking (4 block + 5 slow + 4 tx + 8 ping → victim in slow). noban-alone stays a unit | +| `p2p_inbound_full_rejects_extra` | P2P (**default**) | `max_inbound=1`: second follow is refused; first inbound stays. Same test: `select_inbound_eviction` 21-cand ranking (4 block + 5 slow + 4 tx + 8 ping → victim in slow). A peer that is only noban is not the eviction victim | | `badprev_orphan_does_not_blacklist_then_reorg_reconstructs` | P2P/chain (default) | Orphan whose prev is not on the tip is held (not `BLOCK_FAILED`); winner branch reconstructs. A mutated child of a held sibling is not `BLOCK_FAILED`, forgets the ask, and the honest body reorgs onto it | | `serve_after_restart_via_reconstruct` | P2P (**default**) | Cold serve via reconstruct. Restart RAM body queue is empty. Same-process `rehydrate_block_queue_residue` drops at/below tip, skips empty payloads, keeps above-tip wire, unknown height stays queued. `has_block` / known-archived keep and tip+1 gap `missing` stay `bq_rehydrate_residue_keep_drop_gap_and_unknown` | | `ibd_skips_dead_peer` | P2P (**default**) | Live seeder + `127.0.0.1:1` | @@ -379,7 +399,7 @@ Prefer **one high-level scenario** per behavior cluster. Delete lower-level test | `end_of_ibd_follow` | P2P (**default**) | Miner plus `--sh-index` syncer. One mature regtest: coinbases pay script A, one spend pays script B. IBD reaches that tip, leaves IBD, and Electrum history matches. The next block tip-follows. Restart does not rewrite the scripthash pack mark; one more block arrives by write-behind. Cancelling IBD once the height is below the miner, then `run_p2p`, finishes the same tip and history. With the syncer caught up, dropping the miner leaves tip follow (not IBD). A partial datadir whose miner is down does not open Electrum and stays short of that tip; the same miner address coming back lets that datadir finish | | `end_of_ibd_sh_interrupt` | P2P (**default**) | Same miner and scripts. The syncer first catches up with `--sh-index` off, then the datadir is frozen after pass 1 (`DONE.keys`, no `DONE.post`). Resume builds the index and Electrum's first answer is the full A/B history; restart does not rewrite the pack mark. A block mined after the freeze is in that history. A copy that already has the block, with `include_hwm` at the new tip and that create appended on the unsealed head, still serves the same history and does not open Electrum early | | `end_of_ibd_work_fork` | P2P (**default**) | Two miners on a regtest with retargeting and min-difficulty. One chain retargets harder and stops shorter. The other forks after that retarget, resets to the pow limit, and grows taller with less work. The syncer IBD-adopts the heavier tip, keeps it across a reopen, and follows one more heavy block while the tall chain is still ahead | -| `node_run_p2p_short` | Node (**default**) | Product `run_p2p` `--blocks-only` `--connect` to a live seeder (`--max-tip-age` so the 3-block pad is not stale IBD); process `getpeerinfo` / `getconnectioncount` / `getnetworkinfo` / `getnettotals` / `ping` while connected (v2 `manual`, as Core reports `-connect`; handshake `startingheight` equals the seeder tip; `timeoffset` present; `synced_headers`/`synced_blocks` stay `-1` until the peer announces a header hash (empty getheaders at tip does not copy VERSION height); `servicesnames` present; `getnetworkinfo.timeoffset` present); after catch-up `localrelay` / mempool `relay_enabled` stay false and `sendrawtransaction` is not `relay disabled`; Electrum `broadcast` and Esplora `POST /tx` admit decode/consensus errors (not hub-missing / not `relay disabled`); `addconnection inbound` refuses; `disconnectnode` unknown `nodeid` / empty params error then a real addr drops that row from the next `getpeerinfo`, the seeder sees it go, and a second `disconnectnode` of that `nodeid` (or of an address never connected) is `-29`; the `--connect` peer is redialled as `manual` under a new id and the seeder sees the inbound; seeder inbound `tx` then ends that session. Exit via `stop`. `max_run_secs=0` is `node_listen_and_exit`. Mock-clock `timeoffset` median (odd N, even N upper-middle, inbound-only 0, peer clock behind), connecting dummy `-1`, header-only vs connected `synced_blocks`, query-without-chain, `pingwait` / `NETWORK_LIMITED` / `noban` stay RPC guts | +| `node_run_p2p_short` | Node (**default**) | Product `run_p2p` `--blocks-only` `--connect` to a live seeder (`--max-tip-age` so the 3-block pad is not stale IBD); process `getpeerinfo` / `getconnectioncount` / `getnetworkinfo` / `getnettotals` / `ping` while connected (v2 `manual`, as Core reports `-connect`; handshake `startingheight` equals the seeder tip; `timeoffset` present; `synced_headers`/`synced_blocks` stay `-1` until the peer announces a header hash (empty getheaders at tip does not copy VERSION height); `servicesnames` present; `getnetworkinfo.timeoffset` present); after catch-up `localrelay` / mempool `relay_enabled` stay false and `sendrawtransaction` is not `relay disabled`; Electrum `broadcast` and Esplora `POST /tx` admit decode/consensus errors (not hub-missing / not `relay disabled`); `addconnection inbound` refuses; `disconnectnode` unknown `nodeid` / empty params error then a real addr drops that row from the next `getpeerinfo`, the seeder sees it go, and a second `disconnectnode` of that `nodeid` (or of an address never connected) is `-29`; the `--connect` peer is redialled as `manual` under a new id and the seeder sees the inbound; seeder inbound `tx` then ends that session. Exit via `stop`. `max_run_secs=0` is `node_listen_and_exit`. Mock-clock `timeoffset` is the median (odd N, even N upper-middle, inbound-only 0, peer clock behind). A connecting dummy reports `-1`. Header-only and connected peers report `synced_blocks` differently. A query without a chain errors. `pingwait`, `NETWORK_LIMITED`, and `noban` are on the peer row | Removed (covered by the rows above): `confirm_cross_block_prevout_without_tx_head`, `double_archive_keeps_tx_height_for_coinbase_maturity`, `mega_batch_duplicate_header_is_idempotent`, @@ -593,8 +613,11 @@ Do not treat a package-only `cargo mutants` run as the gate: a journey outside the mutated crate must be able to catch the mutant. A new production behavior still needs a test that fails when that behavior -is removed or inverted. Put that assert on a catalog journey, or on a -store-less unit when the journey cannot see it. `MISSED` in the nightly -artifact means no workspace test cared. Extend the journey. A same-crate +is removed or inverted. Put that assert on a catalog journey. A `MISSED` +mutant that changes a result a peer, client, or operator can observe is +killed by extending that journey. A `MISSED` mutant on an expression no +session can flip into a different client-visible result is a candidate to +delete the expression, not a candidate for a new unit. The nightly job +stays an oracle. It does not become a pull-request check. A same-crate twin that only existed to satisfy a package-local mutant is a deletion candidate once a workspace run shows the journey catching it. diff --git a/crates/rbitcoin-sv2/src/lib.rs b/crates/rbitcoin-sv2/src/lib.rs index 0603ecc2d..758209eda 100644 --- a/crates/rbitcoin-sv2/src/lib.rs +++ b/crates/rbitcoin-sv2/src/lib.rs @@ -132,6 +132,12 @@ pub async fn run_sv2_tp(config: Sv2TpConfig) -> io::Result { continue; } }; + // Linux autotunes SO_SNDBUF up to tcp_wmem max. write() keeps + // completing into that buffer after the peer stops reading, so + // the deadline does not start. The write-deadline test pins a + // small buffer on the accepted socket. + #[cfg(all(test, target_os = "linux"))] + test_send_buffer::apply(&stream); let Ok(slot) = slots.clone().try_acquire_owned() else { rbitcoin_log::warn!("sv2: reject {peer} (at max_sessions={MAX_SESSIONS})"); drop(stream); @@ -180,6 +186,63 @@ pub async fn run_sv2_tp(config: Sv2TpConfig) -> io::Result { }) } +/// Accepted-socket `SO_SNDBUF` for +/// `client_that_stops_reading_is_dropped_at_the_write_deadline`. +/// +/// `write()` returns as soon as the bytes fit in the send buffer. With +/// autotune that is several MiB, so a client that never reads still looks +/// like a live writer and the deadline never starts. +#[cfg(all(test, target_os = "linux"))] +mod test_send_buffer { + use std::os::fd::AsRawFd; + use std::sync::atomic::{AtomicU32, Ordering}; + use tokio::net::TcpStream; + + static BUF: AtomicU32 = AtomicU32::new(0); + + pub(crate) struct Guard; + + pub(crate) fn pin(bytes: u32) -> Guard { + BUF.store(bytes, Ordering::SeqCst); + Guard + } + + impl Drop for Guard { + fn drop(&mut self) { + BUF.store(0, Ordering::SeqCst); + } + } + + pub(super) fn apply(stream: &TcpStream) { + let n = BUF.load(Ordering::SeqCst); + if n == 0 { + return; + } + unsafe extern "C" { + fn setsockopt(fd: i32, level: i32, opt: i32, val: *const i32, len: u32) -> i32; + fn getsockopt(fd: i32, level: i32, opt: i32, val: *mut i32, len: *mut u32) -> i32; + } + // linux/asm-generic/socket.h: SOL_SOCKET = 1, SO_SNDBUF = 7. + let fd = stream.as_raw_fd(); + let val = n as i32; + // SAFETY: `fd` is the accepted stream. `val` is one i32 and `len` is + // its size. SOL_SOCKET / SO_SNDBUF take that pointer. + let rc = unsafe { setsockopt(fd, 1, 7, &val, 4) }; + assert_eq!(rc, 0, "SO_SNDBUF: {}", std::io::Error::last_os_error()); + let mut got = 0i32; + let mut len = 4u32; + // SAFETY: same socket. `got` is one i32 and `len` is in-out its size. + let rc = unsafe { getsockopt(fd, 1, 7, &mut got, &mut len) }; + assert_eq!(rc, 0, "SO_SNDBUF get: {}", std::io::Error::last_os_error()); + // The kernel doubles the value for bookkeeping. Above 64KiB means the + // pin did not stick and autotune is still in effect. + assert!( + got > 0 && got <= 64 * 1024, + "SO_SNDBUF stayed {got} after requesting {n}" + ); + } +} + #[cfg(test)] mod listener_tests; #[cfg(test)] diff --git a/crates/rbitcoin-sv2/src/listener_tests.rs b/crates/rbitcoin-sv2/src/listener_tests.rs index a74393041..1920cc04e 100644 --- a/crates/rbitcoin-sv2/src/listener_tests.rs +++ b/crates/rbitcoin-sv2/src/listener_tests.rs @@ -5,6 +5,7 @@ use common_messages_sv2::{ SetupConnectionError, SetupConnectionSuccess, MESSAGE_TYPE_SETUP_CONNECTION_ERROR, MESSAGE_TYPE_SETUP_CONNECTION_SUCCESS, }; +use std::io; use std::net::SocketAddr; use std::time::{Duration, Instant}; use template_distribution_sv2::MESSAGE_TYPE_REQUEST_TRANSACTION_DATA_ERROR; @@ -226,6 +227,9 @@ async fn session_without_constraints_is_dropped_at_the_setup_deadline() { async fn client_that_stops_reading_is_dropped_at_the_write_deadline() { let tc = padded_chain("sv2-write-deadline", 0); let write_timeout = Duration::from_millis(200); + // Pin before accept. Drop clears it so other tests keep autotune. + #[cfg(target_os = "linux")] + let _send_buf = crate::test_send_buffer::pin(8 * 1024); let tp = run_sv2_tp(Sv2TpConfig { listen: "127.0.0.1:0".parse().unwrap(), chain: std::sync::Arc::clone(&tc.chain), @@ -237,7 +241,9 @@ async fn client_that_stops_reading_is_dropped_at_the_write_deadline() { }) .await .expect("listen"); - let mut c = TpClient::connect(tp.local_addr, tp.authority_pubkey) + // Pin the receive window too. A small SO_SNDBUF still drains while this + // socket ACKs into an autotuned window, so write() never stalls. + let mut c = TpClient::connect_recv_buffer(tp.local_addr, tp.authority_pubkey, Some(2048)) .await .expect("handshake"); c.setup_connection(TDP, 2, 2, 0).await.unwrap(); @@ -247,29 +253,26 @@ async fn client_that_stops_reading_is_dropped_at_the_write_deadline() { // stale tip holds the template, so constraints add no traffic. c.coinbase_output_constraints(0, 0).await.unwrap(); - // Each unknown id answers RequestTransactionData.Error, which the - // client never reads: the TP blocks on write, then stops reading. - let mut jammed = false; - for id in 1..=1_000_000u64 { - let sent = tokio::time::timeout(write_timeout, c.request_transaction_data(id)).await; - // A stall past the write deadline, or a fast write error once the - // closed session turns sends into EPIPE: the pipe is dead either way. - if !matches!(sent, Ok(Ok(()))) { - jammed = true; - break; + // Each unknown id answers RequestTransactionData.Error. This client never + // reads those frames. Keep sending until the stalled write closes the + // socket: a cancelled send is not a full window, and stopping there + // leaves the session idle so the deadline never starts. Yield so the + // session task runs; a ready send does not, and the flood only fills + // the kernel buffer. + let outcome = tokio::time::timeout(Duration::from_secs(5), async { + for id in 1..=1_000_000u64 { + c.request_transaction_data(id).await?; + tokio::task::yield_now().await; } - } - assert!(jammed, "socket buffers never filled"); - tokio::time::sleep(write_timeout * 3).await; - - let drained = tokio::time::timeout(Duration::from_secs(10), async { - while c.recv().await.is_ok() {} + Ok::<(), io::Error>(()) }) .await; - assert!( - drained.is_ok(), - "session must close once its write stalls past the deadline" - ); + match outcome { + Ok(Err(e)) if e.kind() != io::ErrorKind::TimedOut => {} + Ok(Err(e)) => panic!("client write timed out; session stayed open: {e}"), + Ok(Ok(())) => panic!("socket buffers never filled"), + Err(_) => panic!("session must close once its write stalls past the deadline"), + } tp.shutdown().await; } diff --git a/crates/rbitcoin-sv2/src/testutil.rs b/crates/rbitcoin-sv2/src/testutil.rs index 306eb5b16..e6c7a7cb1 100644 --- a/crates/rbitcoin-sv2/src/testutil.rs +++ b/crates/rbitcoin-sv2/src/testutil.rs @@ -10,7 +10,7 @@ use template_distribution_sv2::{ MESSAGE_TYPE_COINBASE_OUTPUT_CONSTRAINTS, MESSAGE_TYPE_REQUEST_TRANSACTION_DATA, MESSAGE_TYPE_SUBMIT_SOLUTION, }; -use tokio::net::TcpStream; +use tokio::net::TcpSocket; pub struct TpClient { conn: NoiseConn, @@ -19,7 +19,35 @@ pub struct TpClient { impl TpClient { /// TCP connect and complete the NX handshake against `authority_pubkey`. pub async fn connect(addr: SocketAddr, authority_pubkey: [u8; 32]) -> io::Result { - let stream = TcpStream::connect(addr).await?; + Self::connect_recv_buffer(addr, authority_pubkey, None).await + } + + /// [`Self::connect`] with `SO_RCVBUF` pinned when `recv_buffer` is set. + /// + /// An unset buffer autotunes up to `tcp_rmem` max. The peer's `write()` + /// then keeps completing, because this socket ACKs everything into that + /// window. Setting the option turns autotune off. The kernel doubles the + /// value; the handshake reply still fits. + pub(crate) async fn connect_recv_buffer( + addr: SocketAddr, + authority_pubkey: [u8; 32], + recv_buffer: Option, + ) -> io::Result { + let socket = match addr { + SocketAddr::V4(_) => TcpSocket::new_v4()?, + SocketAddr::V6(_) => TcpSocket::new_v6()?, + }; + if let Some(n) = recv_buffer { + socket.set_recv_buffer_size(n)?; + // Doubled for bookkeeping, and at least `tcp_rmem` min. + let got = socket.recv_buffer_size()?; + if got > 64 * 1024 { + return Err(io::Error::other(format!( + "SO_RCVBUF stayed {got} after requesting {n}" + ))); + } + } + let stream = socket.connect(addr).await?; let initiator = noise_sv2::Initiator::from_raw_k(authority_pubkey) .map_err(|e| io::Error::new(io::ErrorKind::InvalidInput, format!("{e:?}")))?; Ok(Self { diff --git a/docs/how-we-plan.md b/docs/how-we-plan.md index 857a85530..670dccc33 100644 --- a/docs/how-we-plan.md +++ b/docs/how-we-plan.md @@ -283,8 +283,8 @@ Release / multi-day work = ordered stories. One agent turn ≈ one step | Prefer for Red | When | |----------------|------| -| Extend an existing default [catalog](../TESTING.md#scenario-catalog) journey | Operator/peer-visible RPC, Electrum, Esplora, BIP324 P2P | -| Focused unit next to shipped fn | Pure helper, fast loop, expensive full path | +| Extend an existing default [catalog](../TESTING.md#scenario-catalog) journey | Operator/peer-visible RPC, Electrum, Esplora, BIP324 P2P. This is the default | +| Narrow unit ([`TESTING.md`](../TESTING.md)) | The return is the consensus or schema result, and a session cannot reach it without an absurd chain | | Slim scenario / integration | Stage boundaries, IBD/confirm wiring, store publish order | | One pin per contract | Not unit + twin integration for the same lines | @@ -297,7 +297,7 @@ until it passes — do not use that CI job as the inner loop | Plan-time rules | | |-----------------|--| | Each step declares Red tests **before** Green work | | -| New scenarios must justify cost (what unit cannot catch) | | +| The journey is the default. A new unit must be the narrow [`TESTING.md`](../TESTING.md) exception | | | No step that “adds coverage later” | | | Prefer synthetic `/tmp` fixtures; no agent-VM mainnet open | | | Hot-path Contract includes the cost model | [`CONTRIBUTING.md`](../CONTRIBUTING.md) principle 9 | @@ -321,7 +321,8 @@ even if the slices are “vertical.” (return value, store after reopen, peer / RPC / log line). It fails with the same class of error the bug would produce, not a compile error. - Prefer extending a default [catalog](../TESTING.md#scenario-catalog) - journey. A unit belongs next to a pure helper. One pin per contract. + journey. A new unit is the narrow [`TESTING.md`](../TESTING.md) exception. + One pin per client-visible result. - Watch it fail once. A test that never failed proves nothing. ### Green: smallest change that passes @@ -356,7 +357,17 @@ Test moves ([`TESTING.md`](../TESTING.md) owns the budget): - Lift guts asserts up to the journey once the journey hits the same shipped path, then delete the twin unit. +- If the journey cannot hit the line, delete the production branch in this + refactor and delete the test that only painted it. The 92% floor still + applies. It passes because the dead lines left both `LH` and `LF`, or + because the journey hits them. +- A CRAP failure on a function this step made more branched is a simplify, + not a new test and not a new allowlist row. - Delete tests that pin implementation shape rather than behavior. +- Dropping the twin after the journey hits the path, and dropping the test + that only painted a deleted branch, are the duplicate and empty pins the + plan-time rule allows. Dropping the only witness of a live line is not a + refactor. - Replace a `*_for_test` hook or hot-path probe with an instance stat or an on-disk assert, then delete the hook. - Fold duplicate fixtures into the shared `testutil`; shrink N to the @@ -367,6 +378,7 @@ Test moves ([`TESTING.md`](../TESTING.md) owns the budget): | After Green | Refactor move | |-------------|---------------| | A unit drove a private helper; the journey now covers that path | Move the assert to the journey, delete the unit, inline or `pub(crate)` the helper | +| The journey cannot execute the line | Delete that production branch and the test that only painted it. The 92% floor still applies | | Green added a second branch beside the old one | Collapse to one path and delete the old; the same test still passes | | Green needed a test-only hook on production | Assert the session/table stat or file state instead; delete the hook | | A new scenario re-mines a pad the journey already has | Reuse that journey’s pad; one open per binary |