Skip to content

feat(vara): add BEEFY finality and MMR commitments - #5642

Draft
ukint-vs wants to merge 12 commits into
gear-tech:masterfrom
ukint-vs:beefy-mmr-phase-0-1
Draft

ukint-vs wants to merge 12 commits into
gear-tech:masterfrom
ukint-vs:beefy-mmr-phase-0-1

Conversation

@ukint-vs

@ukint-vs ukint-vs commented Sep 19, 2026

Copy link
Copy Markdown
Member

Summary

  • Add Vara BEEFY consensus and BEEFY-backed MMR commitments. Development and local testnet presets activate BEEFY at block 1; production remains inactive for the rollout.
  • Migrate stored four-key session records to the five-key layout at runtime spec 20100, including key ownership and queued-key state. The one-shot migration is gated by the previous on-chain spec version and is covered by try-runtime checks.
  • Commit authoritative Ethereum bridge lifecycle state into the MMR leaf, preserve the session-boundary ordering contract, expose bounded MMR proof RPCs, and register benchmark/runtime metadata surfaces.
  • Permit the ECDSA key-generation host import introduced by five-key session generation.
  • Backport Polkadot SDK #12812 into the pinned sc-consensus-beefy implementation so malformed justification requests are rejected and penalized without terminating the request handler. The rest of the SDK remains pinned at 298f676c91d64f15f38ea7fd78f125c5889ab09c.
  • Repair fork pull-request CI de-duplication, pin the typos action to v1.50.2, and fix the typo exposed by the restored check.

Architecture

This PR provides the source-chain commitment and proof-serving layer. Gear #5644 adds only an opt-in fast local cadence; gear-bridges #860 consumes these commitments and proves delivery on Ethereum.

flowchart LR
    Message["send_eth_message"] --> Queue["GearEthBridge queue and root"]
    Queue --> Snapshot["bridge_snapshot: queue ID and root"]
    Snapshot --> Extra["VaraBridgeProvider: 45-byte snapshot hash"]
    Session["Five session keys, including BEEFY ECDSA"] --> Authorities["BEEFY current and next authority sets"]
    Authorities --> Leaf["pallet_beefy_mmr leaf"]
    Extra --> Leaf
    Leaf --> MMR["pallet_mmr: Keccak tree and header digest"]
    MMR --> Worker["Node BEEFY worker: signed commitments"]
    MMR --> Index["Offchain indexed MMR nodes"]
    Worker --> RPC["BEEFY subscription and proof RPCs"]
    Index --> RPC
    RPC --> Consumer["External Rust relay and Ethereum client"]
Loading

Responsibilities and trust boundary

  • BABE authors blocks; GRANDPA finalizes the chain. BEEFY adds ECDSA-signed commitments to the MMR root for external consumers. It does not replace either existing consensus role.
  • The bridge pallet remains the queue owner. Its new bridge_snapshot() accessor exposes the authoritative initialized state, queue ID and queue root. This PR does not replace send_eth_message, message hashing or the queue-clearing mechanism.
  • VaraBridgeProvider commits the bridge snapshot. The preimage is version:u8 || "vara" || queueId:u64-LE || queueRoot:bytes32 (45 bytes); its Keccak hash is the MMR leaf's 32-byte extra field. Version 255 distinguishes uninitialized state from initialized/empty version 0.
  • **Runtime BEEFY/MMR configuration combines this snapshot with the parent block and next authority-set commitment in the standard BEEFY MMR leaf. The runtime uses Keccak, converts BEEFY authorities into Ethereum-address Merkle leaves, and deposits the MMR root in a BEEFY header digest. The node signs that root through the BEEFY worker.
  • RPC/indexing serves evidence, not authority. Offchain indexing retains the MMR data needed for historical proofs. The consumer must still verify signatures and inclusion; an RPC root alone is not a consensus proof. Enabling indexing after relevant blocks were imported cannot recover data that was never indexed.

Session boundaries and snapshot timing

The five-key session layout is BABE, GRANDPA, im-online, authority discovery and BEEFY. The GrandpaAndGearEthBridge handler preserves delivery of session callbacks to both existing GRANDPA and bridge logic, then includes BEEFY in the session-handler tuple.

Declaration/hook order is part of the bridge commitment contract: Session runs before MMR leaf construction, and MMR construction runs before the bridge's on_initialize can perform a due queue clear. The bridge updates a changed queue root in on_finalize. Thus a queue snapshot from source block B is proven using the leaf inserted at L=B+1, under a signed anchor C>=L. Consumers must track the first MMR insertion block A separately: index=L-A, leafCount=C-A+1.

This ordering lets a consumer retain a message from an old queue and authenticate its historical snapshot after a natural session-driven clear. It does not require resetting runtime storage or simulating session callbacks.

Migration and activation are separate

The spec-20100 migration expands stored four-key records to five keys, including queued keys and ownership bookkeeping. Deterministic placeholder BEEFY public bytes preserve the production state layout; they are not usable signing keys or permission to activate production BEEFY. The one-shot migration is gated by the previous on-chain spec version.

Dev/local presets activate BEEFY at block 1. Production keeps genesis_block: None; validators must register real BEEFY keys before a separately governed activation. Ethereum deployment, queue-root registration and message delivery are outside this PR and are explained in gear-bridges #860.

For review, start with the snapshot accessor/provider and runtime hook order above, then the session migration and node/RPC wiring. Existing test and rollout evidence remains below; production weight benchmarking remains a separate gate.

How to test

  • make pre-commit
  • cargo nextest run -p sc-consensus-beefy malformed_requests_do_not_stop_handler
  • cargo nextest run -p vara-runtime
  • cargo nextest run -p gear-service -p gear-node-testing -p runtime-fuzzer
  • cargo check -p vara-runtime --features runtime-benchmarks
  • cargo check -p vara-runtime --features try-runtime
  • cargo metadata --locked --no-deps
  • make node
  • GitHub PR checks at e648518c5e57795df1f7ec432c08ce209de8d954: 24 passed, 0 failed. Docker image, Windows build, fuzzer, and production jobs were skipped by workflow configuration.
  • Run try-runtime v0.8.0 against a Vara mainnet snapshot: migration runs once, remains idempotent, and all post-upgrade state checks pass.
  • Run three local authorities with offchain indexing from genesis: BEEFY produces three-signature justifications; the header contains a BEEFY/MMR digest; an N-3 proof verifies at N; a 257-leaf request is rejected at the 256-leaf limit; the exact historical proof remains identical and valid after restarting a node on the same base path.
  • Run the committed --dev --alice --tmp --enable-offchain-indexing true preset: BEEFY finalizes from block 1, finalized headers contain BEEFY/MMR digests, and beefy_subscribeJustifications receives a signed beefy_justifications notification.

Notes

  • Production BEEFY remains inactive (genesis_block: None). The malformed-request code blocker is backported locally; production activation still requires validator key rotation plus the normal governance and deployment timing.
  • Existing production validators receive deterministic, distinct placeholder BEEFY public bytes during migration. They never sign while production BEEFY is inactive. Validators must rotate to real BEEFY keys before activation.
  • Proof-serving nodes must start with --enable-offchain-indexing before the first imported MMR block. Restarting later cannot reconstruct leaves omitted while indexing was disabled.
  • The MMR benchmark runner is unavailable in this environment, so VaraMmrWeight conservatively wraps the upstream weight and accounts for the Vara-specific five reads and one write. No workstation measurements are committed.
  • The originally proposed uninitialized bridge vector was mislabeled. The fixture records the cryptographically verified values: initialized preimage hash 5181adf4…9980; uninitialized domain plus zero payload hash c2fe11d2…c2de.

Checklist

  • PR title follows Conventional Commits (type(scope): description)
  • Single logical change
  • Tests added or updated (if logic changed)
  • Docs updated (if needed)

ukint-vs and others added 5 commits September 16, 2026 20:27
…ition

Phase 0+1 of the BEEFY/MMR bridge migration: adds pallet-beefy, pallet-mmr,
and pallet-beefy-mmr to the Vara runtime at indices 32-34 (declared after
Session, before GearEthBridge, to preserve the leaf-capture ordering from
polkadot-fellows/runtimes#160), with a bridge-snapshot leaf-extra provider
and a session-key upgrade for `beefy`. BEEFY ships inactive
(`genesis_block: None`); node-side gadget/RPC wiring is deferred to a later
phase.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Reuse the existing MaxSetIdSessionEntries constant for pallet_beefy
  instead of duplicating it as BeefySetIdSessionEntries (same formula,
  now a single source of truth for babe/grandpa/beefy).
- Document the SessionKeys re-encoding hazard on the struct: appending
  `beefy` changes NextKeys/QueuedKeys' SCALE encoding, so a live-chain
  upgrade needs the Session::upgrade_keys migration (Phase 2, tracked
  separately) landed in the same runtime upgrade.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- bridge_leaf.rs: build the leaf preimage in a fixed-size stack array
  instead of a heap-allocated Vec (this runs on every block's MMR leaf
  construction).
- gear-eth-bridge: bridge_snapshot() drops the redundant Initialized
  read — QueueMerkleRoot is only ever `put`, never `kill`ed, once the
  bridge initializes, so its presence already implies Initialized.
- integration_tests.rs: factor the 11 repeated
  ecdsa::Pair::from_string("//<Name>", None) call sites into a
  beefy_key() helper.
- lib.rs: pallet_mmr::Config::INDEXING_PREFIX reuses
  sp_mmr_primitives::INDEXING_PREFIX instead of retyping the literal.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…e 2)

Session::upgrade_keys translates existing 4-key NextKeys/QueuedKeys
entries into the current 5-key SessionKeys, assigning each validator a
deterministic placeholder beefy key. Closes the SCALE-decode hazard the
BEEFY/MMR runtime composition (2e768ce) introduced for already-bonded
validators, and is safe to deploy in the same upgrade since BEEFY stays
inactive (genesis_block: None) so the placeholder never has to sign.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds sc-consensus-beefy, sc-consensus-beefy-rpc, mmr-gadget, and mmr-rpc
to the node service, following the reference wiring in
substrate/bin/node/cli/src/service.rs (pinned SDK rev 298f676c9):

- lib.rs: wrap the BABE/GRANDPA import stack with a BEEFY block import,
  start the BEEFY gossip/justification-request network protocols, and
  spawn the BEEFY voter and MMR gadgets alongside GRANDPA. The gadgets
  run unconditionally; runtime activation (GenesisBlock) gates signing.
- client.rs: add BeefyApi and MmrApi to RuntimeApiCollection.
- rpc/mod.rs: add BeefyDeps and wire sc_consensus_beefy_rpc::Beefy and
  mmr_rpc::Mmr into create_full, giving beefy_subscribeJustifications,
  beefy_getFinalizedHead, and mmr_generateProof.
- runtime lib.rs: implement pallet_beefy_mmr::BeefyMmrApi (authority-set
  proofs for BEEFY light clients/relayers), which Phase 1 left out.

CLI key registration and node/testing BEEFY key provisioning needed no
changes: sc-cli's generic `key insert` already accepts any --key-type,
and vara/node/testing/src/keyring.rs already derives a BEEFY key
alongside the others.

make workspace-hack run after the new dependencies per repo convention.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@ukint-vs ukint-vs added the ci: production Run production build (cargo --profile production) label Sep 19, 2026
@ukint-vs
ukint-vs marked this pull request as ready for review September 19, 2026 11:41
@ukint-vs
ukint-vs marked this pull request as draft September 19, 2026 11:41
@ukint-vs
ukint-vs marked this pull request as ready for review September 19, 2026 11:42
@ukint-vs
ukint-vs marked this pull request as draft September 19, 2026 11:43
@ukint-vs
ukint-vs force-pushed the beefy-mmr-phase-0-1 branch 2 times, most recently from 0dffb97 to 6382de8 Compare September 19, 2026 14:53
Comment thread utils/gear-workspace-hack/Cargo.toml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci: production Run production build (cargo --profile production)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants