Skip to content

restore: validate sysvars before accepting snapshots - #11498

Draft
ripatel-fd with Copilot wants to merge 8 commits into
mainfrom
copilot/restore-comprehensive-sysvar-sanity-checking
Draft

ripatel-fd with Copilot wants to merge 8 commits into
mainfrom
copilot/restore-comprehensive-sysvar-sanity-checking

Conversation

Copilot AI commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Restore validates SlotHistory but allows other malformed sysvars to reach replay assertions. Extend validation using Agave invariants and reject invalid snapshots through the existing retry protocol.

  • Validation: Check all nine cached sysvars for required presence, ownership, decoding, and runtime layout constraints. Enforce rent-rate bounds and active reward total, partition-count, and height-overflow checks before NEXT/DONE.
  • Capture and recovery: Validate stream captures without racing snapwr. Match accdb replacement precedence, including malformed replacements and deletions; restore the full-snapshot baseline on incremental retries.
  • Compatibility: Preserve optional sysvars, empty/gapped histories, nonstandard rent thresholds, and zero reward points.
  • Scope: Cross-account reward-reconstruction invariants remain outside this patch.

Copilot AI balanced review requested due to automatic review settings September 15, 2026 17:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-authored-by: ripatel-fd <113896534+ripatel-fd@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 15, 2026 17:32
Co-authored-by: ripatel-fd <113896534+ripatel-fd@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Missing semantic validation can permit crashes, incorrect rent calculations, and unsafe epoch-reward state.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (3)

src/discof/restore/fd_snapin_tile.c:450

  • FD_SYSVAR_FLAG_VALID only checks that EpochRewards is structurally decodable (including that active is boolean). An active record with zero/oversized num_partitions, distributed_rewards > total_rewards, or overflowing distribution bounds still passes; reward reconstruction later casts the count to uint and indexes fixed MAX_PARTITIONS_PER_EPOCH arrays, so a crafted snapshot can cause out-of-bounds access. Validate the active-reward invariants before accepting the snapshot.
  for( ulong i=0UL; i<FD_SYSVAR_CACHE_ENTRY_CNT; i++ ) {
    /* Agave restore requires Rent; Firedancer also requires Clock. */
    if( FD_UNLIKELY( (i==FD_SYSVAR_clock_IDX || i==FD_SYSVAR_rent_IDX) &&

src/discof/restore/fd_snapin_tile.c:450

  • A structurally valid Rent value can still overflow runtime rent arithmetic. For thresholds 1.0 and 2.0, fd_rent_exempt_minimum_balance directly multiplies lamports_per_uint8_year by account size, so values above the supported limits wrap and produce incorrect balances. Reject those values during snapshot verification.
    if( FD_UNLIKELY( (i==FD_SYSVAR_clock_IDX || i==FD_SYSVAR_rent_IDX) &&

src/discof/restore/fd_snapin_tile.c:455

  • The generic validator accepts compact SlotHashes encodings (for example, an 8-byte zero-length vector), but fd_sysvar_slot_hashes_update later requires the full FD_SYSVAR_SLOT_HASHES_BINCODE_SZ backing account and terminates otherwise. Require the full size here so a malformed snapshot is rejected gracefully instead of crashing on the first update.
    if( FD_UNLIKELY( ctx->sysvars.accounts[ i ].present &&
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread src/discof/restore/fd_snapin_tile.c
Copilot AI review requested due to automatic review settings September 15, 2026 17:39
Co-authored-by: ripatel-fd <113896534+ripatel-fd@users.noreply.github.com>
Copilot AI changed the title [WIP] Restore comprehensive sysvar sanity checking restore: validate sysvars before accepting snapshots Sep 15, 2026
Copilot AI requested a review from ripatel-fd September 15, 2026 17:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Security-sensitive snapshot validation remains WIP and targeted test execution is not yet confirmed.

Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Copilot AI review requested due to automatic review settings September 15, 2026 17:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Snapshot validation is security-sensitive and affects restore-to-replay state transitions.

Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@ripatel-fd ripatel-fd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

everything looks good except asan guard, and vertical alignment for multiline ifs (off-by-one). don't fix anything else

Comment thread src/discof/restore/fd_snapin_tile.c Outdated
Comment thread src/discof/restore/fd_snapin_tile.c Outdated
Comment thread src/discof/restore/fd_snapin_tile.c Outdated
Comment thread src/discof/restore/fd_snapin_tile.c Outdated
Comment thread src/discof/restore/fd_snapin_tile.c Outdated
Comment thread src/discof/restore/fd_snapin_tile.c Outdated
Comment thread src/discof/restore/test_snapin_tile.c Outdated
Comment thread src/discof/restore/test_snapin_tile.c Outdated
Co-authored-by: ripatel-fd <113896534+ripatel-fd@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 15, 2026 19:13
Copilot AI requested a review from ripatel-fd September 15, 2026 19:13

@ripatel-fd ripatel-fd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still 2 vertical alignment issues

Comment thread src/discof/restore/fd_snapin_tile.c Outdated
Comment thread src/discof/restore/fd_snapin_tile.c Outdated
Co-authored-by: ripatel-fd <113896534+ripatel-fd@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 15, 2026 19:21
Copilot AI requested a review from ripatel-fd September 15, 2026 19:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Snapshot acceptance and incremental recovery are consensus-critical paths requiring final human validation.

Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@ripatel-fd
ripatel-fd marked this pull request as ready for review September 15, 2026 20:14
@github-actions

github-actions Bot commented Sep 15, 2026

Copy link
Copy Markdown
 ┌─ ⚡ PERF · 204a4ec vs main@c102002 ─────────────────────────────────
 │ SUITE                               BASELINE          NEW         Δ
 │ replay tps, mainnet               30,872 tps   30,918 tps  ·  +0.15%
 │ bench tps, localnet              830,926 tps  831,270 tps  ·  +0.04%
 │ snapshot load, testnet               15.16 s      15.16 s  ·  -0.02%
 │ mem total, mainnet                164.38 GiB   164.38 GiB  ·   0.00%
 │ mem total, testnet                 95.38 GiB    95.38 GiB  ·   0.00%
 │ clean compile, firedancer        305.2 cpu·s  305.5 cpu·s  ·  +0.07%
 │ binary size, firedancer             68.42 MB     68.43 MB  ·  +0.01%
 ├─────────────────────────────────────────────────────────────────────
@@ 0 REGRESSIONS · 0 WARNINGS · 0 IMPROVED · 7 NOISE @@
 └─────────────────────────────────────────────────────────────────────
history · 2 pushes
 ┌─ HISTORY · Δ vs main, per push, newest first ─────────────────────────
 │ HEAD         TPS    BENCH     SNAP    MEM·M    MEM·T  COMPILE   BINARY
 │ 204a4ec   +0.15%   +0.04%   -0.02%    0.00%    0.00%   +0.07%   +0.01%
 │ 6a61060   -0.07%   +0.16%   +0.02%    0.00%    0.00%   +0.23%   +0.01%
 └───────────────────────────────────────────────────────────────────────

@ripatel-fd

Copy link
Copy Markdown
Contributor

@copilot fix all the issues in resolved comments they have been ignored

Co-authored-by: ripatel-fd <113896534+ripatel-fd@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 15, 2026 20:21

Copilot AI commented Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

@copilot fix all the issues in resolved comments they have been ignored

Addressed all resolved feedback, including removing the ASAN pragma guard, in 204a4ec.

@ripatel-fd

Copy link
Copy Markdown
Contributor

@copilot the asan failure is too much stack usage. declare big variables as globals (static) instead of on the stack to fix it. targeted approach >1MB or so vars only not everything

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Security-critical snapshot validation and reward recovery behavior warrant final human review.

Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Co-authored-by: ripatel-fd <113896534+ripatel-fd@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 15, 2026 20:42
auto-merge was automatically disabled September 15, 2026 20:42

Head branch was pushed to by a user without write access

Copilot AI commented Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

@copilot the asan failure is too much stack usage. declare big variables as globals (static) instead of on the stack to fix it. targeted approach >1MB or so vars only not everything

Fixed in 66d176c. Moved only the test contexts to static storage; smaller locals are unchanged. These contexts are ~353 KiB each, but inlining accumulated them into the ~13 MiB stack frame. Normal and ASAN/UBSAN targeted tests now pass with an 8 MiB stack limit.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Snapshot acceptance and reward reconstruction are consensus-critical despite the strong targeted test coverage.

Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@ripatel-fd
ripatel-fd marked this pull request as draft September 15, 2026 21:08
@ripatel-fd

Copy link
Copy Markdown
Contributor

Blocked on Manik's snapshot loader refactor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

restore: comprehensive sysvar sanity checking

3 participants