Skip to content

perf(rallocator): consolidate v1 and Seismograph improvements - #764

Draft
Ralf Biedert (ralfbiedert) wants to merge 39 commits into
mainfrom
users/ralfbiedert/rallocator-v1-integration-20260915
Draft

Ralf Biedert (ralfbiedert) wants to merge 39 commits into
mainfrom
users/ralfbiedert/rallocator-v1-integration-20260915

Conversation

@ralfbiedert

Copy link
Copy Markdown
Collaborator

Summary

Consolidates the Rallocator v1 performance work and Seismograph snapshot/monitor improvements: medium-span caching and reclamation, eligible in-place reallocation, remote-accounting improvements, independently owned snapshot storage, and an offline snapshot viewer.

Public API changes

  • Exposes rallocator::tuning_telemetry behind the existing tuning-telemetry feature, with explicit session control and allocation-free counter observations.
  • Adds PeakLiveBytesScope, Stats::peak_live_bytes_scope, and optional lifetime/sample peak accessors. Rallocator reports aggregate-query sample peaks, not lifetime high-water marks.
  • Requires allocator statistics section version 2. Legacy version 1 statistics are rejected; backward compatibility is intentionally not retained.
  • Adds the documentation-hidden integration hook seismograph::snapshot::snapshot_collection_active(); no snapshot_arena_active() alias is retained.
  • Adds seismograph view <SNAPSHOT-FILE> for read-only interactive inspection of saved captures.
  • Propagates Cachet's Seismograph feature to BytesBuf when that optional dependency is present.

Existing allocator entry points and configuration remain unchanged. Eligible allocations can resize in place through the existing GlobalAlloc::realloc interface.

Validation so far

  • Windows: 8,211 all-feature tests passed (4 skipped), excluding only the unchanged msvc_spectre_libs package because this machine lacks Visual Studio Spectre-mitigated CRT libraries.
  • Windows: all-workspace no-default-feature tests passed (6,904 passed, 4 skipped).
  • Workspace doctests passed: 1,262 with all features (same Spectre-package exclusion), 1,057 with default features.
  • Rust 1.95 tests for all 11 changed crates passed: 1,314 all-feature and 1,261 default-feature tests.
  • All 36 each-feature builds passed; Rallocator all-target/all-feature debug and release builds passed.
  • Workspace Clippy passed with the same Spectre-package exclusion. Formatting, spelling, README, license, dependency-policy and branch-crate unused-dependency checks passed.

Draft while validation continues. The complete local just anvil-pr gate stops at the missing Spectre libraries; coverage, runtime-analysis, mutation, Linux and ARM results are not yet all confirmed. Hosted CI is the next source of actionable failures.

Integrate accepted medium allocation, retention, remote accounting, realloc, cached-region lookup, and snapshot bitmap improvements on current main. Reconcile snapshot ownership and registry maintenance while retaining bounded warm tables. Remove snapshot response copies, correct runtime accounting and feature wiring, and preserve isolated nextest-compatible mapping coverage.

The final combined Linux model and performance qualification remains pending; historical component results are not a claim of an overall combined speedup. Rejected TLS and experimental v2 changes are excluded.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@github-advanced-security github-advanced-security AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Exercise arena cleanup through a linked-library snapshot source and validate peak labels, accessors, and short output buffers. Keep manual profiling probes outside automated coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Align both payload placement and discriminator shifts for the stored header pointer, including conservative mapping-size checks. Guard the fallible heap allocation in the owner-exit fixture.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cover loader progress, source precedence, event stack release, and offline UI states without reducing production coverage requirements.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Inject memory queries and cold-path failures without changing allocator defaults. Consolidate equivalent resize rejection conditions while preserving every defensive check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@codecov

codecov Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.0%. Comparing base (ec1f6bc) to head (325a3ad).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##             main     #764     +/-   ##
=========================================
  Coverage   100.0%   100.0%             
=========================================
  Files         640      648      +8     
  Lines       85528    91129   +5601     
=========================================
+ Hits        85528    91129   +5601     
Flag Coverage Δ
linux 99.9% <100.0%> (-0.1%) ⬇️
linux-arm 99.9% <100.0%> (-0.1%) ⬇️
scheduled ?
windows 100.0% <100.0%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cover monitor contention, deadlines, spawn failures, and recorder recovery without weakening coverage gates.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Exercise custom modes, sampling, backtraces, navigation, and rendered settings. Use exhaustive field dispatch instead of duplicate mode checks and omit unreachable empty-value formatting.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the single-use const constructor without changing retirement behavior or coverage policy.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep native stress and I/O coverage while preventing unsupported or disproportionately expensive Miri execution. Localize the older allocator-api dependency so cargo-delta no longer marks the whole workspace modified, and make deadline-sensitive lock acquisition immune to spurious CAS failure.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep medium-allocation internals mutation-tested while excluding the synthetic permanent-null boundary, and exclude scheduler-dependent monitor polling from line coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Centralize medium-cache eligibility and exclude performance-only size mutations that make workspace mutation runs exceed their process timeout.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use checked size arithmetic and a nonzero batch count so mutation testing cannot turn reservation metadata into invalid pointer or extent state.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Prevent generated mutations from creating non-progressing bitmap scans or null-pointer traversal while preserving the allocator's behavior and reference tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove redundant let-scrutinee parentheses around unsafe pointer traversal expressions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread crates/rallocator/src/allocator/drain_batch_tests.rs Dismissed
Comment thread crates/rallocator/src/allocator/drain_batch_tests.rs Dismissed
Comment thread crates/rallocator/src/allocator/drain_batch_tests.rs Dismissed
Comment thread crates/rallocator/src/allocator/drain_batch_tests.rs Dismissed
Comment thread crates/rallocator/src/allocator/medium.rs Dismissed
Comment thread crates/rallocator/src/allocator/realloc/tests.rs Dismissed
Comment thread crates/rallocator/src/allocator/realloc/tests.rs Dismissed
Comment thread crates/rallocator/src/allocator/realloc/tests.rs Dismissed
Comment thread crates/rallocator/src/allocator/realloc/tests.rs Dismissed
Comment thread crates/rallocator/src/allocator/realloc/tests.rs Dismissed
Avoid whole-structure pointer retags during concurrent region scans, keep unsupported fault injection and stress paths out of Miri, and make UI rendering tests independent of the host realtime clock. Exclude the allocator corruption abort path from in-process coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread crates/rallocator/src/allocator/realloc/tests.rs Dismissed
Bound telemetry bitmap iteration to the platform-specific region slice count and avoid the profiling allocator wrapper under Miri.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Clear a stale admission deadline when a later quota acquisition fully satisfies the requested retention credit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants