Skip to content

perf: large-page reuse CPU investigation and draft V7 (#614) - #616

Closed
zackees wants to merge 44 commits into
mainfrom
perf/614-large-reuse-cpu
Closed

zackees wants to merge 44 commits into
mainfrom
perf/614-large-reuse-cpu

Conversation

@zackees

@zackees zackees commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

Draft: V7 large-page reuse CPU candidate

Tracks #614. Benchmark prerequisite #615 is complete through merged PR #617.

Not ready to merge. V7 has not been performance-tested. Validation blockers are tracked in #618; no CPU, throughput or RSS gain is claimed.

Implementation

Specialize removal of a selected retired large singleton at the existing call site after unpublication. Selection proves normal large bin, singleton links and clear full-queue flag. The new helper avoids generic linked-list checks and a redundant atomic flag clear, preserving queue/page counts and all geometry, retention, aging, purge, ownership and failure/callback ordering. Other callers retain the original generic remover.

C commit 2ec4d32e; separately generated Rust C 28ef6ad7. Frozen performance baseline 227eef06ff03f92966f38508441d6c2f60a75135.

Verified and unresolved

  • Focused original-path RED: one redundant full-flag write; two test cases fail. Actual executable/log preserved.
  • Candidate focused C, secure C++ and actual ASan + owner-gate large-span tests PASS. Checks cover generic control, flags, owner, queue/page/full-size accounting, neighboring queues, direct cache and 121 ordered geometry recarves.
  • Design, implementation and draft-evidence reviews clean; Rust amalgamation regenerated.
  • Secure C++ purge test compilation fails: goto crosses initialized scalar locals. Reproduced on identical frozen-baseline test with matching flags.
  • ASan + owner-gate purge suite failed one of 35 cases: new-thread-does-not-sweep-at-once observed six global discards, expected zero. Matching baseline passed. This is retained, unresolved and not waived.
  • Diagnostics show zero successful repurposes in both purge runs: V7's specialized path was not executed. The global discard delta may include scavenger work; this is a hypothesis, not a proven root cause.
  • No unchanged failed-candidate rerun, V7 performance campaign, full V7 platform acceptance or merge authorization implied.

Evidence and handoff

See V7 handoff, v7-validation-evidence.zip and its adjacent digest in .github/assets/perf-614/. Archive: 46 members, 4,864,167 bytes; SHA256 c1a9c4ff57f1694031ed38aeee4b1e1dc9f9f05802fe925b76afe642d06465e1. Exact binaries, commands, source patches/headers, logs, configurations and member hashes are retained.

V1/V2/V4/V6 lacked supported CPU/throughput gains; V3 had confirmed regressions; V5 was rejected for a reentrant collection risk. All rejected code was removed and original evidence/ledgers retained.

Resolve validation blockers before resuming #614's predeclared measurement and platform gates. Final merge, cleanup and synchronization with origin/main remain outstanding.

Local pre-push gate: 45/45 passed; stamped head 6155a79f6701f930ea799e66bc08a22c75ca3a30. This does not waive the recorded test failures.

@zackees zackees added the ci-full Opt-in complete platform and release validation on PR SHA label Oct 22, 2026
Local-Gate: v1 tree=a910b85e21192b946f9d7acda1096c60d437c582 secs=38 lanes=lint:run
Ci-Attestation: {"at":1792687774,"gate":"python/all/lint","host":"linux-x86_64","key":"64dd0c67bab2c8e0cfc15640286a0ef8c016e39afe06325825e1bcc810ec1bb0","lane":"lint","parents":["293cbf807f5753214a0c212320da1988bc99b0dd"],"secs":38,"stamp":"3b83d582b7b528eaa5a64f0fcc59d2b8","tree":"a910b85e21192b946f9d7acda1096c60d437c582","v":1,"via":"run"}
Local-Gate: v1 tree=af9478ba418852877af7fcf8914fe7f96d1b9a4c secs=55 lanes=lint:run
Ci-Attestation: {"at":1792693563,"gate":"python/all/lint","host":"linux-x86_64","key":"a86bfdc466bdc6a7a1ef1d39b5585696309759897681ccb9ff5afe2fa87991e0","lane":"lint","parents":["224f29c3f83034a0965a1be791094b6dd005a8e3"],"secs":55,"stamp":"7bf86144c5411f1bbe813c20d5f7cd60","tree":"af9478ba418852877af7fcf8914fe7f96d1b9a4c","v":1,"via":"run"}
@zackees
zackees force-pushed the perf/614-large-reuse-cpu branch from 287cac2 to 59d3696 Compare October 22, 2026 18:26
Local-Gate: v1 tree=1d0585c56d062fe6a32a2b51a9462195b14f61aa secs=59 lanes=lint:run
Ci-Attestation: {"at":1792694310,"gate":"python/all/lint","host":"linux-x86_64","key":"8c80351474c69b33f5060c32ff132be34c1f04cf0b56011cbcf755e1354237a7","lane":"lint","parents":["24d291e0927520b5de35876697c57c947f0b6703"],"secs":59,"stamp":"eccb7bea086448bda04b4f066143f90d","tree":"1d0585c56d062fe6a32a2b51a9462195b14f61aa","v":1,"via":"run"}
Local-Gate: v1 tree=71bd2cb92cfd9ffc9112073a85176a3074c7c692 secs=88 lanes=lint:run
Ci-Attestation: {"at":1792697089,"gate":"python/all/lint","host":"linux-x86_64","key":"56675ffd527620a69bb2afdb06dc84a467116068c1b1e6916e243a688244790a","lane":"lint","parents":["9358cf689bd0e368d4fa4a4c4960f88ce2b3d28b"],"secs":88,"stamp":"b8e62a774b9f7c1be0de89e4aea320db","tree":"71bd2cb92cfd9ffc9112073a85176a3074c7c692","v":1,"via":"run"}
Local-Gate: v1 tree=d6432b7f739a0b8ddca84f57da728b2b981ca59a secs=47 lanes=lint:run
Ci-Attestation: {"at":1792699464,"gate":"python/all/lint","host":"linux-x86_64","key":"9f510f1f88ec3b8cb5a43774ef0e109f9dbcf9269db5c4d34a4b6c79f80d8a7f","lane":"lint","parents":["3c2d1ce2a43906a71454d3267a3c7411d3f296cd"],"secs":47,"stamp":"7b7a5dfa7bab6e55bbf7f453a038c67c","tree":"d6432b7f739a0b8ddca84f57da728b2b981ca59a","v":1,"via":"run"}
Local-Gate: v1 tree=91746d781326114363d0837b5223bd5a7ec7d5d6 secs=57 lanes=lint:run
Ci-Attestation: {"at":1792702180,"gate":"python/all/lint","host":"linux-x86_64","key":"47ce08ab76127467ca8a391c63c54c3f1905818a17a2fbe331a252c6d1b41119","lane":"lint","parents":["6ad865c959a8d234de16d3c05a3197f3aa10e2ce"],"secs":57,"stamp":"591c3faa7c53002ccb6989275cc0efa7","tree":"91746d781326114363d0837b5223bd5a7ec7d5d6","v":1,"via":"run"}
Local-Gate: v1 tree=8ad8e44023234457bffd4031b794fb5846bbba23 secs=50 lanes=lint:run
Ci-Attestation: {"at":1792702875,"gate":"python/all/lint","host":"linux-x86_64","key":"785df5103e364ca78fa5d42b9c5e2bc3ffab26c5609148d495e86dde76ee4fa4","lane":"lint","parents":["73e47a0a15f21f2df68cc79cff6f032445fbdc6f"],"secs":50,"stamp":"38ad0c4997db7a43e69d1c2a9b8b4544","tree":"8ad8e44023234457bffd4031b794fb5846bbba23","v":1,"via":"run"}
Local-Gate: v1 tree=555f1942e6fcba02caa33a916d63104a444c01c5 secs=78 lanes=lint:run
Ci-Attestation: {"at":1792703460,"gate":"python/all/lint","host":"linux-x86_64","key":"f725d9696a67ab8e5e281aad535a0f5924fa11ff22e5d1d2bc2318061b0e1802","lane":"lint","parents":["886655b0aec06bdf702adb9d581cb7144b39c35a"],"secs":78,"stamp":"6eca13d63df2875c6003fc6483e9f38a","tree":"555f1942e6fcba02caa33a916d63104a444c01c5","v":1,"via":"run"}
Local-Gate: v1 tree=364256726d07eaa9d7809f9c27a2f5c34ed24548 secs=65 lanes=lint:run
Ci-Attestation: {"at":1792705969,"gate":"python/all/lint","host":"linux-x86_64","key":"623225000f49f2ffa90775d63bb6527e86d621f990095ad1db6af76a5f00c347","lane":"lint","parents":["31c92eeae5b8e51871c6ea4e609e7fc98ce03213"],"secs":65,"stamp":"fb5db3e1c8bf41350e959cd08b895154","tree":"364256726d07eaa9d7809f9c27a2f5c34ed24548","v":1,"via":"run"}
Local-Gate: v1 tree=cc13e8c16e3a16c7dd2c463a9c6b9cc6c3b4db71 secs=118 lanes=lint:run
Ci-Attestation: {"at":1792724034,"gate":"python/all/lint","host":"linux-x86_64","key":"e5108e93c24139caeaeead35540fde18f6136c0a8fa39163cfcfa88caf6ce779","lane":"lint","parents":["28ef6ad75a97e0737f75046a62329ced4ba2df15"],"secs":117,"stamp":"5f1af1e1a982d2dbc48d1e0df75cfd0d","tree":"cc13e8c16e3a16c7dd2c463a9c6b9cc6c3b4db71","v":1,"via":"run"}
@zackees zackees changed the title perf: bypass unpurge restoration for clean pages (#614) perf: large-page reuse CPU investigation and draft V7 (#614) Oct 23, 2026
- page-queue-remove.h: MI_UNUSED(page) in the retired remover; release
  builds (asserts compiled out) failed -Werror=unused-parameter.
- test-purge-holes.c: scope test_owner_sweep_pacing's measurement locals
  so the early goto done is valid C++ (MI_USE_CXX); reads unchanged.
- page-holes.c: mi_page_discard_slack excluded nothing for the MI_SECURE>=5
  trailing guard page, so the scavenger's eager-zero memset hit PROT_NONE
  (SIGSEGV in idle-thread-releases-large-page; also on baseline 227eef0,
  hidden because the file never compiled as C++).
- new-thread-does-not-sweep-at-once counts process-wide discards; the
  background scavenger's sweep of thread A's abandoned page was charged to C
  (loaded runs: baseline 33/144, candidate 10/144 fail; with scavenger off
  0/144). Run it in its own process with no scanner (test-new-thread-pacing),
  as test-reserved-page-reuse does.
Local-Gate: v1 tree=014360df1f9a7f8bdc27ae7f4ac70308aae073b8 secs=117 lanes=lint:run
Ci-Attestation: {"at":1792726984,"gate":"python/all/lint","host":"linux-x86_64","key":"098c419ed124f70b7bcdfa9570aa5c6b2a83ce1bf4f46b8a56d14ce9940959c4","lane":"lint","parents":["bd05be78f40c0ae5036ec741264128e6b26f3a9c"],"secs":117,"stamp":"a7ab40e4b8f82de76a1bf07d379360d6","tree":"014360df1f9a7f8bdc27ae7f4ac70308aae073b8","v":1,"via":"run"}
@zackees

zackees commented Oct 23, 2026

Copy link
Copy Markdown
Owner Author

#618 disposition — fixed on #616 head f0433230 (C bd05be78, Rust vendor 75e4afc1)

1. Secure C++ compile. RED: frozen baseline 227eef06 with MI_USE_CXX=ON, MI_SECURE=5 MI_DEBUG=3, C++17 fails with jump to label 'done' crosses initialization (line 1939). Fix: the measurement section of test_owner_sweep_pacing sits in an inner scope, so the early goto done skips it whole. Clock and counter reads keep their order, every check is unchanged, and cleanup is unchanged. GREEN: compiles.

1b. Allocator defect found once the file compiled as C++ (also on baseline). The secure-C++ suite segfaulted every time (3/3 runs on baseline and candidate) in idle-thread-releases-large-page. Backtrace: scavenger → _mi_pages_release_retired → mi_page_discard_slack → _mi_os_discard → eager-zero memset on the MI_SECURE≥5 trailing guard page (PROT_NONE). The slack range ran to the end of the slices and included the guard page. Fix (src/page-holes.c): subtract _mi_os_secure_guard_page_size() from hi, matching page_noguard_size in arena.c. GREEN: secure C++ purge-holes 32/32, 3/3 runs.

2. Pacing assertion: attributed to the background scavenger, not to thread C.

build, 144 runs, 48 concurrent fails (discards)
candidate, scavenger on 10/144 (8 discards)
baseline 227eef0, scavenger on 33/144 (8 discards)
baseline, MIMALLOC_SCAVENGER=0 0/144
fix (--new-thread-pacing, own process, scavenger off) 0/144

The test reads a process-wide counter, so the scavenger's sweep of thread A's abandoned page during C's life was charged to C. The defect predates V7: the baseline fails more often. V7's path was not exercised (0 repurposes). #478 is about the owner thread's first tick, so the case now runs in its own process with no scanner (test-new-thread-pacing, MIMALLOC_SCAVENGER=0;MIMALLOC_PURGE_HOLES=1). That mirrors test-reserved-page-reuse. The background-release cases stay in the default invocation with the scavenger on. No threshold changed. The original candidate failure and the baseline pass from the evidence archive stay on record. The baseline passing once was luck under low load.

Also fixed: PR CI build (release) and the other Release rows failed with -Werror=unused-parameter on page in mi_page_queue_remove_retired, because the asserts compile out. Added MI_UNUSED(page).

Local validation: owner-gate Debug (MI_DEBUG_FULL + MI_DIAGNOSTICS + MI_PPROF; no ASan headers on this host, CI's asan row covers ASan) 86/86 passed. Release MI_PPROF 70/70 passed. Secure C++: purge-holes passes. Local gate 45/45.

Not fixed by this change:

Native MSVC and win-gnu run in PR CI.

Unblock decision: #618's validation blockers are resolved. The V7 CPU measurement under #614 may resume. No CPU, throughput or RSS gain is claimed. #616 stays a draft until #614's perf-ab campaign and platform gates pass.

Local-Gate: v1 tree=e9507dcd890ebfa68486fabba474edb2aeb10981 secs=135 lanes=lint:run
Ci-Attestation: {"at":1792728348,"gate":"python/all/lint","host":"linux-x86_64","key":"c13049b70c7ef4ea36465b2fc6a6c46595db75269740c0568f86a2a3351c0581","lane":"lint","parents":["f04332301b27eb4c964361a84d173614097f4d07"],"secs":135,"stamp":"e96f7fa851abf27ea632d99e10b5bb5a","tree":"e9507dcd890ebfa68486fabba474edb2aeb10981","v":1,"via":"run"}
@zackees

zackees commented Oct 23, 2026

Copy link
Copy Markdown
Owner Author

V7 verdict: REJECTED (rule 12)

Complete perf-ab runs comparing head f0433230 with frozen base 227eef06, 15 reps (up to 25), with a null arm: group 1 39905247635 on AMD EPYC 7763, group 2 39909998440 on Xeon 8370C. Rows were judged only within their own VM; nothing was averaged across hosts.

  • Confirmed FAIL: large-class-ephemeral/8 CPU/op +0.7% [+0.4, +1.0] against a 0.3% allowance, with zero memory credit.
  • Target cells lean toward regression: large-class/8 CPU +0.8% [+0.3, +1.2]; random-large/8 ops/s −1.9% [−2.5, −1.0] and CPU +0.8% [+0.5, +1.5].
  • No cell shows a statistically supported CPU or throughput gain. Per perf: reduce large-page reuse CPU after #611 while preserving RSS and p99 #614, no rerun was made to turn the FAIL green.

Earlier attempts, kept as evidence: 39896572843 (sanity gate tripped on base noise, #620) and 39897074404 (90-min timeout, incomplete; diagnosed and fixed in #621).

Disposition: V7 joins V1–V6 as rejected. The #618 validation fixes do not depend on V7 and continue separately in #622. Closing this draft.

Full ledgers

perf-ab run 39905247635: https://github.com/zackees/mimalloc-pprof/actions/runs/39905247635
base 227eef06ff03 head f04332301b27 on AMD EPYC 7763 64-Core Processor, 15 reps (up to 25); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 0% INCONCLUSIVE -1.3% [-2.0, +1.0] / 1.2% INCONCLUSIVE +0.8% [+0.3, +1.2] / 0.6% INCONCLUSIVE -5.5% [-34.2, -0.9] / 32.2% PASS -1.2% [-3.2, +0.5] / 3.3% PASS +0.0% [-2.5, +2.7] / 2.7% PASS +1.7% [-0.8, +3.2] / 2.3% INCONCLUSIVE
large-class-ephemeral/8 0% FAIL -0.1% [-0.9, +0.6] / 0.6% INCONCLUSIVE +0.7% [+0.4, +1.0] / 0.3% FAIL +0.4% [-0.4, +1.6] / 1.1% INCONCLUSIVE +0.0% [-2.1, +0.6] / 1.1% PASS -0.0% [-2.6, +2.5] / 2.7% PASS -0.7% [-11.6, +14.1] / 14.1% INCONCLUSIVE
random-large/8 0% INCONCLUSIVE -1.9% [-2.5, -1.0] / 1.7% INCONCLUSIVE +0.8% [+0.5, +1.5] / 0.8% INCONCLUSIVE -0.7% [-1.8, +0.7] / 1.8% PASS -0.4% [-1.5, +2.0] / 3.5% PASS -1.1% [-2.4, +2.2] / 3.3% PASS -1.1% [-2.6, +2.4] / 3.3% PASS
random-large-bursty/8 0% INCONCLUSIVE +0.0% [-0.2, +0.2] / 0.3% PASS +0.5% [+0.1, +1.0] / 0.7% INCONCLUSIVE +17.9% [-42.2, +71.2] / 44.7% INCONCLUSIVE -1.0% [-1.4, +0.3] / 1.0% PASS +0.0% [-4.9, +3.7] / 5.0% PASS -5.2% [-9.0, -2.3] / 8.5% PASS
random-large/1 0% PASS -0.5% [-0.9, +0.2] / 1.5% PASS +0.5% [-1.8, +2.0] / 3.5% PASS +0.0% [-0.7, +0.7] / 0.7% PASS -0.3% [-0.9, +0.9] / 1.2% PASS +0.0% [-0.0, +0.0] / 0.0% PASS +0.0% [+0.0, +0.0] / 0.0% PASS
small/8 (control) 0% INCONCLUSIVE -1.6% [-3.5, +0.7] / 4.5% PASS +2.5% [-2.7, +4.5] / 3.9% INCONCLUSIVE -0.6% [-1.2, +0.6] / 1.2% PASS +0.0% [+0.0, +0.0] / 0.8% PASS +0.0% [-0.0, +0.0] / 0.0% PASS +0.0% [-0.0, +0.0] / 0.0% PASS

perf-ab run 39909998440: https://github.com/zackees/mimalloc-pprof/actions/runs/39909998440
base 227eef06ff03 head f04332301b27 on Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz, 15 reps (up to 25); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
larson/1 0% INCONCLUSIVE -3.4% [-4.6, -1.2] / 4.2% INCONCLUSIVE +3.5% [+1.1, +5.8] / 4.9% INCONCLUSIVE +0.0% [-0.9, +1.0] / 0.5% INCONCLUSIVE +0.1% [-1.4, +2.8] / 1.8% INCONCLUSIVE -1.3% [-1.4, +0.0] / 1.3% PASS +0.0% [+0.0, +0.0] / 0.0% PASS
larson/8 0% PASS -1.3% [-2.9, +3.4] / 3.3% PASS +1.4% [-1.8, +3.9] / 4.6% PASS -15.4% [-29.5, +40.1] / 74.9% PASS -2.7% [-6.0, -0.7] / 3.2% PASS +0.7% [-1.3, +4.6] / 4.7% PASS +0.7% [-1.4, +2.1] / 2.4% PASS
larson/8 (chart build) 0% PASS -0.3% [-2.6, +3.2] / 3.5% PASS +0.4% [-4.1, +2.2] / 5.0% PASS +14.6% [-10.5, +64.5] / 79.0% PASS +0.0% [-2.0, +2.1] / 4.7% PASS -1.5% [-6.6, +4.9] / 5.2% PASS +1.0% [-1.0, +340.9] / 342.2% PASS
large-class/8 (profiler on) 0% PASS +0.2% [-0.6, +2.7] / 1.4% PASS -0.5% [-2.4, +0.7] / 1.1% PASS +0.0% [-6.8, +1.0] / 1.7% PASS -1.0% [-4.4, +1.4] / 4.3% PASS -0.7% [-1.5, +1.3] / 2.0% PASS -0.3% [-2.0, +1.4] / 2.0% PASS
large-class-ephemeral/8 (chart build) 0% INCONCLUSIVE +0.3% [-0.6, +0.8] / 0.7% PASS -0.3% [-0.9, +0.5] / 0.5% INCONCLUSIVE +0.0% [-1.1, +1.1] / 1.5% PASS -0.2% [-1.8, +1.8] / 1.8% INCONCLUSIVE +0.0% [-2.3, +2.6] / 2.6% INCONCLUSIVE -9.8% [-11.2, -0.0] / 11.2% PASS
large-class-ephemeral/8 short generations (chart build) 0% INCONCLUSIVE +0.9% [-0.7, +2.2] / 1.2% PASS -0.9% [-1.3, +0.3] / 1.2% PASS -0.4% [-1.1, +0.8] / 2.2% PASS +0.2% [-1.1, +2.7] / 2.1% INCONCLUSIVE +0.0% [-2.7, +2.5] / 2.7% PASS +0.0% [-2.7, +2.5] / 2.7% PASS

@zackees zackees closed this Oct 23, 2026
Nosenzor pushed a commit to Nosenzor/mimalloc-pprof that referenced this pull request Oct 28, 2026
…luted pacing test (zackees#618)

- page-holes.c: mi_page_discard_slack ran its range to the end of the
  slices, over the MI_SECURE>=5 trailing PROT_NONE guard page; the
  scavenger's eager-zero memset there segfaulted (idle-thread-releases-
  large-page, deterministic in a secure C++ build). Stop before the guard,
  as arena.c's page_noguard_size does.
- test-purge-holes.c: scope test_owner_sweep_pacing's measurement locals
  so its early goto done is valid C++ (MI_USE_CXX); reads unchanged.
- new-thread-does-not-sweep-at-once counted process-wide discards, so a
  background scavenger sweep of thread A's page during C's life was
  charged to C (loaded: 33/144 fail at 227eef0; 0/144 with the scavenger
  off). Run it in its own process with no scanner (test-new-thread-pacing,
  static builds), as test-reserved-page-reuse does.

Split out of the rejected V7 draft zackees#616 so the fixes land on their own.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-full Opt-in complete platform and release validation on PR SHA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant