Repository navigation
Conversation
Local-Gate: v1 tree=a910b85e21192b946f9d7acda1096c60d437c582 secs=38 lanes=lint:run
Ci-Attestation: {"at":1792687774,"gate":"python/all/lint","host":"linux-x86_64","key":"64dd0c67bab2c8e0cfc15640286a0ef8c016e39afe06325825e1bcc810ec1bb0","lane":"lint","parents":["293cbf807f5753214a0c212320da1988bc99b0dd"],"secs":38,"stamp":"3b83d582b7b528eaa5a64f0fcc59d2b8","tree":"a910b85e21192b946f9d7acda1096c60d437c582","v":1,"via":"run"}
Local-Gate: v1 tree=af9478ba418852877af7fcf8914fe7f96d1b9a4c secs=55 lanes=lint:run
Ci-Attestation: {"at":1792693563,"gate":"python/all/lint","host":"linux-x86_64","key":"a86bfdc466bdc6a7a1ef1d39b5585696309759897681ccb9ff5afe2fa87991e0","lane":"lint","parents":["224f29c3f83034a0965a1be791094b6dd005a8e3"],"secs":55,"stamp":"7bf86144c5411f1bbe813c20d5f7cd60","tree":"af9478ba418852877af7fcf8914fe7f96d1b9a4c","v":1,"via":"run"}
287cac2 to
59d3696
Compare
Local-Gate: v1 tree=1d0585c56d062fe6a32a2b51a9462195b14f61aa secs=59 lanes=lint:run
Ci-Attestation: {"at":1792694310,"gate":"python/all/lint","host":"linux-x86_64","key":"8c80351474c69b33f5060c32ff132be34c1f04cf0b56011cbcf755e1354237a7","lane":"lint","parents":["24d291e0927520b5de35876697c57c947f0b6703"],"secs":59,"stamp":"eccb7bea086448bda04b4f066143f90d","tree":"1d0585c56d062fe6a32a2b51a9462195b14f61aa","v":1,"via":"run"}
Local-Gate: v1 tree=71bd2cb92cfd9ffc9112073a85176a3074c7c692 secs=88 lanes=lint:run
Ci-Attestation: {"at":1792697089,"gate":"python/all/lint","host":"linux-x86_64","key":"56675ffd527620a69bb2afdb06dc84a467116068c1b1e6916e243a688244790a","lane":"lint","parents":["9358cf689bd0e368d4fa4a4c4960f88ce2b3d28b"],"secs":88,"stamp":"b8e62a774b9f7c1be0de89e4aea320db","tree":"71bd2cb92cfd9ffc9112073a85176a3074c7c692","v":1,"via":"run"}
Local-Gate: v1 tree=d6432b7f739a0b8ddca84f57da728b2b981ca59a secs=47 lanes=lint:run
Ci-Attestation: {"at":1792699464,"gate":"python/all/lint","host":"linux-x86_64","key":"9f510f1f88ec3b8cb5a43774ef0e109f9dbcf9269db5c4d34a4b6c79f80d8a7f","lane":"lint","parents":["3c2d1ce2a43906a71454d3267a3c7411d3f296cd"],"secs":47,"stamp":"7b7a5dfa7bab6e55bbf7f453a038c67c","tree":"d6432b7f739a0b8ddca84f57da728b2b981ca59a","v":1,"via":"run"}
Local-Gate: v1 tree=91746d781326114363d0837b5223bd5a7ec7d5d6 secs=57 lanes=lint:run
Ci-Attestation: {"at":1792702180,"gate":"python/all/lint","host":"linux-x86_64","key":"47ce08ab76127467ca8a391c63c54c3f1905818a17a2fbe331a252c6d1b41119","lane":"lint","parents":["6ad865c959a8d234de16d3c05a3197f3aa10e2ce"],"secs":57,"stamp":"591c3faa7c53002ccb6989275cc0efa7","tree":"91746d781326114363d0837b5223bd5a7ec7d5d6","v":1,"via":"run"}
Local-Gate: v1 tree=8ad8e44023234457bffd4031b794fb5846bbba23 secs=50 lanes=lint:run
Ci-Attestation: {"at":1792702875,"gate":"python/all/lint","host":"linux-x86_64","key":"785df5103e364ca78fa5d42b9c5e2bc3ffab26c5609148d495e86dde76ee4fa4","lane":"lint","parents":["73e47a0a15f21f2df68cc79cff6f032445fbdc6f"],"secs":50,"stamp":"38ad0c4997db7a43e69d1c2a9b8b4544","tree":"8ad8e44023234457bffd4031b794fb5846bbba23","v":1,"via":"run"}
Local-Gate: v1 tree=555f1942e6fcba02caa33a916d63104a444c01c5 secs=78 lanes=lint:run
Ci-Attestation: {"at":1792703460,"gate":"python/all/lint","host":"linux-x86_64","key":"f725d9696a67ab8e5e281aad535a0f5924fa11ff22e5d1d2bc2318061b0e1802","lane":"lint","parents":["886655b0aec06bdf702adb9d581cb7144b39c35a"],"secs":78,"stamp":"6eca13d63df2875c6003fc6483e9f38a","tree":"555f1942e6fcba02caa33a916d63104a444c01c5","v":1,"via":"run"}
Local-Gate: v1 tree=364256726d07eaa9d7809f9c27a2f5c34ed24548 secs=65 lanes=lint:run
Ci-Attestation: {"at":1792705969,"gate":"python/all/lint","host":"linux-x86_64","key":"623225000f49f2ffa90775d63bb6527e86d621f990095ad1db6af76a5f00c347","lane":"lint","parents":["31c92eeae5b8e51871c6ea4e609e7fc98ce03213"],"secs":65,"stamp":"fb5db3e1c8bf41350e959cd08b895154","tree":"364256726d07eaa9d7809f9c27a2f5c34ed24548","v":1,"via":"run"}
Local-Gate: v1 tree=cc13e8c16e3a16c7dd2c463a9c6b9cc6c3b4db71 secs=118 lanes=lint:run
Ci-Attestation: {"at":1792724034,"gate":"python/all/lint","host":"linux-x86_64","key":"e5108e93c24139caeaeead35540fde18f6136c0a8fa39163cfcfa88caf6ce779","lane":"lint","parents":["28ef6ad75a97e0737f75046a62329ced4ba2df15"],"secs":117,"stamp":"5f1af1e1a982d2dbc48d1e0df75cfd0d","tree":"cc13e8c16e3a16c7dd2c463a9c6b9cc6c3b4db71","v":1,"via":"run"}
- page-queue-remove.h: MI_UNUSED(page) in the retired remover; release builds (asserts compiled out) failed -Werror=unused-parameter. - test-purge-holes.c: scope test_owner_sweep_pacing's measurement locals so the early goto done is valid C++ (MI_USE_CXX); reads unchanged. - page-holes.c: mi_page_discard_slack excluded nothing for the MI_SECURE>=5 trailing guard page, so the scavenger's eager-zero memset hit PROT_NONE (SIGSEGV in idle-thread-releases-large-page; also on baseline 227eef0, hidden because the file never compiled as C++). - new-thread-does-not-sweep-at-once counts process-wide discards; the background scavenger's sweep of thread A's abandoned page was charged to C (loaded runs: baseline 33/144, candidate 10/144 fail; with scavenger off 0/144). Run it in its own process with no scanner (test-new-thread-pacing), as test-reserved-page-reuse does.
Local-Gate: v1 tree=014360df1f9a7f8bdc27ae7f4ac70308aae073b8 secs=117 lanes=lint:run
Ci-Attestation: {"at":1792726984,"gate":"python/all/lint","host":"linux-x86_64","key":"098c419ed124f70b7bcdfa9570aa5c6b2a83ce1bf4f46b8a56d14ce9940959c4","lane":"lint","parents":["bd05be78f40c0ae5036ec741264128e6b26f3a9c"],"secs":117,"stamp":"a7ab40e4b8f82de76a1bf07d379360d6","tree":"014360df1f9a7f8bdc27ae7f4ac70308aae073b8","v":1,"via":"run"}
#618 disposition — fixed on #616 head
|
| build, 144 runs, 48 concurrent | fails (discards) |
|---|---|
| candidate, scavenger on | 10/144 (8 discards) |
| baseline 227eef0, scavenger on | 33/144 (8 discards) |
baseline, MIMALLOC_SCAVENGER=0 |
0/144 |
fix (--new-thread-pacing, own process, scavenger off) |
0/144 |
The test reads a process-wide counter, so the scavenger's sweep of thread A's abandoned page during C's life was charged to C. The defect predates V7: the baseline fails more often. V7's path was not exercised (0 repurposes). #478 is about the owner thread's first tick, so the case now runs in its own process with no scanner (test-new-thread-pacing, MIMALLOC_SCAVENGER=0;MIMALLOC_PURGE_HOLES=1). That mirrors test-reserved-page-reuse. The background-release cases stay in the default invocation with the scavenger on. No threshold changed. The original candidate failure and the baseline pass from the evidence archive stay on record. The baseline passing once was luck under low load.
Also fixed: PR CI build (release) and the other Release rows failed with -Werror=unused-parameter on page in mi_page_queue_remove_retired, because the asserts compile out. Added MI_UNUSED(page).
Local validation: owner-gate Debug (MI_DEBUG_FULL + MI_DIAGNOSTICS + MI_PPROF; no ASan headers on this host, CI's asan row covers ASan) 86/86 passed. Release MI_PPROF 70/70 passed. Secure C++: purge-holes passes. Local gate 45/45.
Not fixed by this change:
- In the secure C++ build,
test-fork-locks*fails to link on the baseline too. Filed as test-fork-locks fails to link under MI_USE_CXX (C++ linkage of _mi_test_* hooks) #619. - One
test-no-process-detach-skippedfailure under heavy concurrent host load: 30/30 passes on both baseline and fix when rerun. Untouched by this diff; recorded, not waived.
Native MSVC and win-gnu run in PR CI.
Unblock decision: #618's validation blockers are resolved. The V7 CPU measurement under #614 may resume. No CPU, throughput or RSS gain is claimed. #616 stays a draft until #614's perf-ab campaign and platform gates pass.
Local-Gate: v1 tree=e9507dcd890ebfa68486fabba474edb2aeb10981 secs=135 lanes=lint:run
Ci-Attestation: {"at":1792728348,"gate":"python/all/lint","host":"linux-x86_64","key":"c13049b70c7ef4ea36465b2fc6a6c46595db75269740c0568f86a2a3351c0581","lane":"lint","parents":["f04332301b27eb4c964361a84d173614097f4d07"],"secs":135,"stamp":"e96f7fa851abf27ea632d99e10b5bb5a","tree":"e9507dcd890ebfa68486fabba474edb2aeb10981","v":1,"via":"run"}
V7 verdict: REJECTED (rule 12)Complete perf-ab runs comparing head
Earlier attempts, kept as evidence: 39896572843 (sanity gate tripped on base noise, #620) and 39897074404 (90-min timeout, incomplete; diagnosed and fixed in #621). Disposition: V7 joins V1–V6 as rejected. The #618 validation fixes do not depend on V7 and continue separately in #622. Closing this draft. Full ledgersperf-ab run 39905247635: https://github.com/zackees/mimalloc-pprof/actions/runs/39905247635 Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py):
perf-ab run 39909998440: https://github.com/zackees/mimalloc-pprof/actions/runs/39909998440 Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py):
|
…luted pacing test (zackees#618) - page-holes.c: mi_page_discard_slack ran its range to the end of the slices, over the MI_SECURE>=5 trailing PROT_NONE guard page; the scavenger's eager-zero memset there segfaulted (idle-thread-releases- large-page, deterministic in a secure C++ build). Stop before the guard, as arena.c's page_noguard_size does. - test-purge-holes.c: scope test_owner_sweep_pacing's measurement locals so its early goto done is valid C++ (MI_USE_CXX); reads unchanged. - new-thread-does-not-sweep-at-once counted process-wide discards, so a background scavenger sweep of thread A's page during C's life was charged to C (loaded: 33/144 fail at 227eef0; 0/144 with the scavenger off). Run it in its own process with no scanner (test-new-thread-pacing, static builds), as test-reserved-page-reuse does. Split out of the rejected V7 draft zackees#616 so the fixes land on their own.
Draft: V7 large-page reuse CPU candidate
Tracks #614. Benchmark prerequisite #615 is complete through merged PR #617.
Not ready to merge. V7 has not been performance-tested. Validation blockers are tracked in #618; no CPU, throughput or RSS gain is claimed.
Implementation
Specialize removal of a selected retired large singleton at the existing call site after unpublication. Selection proves normal large bin, singleton links and clear full-queue flag. The new helper avoids generic linked-list checks and a redundant atomic flag clear, preserving queue/page counts and all geometry, retention, aging, purge, ownership and failure/callback ordering. Other callers retain the original generic remover.
C commit
2ec4d32e; separately generated Rust C28ef6ad7. Frozen performance baseline227eef06ff03f92966f38508441d6c2f60a75135.Verified and unresolved
new-thread-does-not-sweep-at-onceobserved six global discards, expected zero. Matching baseline passed. This is retained, unresolved and not waived.Evidence and handoff
See V7 handoff,
v7-validation-evidence.zipand its adjacent digest in.github/assets/perf-614/. Archive: 46 members, 4,864,167 bytes; SHA256c1a9c4ff57f1694031ed38aeee4b1e1dc9f9f05802fe925b76afe642d06465e1. Exact binaries, commands, source patches/headers, logs, configurations and member hashes are retained.V1/V2/V4/V6 lacked supported CPU/throughput gains; V3 had confirmed regressions; V5 was rejected for a reentrant collection risk. All rejected code was removed and original evidence/ledgers retained.
Resolve validation blockers before resuming #614's predeclared measurement and platform gates. Final merge, cleanup and synchronization with origin/main remain outstanding.
Local pre-push gate: 45/45 passed; stamped head
6155a79f6701f930ea799e66bc08a22c75ca3a30. This does not waive the recorded test failures.