Skip to content

perf: reduce large-page bookkeeping and complete measurement gates (#585) - #609

Draft
zackees wants to merge 20 commits into
mainfrom
perf/585-large-path-costs
Draft

zackees wants to merge 20 commits into
mainfrom
perf/585-large-path-costs

Conversation

@zackees

@zackees zackees commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

#585: large-page CPU costs and measurement gaps

Draft; 12 of 13 original performance cells have PASS support. Chart Larson remains INCONCLUSIVE; one bounded follow-up is running; full platform CI has completed. Current head is config/docs-only 3db601e13aace89ca44b2be9dbd59b2ee8f5f034, activating historical Windows memory references. Allocator/header/test/CMake/Rust/benchmark/harness source is unchanged from measured da1fdb98; that source includes word-hash C 0331d770, separate Rust regeneration 6f7be38f, and the serial test fixture correction. Fixed baseline is ee3623f6b7a485754a0a0aa3462d8cba43390e8b. Previous 15bfcba0 profiler p99 FAIL remains recorded.

Gate Current status
Platform CI da1fdb9 completed 104 SUCCESS/five intentional SKIPs. Detailed logs exposed missing Windows memory references: comparisons/controls skipped inside green steps. Config-only 3db601e fixes that from historical merged-source data; current full CI completed104SUCCESS/fiveintentionalSKIPs, with actual Linux/Windows memory comparators and leak controls verified
Local verification Current config commit 45/45 local gate PASS in35s; both Windows normal eight-run sets pass and both leak sets trigger against historical references; same sole reviewer checked provenance
Performance Six plain PASS cells carried through verified whole-executable identity; profiler PASS60/arm; short chart PASS60/arm; random/8 and standard chart PASS80/arm. Small/8 PASS110/arm and plain Larson/8 PASS115/arm; chart Larson INCONCLUSIVE120/arm
Memory experiments No new policy retained. Faster aging rejected for p99; geometry/capacity-fit failed fungibility; singleton compact worsened RSS; resident retry had no useful benefit

Earlier failed and incomplete runs remain recorded below. The new prototype changes only profiler stack hashing. Harness checkpoints now survive forced termination; previous candidate failures remain visible. A rebase onto unchanged origin/main completed; final revalidation and scratch cleanup remain pending.

Allocator candidate

Frozen baseline: ee3623f6b7a485754a0a0aa3462d8cba43390e8b. Old timed candidate: 9b67e411fc8c8899e07e05d528a4c5b37ab20417. Its results below do not validate the new byte-threshold prototype in C commit fc1ee95e. The prototype precomputes the minimum capacity in bytes with overflow handling, removing a per-candidate division while preserving eligibility and first-best selection; Rust regeneration is a separate commit.

  • Stop the retired-page best-fit search at its first exact fit, preserving the first-best selection.
  • Preserve the valid page-map prefix when an empty, owned, separated-metadata arena page is re-carved. Shrinking clears the stale tail; growing retains registration and failure cleanup.
  • Extend the large-span structural test for stale tails and registration budgets: RED before the map change, GREEN afterward. Rust amalgamation regeneration is a separate commit.

No new memory policy has been retained. Geometry trimming, singleton compaction and resident-retry probes were rejected for failed fungibility, worse RSS or no useful benefit. A new capacity-based fit variant also failed the unchanged fungibility check (four pages vs limit three) and was rejected before RSS/timing: diagnostic patch and outcomes.

New word-hash prototype validation

Current 6f7be38f replaces byte-wise FNV rounds with the existing stateless word mixer in stack_hash; full depth/PC equality remains the collision check. No layout, capture, sampling/PRNG, arena or record behavior changes. Six saved local cycle annotations place 39.1–55.7% of intern samples in the old hash loop under the global profiler lock; this is a CPU target, not an established CI failure cause.

Debug profiler tests: 10/10 PASS; soldr regeneration/check PASS; same sole primary review clean; 45/45 local gate PASS in 32s. Platform CI failed the musl page-reuse assertion; no retry or relaxed assertion is being used to waive it. Fixed-baseline exact-profiler acceptance 39532706232 compares EE → 6f7be38; hash attribution 39532708955 compares 15bfcba → 6f7be38. Both retain 60-to-120/null/sanity/original trace. Fixed-baseline run 39532706232 completed with 60 actual samples per arm, all six metrics PASS on Xeon 8573C: CPU -0.9% [-2.7%,+1.4%], p99 -1.4% [-3.7%,+0.7%], zero memory credit. Four actual executable hashes verified. This one-row result does not establish an RSS improvement or resolve the musl correctness failure. Hash-step attribution completed at 120 actual samples/arm on EPYC9V74: CPU -1.84% [-2.11%,-1.55%], p99 -2.33% [-4.57%,-1.04%]; after-drain RSS remains INCONCLUSIVE, so the row is not an overall PASS. Full ledger. The six individual Linux measurements have completed: five PASS, chart Larson INCONCLUSIVE. Acceptance still requires all full CI gates and resolution of that cell. Controlled baseline interleaving and fixture verification: EE and 6f both create one new page if the scanner owns A’s reserved page during B’s allocation; both reuse A with no scanner. This proves a possible mechanism, not the historical CI cause. The original same-page and zero-created assertions run in fresh processes with scavenging disabled and verified; background release cases remain enabled. Native/musl verification and four actual CTest entries passed; 45/45 local gate passed in 35s; same reviewer clean.

Current plain executable identity and remaining validation

Build-only 39536056572 freshly built EE and da1fdb9. All four actual linked plain timing/latency files are byte-for-byte identical to the earlier timed EE/15 pair; six report file hashes verified. This carries forward six distinct plain PASS rows (main, plain ephemeral, sparse, bursty, Larson/1, random/1) with their original refs/hosts/counts/verdicts; no new timing, relabeling or pooling. It does not resolve plain small/8, Larson/8 or random/8. Chart enables PPROF and its binaries changed, so Larson/8 plus both ephemeral chart rows need new measurements. All six individual measurements completed under the original 60-to-120/null/sanity plans: five PASS and chart Larson INCONCLUSIVE. Linux parallel/serial, both native Macs, native MSVC, Rust/cross/ASan passed on current 3db; Windows bundles subsequently completed SUCCESS. Removing the assistant-imposed scheduling dependency changes no final acceptance gate. Six live run handles and scheduling scope. Identity evidence and scope.

Windows memory baseline verification gap fixed

The completed da1fdb9 CI collected Windows data but skipped both compiler-specific baseline checks and leak-positive controls. The new references use only eight historical normal runs per compiler from merged-source80a513/run36812458448, whose allocator/test/build paths are byte-identical to frozen EE. No candidate data or changed tolerance initializes a reference. Current minimum-of-eight sets PASS at71.6MB peak commit, and GNU/MSVC leak sets fail at125.7/131.6MB. Current3db CI has now executed all four memory checks and controls: normal minimum71.6MB for each compiler, GNU leak131.6MB and MSVC leak126.8MB correctly rejected. All32 actual input hashes verified, unchanged comparator independently reproduced, same sole reviewer clean; fresh artifact results. Overall Windows job subsequently completed SUCCESS; current full CI is104SUCCESS/fiveintentionalSKIPs. New3db601e1 activates those previously skipped CI paths; full provenance, output and input hashes. This is historical Windows memory-gate validation, not a same-host performance ledger or RSS saving.

Current remaining-row outcomes

  • Short chart39537470652: PASS all six,60actual/arm, EPYC7763; CPU-1.2%, p99-1.1%,0memorycredit. Full ledger.
  • Random/8 39537464950: PASS all six,80actual/arm, EPYC9V45; CPU interval contains0, no CPU improvement claim,0credit.
  • Standard chart39537468554: PASS all six,80actual/arm, EPYC7763; CPU-1.1%, p99-1.6%,0credit. Drain upper bound equals original null-noise allowance exactly; comparison uses full precision.
  • Small/8 39537461436: PASS all six, 110 actual/arm, EPYC9V45, zero credit.
  • Plain Larson/8 39537463373: PASS all six, 115 actual/arm, EPYC7763, zero credit. P99 upper interval equals the null allowance at full precision.
  • Chart Larson/8 39537466521: INCONCLUSIVE faults/p99/drain at 120 actual/arm, EPYC7763; CPU/throughput/peak PASS, zero credit. P99 upper 0.4166667% exceeds 0.3984064% null noise; rounded 0.4% values do not establish PASS. All three complete ledgers. One predeclared 120-to-180 follow-up runs with identical source, workload and gates. No third automatic retry; original uncertainty and any future failures remain in the record.

Every completed new row's four actual timing/latency file hashes are verified. Old15 INCONCLUSIVE outcomes stay attached to their original source/hosts/counts; they are not relabeled or pooled. Chart executables changed with word-hash compiled in but profiling runtime-unarmed, so chart CPU deltas are not attributed to executing that loop. Current3db config changes no measured program source.

Previous byte-threshold candidate validation

15bfcba0 against frozen ee3623f6: exact plain ephemeral/8 PASS, 65 actual samples/arm, EPYC 7763, all four actual executable hashes verified. Baseline executables match the prior failed run byte for byte; separate VMs/source pairs are not pooled. The old candidate's failure remains below.

The following original-cell validations retain 60-to-120 samples/arm, original traces and unchanged gates:

  • main + profiler: 39520753532: plain PASS all six, 60/arm; profiler FAIL p99 +11.3% [4.2%,15.7%] versus 3.9% noise, 120/arm, EPYC 9V45. Both have zero memory credit. Full failure ledger. The failed pair is not being rerun for acceptance; 39526717341 isolates old prefix-reuse → byte-threshold, exact profiler row: INCONCLUSIVE p99/drain at 120/arm on EPYC 7763, other four metrics PASS; full attribution. Earlier-step profiler isolation completed: baseline → exact-fit PASS all six at 95 actual samples/arm in 39531382092 and exact-fit → prefix reuse INCONCLUSIVE faults at 120 actual samples/arm, other five PASS, in 39531384408, unchanged 60-to-120/null/sanity plans. These are attribution, not acceptance retries; no cross-host pooling or addition of deltas.
  • chart ephemeral variants: 39520756150: standard PASS all six, 80/arm; short generations INCONCLUSIVE faults/peak/drain, 120/arm, EPYC 7763. Full ledger; exact short-generation follow-up 39526576845 remains INCONCLUSIVE on minor faults at 240 actual samples/arm, EPYC 9V45; CPU -0.9%, p99 upper bound zero, other five metrics PASS. All four actual binaries match the first chart batch; different CPUs are not pooled. This remains old-15 evidence, not new-hash acceptance.
  • Original four-control screen: 39520758674, retained incomplete after one repetition, with no verdict. Full raw data and baseline-only attribution; thresholds unchanged.
  • Three-control replacement (small/8 and Larson/8 plain/chart): 39521491231, cancelled by its 90-minute CI limit; no artifacts and no verdict. Logs and evidence-loss report. That historical run provides no control verdict; later individual outcomes are recorded above. The three-row group was not restarted.
  • Exact Larson/1 replacement: 39521493862, PASS all six, 65 samples/arm on EPYC 9V45; zero credit. Faults/drain intervals are exactly [0,0]; peak interval [-0.0023213%,0%], with 0.0023212% noise, has zero regression upper bound and PASSes.
  • random-large /8 + /1: 39520761595: /1 PASS all six, 60/arm; /8 INCONCLUSIVE faults/peak, 120/arm, Xeon 8573C. Full ledger; exact /8 follow-up 39526533882 also remains INCONCLUSIVE at 120/arm, EPYC 9V45, on throughput/drain RSS; other four metrics PASS. Its binaries match the first run, but different CPUs are not pooled. No further acceptance retry of the rejected candidate is planned while the profiler failure is being attributed.
  • bursty: run 39520763939 PASS all six, 60 actual samples/arm, EPYC 7763; CPU -0.8%, p99 upper bound 0%, zero memory credit. All four actual binaries match the other ordinary runs.
  • sparse 64-slot: run 39520766570 PASS all six, 65 actual samples/arm, EPYC 9V74; CPU -1.7%, ops +1.4%, p99 within noise; zero memory credit. All four actual binaries match new ephemeral byte for byte.

The isolated aging-setting trial 39521589663 finished at 30 samples/arm on Xeon 8573C. Default 4 PASSed all six metrics, with CPU -1.9%; its executables match the current candidate byte for byte. Setting 2 is rejected: peak RSS -3.9% [-5.3%, -2.6%], but p99 +9.8% [7.6%, 10.8%] versus 1.4% noise FAIL. It also costs 2.15% CPU relative to the default on the same VM. Both ledgers retain the frozen baseline; no setting was changed or retried after failure. The separate ordinary main/profiler run passed plain but failed profiler p99; the four-arm default-trial PASS does not waive that failure. All local screen pairs and scope limits.

Corrected old-candidate measurements

All rows use the exact source pair above, a same-run null arm, balanced arm positions, original work traces and unchanged sanity/ledger thresholds. Actual timing and separate latency executables are preserved with verified SHA256 manifests. Each result is a same-host paired comparison; CPU models are listed because these runs are not pooled across models.

cells corrected evidence status
large-class/8 60/arm, EPYC 7763, 39502531125 INCONCLUSIVE p99/peak/drain
plain ephemeral/8 60/arm, same run FAIL p99 +2.7% [1.6%, 3.3%] against 1.1% noise, despite CPU/op -1.5%
profiler-on large-class/8 60/arm, same run INCONCLUSIVE p99
chart ephemeral/8 60/arm, same run INCONCLUSIVE peak RSS
chart short generations/8 55/arm, same run PASS all six metrics
small/8 60/arm, EPYC 7763, 39509436788 PASS all six metrics
Larson/1 60/arm, EPYC 7763, 39509440439 PASS all six metrics
Larson/8 plain and chart 30 and 35/arm, EPYC 9V74, 39502534567 both PASS
random-large/8 and /1 30 and 60/arm, EPYC 9V45, 39502538433 both PASS
random-large-bursty/8 75/arm, Xeon Platinum 8573C, 39511940176 PASS all six metrics; original 9V45 uncertainty retained
sparse 64-slot/8 95/arm, EPYC 7763, 39511942928 PASS all six metrics; original 9V45 uncertainty retained

The original EPYC 9V74 small/8 and Larson/1 rows remain INCONCLUSIVE; independent 7763 passes do not reclassify those rows. Every control now has a conclusive PASS in a corrected run. The focused control binaries match the original control binaries byte for byte. All old-candidate control/random follow-ups are terminal; different CPU models are not pooled.

The target failure is preserved with all twelve actual timing/latency executable hashes verified: full target ledger and binary evidence. The candidate remains unaccepted. Source attribution is terminal: baseline → exact-fit-only (39514174840) passed all six metrics with 60 samples/arm on EPYC 9V74. Exact-fit-only → mapped-prefix reuse (39514578328) remains INCONCLUSIVE with 60/arm on EPYC 9V45. An earlier prefix diagnostic (39514177655) failed its first sanity screen, including the unchanged-baseline null executable: all raw data is retained as incomplete, with no verdict or pooling. None of these comparisons erases the combined-candidate p99 failure. Full attribution and limits.

Full ledgers and scope limits: initial controls and untimed residency attribution, focused controls, random/sparse and follow-ups.

Earlier failures remain in the record

Previous head c54e2a26 had 102 successes, five skips and two failed checks from the GCC ASan idle-release deadline assertion and verifier. Its cause remains unexplained; the current source subsequently passed that lane and all 104 platform checks. Full current platform evidence.

The previous perf-ab arm schedule only reversed endpoints, leaving head in the middle every repetition. Those runs remain diagnostic; their order protocol cannot be pooled with corrected runs.

  • Legacy exact-fit-only run 39489674226 failed small/8 at +20.3% CPU/op. Corrected exact-fit-only run 39502542281 did not reproduce the increase but remains INCONCLUSIVE overall. It is not combined-candidate acceptance.
  • Legacy combined run 39491166866 failed the chart drain-RSS sanity screen, 9.9 → 76.7 MiB. Local unchanged-baseline runs reproduce substantial 0.5-second drain variability, but do not establish that original failure's cause. The original threshold remains unchanged.
  • Those original failed timing binaries were discarded by the old harness. Later fresh builds cannot establish their layout or explain the failures.

Failure evidence, corrected exact-fit diagnostic and baseline-only drain observations.

Tooling delivered

  • Sample stream-operation p99 in a separate replay executable; throughput/RSS binaries contain no sampling code. Include p99 without memory credit, and report logical CPUs and oversubscription.
  • Rotate arms through every position and reverse alternate blocks. Three arms cover all six permutations per complete six-repetition cycle. Artifacts record the protocol; pooling rejects incompatible or missing protocols, CPU identity/count, source refs and latency definitions.
  • Queue PR pushes behind live perf-ab runs. Preserve all first-repetition data on a sanity failure, mark it incomplete and refuse a regression verdict or pooling.
  • Preserve built timing/latency executables before measurement with SHA256 manifests. Atomic incomplete checkpoints follow completed measurements; flushed parent progress identifies the current arm/row. Four new forced-interruption/pre-copy tests RED before, 91 focused tests GREEN afterward; 45/45 local gate and same sole reviewer clean. No measurement/threshold change. A separate build-only diagnostic retains compiler settings, symbols and disassembly without claiming measured-binary identity.
  • Enforce 15 blocks for diagnostic scaling dispatches and correct both workflow descriptions.
  • Support '=full workload name' for exact per-cell attribution; existing substring and OR filters remain valid. Integration RED/GREEN and 79 focused tests pass; no row parameters change.
  • Keep cross Rust setup and builds inside soldr. Missing Windows ARM64 std persisted with target installation; the documented fresh-toolchain cache opt-out is scoped to Windows ARM64. Evidence and mitigation.

No new workflow file was added. C-core and Rust paths are separate in every commit.

Historical CPU attribution

Corrected pre/post-merge runs preserve exact binaries and null arms. Separate CPU models and VMs prevent adding these deltas into a decomposition of the whole fork/upstream gap.

merge actual samples per arm result
#570 60 plain and profiler, EPYC 7763 CPU/op -2.1% plain and -1.0% profiler; overall INCONCLUSIVE on remaining metrics
#569 30 plain, 60 profiler, EPYC 9V74 both PASS; plain peak RSS -5.3%
#572 30 plain and profiler, EPYC 9V74 plain p99 +5.2% [4.4%, 5.2%] against 0.7% noise: FAIL; profiler PASS

Full corrected historical ledgers and limits. #544 is the issue for refining demand-sized spans; #569 and #570 are two merged implementations within that issue. Its broader acceptance checklist is not independently certified by these narrow comparisons. Historical results do not substitute for current-candidate acceptance.

Remaining memory evidence

Same-run competitor scaling used 15 balanced blocks, 1,800 samples and zero paired trace/checksum mismatches. Allocator C matches the old timed candidate; its temporary harness also carried the tested benchmark-builder dependency from #608. No chart was published.

8-worker pattern fork/TCMalloc peak-RSS ratio, 95% interval
sparse large buffers 0.988 [0.969, 1.041]
persistent 1.224 [1.169, 1.275]
power of two 1.206 [1.176, 1.333]

The prior sparse gap was not reproduced. Persistent and power-of-two gaps remain. Scaling CPU/op, faults and p99 were not measured; this is diagnostic evidence, not a fixed-baseline acceptance ledger. Persistent fork/upstream throughput remains 0.874 [0.860, 0.905]. Scaling report.

Independent prefix attribution locates about 27 MiB of resident unformed capacity in persistent/8, already present early. These are medians of three separate truncated processes per prefix, not continuous-run snapshots. Completed reuse intervals cluster near 14–16 microseconds, with censoring limits documented; no discard policy was retained. Attribution and rejected-probe context.

Verification and remaining work

Local debug C suite: 89/89. Default fast-path identity passed against pinned upstream and frozen baseline; structural RED/GREEN, same-build layout checks and Rust regeneration are covered. Previous-head full platform checks include Linux ON/OFF, native MSVC cl, win-gnu/MSVC bundles, native macOS ARM64/x64, Rust cross targets, ASan, TSan and fuzz. The selector head subsequently failed the GCC ASan release-deadline assertion. Matching GCC 13.3 local probes passed 10 unchanged-baseline and 10 unchanged-old-candidate runs; 20 additional diagnostic runs per arm also passed with used=1, capacity=1 and reserved=3 before free. Those local observations do not explain the prior CI failure. The new candidate subsequently passed that GCC ASan lane; all 104 platform checks subsequently passed (five intentional skips). Failure evidence.

Remaining: resolve chart Larson performance uncertainty; archive the now-verified actual Windows/Linux memory comparisons and leak controls; record decisions for remaining memory gaps; finish the requirement audit; fetch and rebase onto origin/main again; archive final evidence and remove owned scratch artifacts. Temporary remote branches and extra worktrees are already removed. This PR does not claim that #585 is complete.

Free-capacity time attribution

Existing trace reanalysis reproduces completed reuse counts and independently conserves observed free-slice time. About 99% of completed intervals are under 1ms, but at the provisional last-arena-claim cutoff, 37–51% of virtual free-slice time has age over 1ms. This is not resident-byte-time, a peak/work-phase marker, or evidence for a new discard threshold. Phase and residency events are required before a cold-release policy. All raw trace hashes, observed censored intervals, clock-order inversions, full-window results and analysis code are recorded; no new memory policy was retained.

Local-Gate: v1 tree=6703d5a7b82b1192cc68aa8bb32fb8a5f8b1543a secs=31 lanes=lint:run
Ci-Attestation: {"at":1792526371,"gate":"python/all/lint","host":"linux-x86_64","key":"faac07e785ec1e972423557bb7cb2eca9b94e7a6fcf90b0689fc9110dbe1b148","lane":"lint","parents":["ee3623f6b7a485754a0a0aa3462d8cba43390e8b"],"secs":31,"stamp":"c07bd0cdfb89dc89d89c28e9bc518744","tree":"6703d5a7b82b1192cc68aa8bb32fb8a5f8b1543a","v":1,"via":"run"}
Local-Gate: v1 tree=20d414c77126a74712c5597a430e88a2ce1c6622 secs=40 lanes=lint:run
Ci-Attestation: {"at":1792526844,"gate":"python/all/lint","host":"linux-x86_64","key":"8b72b190f62addb335eee8fef10fb3cb1f7626ca542907a9438ed081ff60e25a","lane":"lint","parents":["f631fd48f79377d1738b8cafe380c40a3d8d7ab1"],"secs":40,"stamp":"7fcf2a6f9e500cca4e56b56f21080b95","tree":"20d414c77126a74712c5597a430e88a2ce1c6622","v":1,"via":"run"}
)

Local-Gate: v1 tree=39069b2fd2eb2b32a95301905f562c4370416584 secs=45 lanes=lint:run
Ci-Attestation: {"at":1792526986,"gate":"python/all/lint","host":"linux-x86_64","key":"acc1290a4c74a80341ac15c2697de2c7b80edc356c6d8db88fb4ef032c692712","lane":"lint","parents":["1d4d4ba4a50976e85cb27915cbd0c03aa90e3111"],"secs":45,"stamp":"219c5e7f901beb08a334be8c4ecb0d4e","tree":"39069b2fd2eb2b32a95301905f562c4370416584","v":1,"via":"run"}
Local-Gate: v1 tree=185a2b2a427e55aee2ed4db0827572e4c8faa085 secs=71 lanes=lint:run
Ci-Attestation: {"at":1792527471,"gate":"python/all/lint","host":"linux-x86_64","key":"dd2d975bf1c252bb5eb89e902bfca56e93ad6f746359921e24b974dc7bc30457","lane":"lint","parents":["d3e043675b2d374f3643e4c8d99481548386ecc5"],"secs":71,"stamp":"298b93679c3f27a691eb38571a369605","tree":"185a2b2a427e55aee2ed4db0827572e4c8faa085","v":1,"via":"run"}
@zackees zackees added the ci-full Opt-in complete platform and release validation on PR SHA label Oct 20, 2026
…ts (#585)

Local-Gate: v1 tree=ae356b3f151f53ade04098d16d3a7b8d48f3a6d8 secs=32 lanes=lint:run
Ci-Attestation: {"at":1792528190,"gate":"python/all/lint","host":"linux-x86_64","key":"ca75dca180e65fd2e2c8efc8baed969d3d309afa476a228a6b4ec50a41482745","lane":"lint","parents":["8b41221106917ea66cceec4d08e01e1f8736c998"],"secs":32,"stamp":"47b752957e8c7cae7c3a04a8b8d22edc","tree":"ae356b3f151f53ade04098d16d3a7b8d48f3a6d8","v":1,"via":"run"}
Local-Gate: v1 tree=72f146723f4e459def99022e96846c349876e48f secs=42 lanes=lint:run
Ci-Attestation: {"at":1792528576,"gate":"python/all/lint","host":"linux-x86_64","key":"d01156bffae95535acf8d582ba8ebb0a41e0a3a473b61f1e6046beea40d4a6a6","lane":"lint","parents":["26281e248fcd8569e5f09142f6ae37d678482aab"],"secs":42,"stamp":"45b3f9478fde95870d8a46ddd10304b1","tree":"72f146723f4e459def99022e96846c349876e48f","v":1,"via":"run"}
@zackees

zackees commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

#585 progress: historical attribution and rejected geometry experiment

This is an interim research report, not candidate acceptance or completion. Frozen candidate baseline remains ee3623f6b7a485754a0a0aa3462d8cba43390e8b. Historical comparisons below isolate earlier merges and must not be combined across hosts or treated as candidate credit.

Historical paired comparisons

Each run used 15 paired repetitions plus a base-vs-base null arm, four logical CPUs, and the unchanged trace. Eight-worker cells are 2:1 oversubscribed. p99 is sampled stream-operation latency from a separate replay, leaving throughput/RSS runs uninstrumented. CPU/op below is process CPU divided by the fixed stream-operation count, including the harness’s existing drain CPU accounting; it is not active-worker-only CPU. Raw samples are in each linked run’s perf-ab-samples artifact.

#570: run 39489801571

Baseline 3672dbd66e39526c712bb5cea52b4a70f026ade3 → cb142ae2a27c4fd55570cc2a20e0158dd5910e51; AMD EPYC 9V45 96-Core Processor.

workload CPU/op baseline → revision (ns) peak RSS baseline → revision (MiB) drain RSS baseline → revision (MiB) p99 baseline → revision (ns)
large-class/8 531.62 → 530.16 187.21 → 190.45 173.41 → 172.79 1823.00 → 1822.00
larson/8 34.12 → 33.84 294.68 → 302.75 211.82 → 213.06 200.00 → 200.00
larson/8 (chart build) 34.69 → 34.02 296.71 → 294.77 210.84 → 211.59 200.00 → 200.00
large-class/8 (profiler on) 845.50 → 840.47 192.98 → 193.29 172.55 → 173.98 36736.00 → 37988.00

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 0% INCONCLUSIVE +1.8% [+0.2, +2.9] / 2.2% PASS -0.3% [-1.6, +0.4] / 1.8% PASS +20.9% [-18.4, +74.6] / 61.2% INCONCLUSIVE +0.5% [-2.1, +2.2] / 2.7% PASS +1.1% [-0.8, +1.7] / 3.4% PASS -0.5% [-1.5, +1.4] / 3.8% PASS
larson/8 0% INCONCLUSIVE +0.5% [-2.5, +3.4] / 4.4% PASS -2.0% [-3.0, +3.6] / 4.8% PASS +1.0% [-0.9, +2.8] / 3.8% PASS -0.5% [-5.0, +0.0] / 5.0% PASS +2.0% [-1.4, +6.8] / 6.5% INCONCLUSIVE +0.9% [-0.1, +1.4] / 1.2% INCONCLUSIVE
larson/8 (chart build) 0% INCONCLUSIVE +2.1% [-0.0, +3.6] / 2.9% PASS +0.3% [-3.1, +2.7] / 2.8% PASS +1.0% [-1.9, +2.2] / 1.9% INCONCLUSIVE +0.0% [-0.5, +5.5] / 5.0% INCONCLUSIVE +0.0% [-5.8, +7.7] / 4.0% INCONCLUSIVE +0.8% [-0.8, +2.1] / 1.7% INCONCLUSIVE
large-class/8 (profiler on) 0% INCONCLUSIVE -0.2% [-1.6, +1.2] / 1.6% PASS -0.6% [-1.5, +1.6] / 1.9% PASS +2.9% [-6.0, +12.1] / 30.1% PASS +5.3% [-4.2, +9.9] / 12.0% PASS -0.4% [-2.9, +4.3] / 1.8% INCONCLUSIVE +1.0% [-1.9, +2.3] / 4.0% PASS

#569: run 39489808894

Baseline cb142ae2a27c4fd55570cc2a20e0158dd5910e51 → 80936c9632b3f60fdf4d8cf397e2a615fc635f83; AMD EPYC 7763 64-Core Processor.

workload CPU/op baseline → revision (ns) peak RSS baseline → revision (MiB) drain RSS baseline → revision (MiB) p99 baseline → revision (ns)
large-class/8 415.81 → 418.56 191.21 → 178.30 177.08 → 159.13 1343.00 → 1352.00
larson/8 57.09 → 57.73 282.75 → 279.12 208.00 → 208.00 240.00 → 241.00
larson/8 (chart build) 60.28 → 61.82 286.38 → 283.16 49.03 → 45.71 250.00 → 250.00
large-class/8 (profiler on) 920.72 → 919.94 191.31 → 184.11 169.48 → 157.37 43802.00 → 45635.00

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 7% PASS -0.1% [-1.4, +0.5] / 2.3% PASS +0.5% [-0.5, +1.1] / 2.3% PASS -24.4% [-58.8, +34.6] / 120.4% PASS +0.0% [-0.7, +0.7] / 1.4% PASS -6.4% [-8.7, -3.2] / 1.9% PASS -10.0% [-11.8, -8.3] / 2.6% PASS
larson/8 0% PASS +0.5% [-3.1, +3.6] / 4.3% PASS +0.6% [-2.1, +2.2] / 3.1% PASS +32.7% [-7.0, +60.0] / 87.9% PASS +3.4% [+0.0, +4.2] / 4.3% PASS -0.3% [-5.0, +5.4] / 6.8% PASS -1.5% [-7.2, +2.2] / 30.6% PASS
larson/8 (chart build) 0% INCONCLUSIVE +0.0% [-3.6, +1.7] / 3.1% INCONCLUSIVE +1.7% [-1.3, +3.5] / 2.7% INCONCLUSIVE +6.2% [-6.1, +75.0] / 50.5% INCONCLUSIVE +0.0% [-3.7, +3.3] / 3.7% PASS -0.2% [-6.9, +7.5] / 9.7% PASS -5.0% [-8.6, -2.8] / 8.0% PASS
large-class/8 (profiler on) 4% INCONCLUSIVE -0.2% [-0.9, +0.7] / 1.4% PASS +0.0% [-0.7, +1.2] / 1.3% PASS -10.3% [-32.9, +25.4] / 19.3% INCONCLUSIVE +3.8% [-11.6, +15.8] / 11.8% INCONCLUSIVE -4.3% [-5.7, -1.1] / 2.3% PASS -7.7% [-10.1, -4.3] / 1.9% PASS

#572: run 39489815787

Baseline ad957541ccebe36aa4396cdae26f0fe665f55c4d → 668b42e021ee7a18a6ba206ef7b7067fddaaa3a8; AMD EPYC 9V74 80-Core Processor.

workload CPU/op baseline → revision (ns) peak RSS baseline → revision (MiB) drain RSS baseline → revision (MiB) p99 baseline → revision (ns)
large-class/8 318.25 → 325.97 180.21 → 154.52 180.21 → 154.52 1052.00 → 1111.00
larson/8 43.66 → 43.54 292.75 → 290.63 208.44 → 208.95 221.00 → 230.00
larson/8 (chart build) 44.59 → 45.01 286.71 → 290.78 208.84 → 208.64 230.00 → 221.00
large-class/8 (profiler on) 862.22 → 830.19 183.20 → 159.86 160.83 → 157.51 37385.00 → 37185.00

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 14% FAIL -3.6% [-4.7, -0.8] / 4.8% PASS +2.0% [+1.7, +3.2] / 4.8% PASS -33.4% [-59.7, -6.5] / 95.6% PASS +5.6% [+4.8, +6.8] / 1.0% FAIL -14.7% [-16.5, -10.9] / 2.2% PASS -14.7% [-16.5, -10.9] / 3.4% PASS
larson/8 0% INCONCLUSIVE +0.0% [-1.2, +6.1] / 3.3% PASS -1.4% [-5.4, +2.2] / 2.5% PASS +0.2% [-1.5, +6.3] / 22.5% PASS +0.5% [-0.4, +4.5] / 8.6% PASS +0.7% [-6.6, +7.8] / 7.2% INCONCLUSIVE +0.1% [-0.7, +0.6] / 1.1% PASS
larson/8 (chart build) 0% INCONCLUSIVE -0.4% [-2.5, +1.7] / 1.6% INCONCLUSIVE +0.1% [-1.8, +2.0] / 1.7% INCONCLUSIVE -1.2% [-27.3, +34.0] / 52.7% PASS -3.9% [-4.5, +4.5] / 4.3% INCONCLUSIVE -0.7% [-7.7, +5.0] / 9.8% PASS -0.2% [-0.9, +0.4] / 0.9% PASS
large-class/8 (profiler on) 13% INCONCLUSIVE +4.3% [+3.4, +4.5] / 4.3% PASS -3.8% [-4.5, -2.7] / 4.3% PASS -73.4% [-76.0, -70.6] / 20.4% PASS +1.2% [-6.3, +5.9] / 5.8% INCONCLUSIVE -13.4% [-15.9, -9.3] / 5.6% PASS -2.5% [-5.1, +0.3] / 1.7% PASS

Interpretation

Fresh local resident-byte attribution (5 repetitions, untimed)

workload RSS MiB live requested MiB unformed partly-used / empty pages MiB queued free slices MiB geometry slack MiB
persistent/8 66.6 10.5 10.0 / 16.5 12.5 3.8
sparse/8 176.7 45.8 5.1 / 17.2 94.9 2.2

These are resident snapshots, not a proof that every free byte is cold. #531 already refutes simply reducing the retention window or assuming first-fit fragmentation. Retired-body discard remains rejected because of refault CPU.

Rejected: trim span geometry to the last complete block

A scratch experiment removed trailing whole slices without losing block capacity (e.g. 384-KiB blocks: 16 → 12 slices). Capacity checks and test-large-span passed, but the existing test-large-fungible guard failed: 160 allocations rotating through ten bins retained 4 pages rather than at most 3. Smaller, different spans reduced cross-bin reuse. I rejected the change, preserved the guard, and removed the experimental worktree and branch. No experimental allocator code remains in the candidate.

Resident-claim retry-budget screen (untimed, 5 repetitions each)

I also tested the existing MI_RESIDENT_FIRST_MAX_TRIES knob at 8 / 32 / 128, with identical sparse/8 traces and current candidate source. Median snapshot RSS was 174.49 / 174.27 / 172.67 MiB; queued resident slices were 91.23 / 91.12 / 90.06 MiB. This does not support retry-budget exhaustion as a large contributor to the remaining pool. These local byte measurements are not timing acceptance, and no default was changed. A larger retry budget would add scan cost without an established memory benefit.

Current candidate and gates

Draft #609 contains exact-fit search termination and reduced page-map rewriting, plus the measurement gaps requested here. Local debug suite: 89/89; fast-path identity against the fixed baseline and pinned upstream: passed. The combined candidate has not passed performance acceptance: run 39491166866 tripped the first-sample chart-ephemeral drain-RSS gate (9.9 → 76.7 MiB). Run 39492581314 retains the same sanity threshold and now preserves all raw first-sample arms if it fails, so a failure cannot be hidden by a rerun.

Full platform validation also found a missing Windows ARM64 Rust standard-library target (E0463, cross run 39491173013). An explicit matrix-target installation is committed and locally gated; remote validation remains required. Other in-flight runs are being preserved.

@zackees

zackees commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

Reuse-distance evidence and chart-ephemeral diagnostic

Local slice-lifetime trace: diagnostic only

Temporary instrumentation at arena claim/free recorded address, slice count, timestamp, and event order into a bounded static buffer (no hooked allocations). Source policy matched candidate 8b41221106917ea66cceec4d08e01e1f8736c998. Three identical sparse/8 traces and one persistent/8 trace completed without buffer overflow or allocation/free ordering inconsistencies. The instrumentation was restored afterward; it is not in the shipping candidate.

trace arena claims reused 64-KiB slices median / p99 reuse time (µs) median / p99 intervening arena claims reused within 1 ms
sparse 60376 2249519 14.6 / 988.4 5 / 350 99.02%
sparse-2 60380 2249524 16.2 / 1178.2 5 / 317 98.79%
sparse-3 60375 2249443 16.3 / 1141.0 5 / 353 98.83%
persistent 42722 687745 13.2 / 1017.7 1 / 146 98.99%

Limits: these lifetime quantiles count completed reuse intervals, weighted by slices, and exclude intervals still free when tracing ends. They do not prove that all queued memory is hot, and the tracing overhead affects wall time. The allocation-count distances are also reported so the inference does not depend only on instrumented clocks. Never-reused free memory and OS residency require separate treatment.

The same claim/free streams were replayed through two offline interval allocators, both respecting 512-slice chunk boundaries and preserving every request/free. First-fit versus best-fit virtual high-water footprints (MiB) were 164.06 / 168.13, 168.13 / 164.06, and 164.06 / 160.00 across the three sparse traces. Best-fit changes sign across runs and saves at most one 4.06-MiB increment here. This model ignores OS purges and physical residency; it is not a performance benchmark or proof of an implementation benefit. It gives no basis to revive #531’s best-fit proposal as the solution to the ~95-MiB snapshot queue.

The traces support preserving hot reuse. They do not justify a discard policy, and none was added.

Temporary tracing patch for reproduction
diff --git a/src/arena.c b/src/arena.c
index 57db5935..6cf90517 100644
--- a/src/arena.c
+++ b/src/arena.c
@@ -231,6 +231,51 @@ static size_t mi_page_full_size(mi_page_t* page) {
   Arena Allocation
 ----------------------------------------------------------- */
 
+// Scratch diagnostic only: record arena-slice lifetimes without hooked allocations.
+#include <time.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <fcntl.h>
+#include <unistd.h>
+#ifndef MI_SCRATCH_TRACE_CAPACITY
+#define MI_SCRATCH_TRACE_CAPACITY (1U << 20)
+#endif
+typedef struct mi_scratch_trace_event_s {
+  uint64_t address;
+  uint64_t time_ns;
+  uint32_t slices;
+  uint32_t freed;
+} mi_scratch_trace_event_t;
+static mi_scratch_trace_event_t mi_scratch_events[MI_SCRATCH_TRACE_CAPACITY];
+static _Atomic(size_t) mi_scratch_event_count;
+static void mi_scratch_record(mi_arena_t* arena, size_t index, size_t count, bool freed) {
+  const size_t slot = mi_atomic_add_relaxed(&mi_scratch_event_count, (size_t)1);
+  if (slot >= MI_SCRATCH_TRACE_CAPACITY) return;
+  struct timespec now;
+  clock_gettime(CLOCK_MONOTONIC, &now);
+  mi_scratch_events[slot].address = (uint64_t)(uintptr_t)mi_arena_slice_start(arena, index);
+  mi_scratch_events[slot].time_ns = (uint64_t)now.tv_sec * UINT64_C(1000000000) + (uint64_t)now.tv_nsec;
+  mi_scratch_events[slot].slices = (uint32_t)count;
+  mi_scratch_events[slot].freed = freed ? 1 : 0;
+}
+__attribute__((destructor)) static void mi_scratch_dump(void) {
+  const char* path = getenv("MI_SCRATCH_TRACE");
+  if (path == NULL) return;
+  const size_t count = mi_atomic_load_relaxed(&mi_scratch_event_count);
+  if (count > MI_SCRATCH_TRACE_CAPACITY) { fprintf(stderr, "SCRATCH TRACE OVERFLOW %zu\n", count); return; }
+  const int fd = open(path, O_CREAT | O_TRUNC | O_WRONLY, 0600);
+  if (fd < 0) return;
+  const char* data = (const char*)mi_scratch_events;
+  size_t remaining = count * sizeof(mi_scratch_events[0]);
+  while (remaining > 0) {
+    const ssize_t wrote = write(fd, data, remaining);
+    if (wrote <= 0) break;
+    data += wrote;
+    remaining -= (size_t)wrote;
+  }
+  close(fd);
+}
+
 // #493 (strategy 9): resident-first claiming. A freed range sits in `slices_purge` (young) or
 // `slices_purge_aged` for the #486 retention window: free, and still resident. The plain search
 // (`mi_bbitmap_try_find_and_clearN`) knows nothing of that and, measured on short-lived threads,
@@ -372,6 +417,7 @@ static mi_decl_noinline void* mi_arena_try_alloc_at(
   size_t slice_index;
   const bool resident = mi_arena_try_claim_resident(arena, slice_count, &slice_index);
   if (!resident && !mi_bbitmap_try_find_and_clearN(arena->slices_free, tseq, slice_count, &slice_index)) return NULL;
+  mi_scratch_record(arena, slice_index, slice_count, false);
   if (!arena->memid.is_pinned) { mi_arena_unqueue_purge(arena, slice_index, slice_count); }
   #if MI_DIAGNOSTICS
   mi_arena_count_claim(arena, slice_index, slice_count, resident);   // before the dirty bits are set below
@@ -1956,6 +2002,7 @@ static void mi_arenas_free_ex(mi_subproc_t* subproc, void* p, size_t size, mi_me
     }
 
     // and make it available to others again
+    mi_scratch_record(arena, slice_index, slice_count, true);
     bool all_inuse = mi_bbitmap_setN(arena->slices_free, slice_index, slice_count);
     if (!all_inuse) {
       _mi_error_message(EAGAIN, "trying to free an already freed arena block: %p, size %zu\n", mi_arena_slice_start(arena,slice_index), mi_size_of_slices(slice_count));

Chart-ephemeral diagnostic: still not accepted

Run 39492581314 compared frozen baseline ee3623f6b7a485754a0a0aa3462d8cba43390e8b with candidate 8b41221106917ea66cceec4d08e01e1f8736c998, on AMD EPYC 9V45 with four logical CPUs (8 workers = 2:1 oversubscription). It requested 15 repetitions and adaptively collected 25 actual repetitions per arm. Raw samples remain in the run artifact. The original sanity threshold was retained.

This run did not reproduce run 39491166866’s extreme first-sample drain jump (9.9 → 76.7 MiB). Across these 25 samples, drain RSS ranged 35.8–66.0 MiB in the baseline, 35.2–63.0 in the candidate, and 30.0–71.4 in the null. That variability is evidence to investigate the original failure, not permission to erase it or declare a rerun sufficient.

metric baseline median candidate median
ops/s 7126780.500 7170974.500
process CPU ns / stream operation 553.000 549.750
minor faults 252.000 253.000
p99 operation ns 1883.000 1892.000
peak RSS MiB 74.707 74.699
RSS 0.5 s after drain MiB 50.289 52.293

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class-ephemeral/8 (chart build) 0% INCONCLUSIVE +0.6% [-0.4, +2.1] / 1.7% PASS -0.7% [-1.9, +0.6] / 2.0% PASS +0.0% [-0.4, +0.8] / 1.2% PASS +0.0% [-1.6, +1.1] / 1.1% INCONCLUSIVE -2.5% [-2.6, +0.0] / 2.6% PASS +3.8% [-3.2, +13.8] / 19.0% PASS

The overall verdict remains INCONCLUSIVE, because the p99 interval slightly exceeds its null-derived allowance. No memory credit is used for latency. The broader candidate/control run is still in flight.

Local-Gate: v1 tree=2a860088093b035658f80003fdf54ca50f204512 secs=42 lanes=lint:run
Ci-Attestation: {"at":1792529727,"gate":"python/all/lint","host":"linux-x86_64","key":"952590386b98d9a41d039814c586d336303ba383051ce1306abf12e022169e7b","lane":"lint","parents":["9b67e411fc8c8899e07e05d528a4c5b37ab20417"],"secs":42,"stamp":"f082fcdd98116d222fc4528c2dbef13c","tree":"2a860088093b035658f80003fdf54ca50f204512","v":1,"via":"run"}
Local-Gate: v1 tree=6f35648090e34aee7fc5f7eef5b3957514301069 secs=39 lanes=lint:run
Ci-Attestation: {"at":1792530060,"gate":"python/all/lint","host":"linux-x86_64","key":"1d9079fff997c8f9566db08126eb75d6b20caf8008a81e5afa7f5e46b921362c","lane":"lint","parents":["6310bebeaea2961170bf83175cef0065ca7d3ecd"],"secs":39,"stamp":"52b1d7992e5081160c35fd0d21029ab4","tree":"6f35648090e34aee7fc5f7eef5b3957514301069","v":1,"via":"run"}
@zackees

zackees commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

Exact-fit-only acceptance failure; compact-singleton experiment rejected

Exact-fit-only paired run: FAIL, despite workflow success

Run 39489674226 compared frozen baseline ee3623f6b7a485754a0a0aa3462d8cba43390e8b against a9d7d2e372dd321d55cec0cc24d32b7c59a0a566 (exact-fit search only; not the later page-map candidate). CPU: AMD EPYC 7763 64-Core Processor. Four logical CPUs: eight workers are 2:1 oversubscribed. The run requested 15 repetitions and extended individual inconclusive rows to 20 or 25. Actual counts are below; raw samples are in the run artifact.

The small control fails: CPU/op +20.3% [+16.7%, +26.9%] against a 4.8% null allowance; throughput −18.1% [−22.6%, −14.1%] against a 5.2% allowance. Fast-path byte identity does not override this measured regression. I have not established its cause, and the exact-fit-only revision is not accepted.

Large-class/8 does improve: CPU −1.4% [−1.8%, −0.9%] and throughput +2.5% [+1.2%, +3.5%], with that row’s full ledger PASS. Several ephemeral/chart/profiler latency or control intervals remain INCONCLUSIVE. Those results cannot be called a global pass.

workload actual paired reps process CPU ns/op baseline → head ops/s baseline → head peak RSS MiB baseline → head drain RSS MiB baseline → head sampled p99 ns baseline → head
large-class/8 20 480.92 → 474.22 8090441.00 → 8290443.35 75.33 → 74.31 51.52 → 50.43 1844.00 → 1828.50
large-class-ephemeral/8 25 481.12 → 474.09 8178149.10 → 8344729.70 74.54 → 74.54 8.52 → 8.52 1853.00 → 1863.00
small/8 (control) 15 40.11 → 48.48 95076318.60 → 79667471.60 16.36 → 16.36 16.36 → 16.36 111.00 → 111.00
larson/8 20 57.95 → 58.59 65775185.05 → 65125758.35 284.82 → 282.03 204.26 → 66.75 250.50 → 260.00
larson/8 (chart build) 25 62.24 → 62.33 61158969.80 → 60986521.50 264.16 → 277.62 45.97 → 46.41 270.00 → 261.00
large-class/8 (profiler on) 25 910.97 → 908.81 4306880.50 → 4316676.60 75.56 → 75.50 54.27 → 55.12 32462.00 → 32772.00
large-class-ephemeral/8 (chart build) 25 494.69 → 491.50 8005936.70 → 8077269.60 76.51 → 76.57 8.85 → 8.85 1883.00 → 1893.00
large-class-ephemeral/8 short generations (chart build) 25 543.25 → 535.25 7577494.70 → 7643054.50 74.60 → 72.61 74.60 → 72.61 1913.00 → 1894.00

CPU/op is the artifact’s existing process CPU measurement divided by fixed stream operations, including the existing short-drain CPU accounting. It is not active-worker-only CPU. The ledger below is computed from all recorded paired samples with the same-run null noise; latency earns no memory credit.

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 0% PASS +2.5% [+1.2, +3.5] / 1.2% PASS -1.4% [-1.8, -0.9] / 0.9% PASS +1.6% [-3.2, +47.2] / 48.8% PASS -0.6% [-3.3, +1.1] / 2.7% PASS -2.6% [-2.7, +0.0] / 5.2% PASS -1.8% [-2.7, +0.6] / 4.1% PASS
large-class-ephemeral/8 0% INCONCLUSIVE +1.9% [+1.2, +2.5] / 1.6% PASS -1.3% [-2.0, -0.8] / 0.7% PASS +0.0% [-1.2, +1.2] / 1.6% PASS +0.5% [-2.1, +3.3] / 1.6% INCONCLUSIVE +0.0% [-2.5, +0.1] / 2.7% PASS +0.0% [-11.0, +13.3] / 14.0% PASS
small/8 (control) 0% FAIL -18.1% [-22.6, -14.1] / 5.2% FAIL +20.3% [+16.7, +26.9] / 4.8% FAIL +0.6% [-0.6, +1.3] / 0.6% INCONCLUSIVE +0.0% [-7.5, +0.9] / 8.1% PASS +0.0% [-0.0, +0.0] / 0.0% PASS +0.0% [-0.0, +0.0] / 0.0% PASS
larson/8 0% PASS +0.4% [-3.0, +1.8] / 3.6% PASS +0.9% [-0.8, +2.6] / 3.0% PASS +10.0% [-29.4, +68.2] / 88.6% PASS +0.0% [-3.6, +4.0] / 7.5% PASS -0.5% [-3.4, +2.2] / 7.2% PASS -1.7% [-77.3, +1.4] / 77.4% PASS
larson/8 (chart build) 0% INCONCLUSIVE -0.5% [-1.7, +0.9] / 2.9% PASS +0.4% [-2.2, +2.9] / 2.2% INCONCLUSIVE -19.9% [-24.3, +2.3] / 28.7% PASS +0.0% [-3.5, +3.3] / 0.4% INCONCLUSIVE +2.4% [-1.4, +5.7] / 8.2% PASS +0.4% [-1.6, +3.1] / 3.0% INCONCLUSIVE
large-class/8 (profiler on) 0% INCONCLUSIVE +0.1% [-0.5, +0.6] / 0.4% INCONCLUSIVE -0.2% [-0.7, +0.3] / 0.3% INCONCLUSIVE -0.3% [-0.9, +0.3] / 0.9% PASS -2.8% [-6.1, +8.7] / 8.1% INCONCLUSIVE -0.2% [-2.6, +0.1] / 4.9% PASS +0.4% [-1.1, +2.0] / 1.9% INCONCLUSIVE
large-class-ephemeral/8 (chart build) 0% INCONCLUSIVE +1.2% [-0.1, +1.6] / 0.9% PASS -0.9% [-1.2, -0.4] / 0.6% PASS +0.4% [-0.8, +2.0] / 2.8% PASS +1.1% [-0.6, +2.2] / 1.6% INCONCLUSIVE +0.1% [-0.1, +2.7] / 2.7% PASS -11.3% [-25.8, +17.2] / 33.9% PASS
large-class-ephemeral/8 short generations (chart build) 0% INCONCLUSIVE +1.7% [-0.6, +2.2] / 1.4% PASS -1.2% [-1.8, -0.7] / 1.0% PASS -0.4% [-1.5, +2.1] / 2.0% INCONCLUSIVE -0.1% [-2.5, +1.6] / 2.7% PASS +0.0% [-2.7, +2.7] / 2.7% PASS -0.0% [-2.7, +2.7] / 2.7% PASS

Compact-singleton prototype: structural success, memory failure

The page layout currently reserves an inline-header slice for singleton allocations even when arena page metadata is separate. A prototype reduced only arena singleton reservations to their block extent, while keeping the original OS-fallback reservation and excluding large-alignment/secure cases. The structural probe failed on the original geometry (all five tested sizes had one extra slice) and passed with compaction, including last-byte lookup/write/free and an OS-fallback allocation.

However, five untimed byte-attribution repetitions per arm showed the wrong memory direction:

workload compact reservations median RSS MiB median queued resident MiB
sparse/8 0 177.07 95.06
sparse/8 1 188.82 108.43
persistent/8 0 66.70 13.25
persistent/8 1 66.70 14.37

Rejected: the sparse snapshot rose about 6.6%, from 177.07 to 188.82 MiB, despite a smaller reservation per singleton. The queue grew too. Tighter geometry is insufficient evidence of lower resident memory; placement/reuse/THP interactions need attribution before a redesigned policy. No defaults were changed, no discard policy was introduced, and the prototype has been restored out of the worktree. These local byte probes are not timing acceptance.

Rejected compact-singleton patch for reproduction
diff --git a/src/arena.c b/src/arena.c
index 57db5935..719f8985 100644
--- a/src/arena.c
+++ b/src/arena.c
@@ -941,16 +941,22 @@ static uint8_t* mi_arenas_page_alloc_fresh_area(mi_theap_t* theap, size_t slice_
   uint8_t* start = NULL;
   *memid = _mi_memid_none();
   const size_t alloc_size = mi_size_of_slices(slice_count);
+  size_t arena_slice_count = slice_count;
+  #if MI_PAGE_META_IS_SEPARATED && MI_SECURE < 2 && defined(MI_SINGLETON_COMPACT_ARENA) && MI_SINGLETON_COMPACT_ARENA
+  if (!os_align && block_size > MI_LARGE_MAX_OBJ_SIZE) {
+    arena_slice_count = mi_slice_count_of_size(block_size);
+  }
+  #endif
   if (!mi_option_is_enabled(mi_option_disallow_arena_alloc) &&       // allowed to allocate from arena's?
       !os_align &&                                                   // not large alignment
       slice_count <= mi_arena_max_object_size()/MI_ARENA_SLICE_SIZE) // and not too large
   {
-    start = (uint8_t*)mi_arenas_try_alloc(heap, slice_count, page_alignment, commit, allow_large, req_arena, tld->thread_seq, numa_node, memid);
+    start = (uint8_t*)mi_arenas_try_alloc(heap, arena_slice_count, page_alignment, commit, allow_large, req_arena, tld->thread_seq, numa_node, memid);
     if (start != NULL) {
       mi_arena_pages_t* const arena_pages = mi_heap_ensure_arena_pages(heap, memid->mem.arena.arena);
       *parena_pages = arena_pages;
       if (arena_pages==NULL) {
-        _mi_arenas_free(heap->subproc, start, mi_size_of_slices(slice_count), *memid); // roll back
+        _mi_arenas_free(heap->subproc, start, mi_size_of_slices(arena_slice_count), *memid); // roll back
         start = NULL;
       }
       else {
@@ -1025,11 +1031,14 @@ static void mi_arenas_free_ex(mi_subproc_t* subproc, void* p, size_t size, mi_me
 static mi_page_t* mi_arenas_page_alloc_fresh(mi_theap_t* theap, size_t slice_count, size_t block_size, size_t block_alignment, bool commit)
 {
   const bool os_align           = (block_alignment > MI_PAGE_MAX_OVERALLOC_ALIGN);
-  const size_t alloc_size       = mi_size_of_slices(slice_count);
   mi_memid_t memid              = _mi_memid_none();
   mi_arena_pages_t* arena_pages = NULL;
   uint8_t* const slice_start    = mi_arenas_page_alloc_fresh_area(theap,slice_count,block_size,block_alignment,os_align,commit,&memid,&arena_pages);
   if (!slice_start) return NULL;
+  #if MI_PAGE_META_IS_SEPARATED && MI_SECURE < 2 && defined(MI_SINGLETON_COMPACT_ARENA) && MI_SINGLETON_COMPACT_ARENA
+  if (memid.memkind == MI_MEM_ARENA) { slice_count = memid.mem.arena.slice_count; }
+  #endif
+  const size_t alloc_size = mi_size_of_slices(slice_count);
 
   // guard page at the end of mimalloc page?
   #if MI_SECURE>=5

Cross gate and ongoing evidence

Cross run 39496948660 passed, including the Windows ARM64 build, after disabling only its solo toolchain cache. All toolchain/Cargo operations remain through soldr. This validates the documented fresh-toolchain mitigation for this run; the upstream cache defect remains tracked separately.

Combined-candidate target/control/random-sparse runs 39495904856 / 39495909935 / 39495916158 and same-run competitor scaling 39494686018 remain active. They measure different candidate C code from this exact-fit-only run. Their results are still required; none of the failed or inconclusive evidence above is erased. The original combined-candidate drain-RSS sanity failure also remains unresolved.

Local-Gate: v1 tree=d5da40744be8c8b1cadd5f4816e4b21a4e882420 secs=40 lanes=lint:run
Ci-Attestation: {"at":1792531183,"gate":"python/all/lint","host":"linux-x86_64","key":"a85874d7c9d12ba8608efddba2a3d24899f43c6fc895c4f8478d67ac1477fff4","lane":"lint","parents":["2adb4c009882f752f5f6146f871f85133c05e61a"],"secs":40,"stamp":"c0bdfb136b38cbd62216b72d1d29a49f","tree":"d5da40744be8c8b1cadd5f4816e4b21a4e882420","v":1,"via":"run"}
@zackees

zackees commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

#585 combined-candidate controls and random/sparse results

Frozen baseline ee3623f6b7a485754a0a0aa3462d8cba43390e8b, candidate 9b67e411fc8c8899e07e05d528a4c5b37ab20417 (C source identical to the previously reported 8b412211). The candidate remains unaccepted. Every run uses the unchanged sanity screen and a base-vs-base null arm. Eight-worker cells are 2:1 oversubscribed on four logical CPUs. Different CPU models are reported separately.

Run 39495909935

Run39495909935: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz, 4 logical CPUs; base=ee3623f6b7a485754a0a0aa3462d8cba43390e8b; head=9b67e411fc8c8899e07e05d528a4c5b37ab20417
small/8 (control): n=25; medians {'ops/s': (89414452.9, 92152704.4), 'cpu s': (1.6762, 1.6308), 'minor faults': (160.0, 160.0), 'p99 operation ns': (208.0, 203.0), 'peak RSS MiB': (16.421875, 16.421875), 'RSS 0.5 s after drain MiB': (16.421875, 16.421875)}
larson/1: n=25; medians {'ops/s': (36121809.6, 36054628.5), 'cpu s': (0.2939, 0.2939), 'minor faults': (202.0, 202.0), 'p99 operation ns': (266696.0, 268904.0), 'peak RSS MiB': (146.28125, 146.28125), 'RSS 0.5 s after drain MiB': (85.359375, 85.35546875)}
larson/8: n=20; medians {'ops/s': (48188078.25, 48218216.65), 'cpu s': (1.58035, 1.59015), 'minor faults': (990.5, 795.0), 'p99 operation ns': (479.5, 480.5), 'peak RSS MiB': (286.943359375, 283.037109375), 'RSS 0.5 s after drain MiB': (204.505859375, 204.728515625)}
larson/8 (chart build): n=25; medians {'ops/s': (43700473.2, 44906597.9), 'cpu s': (1.7443, 1.7003), 'minor faults': (1025.0, 1137.0), 'p99 operation ns': (460.0, 464.0), 'peak RSS MiB': (272.09765625, 269.18359375), 'RSS 0.5 s after drain MiB': (205.41015625, 205.64453125)}

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
small/8 (control) 0% INCONCLUSIVE +1.7% [-4.3, +7.2] / 4.2% INCONCLUSIVE -0.1% [-5.4, +2.5] / 3.2% PASS +0.0% [-0.6, +0.6] / 0.6% PASS +0.5% [-11.2, +8.8] / 6.1% INCONCLUSIVE +0.0% [-0.0, +0.0] / 0.0% PASS +0.0% [-0.0, +0.0] / 0.0% PASS
larson/1 0% INCONCLUSIVE -1.3% [-3.2, +2.8] / 2.8% INCONCLUSIVE +1.6% [-3.2, +2.5] / 3.4% PASS +0.0% [+0.0, +0.0] / 1.5% PASS +0.5% [-0.7, +1.7] / 2.6% PASS +0.0% [-1.3, +0.0] / 1.3% PASS +0.0% [-0.0, +0.0] / 0.0% PASS
larson/8 0% PASS +0.3% [-2.9, +3.3] / 3.7% PASS -0.3% [-2.6, +3.3] / 3.8% PASS -12.2% [-29.6, -0.2] / 51.8% PASS +2.0% [-0.4, +3.9] / 4.8% PASS -1.7% [-3.7, +3.0] / 4.1% PASS +0.6% [-1.8, +3.1] / 354.1% PASS
larson/8 (chart build) 0% INCONCLUSIVE +1.8% [-0.4, +4.1] / 4.1% PASS -2.0% [-3.9, -0.5] / 3.0% PASS +20.2% [-21.1, +47.8] / 35.6% INCONCLUSIVE +1.7% [-2.4, +3.4] / 4.2% PASS -1.2% [-3.8, +2.3] / 4.2% PASS -0.2% [-1.1, +0.4] / 0.8% PASS

Run 39495916158

Run 39495916158: AMD EPYC 9V45 96-Core Processor, 4 logical CPUs (8 workers oversubscribed).
random-large/8: n=25; medians {'ops/s': (978881.9, 965776.7), 'cpu s': (1.2922, 1.2983), 'minor faults': (261.0, 261.0), 'p99 operation ns': (18118.0, 17076.0), 'peak RSS MiB': (178.46875, 180.4140625), 'RSS 0.5 s after drain MiB': (20.578125, 20.578125)}
random-large-bursty/8: n=15; medians {'ops/s': (140025.4, 139826.6), 'cpu s': (1.1727, 1.1739), 'minor faults': (716.0, 1243.0), 'p99 operation ns': (13911.0, 13991.0), 'peak RSS MiB': (168.359375, 165.90625), 'RSS 0.5 s after drain MiB': (105.88671875, 109.32421875)}
random-large/1: n=15; medians {'ops/s': (393488.9, 398200.9), 'cpu s': (0.5346, 0.5272), 'minor faults': (131.0, 131.0), 'p99 operation ns': (9775.0, 9634.0), 'peak RSS MiB': (48.19921875, 48.19921875), 'RSS 0.5 s after drain MiB': (43.01171875, 43.01171875)}
sparse-large-buffers 64-slot budget/8 (#422): n=25; medians {'ops/s': (1394700.0, 1382132.8), 'cpu s': (0.5748, 0.5734), 'minor faults': (252.0, 252.0), 'p99 operation ns': (15123.0, 15113.0), 'peak RSS MiB': (168.4140625, 172.3828125), 'RSS 0.5 s after drain MiB': (168.01171875, 171.01171875)}

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
random-large/8 0% INCONCLUSIVE -0.5% [-2.4, +1.1] / 2.5% PASS +0.8% [-0.4, +2.6] / 2.5% INCONCLUSIVE +0.4% [-0.8, +2.0] / 2.7% PASS -0.8% [-13.0, +4.1] / 12.1% PASS +1.1% [-1.1, +2.3] / 3.5% PASS +0.0% [-0.0, +0.0] / 0.0% PASS
random-large-bursty/8 0% PASS -0.1% [-0.5, +0.4] / 0.5% PASS +0.9% [-1.5, +3.2] / 4.5% PASS +24.0% [-23.9, +76.6] / 99.9% PASS +0.7% [-2.4, +2.4] / 5.3% PASS -2.1% [-5.9, +7.7] / 7.8% PASS +1.2% [-14.2, +10.2] / 13.5% PASS
random-large/1 0% PASS +1.4% [-1.6, +7.7] / 7.6% PASS -1.8% [-5.7, +2.1] / 7.5% PASS +0.0% [+0.0, +0.8] / 0.8% PASS -2.3% [-3.5, +1.1] / 2.4% PASS +0.0% [-0.1, +0.0] / 0.0% PASS +0.0% [-0.6, +0.0] / 0.6% PASS
sparse-large-buffers 64-slot budget/8 (#422) 0% INCONCLUSIVE +0.6% [-2.7, +1.5] / 2.3% INCONCLUSIVE -0.5% [-2.6, +1.5] / 2.6% PASS +0.4% [-1.6, +1.6] / 1.6% PASS -1.1% [-3.9, +0.9] / 1.5% PASS +1.1% [-2.3, +2.4] / 3.6% PASS +0.5% [-2.0, +2.7] / 3.7% PASS

Interpretation and attribution

  • No memory reduction is established in these rows. random-large-bursty/8, random-large/1, and plain larson/8 pass their same-run ledgers. Remaining rows are INCONCLUSIVE, not accepted. Bursty fault noise is very large; the PASS does not establish a fault improvement.
  • The combined candidate's small control ran on Xeon 8370C. It cannot clear the earlier exact-fit-only EPYC 7763 FAIL (+20.3% CPU). No old result is removed or pooled across CPU models.
  • Focused follow-ups, with 25 initial and up to 45 actual paired samples, retain the same thresholds: controls 39500299084, random/sparse 39500303123.
  • Fresh unchanged-flag build-only reports preserve linked executables, hashes, symbols and disassembly: exact-fit 39499808868, combined 39499813370. Both use GCC 13.3.0, binutils 2.42, Ubuntu image 20261012.334.1; their baseline timing binary SHA256 is identical (7f0c70ce2450dfa09126c1674bddab53187614f12d5bd648b4fda33ccbd3c712). These are fresh builds, not the discarded original failed-run binaries, and have no performance verdict.
  • Fresh exact-fit disassembly has identical run_ops, mi_malloc, mi_free addresses and instructions apart from resolved call displacements; only mi_page_fresh_alloc changes normalized instructions. The small-size branch inside it retains the same address/instructions and bypasses the changed search. Shared fallback functions move 16 bytes. All non-text symbol addresses are unchanged. This narrows attribution but does not establish the cause of the old FAIL.
  • Local CPU-clock profiles of these fresh binaries (20M operations per worker; diagnostic only, different host) put ~73% in run_ops, ~10-12% each in malloc/free, and ~1.2-1.5% in _mi_malloc_generic; they do not reproduce a 20% CPU increase and cannot clear the CI failure.
  • Pending original target run 39495904856 and scaling 39494686018 remain live; neither has been restarted or cancelled.

@zackees

zackees commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

#585 combined-candidate large-class results

Run 39495904856 completed against the frozen baseline with the original sanity gate and same-run null. Four of five rows pass. Main large-class/8: paired CPU/op -2.0% [-4.2%, -0.1%], throughput +3.3% [+0.3%, +4.3%], p99 -1.1% [-1.7%, +0.0%]; no established peak-memory reduction. The chart ephemeral row remains INCONCLUSIVE on minor faults, p99 and peak RSS. It is being narrowed to one row in 39500745637, 25 initial/up to 45 paired samples, unchanged thresholds.

Candidate is not accepted yet: control/random/sparse follow-ups, competitor scaling and the earlier failed-run attribution remain outstanding. The older exact-fit-only small-control failure is retained.

Run39495904856: AMD EPYC 9V45 96-Core Processor, 4 logical CPUs; base=ee3623f6b7a485754a0a0aa3462d8cba43390e8b; head=9b67e411fc8c8899e07e05d528a4c5b37ab20417
large-class/8: n=20; medians {'ops/s': (7368753.300000001, 7571871.55), 'cpu s': (1.7038, 1.6729500000000002), 'minor faults': (219.0, 224.0), 'p99 operation ns': (1828.0, 1813.0), 'peak RSS MiB': (72.4453125, 73.46484375), 'RSS 0.5 s after drain MiB': (57.048828125, 57.3984375)}
large-class-ephemeral/8: n=25; medians {'ops/s': (7412191.5, 7485879.8), 'cpu s': (1.7026, 1.6908), 'minor faults': (250.0, 250.0), 'p99 operation ns': (1833.0, 1842.0), 'peak RSS MiB': (74.6796875, 74.671875), 'RSS 0.5 s after drain MiB': (42.765625, 41.94921875)}
large-class/8 (profiler on): n=25; medians {'ops/s': (5002361.6, 5031713.1), 'cpu s': (2.4946, 2.4755), 'minor faults': (1632.0, 1632.0), 'p99 operation ns': (26721.0, 27622.0), 'peak RSS MiB': (74.01171875, 73.76171875), 'RSS 0.5 s after drain MiB': (72.05078125, 71.66015625)}
large-class-ephemeral/8 (chart build): n=25; medians {'ops/s': (7321626.6, 7397945.4), 'cpu s': (1.7277, 1.7068), 'minor faults': (251.0, 253.0), 'p99 operation ns': (1833.0, 1843.0), 'peak RSS MiB': (74.69921875, 74.6953125), 'RSS 0.5 s after drain MiB': (46.84765625, 40.79296875)}
large-class-ephemeral/8 short generations (chart build): n=15; medians {'ops/s': (7165887.7, 7300413.0), 'cpu s': (0.4503, 0.4415), 'minor faults': (253.0, 249.0), 'p99 operation ns': (1833.0, 1823.0), 'peak RSS MiB': (74.73046875, 72.73046875), 'RSS 0.5 s after drain MiB': (74.7265625, 72.73046875)}

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 0% PASS +3.3% [+0.3, +4.3] / 3.8% PASS -2.0% [-4.2, -0.1] / 1.5% PASS +2.3% [-1.5, +5.6] / 53.3% PASS -1.1% [-1.7, +0.0] / 1.7% PASS +0.1% [-2.7, +2.8] / 2.8% PASS -0.4% [-4.4, +3.1] / 3.7% PASS
large-class-ephemeral/8 0% PASS +1.2% [+0.3, +2.7] / 1.8% PASS -0.5% [-1.5, +0.7] / 1.6% PASS +0.0% [-1.2, +0.8] / 1.2% PASS -0.1% [-1.5, +1.1] / 1.6% PASS +0.0% [-2.6, +2.6] / 2.7% PASS +3.1% [-22.7, +15.2] / 24.0% PASS
large-class/8 (profiler on) 0% PASS +0.6% [-0.1, +1.3] / 1.6% PASS -0.8% [-1.6, +0.2] / 1.5% PASS -0.3% [-0.6, +0.6] / 0.9% PASS +2.5% [-5.4, +10.4] / 13.1% PASS -0.0% [-2.9, +2.4] / 2.8% PASS -0.6% [-2.7, +0.7] / 1.8% PASS
large-class-ephemeral/8 (chart build) 0% INCONCLUSIVE +0.8% [-0.5, +2.3] / 1.4% PASS -0.8% [-1.8, -0.2] / 1.6% PASS +1.2% [+0.0, +2.0] / 1.6% INCONCLUSIVE +0.5% [-1.6, +1.7] / 1.1% INCONCLUSIVE +0.0% [-2.6, +2.8] / 2.7% INCONCLUSIVE -6.9% [-30.7, +5.9] / 12.7% PASS
large-class-ephemeral/8 short generations (chart build) 0% PASS +1.0% [-0.3, +7.6] / 6.4% PASS -1.9% [-7.9, -0.4] / 5.2% PASS -1.2% [-2.4, -0.4] / 2.0% PASS -1.7% [-3.8, +0.5] / 4.3% PASS +0.0% [-5.4, +2.8] / 5.4% PASS +0.0% [-5.4, +2.8] / 5.4% PASS

Local-Gate: v1 tree=b29bf3ca7c1551169059eb68f52a73ee7b1c1a44 secs=35 lanes=lint:run
Ci-Attestation: {"at":1792532275,"gate":"python/all/lint","host":"linux-x86_64","key":"cf99b163b801d3af55d58955f96afcc2dd5f608104268a1718524862df58dead","lane":"lint","parents":["3bbe4e727b6b2364b3c8c4b3e4cf219666718014"],"secs":35,"stamp":"0b49181e657c4af4500410ae3dad0379","tree":"b29bf3ca7c1551169059eb68f52a73ee7b1c1a44","v":1,"via":"run"}
Local-Gate: v1 tree=9c3ad8c5176f7fc64846dfaad6ef2ff9725c34de secs=43 lanes=lint:run
Ci-Attestation: {"at":1792532501,"gate":"python/all/lint","host":"linux-x86_64","key":"0da51912007b2d69f8bed1d4a1df35ac25a25869688d5232c8edf4247570d8e7","lane":"lint","parents":["acd8300e5e8c7bdbc2293456c13741502ecb60f3"],"secs":43,"stamp":"8cad2435a94ce537259768a4e5b1dfd1","tree":"9c3ad8c5176f7fc64846dfaad6ef2ff9725c34de","v":1,"via":"run"}
@zackees

zackees commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

#585 measurement-order correction and exact executable preservation

A concrete harness defect was found while investigating the failed exact-fit-only small control: run_rep alternated [base, head, null] and [null, head, base], leaving head in the middle on every repetition. Treatment was confounded with execution position. This does not establish the cause of the earlier +20.3% CPU FAIL, which remains visible and unaccepted. All earlier perf-ab ledgers used this older order; their numbers are retained as diagnostic evidence rather than silently overwritten.

The regression test ran the actual measurement loop: RED head-position counts [0,6,0]; GREEN [2,2,2]. The corrected schedule rotates every arm through every execution position and reverses alternate blocks. Three arms cover all six permutations. Extra variants retain position balance, without a claim of all carryover pairs. Artifacts/header record arm_order_protocol=rotating-reversed-blocks-v1; pooling rejects missing or different order protocols.

CI now also preserves the exact timing and latency executables (per arm/build kind) and SHA256 digests in perf-ab-builds, copied after measurements, including a failed sanity screen. Raw JSON is saved before copying so an artifact-copy error cannot erase measurements. Existing build-only reports remain fresh-build diagnostics, not substitutes for actual timed binaries.

Review: clean from the same single primary reviewer. Local gate on new head 43f0b97d1821: 45/45 passed. Previous head 2adb4c00: 104 successful platform checks, five intentional skips, none pending (native MSVC, win-gnu, Rust, cross Windows ARM64 via soldr, Linux and native Mac included). New-head platform checks are running.

Corrected-order measurements retain baseline ee3623f6b7a485754a0a0aa3462d8cba43390e8b, source candidate 9b67e411fc8c8899e07e05d528a4c5b37ab20417, null arm and original sanity threshold, 30 initial/up to 60 actual samples:

Older live follow-ups 39500299084, 39500303123 and 39500745637 remain intact and will be reported under the legacy protocol. No previous sample/run was cancelled to replace an unfavorable result. Acceptance remains pending.

@zackees

zackees commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner Author

#585 same-host competitor scaling and memory priorities

Run 39494686018 completed successfully: 15 paired blocks, four patterns, worker counts 1/2/3/4/6/8, five allocators, 1,800 samples. Status is diagnostic/not publishable; nothing was published to benchmark-stats. Host: AMD EPYC 9V74, two physical/four logical cores; eight workers are 2:1 oversubscribed on logical CPUs.

The temporary harness merge 67f55aa343ee21c46381085019639b756c357745 carries the tested #608 TCMalloc builder dependency. git diff 8b412211... 67f55aa... -- src include rust/mimalloc-pprof/vendor is empty. The comparison was rechecked against the actual vendor directory, which exists at the temporary merge, and is empty for C, headers and vendor content. Thus this is the same allocator C source as the old combined candidate. The builder merge is not an allocator-policy change.

References are pinned: TCMalloc c316de3ee8ffb5aff6547aa10151bc8dda3a2942, jemalloc 81034ce1f1373e37dc865038e1bc8eeecf559ce8, upstream mimalloc 6def7be9458fb8a97b8323af3fb0b0ae04387065, Bun mimalloc b20b60d959093b1bc0e24306ec72ccacb3e46fb9. The fork uses the existing chart/default benchmark recipe with profiler and memory-events compiled in, both runtime-disabled; this is separate from perf-ab's plain minimal build. Build suite commands use cd rust; soldr cargo ....

Audit: zero operation-count, operations-per-worker, checksum, worker-seed or size-histogram mismatches across all 360 paired blocks. All 120 allocator/cell schedules place each allocator in each of five execution positions exactly three times; this scaling scheduler does not have perf-ab's legacy fixed-middle defect.

Findings

  • Sparse/8 does not reproduce the historical ~1.40x TCMalloc peak gap on this host/protocol/reference set: paired peak ratio 0.988 [0.969,1.041], fork median 154.7 MiB vs TCMalloc 156.2. This is not proof that the candidate saved that memory: no fixed-baseline before/after comparison is present in this scaling run, and its C changes are intended to preserve memory policy.
  • Persistent/8 remains a concrete target: peak ratio 1.224 [1.169,1.275], fork 67.5 MiB vs TCMalloc 56.1 and jemalloc 59.7. Fork/upstream throughput remains 0.874 [0.860,0.905] in this benchmark recipe. The older throughput debt has not been erased by the small re-carve optimization.
  • Power-of-two-large/8 is another memory target: peak ratio 1.206 [1.176,1.333], fork 127.5 MiB vs TCMalloc 104.0. Random-large/8 is lower than TCMalloc here: paired ratio 0.930 [0.884,0.974].
  • The earlier attribution/reuse results still reject a blind queued-body discard: completed reused slices are predominantly sub-millisecond, and tighter geometry / compact singleton prototypes failed the stated structural or memory checks. No new discard policy or memory tuning was retained.
  • Next memory investigation: correlate live/formed/unformed occupancy with the peak-time extent for persistent and power-of-two rows, and distinguish it from the end-of-work queued-free snapshot. Measure this before selecting another geometry or packing policy. Do not assume the old sparse gap is a universal constant.

This run measures memory and throughput; CPU/op, faults and p99: not measured by scaling. It is not a rule-12 acceptance ledger. Corrected-order perf-ab runs supply those separate measurements and remain pending (order correction: #585 (comment)).

Per-cell paired ratios

Medians and 95% sign-test intervals on paired ratios; ratio-of-medians need not equal the median paired ratio. Raw samples and telemetry replays are in the linked run artifacts.

Trace parity audit: 0 mismatches across 360 paired blocks.

pattern workers fork peak MiB TCMalloc peak MiB fork/TC peak [95% CI] jemalloc peak MiB fork/upstream ops ratio [95% CI]
large-class-persistent 1 17.4 24.3 0.714 [0.709, 0.738] 12.2 0.811 [0.791, 0.835]
large-class-persistent 2 25.4 30.5 0.827 [0.815, 0.842] 19.0 0.828 [0.793, 0.847]
large-class-persistent 3 31.4 34.5 0.904 [0.879, 0.916] 26.3 0.872 [0.854, 0.879]
large-class-persistent 4 39.4 40.8 0.948 [0.918, 0.974] 33.7 0.870 [0.847, 0.885]
large-class-persistent 6 53.4 47.4 1.125 [1.082, 1.158] 46.1 0.892 [0.793, 0.918]
large-class-persistent 8 67.5 56.1 1.224 [1.169, 1.275] 59.7 0.874 [0.860, 0.905]
sparse-large-buffers 1 51.1 49.2 1.029 [0.919, 1.069] 47.0 0.932 [0.919, 0.954]
sparse-large-buffers 2 71.3 69.5 1.010 [0.997, 1.124] 85.0 0.940 [0.921, 0.947]
sparse-large-buffers 3 93.3 85.0 1.106 [1.001, 1.153] 126.5 0.942 [0.931, 0.957]
sparse-large-buffers 4 99.0 100.6 0.980 [0.958, 1.025] 159.1 0.958 [0.951, 0.963]
sparse-large-buffers 6 130.7 127.3 1.016 [0.992, 1.066] 230.0 0.992 [0.970, 1.007]
sparse-large-buffers 8 154.7 156.2 0.988 [0.969, 1.041] 300.3 0.953 [0.947, 0.968]
random-large 1 43.4 48.4 0.914 [0.840, 0.960] 45.2 0.960 [0.943, 0.985]
random-large 2 73.4 70.8 1.058 [0.980, 1.161] 81.9 0.972 [0.958, 0.980]
random-large 3 81.4 87.1 0.966 [0.912, 1.133] 115.3 0.999 [0.973, 1.034]
random-large 4 97.3 105.0 0.958 [0.870, 0.993] 157.5 1.001 [0.973, 1.010]
random-large 6 121.4 135.2 0.904 [0.879, 0.944] 228.0 1.012 [0.963, 1.042]
random-large 8 147.5 156.5 0.930 [0.884, 0.974] 282.5 0.994 [0.977, 1.052]
power-of-two-large 1 35.2 38.1 0.895 [0.878, 0.936] 39.1 0.938 [0.931, 0.955]
power-of-two-large 2 57.4 53.4 1.078 [1.014, 1.366] 73.4 0.951 [0.937, 0.961]
power-of-two-large 3 69.5 62.5 1.093 [1.051, 1.169] 106.7 0.947 [0.933, 0.959]
power-of-two-large 4 85.4 72.6 1.178 [1.129, 1.201] 142.0 0.953 [0.945, 0.973]
power-of-two-large 6 103.4 88.1 1.177 [1.131, 1.219] 200.9 0.908 [0.890, 0.955]
power-of-two-large 8 127.5 104.0 1.206 [1.176, 1.333] 268.0 0.948 [0.927, 0.960]

@zackees

zackees commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner Author

#585 corrected exact-fit control and baseline-only drain-RSS investigation

Corrected-order run 39502542281 collected 60 actual paired samples per arm on EPYC 7763 / four logical CPUs. This is the same CPU model as the earlier +20.3% exact-fit-only CPU failure, not the same physical VM. Head remains a9d7d2e..., baseline ee3623f6....

CPU/op passes its same-run allowance at -0.3% [-2.2%, +1.0%]; the 20% CPU increase was not reproduced. Overall the row remains INCONCLUSIVE on throughput and p99. It is not standalone exact-fit acceptance. Old and corrected protocols are kept separate and cannot be pooled.

base ee3623f6b7a4 head a9d7d2e372dd on AMD EPYC 7763 64-Core Processor, 30 reps (up to 60); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
small/8 (control) 0% INCONCLUSIVE -0.5% [-2.6, +1.7] / 2.4% INCONCLUSIVE -0.3% [-2.2, +1.0] / 3.0% PASS +0.0% [-0.6, +0.6] / 0.6% PASS +0.0% [+0.0, +0.9] / 0.9% INCONCLUSIVE +0.0% [+0.0, +0.0] / 0.0% PASS +0.0% [+0.0, +0.0] / 0.0% PASS

The actual timed executables in perf-ab-builds match the fresh GCC-13 build-only report byte for byte: base timing SHA256 7f0c70ce2450dfa09126c1674bddab53187614f12d5bd648b4fda33ccbd3c712, head 6b35cf7b5a2d9cac2fb13f9fb16694c7c26fc12a4e2bdcd418a5e67d8323fc8d; both latency digests match as well. These hashes now identify actual tested binaries. The original discarded failed-run binaries remain unavailable, so this does not establish that failure's cause.

Baseline-only reproduction of drain variation

A local memory diagnostic used one unchanged frozen-baseline chart binary (pprof and memory-events compiled ON, runtime OFF), identical arguments, and identical two-CPU affinity for six consecutive fresh processes. The binary SHA256 is ffcf36c7d1fb53b430b75d456775da02b1eb5d989b4eede77e00bd5e611374ee. This is a different host/compiler from CI; it reproduces large drain-RSS variation in the unchanged baseline. The scavenger's delayed release is a mechanism consistent with that variation; these runs do not establish its cause. No CPU/throughput/p99 acceptance claim is made from these local runs.

repetition peak RSS MiB RSS 0.5 s after drain MiB RSS at 1300-ms bound MiB release ms
0 76.152 6.871 5.871 480
1 71.109 8.934 5.871 1060
2 71.574 68.867 5.867 1050
3 72.996 46.555 5.867 1050
4 75.836 53.746 5.871 1080
5 68.797 59.797 5.871 1040

Consecutive repetitions 1 and 2 changed drain RSS 8.934 → 68.867 MiB, while bound RSS stayed 5.871 → 5.867 MiB. Applying the existing 1.5 × base + 8 MiB screen to those baseline-only observations would flag a drain regression. The earlier mixed-affinity exploratory run also observed 6.871–64.188 MiB at the same two-CPU setting; all nine exploratory samples are retained below. This is a post-hoc diagnostic, not an estimate of the false-positive rate.

Code context: _mi_pages_release_retired stamps a retired page when the scavenger first sees it, then waits purge_delay * MI_RETIRED_RELEASE_MULT (default multiplier 10). Release timing can therefore put the fixed 0.5-second snapshot on either side of the release. This is a supported mechanism consistent with the observations; the original failed-run timing was not preserved. The candidate and the original sanity threshold remain unchanged.

Reproduction: build frozen baseline with the current ci/perf_ab.py chart recipe; run the resulting same executable six times as:

taskset -c 0,1 ./perf_ab 8 8 98304 524288 400000 0 0 uniform 8 1300

Choose two allowed CPUs on the host, keep MIMALLOC_PROF=0 and MIMALLOC_MEMORY_EVENTS=0, and use the identical binary/arguments each time. Results vary with scheduling. The script uses fresh processes and reads the normal child's peak/drain/bound/release fields; no allocator option was tuned.

All nine exploratory observations (same frozen baseline, varied affinity)
[
  {
    "rep": 0,
    "cpus": [
      0
    ],
    "peak_MiB": 58.83984375,
    "drain_MiB": 2.94140625,
    "bound_MiB": 2.94140625,
    "release_ms": 450.0
  },
  {
    "rep": 0,
    "cpus": [
      0,
      1,
      2,
      3
    ],
    "peak_MiB": 69.69140625,
    "drain_MiB": 69.69140625,
    "bound_MiB": 4.75390625,
    "release_ms": 1030.0
  },
  {
    "rep": 0,
    "cpus": [
      0,
      1
    ],
    "peak_MiB": 67.640625,
    "drain_MiB": 64.1875,
    "bound_MiB": 4.75,
    "release_ms": 710.0
  },
  {
    "rep": 1,
    "cpus": [
      0,
      1
    ],
    "peak_MiB": 72.76953125,
    "drain_MiB": 63.44140625,
    "bound_MiB": 4.75390625,
    "release_ms": 1090.0
  },
  {
    "rep": 1,
    "cpus": [
      0,
      1,
      2,
      3
    ],
    "peak_MiB": 71.49609375,
    "drain_MiB": 70.90234375,
    "bound_MiB": 4.75,
    "release_ms": 1060.0
  },
  {
    "rep": 1,
    "cpus": [
      0
    ],
    "peak_MiB": 60.1953125,
    "drain_MiB": 56.30859375,
    "bound_MiB": 4.74609375,
    "release_ms": 1090.0
  },
  {
    "rep": 2,
    "cpus": [
      0
    ],
    "peak_MiB": 62.23828125,
    "drain_MiB": 59.1875,
    "bound_MiB": 4.75,
    "release_ms": 720.0
  },
  {
    "rep": 2,
    "cpus": [
      0,
      1,
      2,
      3
    ],
    "peak_MiB": 71.515625,
    "drain_MiB": 52.4375,
    "bound_MiB": 4.75,
    "release_ms": 1020.0
  },
  {
    "rep": 2,
    "cpus": [
      0,
      1
    ],
    "peak_MiB": 72.6171875,
    "drain_MiB": 6.87109375,
    "bound_MiB": 5.87109375,
    "release_ms": 450.0
  }
]

Independent truncated-prefix attribution

Table values are medians across three independent untimed runs per prefix. Current C source unchanged from 8b412211, MI_DIAGNOSTICS=ON; MI_STAT=1, using ci/attribution_probe.py parsing/buckets. Each prefix is a separate process with its operation count shortened; these are not time samples from one continuous execution. This screen locates early resident unformed capacity and is not a candidate acceptance ledger.

row operations per worker snapshot RSS MiB peak RSS MiB live requested MiB unformed partial MiB unformed empty MiB queued resident MiB
persistent/8 500 54.73 54.77 7.77 11.94 18.88 1.37
persistent/8 2000 60.61 60.79 9.06 10.56 16.38 6.43
persistent/8 10000 60.82 60.86 9.87 10.94 16.09 8.56
persistent/8 50000 66.59 66.86 9.32 8.53 16.56 14.56
persistent/8 100000 64.76 64.87 9.51 12.50 16.81 10.25
persistent/8 200000 64.68 64.87 10.93 13.97 8.56 16.50
persistent/8 400000 64.61 64.86 10.49 10.03 17.56 10.50
sparse/8 500 156.53 156.43 35.14 7.45 14.05 85.68
sparse/8 2000 160.64 160.42 21.40 10.02 14.73 101.18
sparse/8 5000 170.78 170.43 44.28 6.03 18.70 87.37
sparse/8 10000 168.75 168.80 34.21 9.28 13.59 97.43
sparse/8 25000 172.68 172.51 45.79 5.14 17.25 89.56

Persistent unformed resident capacity is already ~27 MiB after 2,000 operations per worker. A policy targeting it still has to preserve cross-bin reuse and pass the full per-cell ledger; the earlier smaller-span and compact-singleton prototypes did not meet those requirements. No such policy was retained.

Corrected candidate runs and corrected historical bisections remain live. All numerical results, including failed/inconclusive ones, remain in the investigation.

@zackees

zackees commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

#585 corrected historical #572 bisection: latency debt remains

Run 39504129574 completed on EPYC 9V74 / four logical CPUs, comparing exact pre-merge ad957541ccebe36aa4396cdae26f0fe665f55c4d with merge 668b42e021ee7a18a6ba206ef7b7067fddaaa3a8. It uses the corrected rotating-reversed-blocks-v1 arm schedule and a same-run baseline-null arm. All three arms have 30 actual samples per row. Eight workers are 2:1 oversubscribed.

The plain large-class/8 row has a p99 regression of +5.2% [4.4%, 5.2%], beyond its 0.7% same-run noise allowance. Throughput, CPU/op, peak RSS, faults and drain RSS pass that historical cell's allowances. The profiler-on row passes all six metrics. This identifies historical latency debt at the #572 merge; it does not isolate a particular function and is not a current-candidate-versus-frozen-baseline failure. Current candidate acceptance remains pending in the three separately listed corrected runs. Earlier fixed-middle-order bisections remain diagnostic and are not pooled with this result.

base ad957541cceb head 668b42e021ee on AMD EPYC 9V74 80-Core Processor, 30 reps (up to 60); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 13% FAIL -1.3% [-2.7, -0.2] / 4.4% PASS +0.9% [+0.1, +1.5] / 4.4% PASS -74.9% [-78.1, -71.8] / 33.5% PASS +5.2% [+4.4, +5.2] / 0.7% FAIL -14.1% [-14.9, -12.4] / 1.9% PASS -20.7% [-21.9, -19.4] / 1.0% PASS
large-class/8 (profiler on) 14% PASS +3.9% [+3.6, +5.0] / 4.6% PASS -3.9% [-4.6, -3.4] / 4.6% PASS -78.1% [-81.8, -71.8] / 17.3% PASS -0.4% [-3.5, +2.6] / 3.8% PASS -13.4% [-14.3, -12.4] / 2.6% PASS +0.5% [-0.9, +1.3] / 2.0% PASS

Actual sample and latency medians:

large-class/8: base n=30, p99 median=1352 ns, head n=30, p99 median=1412 ns, null n=30, p99 median=1352 ns
large-class/8 (profiler on): base n=30, p99 median=53000 ns, head n=30, p99 median=52499.5 ns, null n=30, p99 median=52899.5 ns

Both perf-ab-samples and perf-ab-builds artifacts are retained on this run. The latter contains the actual measured timing/latency executables and SHA256 manifest; the workflow being successful is not a PASS for this historical performance ledger. Original sanity and ledger thresholds were retained.

@zackees

zackees commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

#585 corrected historical #569 bisection: both measured rows pass

Run 39504125686 compares exact pre-merge cb142ae2a27c4fd55570cc2a20e0158dd5910e51 with merge 80936c9632b3f60fdf4d8cf397e2a615fc635f83 on EPYC 9V74 / four logical CPUs, using corrected balanced arm order and a same-run null arm. Actual sample counts are 30 per arm for plain large-class/8 and 60 per arm for profiler-on, not the requested minimum alone. Eight workers are 2:1 oversubscribed.

Both rows pass their full six-metric ledgers. Plain peak RSS improves 5.3%, with CPU/op +0.7% [0.1%, 1.3%] inside its 1.6% allowance; p99 is within same-run noise. This historical result does not validate the combined current candidate, whose acceptance runs are still live. Earlier fixed-middle-order results remain diagnostic and are not pooled.

base cb142ae2a27c head 80936c9632b3 on AMD EPYC 9V74 80-Core Processor, 30 reps (up to 60); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 5% PASS +0.1% [-1.3, +0.9] / 2.0% PASS +0.7% [+0.1, +1.3] / 1.6% PASS -6.5% [-37.6, +5.6] / 39.0% PASS +0.0% [-0.8, +0.9] / 0.9% PASS -5.3% [-6.4, -4.2] / 1.1% PASS -5.5% [-6.4, -4.0] / 1.6% PASS
large-class/8 (profiler on) 5% PASS +0.5% [-0.2, +1.1] / 1.5% PASS -0.3% [-1.1, +0.4] / 1.5% PASS -5.7% [-14.4, +7.2] / 8.4% PASS +0.6% [-1.3, +2.0] / 4.1% PASS -4.1% [-5.7, -3.1] / 0.9% PASS -7.2% [-7.8, -6.5] / 1.7% PASS

The run retains perf-ab-samples and perf-ab-builds; all eight preserved timing/latency executable SHA256 entries were independently verified after download. This narrows historical attribution: the measured #569 large-class rows do not account for a large latency debt, whereas corrected #572 does show one (#585 (comment)). It does not rule out costs on workloads not measured in this bisection. No thresholds changed.

@zackees

zackees commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

#585 corrected historical #570 bisection: CPU improves, full ledger remains inconclusive

Run 39504122084 compares exact pre-merge 3672dbd66e39526c712bb5cea52b4a70f026ade3 with merge cb142ae2a27c4fd55570cc2a20e0158dd5910e51, on EPYC 7763 / four logical CPUs. Both rows collected 60 actual samples per arm, including the same-run null arm, under corrected rotating-reversed-blocks-v1 ordering. Eight workers are 2:1 oversubscribed.

CPU/op is lower beyond noise: -2.1% [-2.5%, -1.8%] plain, -1.0% [-1.4%, -0.7%] profiler-on. Throughput is also higher. This supports the narrow conclusion that moving the retired-page clock read off the owner path paid down CPU cost in these measured rows. It does not establish full-metric equivalence: plain p99 and 0.5-second drain RSS, and profiler-on faults and p99, remain INCONCLUSIVE at the sample cap. Equal rounded p99 medians are not proof of equivalence. This is not a standalone acceptance result.

base 3672dbd66e39 head cb142ae2a27c on AMD EPYC 7763 64-Core Processor, 30 reps (up to 60); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 0% INCONCLUSIVE +2.6% [+2.2, +3.6] / 1.5% PASS -2.1% [-2.5, -1.8] / 0.8% PASS -5.4% [-24.0, +34.8] / 40.4% PASS +0.1% [-0.7, +0.8] / 0.7% INCONCLUSIVE -0.0% [-0.8, +1.1] / 1.5% PASS +0.5% [-0.8, +1.3] / 1.0% INCONCLUSIVE
large-class/8 (profiler on) 0% INCONCLUSIVE +1.1% [+0.7, +1.5] / 0.4% PASS -1.0% [-1.4, -0.7] / 0.4% PASS +8.5% [+0.3, +16.9] / 8.9% INCONCLUSIVE +0.6% [-3.2, +4.0] / 3.8% INCONCLUSIVE +0.0% [-0.8, +1.0] / 1.5% PASS -0.4% [-1.7, +1.3] / 1.3% PASS

The run retains the exact measured timing/latency executables in perf-ab-builds and raw perf-ab-samples. All eight executable hashes match the downloaded SHA256 manifest. Thresholds and the original sanity screen were retained.

Corrected historical attribution so far

These are separate same-run comparisons, not one common physical VM or a pool: #570 ran on EPYC 7763; #569/#572 on EPYC 9V74. Their deltas cannot be added into a causal decomposition of the whole fork/upstream gap. #544 is the coordinating issue for the span follow-ups; #569 and #570 are its merged changes. Earlier fixed-middle-order bisections remain diagnostic only.

The three corrected combined-candidate-versus-frozen-baseline runs remain live. Those results are required for candidate acceptance; historical attribution does not substitute for them.

@zackees

zackees commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner Author

#585 corrected combined-candidate controls: two PASS, two INCONCLUSIVE

Run 39502534567 compares frozen ee3623f6b7a485754a0a0aa3462d8cba43390e8b with candidate 9b67e411fc8c8899e07e05d528a4c5b37ab20417 (allocator source identical to current PR head 43f0b97d). It ran on EPYC 9V74 / four logical CPUs with corrected balanced arm order, same-run null arm and unchanged sanity/ledger thresholds.

Actual samples per arm: small/8 60; larson/1 60; larson/8 30; larson/8 chart 35. Eight workers are 2:1 oversubscribed.

Both Larson/8 rows PASS all six metrics. Small/8 and Larson/1 remain INCONCLUSIVE and are not accepted. Small/8 throughput and CPU pass, but p99, peak and drain RSS remain unresolved; Larson/1 remains unresolved on peak RSS.

base ee3623f6b7a4 head 9b67e411fc8c on AMD EPYC 9V74 80-Core Processor, 30 reps (up to 60); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
small/8 (control) 0% INCONCLUSIVE -1.0% [-2.8, +0.6] / 3.8% PASS +1.6% [-0.4, +3.5] / 4.8% PASS -0.6% [-0.6, +0.6] / 0.6% PASS +0.0% [+0.0, +1.1] / 0.0% INCONCLUSIVE +0.0% [+0.0, +0.0] / 0.0% INCONCLUSIVE +0.0% [+0.0, +0.0] / 0.0% INCONCLUSIVE
larson/1 0% INCONCLUSIVE -0.0% [-0.8, +1.4] / 1.2% PASS -0.1% [-1.6, +0.9] / 1.2% PASS +0.5% [+0.0, +0.5] / 0.5% PASS +0.4% [-0.9, +1.1] / 1.8% PASS +0.0% [-0.0, +1.2] / 1.2% INCONCLUSIVE +0.0% [+0.0, +0.0] / 0.0% PASS
larson/8 0% PASS -0.4% [-1.6, +3.0] / 2.7% PASS -1.1% [-2.9, +0.2] / 1.8% PASS -0.7% [-5.4, +3.8] / 6.4% PASS +0.0% [-0.5, +0.5] / 4.7% PASS +0.1% [-2.2, +2.2] / 3.4% PASS +0.1% [-0.4, +0.4] / 0.7% PASS
larson/8 (chart build) 0% PASS -0.8% [-1.7, +0.2] / 2.2% PASS -0.2% [-1.6, +1.1] / 1.3% PASS +1.8% [-2.8, +12.0] / 13.0% PASS +0.0% [+0.0, +0.5] / 4.1% PASS -1.4% [-3.4, +0.7] / 2.6% PASS +0.3% [-0.0, +0.5] / 0.6% PASS

Do not interpret rounded zeros as equivalence

Small/8 p99 medians are 91 ns for base, head and null, yet the full-precision paired head interval is [0%, 1.111111%], with a zero null median interval. Peak/drain head intervals are [0%, 0.0237023%], with a zero null interval. Peak-RSS medians are base 16.48046875 MiB, head 16.484375 MiB and null 16.484375 MiB; the one-page difference is unresolved, not waived. Larson/1 peak interval is [-0.0356532%, 1.2222090%] against 1.1770021% noise.

All eight actual executable SHA256 hashes were verified. In the plain timing binaries, head text is 64 bytes larger; writable data and BSS are identical, and all LOAD segment addresses/page counts match. That excludes a different mapped segment page count as an explanation; it does not prove which physical pages were resident or explain latency.

Two focused follow-ups retain the same exact source pair, null arm, original work traces and thresholds, requesting 60 samples with adaptive maximum 120:

The original result remains visible. Pooling, if applicable, must first pass the harness compatibility checks for CPU model, logical CPUs, refs, flags, latency protocol and arm-order protocol. No different-host model or legacy-order samples will be pooled. Candidate acceptance is still pending, including the two original live target/random runs.

Untimed residency attribution using the actual control binaries

A local diagnostic replayed the downloaded plain timing executables, unchanged and hash-verified, for three alternating paired blocks with the original small/8 arguments and the same four allowed CPUs. It sampled /proc/<pid>/smaps and per-process pagemap present bits every 100 ms. These are final observed snapshots, not peak RSS, and this is a different host from CI. Timing output is not used for acceptance.

repetition arm executable-mapping RSS KiB anonymous-mapping RSS KiB stack RSS KiB other mappings RSS KiB
0 base 284 14668 40 1960
0 head 284 14668 40 1956
1 head 284 14668 36 1956
1 base 284 14668 36 1960
2 base 240 14668 40 1960
2 head 284 14668 40 1960

The unchanged base itself had 71, 71 and 60 present executable-file pages. In the third base process, file offsets 0x0–0x2000 (read-only) and 0x18000–0x1f000 (executable) were absent from the final snapshot, unlike the first base process. This reproduces baseline process-residency variation; it does not establish the source of the original CI one-page interval. Anonymous mapping RSS is not a measurement of allocator memory alone. No threshold was waived and the control remains INCONCLUSIVE pending the focused CI follow-ups.

@zackees

zackees commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

#585 focused combined-candidate Larson/1: PASS on EPYC 7763

Run 39509440439 collected 60 actual samples per arm, including null, comparing frozen ee3623f6b7a485754a0a0aa3462d8cba43390e8b with unchanged candidate 9b67e411fc8c8899e07e05d528a4c5b37ab20417. It uses corrected balanced arm order and retains the original sanity and ledger thresholds. Host: EPYC 7763 / four logical CPUs.

Larson/1 PASSes all six metrics. This is an independent same-host paired result, not a pool with the earlier EPYC 9V74 control run. That earlier 60-sample row remains INCONCLUSIVE on peak RSS and stays visible: #585 (comment) . It is not reclassified as a pass on EPYC 9V74.

base ee3623f6b7a4 head 9b67e411fc8c on AMD EPYC 7763 64-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
larson/1 0% PASS -0.3% [-1.5, +0.9] / 1.9% PASS +0.4% [-0.4, +1.7] / 2.3% PASS +0.0% [-0.5, +0.0] / 0.0% PASS -0.3% [-0.9, +0.5] / 1.1% PASS -0.0% [-0.0, +0.0] / 1.3% PASS +0.0% [+0.0, +0.0] / 0.0% PASS

The run preserves raw samples and all four actual timing/latency binaries. Each SHA256 digest was verified and each binary matches its counterpart from the original corrected control run byte for byte. Thus no allocator, benchmark binary or work-trace change was made for this follow-up; the result does not establish why the earlier host's RSS interval remained unresolved.

Candidate acceptance still awaits the original large-class and random/sparse runs and the focused small/8 run. The original control ledger and its limitations remain part of the record; no memory credit or threshold exception was given to this control.

@zackees

zackees commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

#585 focused combined-candidate small/8: PASS on EPYC 7763

Run 39509436788 collected 60 actual samples per arm, including null, comparing frozen ee3623f6b7a485754a0a0aa3462d8cba43390e8b with unchanged candidate 9b67e411fc8c8899e07e05d528a4c5b37ab20417. Corrected balanced arm ordering, original work trace, sanity screen and ledger thresholds are retained. Host: EPYC 7763 / four logical CPUs, eight workers (2:1 oversubscribed).

Small/8 PASSes all six metrics. Full-precision paired p99 interval is exactly [0%, 0%], against zero null noise. Peak and drain RSS intervals are [-0.02387205%, 0%], also against zero null noise. CPU/op is -0.7% [-2.0%, +0.5%] and throughput -0.9% [-1.9%, +1.9%], each within 2.3% same-run noise. No memory credit or quantization exception was applied.

base ee3623f6b7a4 head 9b67e411fc8c on AMD EPYC 7763 64-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
small/8 (control) 0% PASS -0.9% [-1.9, +1.9] / 2.3% PASS -0.7% [-2.0, +0.5] / 2.3% PASS +0.0% [-0.6, +0.6] / 0.6% PASS +0.0% [+0.0, +0.0] / 0.0% PASS +0.0% [-0.0, +0.0] / 0.0% PASS +0.0% [-0.0, +0.0] / 0.0% PASS

All four actual timing/latency executable hashes were verified and match the original corrected control run byte for byte. Raw samples and exact executables are retained as this run's artifacts. This is an independent same-host result, not a pool with the earlier EPYC 9V74 run; that run remains INCONCLUSIVE on small p99/RSS and Larson/1 peak RSS: #585 (comment) . The new result does not establish the cause of the earlier host's unresolved intervals or the legacy discarded-binary CPU failure.

The focused Larson/1 run also independently PASSed at 60 samples per arm: #585 (comment) . Both Larson/8 rows PASSed in the original corrected control run. Thus every control has a conclusive PASS in a corrected run, with the earlier unresolved rows retained rather than reclassified. These are per-cell results from separate VMs; no pooled or aggregate cross-host acceptance claim is made.

Combined-candidate acceptance still awaits the original large-class and random/sparse runs. No allocator or benchmark binary was changed to obtain these control follow-ups.

@zackees

zackees commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

#585 corrected combined-candidate random/sparse: two PASS, two INCONCLUSIVE

Run 39502538433 compares frozen ee3623f6b7a485754a0a0aa3462d8cba43390e8b with candidate 9b67e411fc8c8899e07e05d528a4c5b37ab20417 (allocator source identical to current PR head 43f0b97d). Host: EPYC 9V45 / four logical CPUs, eight-worker rows 2:1 oversubscribed. Corrected balanced arm order, same-run null arm, work traces, original sanity screen and ledger thresholds are retained.

Actual samples per arm: random-large/8 30; bursty/8 60; random-large/1 60; sparse 64-slot/8 60.

Random-large/8 and random-large/1 PASS all six metrics. Bursty/8 remains INCONCLUSIVE on p99; sparse 64-slot/8 remains INCONCLUSIVE on CPU/op and faults. All their peak/drain memory metrics pass their same-run noise allowances, but no row earns memory credit. None of the unresolved cells is accepted.

base ee3623f6b7a4 head 9b67e411fc8c on AMD EPYC 9V45 96-Core Processor, 30 reps (up to 60); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
random-large/8 0% PASS +1.6% [-0.1, +3.1] / 2.1% PASS -1.3% [-2.1, +0.6] / 2.4% PASS +0.0% [-0.8, +1.1] / 2.3% PASS +3.1% [-1.7, +6.8] / 19.8% PASS +0.5% [-2.3, +2.1] / 4.5% PASS +0.0% [+0.0, +0.0] / 0.0% PASS
random-large-bursty/8 0% INCONCLUSIVE +0.0% [-0.2, +0.2] / 0.2% PASS -0.1% [-1.1, +1.1] / 1.1% PASS +8.8% [-20.1, +25.1] / 55.2% PASS -0.5% [-1.4, +1.2] / 0.7% INCONCLUSIVE -1.2% [-3.6, +1.3] / 2.5% PASS +2.3% [+0.6, +4.5] / 5.6% PASS
random-large/1 0% PASS -0.4% [-1.7, +0.4] / 2.1% PASS +0.9% [-0.6, +1.7] / 2.6% PASS +0.0% [+0.0, +0.0] / 0.0% PASS +0.3% [-0.9, +1.4] / 1.6% PASS +0.0% [-0.0, +0.0] / 0.0% PASS +0.0% [+0.0, +0.0] / 0.0% PASS
sparse-large-buffers 64-slot budget/8 (#422) 0% INCONCLUSIVE -0.3% [-1.9, +1.4] / 2.1% PASS +0.4% [-1.7, +2.4] / 1.6% INCONCLUSIVE +0.4% [-0.8, +1.6] / 1.2% INCONCLUSIVE +0.6% [-1.1, +1.7] / 1.8% PASS +1.2% [-2.4, +4.5] / 4.8% PASS +1.2% [-2.7, +4.2] / 4.4% PASS

All four preserved actual timing/latency executable hashes were verified and match the original corrected control run byte for byte, despite the different CPU model. These results are not pooled across models and do not establish an explanation for other hosts' intervals.

Focused single-row follow-ups are predeclared at 60 samples, adaptive maximum 120, with the same exact source pair, work traces, original sanity/ledger thresholds, balanced order, null arm and binary preservation:

Only compatible metadata permits pooling; otherwise report independent same-host paired results. The original 60-sample inconclusive rows remain in this record. The large-class target run is still live. No source or benchmark executable was changed for these follow-ups.

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

Chart ephemeral: standard row PASS; short generations remain INCONCLUSIVE

Run 39520756150 completed on AMD EPYC 7763 / four logical CPUs. The standard chart ephemeral row PASSes all six metrics at 80 samples per arm, CPU -1.4%. Short generations remain INCONCLUSIVE at the maximum 120 per arm, on minor faults, peak RSS and after-drain RSS. CPU, throughput and p99 PASS. Both rows have zero memory credit.

Fixed baseline ee3623f6b7a485754a0a0aa3462d8cba43390e8b → candidate 15bfcba0c3e59aea6b22dd04f91db9d95cd63a0a. All actual measured executable hashes verified; row parameters match the original artifacts. Eight workers on four logical CPUs is 2:1 oversubscription. Sampling and balanced arm-order protocols are unchanged.

base ee3623f6b7a4 head 15bfcba0c3e5 on AMD EPYC 7763 64-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class-ephemeral/8 (chart build) 0% PASS +1.6% [+1.1, +2.0] / 0.7% PASS -1.4% [-1.7, -1.2] / 0.1% PASS +0.0% [-0.4, +0.8] / 0.8% PASS -0.5% [-1.1, +0.5] / 1.6% PASS -0.0% [-2.5, +0.1] / 2.5% PASS -0.1% [-11.3, +0.0] / 12.7% PASS
large-class-ephemeral/8 short generations (chart build) 0% INCONCLUSIVE +1.2% [+0.7, +1.6] / 0.7% PASS -1.5% [-1.7, -1.1] / 0.7% PASS +0.2% [-0.4, +1.2] / 0.8% INCONCLUSIVE -0.1% [-1.0, +0.5] / 1.0% PASS +0.0% [-0.0, +2.6] / 2.5% INCONCLUSIVE +0.0% [-0.0, +2.6] / 2.5% INCONCLUSIVE

Focused exact short-generation follow-up: https://github.com/zackees/mimalloc-pprof/actions/runs/39526576845, 120-to-240 samples/arm, same source pair, original trace and null/sanity gates. This raises the sample budget without changing an acceptance threshold. The prior INCONCLUSIVE result remains visible; no cross-host pooling.

Actual executable hashes:

{
  "base/chart/perf_ab": "306d5c8a089d29a7ff276df621226fe2d08ffe7013669627f38dc5c0020d33f7",
  "base/chart/perf_ab_latency": "2f883b21f4d48e9e1b404c3110ca058902c68cba23e99a82edd58026c7495090",
  "head/chart/perf_ab": "51b9dde3f2ce5e446eb8443af7ab56136af1b146bbdb61acad221d2a652a0bff",
  "head/chart/perf_ab_latency": "41d91d58314a4ca202ec6b0e4a8a3d62995a39a3aac891edab560f22c0500240"
}
Full-precision cells and actual arm counts
{
  "large-class-ephemeral/8 (chart build)": {
    "n": {
      "base": 80,
      "head": 80,
      "null": 80
    },
    "saved_percent": 0.0,
    "cells": [
      {
        "metric": "ops/s",
        "base_median": 8090087.300000001,
        "head_median": 8236055.25,
        "change": [
          1.5976350413375746,
          1.0741514386580004,
          2.021272121039053
        ],
        "regression": [
          -2.021272121039053,
          -1.0741514386580004
        ],
        "allowance": 0.7038137898227369,
        "verdict": "PASS"
      },
      {
        "metric": "cpu s",
        "base_median": 1.5591499999999998,
        "head_median": 1.5343499999999999,
        "change": [
          -1.4428242792762553,
          -1.7187400166123523,
          -1.2178619756427695
        ],
        "regression": [
          -1.7187400166123523,
          -1.2178619756427695
        ],
        "allowance": 0.1352134440795822,
        "verdict": "PASS"
      },
      {
        "metric": "minor faults",
        "base_median": 252.0,
        "head_median": 252.0,
        "change": [
          0.0,
          -0.4,
          0.7905138339920948
        ],
        "regression": [
          -0.4,
          0.7905138339920948
        ],
        "allowance": 0.796812749003984,
        "verdict": "PASS"
      },
      {
        "metric": "p99 operation ns",
        "base_median": 1853.0,
        "head_median": 1843.5,
        "change": [
          -0.5396654074473827,
          -1.1154489682097044,
          0.48543689320388345
        ],
        "regression": [
          -1.1154489682097044,
          0.48543689320388345
        ],
        "allowance": 1.6008537886872998,
        "verdict": "PASS"
      },
      {
        "metric": "peak RSS MiB",
        "base_median": 76.466796875,
        "head_median": 74.6328125,
        "change": [
          -0.028803626883975326,
          -2.5002551280742935,
          0.07141399714344011
        ],
        "regression": [
          -2.5002551280742935,
          0.07141399714344011
        ],
        "allowance": 2.482251391797334,
        "verdict": "PASS"
      },
      {
        "metric": "RSS 0.5 s after drain MiB",
        "base_median": 8.7890625,
        "head_median": 7.85546875,
        "change": [
          -0.05361540852510731,
          -11.297440423654017,
          0.0
        ],
        "regression": [
          -11.297440423654017,
          0.0
        ],
        "allowance": 12.729985082048731,
        "verdict": "PASS"
      }
    ]
  },
  "large-class-ephemeral/8 short generations (chart build)": {
    "n": {
      "base": 120,
      "head": 120,
      "null": 120
    },
    "saved_percent": 0.0,
    "cells": [
      {
        "metric": "ops/s",
        "base_median": 7722059.1,
        "head_median": 7802026.4,
        "change": [
          1.2275152243972343,
          0.6723567289540237,
          1.6046831424223438
        ],
        "regression": [
          -1.6046831424223438,
          -0.6723567289540237
        ],
        "allowance": 0.7108754433641795,
        "verdict": "PASS"
      },
      {
        "metric": "cpu s",
        "base_median": 0.423,
        "head_median": 0.4168,
        "change": [
          -1.4744072052498967,
          -1.7009213323883732,
          -1.1461988304093604
        ],
        "regression": [
          -1.7009213323883732,
          -1.1461988304093604
        ],
        "allowance": 0.6850933144342106,
        "verdict": "PASS"
      },
      {
        "metric": "minor faults",
        "base_median": 250.0,
        "head_median": 251.0,
        "change": [
          0.1984126984126984,
          -0.3968253968253968,
          1.1627906976744187
        ],
        "regression": [
          -0.3968253968253968,
          1.1627906976744187
        ],
        "allowance": 0.7843137254901961,
        "verdict": "INCONCLUSIVE"
      },
      {
        "metric": "p99 operation ns",
        "base_median": 1834.0,
        "head_median": 1824.0,
        "change": [
          -0.05437778098439455,
          -1.0341261633919339,
          0.5254860746190226
        ],
        "regression": [
          -1.0341261633919339,
          0.5254860746190226
        ],
        "allowance": 1.0309278350515463,
        "verdict": "PASS"
      },
      {
        "metric": "peak RSS MiB",
        "base_median": 74.6015625,
        "head_median": 74.6015625,
        "change": [
          0.0,
          -0.03058727569331158,
          2.6056804854418436
        ],
        "regression": [
          -0.03058727569331158,
          2.6056804854418436
        ],
        "allowance": 2.544478680051685,
        "verdict": "INCONCLUSIVE"
      },
      {
        "metric": "RSS 0.5 s after drain MiB",
        "base_median": 74.6015625,
        "head_median": 74.6015625,
        "change": [
          0.0,
          -0.005380104374024856,
          2.6056804854418436
        ],
        "regression": [
          -0.005380104374024856,
          2.6056804854418436
        ],
        "allowance": 2.544478680051685,
        "verdict": "INCONCLUSIVE"
      }
    ]
  }
}

Overall acceptance remains pending; PR #609 remains draft. Platform CI is 104 successes / five intentional skips; no source or memory setting changed.

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

Current candidate: main row PASS, profiler p99 FAIL

Run 39520753532, frozen ee3623f6b7a485754a0a0aa3462d8cba43390e8b → current 15bfcba0c3e59aea6b22dd04f91db9d95cd63a0a, on AMD EPYC 9V45 / four logical CPUs. Actual counts: 60 samples/arm plain; 120 samples/arm profiler. Plain large-class/8 PASSes all six metrics. Profiler-enabled large-class/8 FAILs p99, +11.3% [4.2%,15.7%] against 3.9% same-run null noise; the other five metrics PASS. No memory credit in either row. The candidate is unaccepted and PR #609 remains draft. Full platform correctness CI being green does not waive this performance failure.

base ee3623f6b7a4 head 15bfcba0c3e5 on AMD EPYC 9V45 96-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 0% PASS +0.3% [-0.5, +1.2] / 1.0% PASS -0.9% [-1.5, +0.0] / 0.8% PASS +0.0% [-2.7, +2.3] / 15.9% PASS -1.0% [-1.9, +0.0] / 1.5% PASS -0.0% [-2.7, +0.1] / 2.6% PASS -0.9% [-3.8, +1.0] / 3.0% PASS
large-class/8 (profiler on) 0% FAIL +0.2% [-0.3, +0.8] / 0.7% PASS +0.0% [-0.6, +0.3] / 0.7% PASS -0.1% [-0.3, +0.2] / 0.7% PASS +11.3% [+4.2, +15.7] / 3.9% FAIL -0.0% [-2.5, +0.4] / 2.2% PASS -0.3% [-0.9, +0.7] / 1.3% PASS

All eight actual measured executable hashes verify. Plain binaries match all other ordinary new-candidate runs byte for byte; baseline binaries match the previous original-target run. Row parameters, balanced arm positions, separate p99 replay and thresholds are unchanged. Eight workers on four logical CPUs is 2:1 oversubscription. No cross-host pooling.

The failed frozen-baseline comparison will not be reclassified or repeated to seek a pass. The next comparison isolates old prefix-reuse candidate 9b67e411 → new byte-threshold candidate 15bfcba0, exact profiler row, 60-to-120 samples/arm with null and sanity gates. Attribution run 39526717341 is source attribution, not fixed-baseline acceptance. No source change or threshold relaxation is being made.

{
  "base/plain/perf_ab": "7f0c70ce2450dfa09126c1674bddab53187614f12d5bd648b4fda33ccbd3c712",
  "base/plain/perf_ab_latency": "ef2d5d273681a473e253d1dd2080307f1ae9f09d34977bda2ad4cab9cbfe7823",
  "base/pprof/perf_ab": "c6784c2c2cb2619d31ce4bc679f840ad7d9a2cf04aba6ce4dda8c374d5c8ba37",
  "base/pprof/perf_ab_latency": "f4e223e4c1489619ef2cd9831bb88a204931fa207d7ec18a17411e2899eef12f",
  "head/plain/perf_ab": "9ee3617e371e93195de1374ef403455e7ae660848d689ce4bf0ae0ce2f8565b1",
  "head/plain/perf_ab_latency": "c14766d194fd7f3c07652b148d1de18e3e4db90b4414bd563398f5f44ee9b982",
  "head/pprof/perf_ab": "ef1f0135f09bec19ff3cba3c26a211359243cbcb40825818f9bddb3c512771ba",
  "head/pprof/perf_ab_latency": "1f432e1f3a4c857bde03dd44859eac39b92a510806cceb4841087d16e98d7a8e"
}
Full-precision cells and actual per-arm counts
{
  "large-class/8": {
    "n": {
      "base": 60,
      "head": 60,
      "null": 60
    },
    "saved_percent": 0.0,
    "cells": [
      {
        "metric": "ops/s",
        "base_median": 6667628.9,
        "head_median": 6678621.199999999,
        "change": [
          0.3412312144181615,
          -0.5394161115517625,
          1.1655071010572247
        ],
        "regression": [
          -1.1655071010572247,
          0.5394161115517625
        ],
        "allowance": 1.0053539940084428,
        "verdict": "PASS"
      },
      {
        "metric": "cpu s",
        "base_median": 1.8851,
        "head_median": 1.8780999999999999,
        "change": [
          -0.9400186412634518,
          -1.5480674957428158,
          0.04700229789011389
        ],
        "regression": [
          -1.5480674957428158,
          0.04700229789011389
        ],
        "allowance": 0.7734279811410684,
        "verdict": "PASS"
      },
      {
        "metric": "minor faults",
        "base_median": 224.5,
        "head_median": 222.0,
        "change": [
          0.0,
          -2.727272727272727,
          2.2522522522522523
        ],
        "regression": [
          -2.727272727272727,
          2.2522522522522523
        ],
        "allowance": 15.916955017301039,
        "verdict": "PASS"
      },
      {
        "metric": "p99 operation ns",
        "base_median": 2053.0,
        "head_median": 2033.0,
        "change": [
          -0.9862472489756378,
          -1.872299567930869,
          0.0
        ],
        "regression": [
          -1.872299567930869,
          0.0
        ],
        "allowance": 1.4541929229277752,
        "verdict": "PASS"
      },
      {
        "metric": "peak RSS MiB",
        "base_median": 74.43359375,
        "head_median": 72.51171875,
        "change": [
          -0.02623501326673862,
          -2.658486707566462,
          0.07017165065313613
        ],
        "regression": [
          -2.658486707566462,
          0.07017165065313613
        ],
        "allowance": 2.6075550891920254,
        "verdict": "PASS"
      },
      {
        "metric": "RSS 0.5 s after drain MiB",
        "base_median": 63.111328125,
        "head_median": 62.298828125,
        "change": [
          -0.9369928771022332,
          -3.7815889946063876,
          1.0305269298075053
        ],
        "regression": [
          -3.7815889946063876,
          1.0305269298075053
        ],
        "allowance": 3.0231217575097675,
        "verdict": "PASS"
      }
    ]
  },
  "large-class/8 (profiler on)": {
    "n": {
      "base": 120,
      "head": 120,
      "null": 120
    },
    "saved_percent": 0.0,
    "cells": [
      {
        "metric": "ops/s",
        "base_median": 4729425.15,
        "head_median": 4753452.4,
        "change": [
          0.16241125103154308,
          -0.2786631451692534,
          0.7931977420037137
        ],
        "regression": [
          -0.7931977420037137,
          0.2786631451692534
        ],
        "allowance": 0.6542309515856304,
        "verdict": "PASS"
      },
      {
        "metric": "cpu s",
        "base_median": 2.65735,
        "head_median": 2.6506499999999997,
        "change": [
          0.0077255990942455555,
          -0.5575048732943462,
          0.27628434886499675
        ],
        "regression": [
          -0.5575048732943462,
          0.27628434886499675
        ],
        "allowance": 0.6716220805191795,
        "verdict": "PASS"
      },
      {
        "metric": "minor faults",
        "base_median": 1632.5,
        "head_median": 1629.0,
        "change": [
          -0.09200881441042014,
          -0.30731407498463426,
          0.24479804161566704
        ],
        "regression": [
          -0.30731407498463426,
          0.24479804161566704
        ],
        "allowance": 0.676923076923077,
        "verdict": "PASS"
      },
      {
        "metric": "p99 operation ns",
        "base_median": 23811.0,
        "head_median": 26675.5,
        "change": [
          11.288528431618904,
          4.211122651307714,
          15.721972731318527
        ],
        "regression": [
          4.211122651307714,
          15.721972731318527
        ],
        "allowance": 3.946504758779127,
        "verdict": "FAIL"
      },
      {
        "metric": "peak RSS MiB",
        "base_median": 75.6328125,
        "head_median": 74.02734375,
        "change": [
          -0.021476510871790477,
          -2.471042471042471,
          0.4027130139889784
        ],
        "regression": [
          -2.471042471042471,
          0.4027130139889784
        ],
        "allowance": 2.218438838157148,
        "verdict": "PASS"
      },
      {
        "metric": "RSS 0.5 s after drain MiB",
        "base_median": 72.4296875,
        "head_median": 72.029296875,
        "change": [
          -0.27955039925074177,
          -0.9455248737509402,
          0.6826302597194817
        ],
        "regression": [
          -0.9455248737509402,
          0.6826302597194817
        ],
        "allowance": 1.3112415647116915,
        "verdict": "PASS"
      }
    ]
  }
}

@zackees

zackees commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner Author

Profiler p99 failure: local cycle attribution, not acceptance

Current frozen-baseline candidate failed profiler p99 in 39520753532, +11.3% [4.2%,15.7%] against 3.9% noise. The six completed local samples below use the exact failed baseline/candidate latency executables, three alternating pairs on Ryzen 7 3700X with four-CPU affinity, profiler enabled and the original profiler row. User-mode sampled cycles exclude kernel work and perturb execution. No local latency is an acceptance result.

Fresh-page cycle shares are lower in all three candidate runs (3.85/2.79/4.30% baseline vs 1.88/1.83/2.99% candidate); pthread mutex-lock shares are mixed (1.45/2.02/1.42% vs 1.96/1.88/1.22%). Fractions are not per-operation costs. These records do not establish the cause of the CI p99 regression. No source change follows from this local screen.

The source-isolation comparison 39526717341 measures old prefix-reuse candidate 9b67e411 → current byte-threshold candidate 15bfcba0, exact profiler row; it is attribution, not a replacement for the failed fixed-baseline ledger.

Manifest:

{
  "cpus": "0,1,2,3",
  "environment": {
    "MIMALLOC_PROF": "1"
  },
  "purpose": "Local sampled cycles only; perturbed p99 is not acceptance",
  "hashes": {
    "base": "f4e223e4c1489619ef2cd9831bb88a204931fa207d7ec18a17411e2899eef12f",
    "head": "1f432e1f3a4c857bde03dd44859eac39b92a510806cceb4841087d16e98d7a8e"
  },
  "runs": [
    {
      "rep": 0,
      "arm": "base",
      "command": [
        "perf",
        "record",
        "-q",
        "-e",
        "cycles:u",
        "-c",
        "100000",
        "-o",
        "/tmp/mimalloc-585/profiler-cycle-attribution/0-base.data",
        "--",
        "taskset",
        "-c",
        "0,1,2,3",
        "/tmp/mimalloc-585/balanced-byte-threshold-main/builds/base/pprof/perf_ab_latency",
        "8",
        "1",
        "98304",
        "524288",
        "400000",
        "0",
        "0",
        "uniform",
        "8",
        "1300"
      ]
    },
    {
      "rep": 0,
      "arm": "head",
      "command": [
        "perf",
        "record",
        "-q",
        "-e",
        "cycles:u",
        "-c",
        "100000",
        "-o",
        "/tmp/mimalloc-585/profiler-cycle-attribution/0-head.data",
        "--",
        "taskset",
        "-c",
        "0,1,2,3",
        "/tmp/mimalloc-585/balanced-byte-threshold-main/builds/head/pprof/perf_ab_latency",
        "8",
        "1",
        "98304",
        "524288",
        "400000",
        "0",
        "0",
        "uniform",
        "8",
        "1300"
      ]
    },
    {
      "rep": 1,
      "arm": "head",
      "command": [
        "perf",
        "record",
        "-q",
        "-e",
        "cycles:u",
        "-c",
        "100000",
        "-o",
        "/tmp/mimalloc-585/profiler-cycle-attribution/1-head.data",
        "--",
        "taskset",
        "-c",
        "0,1,2,3",
        "/tmp/mimalloc-585/balanced-byte-threshold-main/builds/head/pprof/perf_ab_latency",
        "8",
        "1",
        "98304",
        "524288",
        "400000",
        "0",
        "0",
        "uniform",
        "8",
        "1300"
      ]
    },
    {
      "rep": 1,
      "arm": "base",
      "command": [
        "perf",
        "record",
        "-q",
        "-e",
        "cycles:u",
        "-c",
        "100000",
        "-o",
        "/tmp/mimalloc-585/profiler-cycle-attribution/1-base.data",
        "--",
        "taskset",
        "-c",
        "0,1,2,3",
        "/tmp/mimalloc-585/balanced-byte-threshold-main/builds/base/pprof/perf_ab_latency",
        "8",
        "1",
        "98304",
        "524288",
        "400000",
        "0",
        "0",
        "uniform",
        "8",
        "1300"
      ]
    },
    {
      "rep": 2,
      "arm": "base",
      "command": [
        "perf",
        "record",
        "-q",
        "-e",
        "cycles:u",
        "-c",
        "100000",
        "-o",
        "/tmp/mimalloc-585/profiler-cycle-attribution/2-base.data",
        "--",
        "taskset",
        "-c",
        "0,1,2,3",
        "/tmp/mimalloc-585/balanced-byte-threshold-main/builds/base/pprof/perf_ab_latency",
        "8",
        "1",
        "98304",
        "524288",
        "400000",
        "0",
        "0",
        "uniform",
        "8",
        "1300"
      ]
    },
    {
      "rep": 2,
      "arm": "head",
      "command": [
        "perf",
        "record",
        "-q",
        "-e",
        "cycles:u",
        "-c",
        "100000",
        "-o",
        "/tmp/mimalloc-585/profiler-cycle-attribution/2-head.data",
        "--",
        "taskset",
        "-c",
        "0,1,2,3",
        "/tmp/mimalloc-585/balanced-byte-threshold-main/builds/head/pprof/perf_ab_latency",
        "8",
        "1",
        "98304",
        "524288",
        "400000",
        "0",
        "0",
        "uniform",
        "8",
        "1300"
      ]
    }
  ]
}

Reproduction script:

import hashlib,json,os,subprocess
from pathlib import Path
root=Path('/tmp/mimalloc-585');out=root/'profiler-cycle-attribution';out.mkdir(exist_ok=True)
exes={arm:root/f'balanced-byte-threshold-main/builds/{arm}/pprof/perf_ab_latency' for arm in ('base','head')}
cpus=','.join(map(str,sorted(os.sched_getaffinity(0))[:4]));results=[]
manifest={'cpus':cpus,'environment':{'MIMALLOC_PROF':'1'},'purpose':'Local sampled cycles only; perturbed p99 is not acceptance','hashes':{a:hashlib.sha256(p.read_bytes()).hexdigest() for a,p in exes.items()},'runs':results}
for rep in range(3):
 arms=list(exes) if rep%2==0 else list(reversed(exes))
 for arm in arms:
  data=out/f'{rep}-{arm}.data'
  cmd=['perf','record','-q','-e','cycles:u','-c','100000','-o',str(data),'--','taskset','-c',cpus,str(exes[arm]),'8','1','98304','524288','400000','0','0','uniform','8','1300']
  env=os.environ.copy();env.update(manifest['environment'])
  proc=subprocess.run(cmd,capture_output=True,text=True,env=env,check=True)
  (out/f'{rep}-{arm}.stderr').write_text(proc.stderr);(out/f'{rep}-{arm}.stdout').write_text(proc.stdout)
  report=subprocess.run(['perf','report','--stdio','--percent-limit','0.1','-i',str(data)],capture_output=True,text=True,check=True)
  (out/f'{rep}-{arm}.report').write_text(report.stdout)
  results.append({'rep':rep,'arm':arm,'command':cmd});(out/'manifest.json').write_text(json.dumps(manifest,indent=2)+'\n')
  print(rep,arm,'saved',flush=True)
0-base.report — top user-symbol cycle shares
    49.17%  perf_ab_latency  perf_ab_latency       [.] run_ops
     4.82%  perf_ab_latency  perf_ab_latency       [.] latency_end
     4.74%  perf_ab_latency  perf_ab_latency       [.] mi_free
     3.85%  perf_ab_latency  perf_ab_latency       [.] mi_page_fresh_alloc
     3.44%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_on_alloc
     1.98%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_stack_intern
     1.79%  perf_ab_latency  perf_ab_latency       [.] prof_free_record
     1.45%  perf_ab_latency  libc.so.6             [.] pthread_mutex_lock@@GLIBC_2.2.5
     1.30%  perf_ab_latency  perf_ab_latency       [.] mi_page_queue_find_free_ex
     1.22%  perf_ab_latency  perf_ab_latency       [.] mi_page_map_set_range_prim
     1.16%  perf_ab_latency  perf_ab_latency       [.] _mi_theap_collect_retired
     1.14%  perf_ab_latency  perf_ab_latency       [.] mi_find_page
     1.01%  perf_ab_latency  perf_ab_latency       [.] _mi_page_publish_retired
     0.91%  perf_ab_latency  perf_ab_latency       [.] _mi_page_malloc_zero
     0.87%  perf_ab_latency  perf_ab_latency       [.] _mi_page_retire
     0.75%  perf_ab_latency  perf_ab_latency       [.] mi_page_extend_free
     0.71%  perf_ab_latency  perf_ab_latency       [.] mi_malloc_generic_fallback
     0.67%  perf_ab_latency  perf_ab_latency       [.] _mi_bin
     0.65%  perf_ab_latency  libc.so.6             [.] __GI___pthread_mutex_unlock_usercnt
     0.55%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_on_free
     0.54%  perf_ab_latency  perf_ab_latency       [.] mi_page_free_list_extend
     0.49%  perf_ab_latency  perf_ab_latency       [.] _mi_page_free_collect
     0.46%  perf_ab_latency  perf_ab_latency       [.] mi_large_span_request
     0.41%  perf_ab_latency  perf_ab_latency       [.] mi_page_queue_remove
     0.35%  perf_ab_latency  perf_ab_latency       [.] mi_abandoned_page_try_reclaim
0-head.report — top user-symbol cycle shares
    45.56%  perf_ab_latency  perf_ab_latency       [.] run_ops
     5.05%  perf_ab_latency  perf_ab_latency       [.] mi_free
     4.65%  perf_ab_latency  perf_ab_latency       [.] latency_end
     3.68%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_on_alloc
     2.66%  perf_ab_latency  perf_ab_latency       [.] prof_free_record
     1.96%  perf_ab_latency  libc.so.6             [.] pthread_mutex_lock@@GLIBC_2.2.5
     1.88%  perf_ab_latency  perf_ab_latency       [.] mi_page_fresh_alloc
     1.80%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_stack_intern
     1.78%  perf_ab_latency  perf_ab_latency       [.] _mi_page_malloc_zero
     1.20%  perf_ab_latency  perf_ab_latency       [.] _mi_page_publish_retired
     0.87%  perf_ab_latency  libc.so.6             [.] __GI___pthread_mutex_unlock_usercnt
     0.86%  perf_ab_latency  perf_ab_latency       [.] mi_find_page
     0.86%  perf_ab_latency  perf_ab_latency       [.] mi_page_queue_find_free_ex
     0.79%  perf_ab_latency  perf_ab_latency       [.] _mi_theap_collect_retired
     0.77%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_on_free
     0.75%  perf_ab_latency  libc.so.6             [.] __GI___lll_lock_wait
     0.66%  perf_ab_latency  perf_ab_latency       [.] _mi_page_retire
     0.60%  perf_ab_latency  perf_ab_latency       [.] mi_page_extend_free
     0.57%  perf_ab_latency  perf_ab_latency       [.] _mi_page_free_collect
     0.51%  perf_ab_latency  perf_ab_latency       [.] mi_page_map_set_range_prim
     0.45%  perf_ab_latency  perf_ab_latency       [.] mi_malloc_generic_fallback
     0.45%  perf_ab_latency  perf_ab_latency       [.] _mi_bin
     0.35%  perf_ab_latency  perf_ab_latency       [.] mi_page_free_list_extend
     0.31%  perf_ab_latency  perf_ab_latency       [.] mi_large_span_request
     0.31%  perf_ab_latency  perf_ab_latency       [.] mi_page_queue_remove
1-base.report — top user-symbol cycle shares
    42.33%  perf_ab_latency  perf_ab_latency       [.] run_ops
     5.41%  perf_ab_latency  perf_ab_latency       [.] latency_end
     5.19%  perf_ab_latency  perf_ab_latency       [.] mi_free
     3.83%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_on_alloc
     2.79%  perf_ab_latency  perf_ab_latency       [.] mi_page_fresh_alloc
     2.59%  perf_ab_latency  perf_ab_latency       [.] prof_free_record
     2.02%  perf_ab_latency  libc.so.6             [.] pthread_mutex_lock@@GLIBC_2.2.5
     2.00%  perf_ab_latency  perf_ab_latency       [.] _mi_page_malloc_zero
     1.54%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_stack_intern
     1.38%  perf_ab_latency  perf_ab_latency       [.] _mi_page_publish_retired
     1.01%  perf_ab_latency  libc.so.6             [.] __GI___pthread_mutex_unlock_usercnt
     0.86%  perf_ab_latency  perf_ab_latency       [.] mi_page_map_set_range_prim
     0.85%  perf_ab_latency  perf_ab_latency       [.] mi_page_queue_find_free_ex
     0.84%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_on_free
     0.80%  perf_ab_latency  perf_ab_latency       [.] _mi_theap_collect_retired
     0.77%  perf_ab_latency  perf_ab_latency       [.] mi_find_page
     0.75%  perf_ab_latency  libc.so.6             [.] __GI___lll_lock_wait
     0.72%  perf_ab_latency  perf_ab_latency       [.] _mi_page_retire
     0.64%  perf_ab_latency  perf_ab_latency       [.] mi_page_extend_free
     0.57%  perf_ab_latency  perf_ab_latency       [.] _mi_page_free_collect
     0.43%  perf_ab_latency  perf_ab_latency       [.] mi_malloc_generic_fallback
     0.38%  perf_ab_latency  perf_ab_latency       [.] mi_bitmap_setN
     0.37%  perf_ab_latency  perf_ab_latency       [.] mi_page_free_list_extend
     0.36%  perf_ab_latency  perf_ab_latency       [.] mi_large_span_request
     0.34%  perf_ab_latency  perf_ab_latency       [.] _mi_bin
1-head.report — top user-symbol cycle shares
    44.07%  perf_ab_latency  perf_ab_latency       [.] run_ops
     5.65%  perf_ab_latency  perf_ab_latency       [.] mi_free
     5.10%  perf_ab_latency  perf_ab_latency       [.] latency_end
     3.52%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_on_alloc
     2.75%  perf_ab_latency  perf_ab_latency       [.] prof_free_record
     2.05%  perf_ab_latency  perf_ab_latency       [.] _mi_page_malloc_zero
     1.88%  perf_ab_latency  libc.so.6             [.] pthread_mutex_lock@@GLIBC_2.2.5
     1.83%  perf_ab_latency  perf_ab_latency       [.] mi_page_fresh_alloc
     1.54%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_stack_intern
     1.44%  perf_ab_latency  perf_ab_latency       [.] _mi_page_publish_retired
     0.95%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_on_free
     0.94%  perf_ab_latency  libc.so.6             [.] __GI___pthread_mutex_unlock_usercnt
     0.81%  perf_ab_latency  perf_ab_latency       [.] _mi_theap_collect_retired
     0.81%  perf_ab_latency  perf_ab_latency       [.] mi_find_page
     0.78%  perf_ab_latency  perf_ab_latency       [.] mi_page_queue_find_free_ex
     0.74%  perf_ab_latency  perf_ab_latency       [.] _mi_page_retire
     0.72%  perf_ab_latency  libc.so.6             [.] __GI___lll_lock_wait
     0.67%  perf_ab_latency  perf_ab_latency       [.] _mi_page_free_collect
     0.53%  perf_ab_latency  perf_ab_latency       [.] mi_page_extend_free
     0.47%  perf_ab_latency  perf_ab_latency       [.] mi_page_map_set_range_prim
     0.47%  perf_ab_latency  perf_ab_latency       [.] mi_malloc_generic_fallback
     0.42%  perf_ab_latency  perf_ab_latency       [.] mi_page_free_list_extend
     0.35%  perf_ab_latency  perf_ab_latency       [.] _mi_bin
     0.32%  perf_ab_latency  perf_ab_latency       [.] mi_large_span_request
     0.31%  perf_ab_latency  perf_ab_latency       [.] mi_page_queue_remove
2-base.report — top user-symbol cycle shares
    47.45%  perf_ab_latency  perf_ab_latency       [.] run_ops
     4.40%  perf_ab_latency  perf_ab_latency       [.] latency_end
     4.30%  perf_ab_latency  perf_ab_latency       [.] mi_page_fresh_alloc
     4.27%  perf_ab_latency  perf_ab_latency       [.] mi_free
     4.12%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_on_alloc
     2.41%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_stack_intern
     1.72%  perf_ab_latency  perf_ab_latency       [.] prof_free_record
     1.65%  perf_ab_latency  perf_ab_latency       [.] mi_page_map_set_range_prim
     1.54%  perf_ab_latency  perf_ab_latency       [.] mi_page_queue_find_free_ex
     1.42%  perf_ab_latency  libc.so.6             [.] pthread_mutex_lock@@GLIBC_2.2.5
     1.33%  perf_ab_latency  perf_ab_latency       [.] mi_find_page
     1.29%  perf_ab_latency  perf_ab_latency       [.] _mi_theap_collect_retired
     0.99%  perf_ab_latency  perf_ab_latency       [.] _mi_page_publish_retired
     0.99%  perf_ab_latency  perf_ab_latency       [.] mi_page_extend_free
     0.98%  perf_ab_latency  perf_ab_latency       [.] _mi_page_retire
     0.79%  perf_ab_latency  perf_ab_latency       [.] _mi_bin
     0.76%  perf_ab_latency  perf_ab_latency       [.] mi_malloc_generic_fallback
     0.73%  perf_ab_latency  perf_ab_latency       [.] _mi_page_malloc_zero
     0.71%  perf_ab_latency  perf_ab_latency       [.] mi_page_free_list_extend
     0.66%  perf_ab_latency  libc.so.6             [.] __GI___pthread_mutex_unlock_usercnt
     0.63%  perf_ab_latency  perf_ab_latency       [.] mi_large_span_request
     0.45%  perf_ab_latency  perf_ab_latency       [.] mi_page_queue_remove
     0.42%  perf_ab_latency  perf_ab_latency       [.] mi_option_is_enabled
     0.41%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_on_free
     0.39%  perf_ab_latency  perf_ab_latency       [.] _mi_theap_page_reclaim
2-head.report — top user-symbol cycle shares
    49.96%  perf_ab_latency  perf_ab_latency       [.] run_ops
     4.68%  perf_ab_latency  perf_ab_latency       [.] latency_end
     4.41%  perf_ab_latency  perf_ab_latency       [.] mi_free
     3.88%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_on_alloc
     2.99%  perf_ab_latency  perf_ab_latency       [.] mi_page_fresh_alloc
     2.26%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_stack_intern
     1.71%  perf_ab_latency  perf_ab_latency       [.] prof_free_record
     1.51%  perf_ab_latency  perf_ab_latency       [.] mi_find_page
     1.48%  perf_ab_latency  perf_ab_latency       [.] mi_page_queue_find_free_ex
     1.30%  perf_ab_latency  perf_ab_latency       [.] _mi_theap_collect_retired
     1.22%  perf_ab_latency  libc.so.6             [.] pthread_mutex_lock@@GLIBC_2.2.5
     0.97%  perf_ab_latency  perf_ab_latency       [.] mi_page_extend_free
     0.95%  perf_ab_latency  perf_ab_latency       [.] _mi_page_publish_retired
     0.85%  perf_ab_latency  libc.so.6             [.] __GI___pthread_mutex_unlock_usercnt
     0.81%  perf_ab_latency  perf_ab_latency       [.] _mi_bin
     0.80%  perf_ab_latency  perf_ab_latency       [.] _mi_page_retire
     0.80%  perf_ab_latency  perf_ab_latency       [.] mi_malloc_generic_fallback
     0.69%  perf_ab_latency  perf_ab_latency       [.] mi_large_span_request
     0.67%  perf_ab_latency  perf_ab_latency       [.] _mi_page_malloc_zero
     0.67%  perf_ab_latency  perf_ab_latency       [.] mi_page_map_set_range_prim
     0.49%  perf_ab_latency  perf_ab_latency       [.] _mi_prof_on_free
     0.48%  perf_ab_latency  perf_ab_latency       [.] mi_page_free_list_extend
     0.46%  perf_ab_latency  perf_ab_latency       [.] mi_page_queue_remove
     0.43%  perf_ab_latency  perf_ab_latency       [.] mi_abandoned_page_try_reclaim
     0.41%  perf_ab_latency  perf_ab_latency       [.] _mi_page_free_collect

Static follow-up

Actual old/new profiler executables differ. The listed profiler and generic-allocation symbol sizes match across old/new candidates; fresh-page size is also 1,218 bytes for both, versus 1,162 at frozen baseline. Source isolation changes only the byte predicate in C. Equal symbol sizes do not prove equal instruction sequences, call graphs or performance. This inspection provides no basis for dismissing the p99 failure or changing source before CI attribution.

{
  "symbols": {
    "base": {
      "_mi_prof_stack_capture": {
        "address": "0000000000039990",
        "size": 89
      },
      "_mi_prof_on_alloc": {
        "address": "000000000002f700",
        "size": 855
      },
      "mi_page_queue_find_free_ex": {
        "address": "00000000000230c0",
        "size": 959
      },
      "_mi_malloc_generic": {
        "address": "0000000000023e50",
        "size": 984
      },
      "_mi_prof_stack_intern": {
        "address": "00000000000399f0",
        "size": 984
      },
      "run_ops": {
        "address": "00000000000050d0",
        "size": 1051
      },
      "mi_page_fresh_alloc": {
        "address": "0000000000022050",
        "size": 1162
      }
    },
    "old": {
      "_mi_prof_stack_capture": {
        "address": "00000000000399d0",
        "size": 89
      },
      "_mi_prof_on_alloc": {
        "address": "000000000002f740",
        "size": 855
      },
      "mi_page_queue_find_free_ex": {
        "address": "0000000000023100",
        "size": 959
      },
      "_mi_malloc_generic": {
        "address": "0000000000023e90",
        "size": 984
      },
      "_mi_prof_stack_intern": {
        "address": "0000000000039a30",
        "size": 984
      },
      "run_ops": {
        "address": "00000000000050d0",
        "size": 1051
      },
      "mi_page_fresh_alloc": {
        "address": "0000000000022050",
        "size": 1218
      }
    },
    "head": {
      "_mi_prof_stack_capture": {
        "address": "00000000000399d0",
        "size": 89
      },
      "_mi_prof_on_alloc": {
        "address": "000000000002f740",
        "size": 855
      },
      "mi_page_queue_find_free_ex": {
        "address": "0000000000023100",
        "size": 959
      },
      "_mi_malloc_generic": {
        "address": "0000000000023e90",
        "size": 984
      },
      "_mi_prof_stack_intern": {
        "address": "0000000000039a30",
        "size": 984
      },
      "run_ops": {
        "address": "00000000000050d0",
        "size": 1051
      },
      "mi_page_fresh_alloc": {
        "address": "0000000000022050",
        "size": 1218
      }
    }
  },
  "actual_executable_sha256": {
    "old": {
      "perf_ab": "9249fe01572f8989a024d0f6ee72b6ded65ea6feff2343e2274ae800b23a5cc3",
      "perf_ab_latency": "43e680d121436d9e4d8f61d66e11a0d162c166a8d7bccaa69209c264355f2022"
    },
    "head": {
      "perf_ab": "ef1f0135f09bec19ff3cba3c26a211359243cbcb40825818f9bddb3c512771ba",
      "perf_ab_latency": "1f432e1f3a4c857bde03dd44859eac39b92a510806cceb4841087d16e98d7a8e"
    }
  }
}

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

Exact random-large/8 follow-up: still INCONCLUSIVE

Run 39526533882, frozen ee3623f6b7a485754a0a0aa3462d8cba43390e8b → current 15bfcba0c3e59aea6b22dd04f91db9d95cd63a0a, completed on AMD EPYC 9V45 96-Core Processor / four logical CPUs, with 120 actual samples each for base, head and null. Overall remains INCONCLUSIVE on throughput and after-drain RSS; CPU, faults, p99 and peak RSS PASS. Zero memory credit.

base ee3623f6b7a4 head 15bfcba0c3e5 on AMD EPYC 9V45 96-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
random-large/8 0% INCONCLUSIVE +0.1% [-0.8, +0.9] / 0.7% INCONCLUSIVE -0.1% [-1.2, +0.3] / 1.3% PASS -0.2% [-1.1, +0.4] / 0.4% PASS -0.1% [-1.8, +1.8] / 2.4% PASS -0.0% [-1.1, +1.1] / 1.1% PASS +0.0% [+0.0, +0.3] / 0.0% INCONCLUSIVE

All four actual timing/latency executable hashes verify and are byte-identical to the original new-candidate random batch. Exact row parameters and balanced arm/sampling protocols match. The earlier Intel 8573C run remains INCONCLUSIVE on faults/peak; it is not pooled with or reclassified by this EPYC result. Eight workers on four logical CPUs is 2:1 oversubscription. No trace, threshold or sampling change.

No further acceptance rerun of this already rejected candidate is being launched while the real profiler p99 failure is under source attribution. This control remains unresolved; a successful workflow conclusion is not a performance PASS. PR #609 remains draft and the failed profiler comparison remains in the record.

{
  "base/plain/perf_ab": "7f0c70ce2450dfa09126c1674bddab53187614f12d5bd648b4fda33ccbd3c712",
  "base/plain/perf_ab_latency": "ef2d5d273681a473e253d1dd2080307f1ae9f09d34977bda2ad4cab9cbfe7823",
  "head/plain/perf_ab": "9ee3617e371e93195de1374ef403455e7ae660848d689ce4bf0ae0ce2f8565b1",
  "head/plain/perf_ab_latency": "c14766d194fd7f3c07652b148d1de18e3e4db90b4414bd563398f5f44ee9b982"
}
Full-precision cells
{
  "actual_counts": {
    "base": 120,
    "head": 120,
    "null": 120
  },
  "saved_percent": 0.0,
  "cells": [
    {
      "metric": "ops/s",
      "base_median": 1035977.7,
      "head_median": 1039670.1,
      "change": [
        0.07535061440704756,
        -0.7616121749211847,
        0.9318871090325886
      ],
      "regression": [
        -0.9318871090325886,
        0.7616121749211847
      ],
      "allowance": 0.7041781786584234,
      "verdict": "INCONCLUSIVE"
    },
    {
      "metric": "cpu s",
      "base_median": 1.2324000000000002,
      "head_median": 1.2218,
      "change": [
        -0.11101859806526174,
        -1.1700273836196167,
        0.3362001563721808
      ],
      "regression": [
        -1.1700273836196167,
        0.3362001563721808
      ],
      "allowance": 1.2820512820512813,
      "verdict": "PASS"
    },
    {
      "metric": "minor faults",
      "base_median": 263.0,
      "head_median": 262.0,
      "change": [
        -0.18587360594795538,
        -1.1111111111111112,
        0.3861003861003861
      ],
      "regression": [
        -1.1111111111111112,
        0.3861003861003861
      ],
      "allowance": 0.3861003861003861,
      "verdict": "PASS"
    },
    {
      "metric": "p99 operation ns",
      "base_median": 14848.0,
      "head_median": 14917.5,
      "change": [
        -0.10560582149508456,
        -1.8236444941307994,
        1.8134527852033813
      ],
      "regression": [
        -1.8236444941307994,
        1.8134527852033813
      ],
      "allowance": 2.4077350859453994,
      "verdict": "PASS"
    },
    {
      "metric": "peak RSS MiB",
      "base_median": 182.380859375,
      "head_median": 180.45703125,
      "change": [
        -0.0032476670924719074,
        -1.1469596813028062,
        1.0516976477462077
      ],
      "regression": [
        -1.1469596813028062,
        1.0516976477462077
      ],
      "allowance": 1.1141827929965653,
      "verdict": "PASS"
    },
    {
      "metric": "RSS 0.5 s after drain MiB",
      "base_median": 20.578125,
      "head_median": 20.63671875,
      "change": [
        0.018982536066818528,
        0.0,
        0.28468400075915734
      ],
      "regression": [
        0.0,
        0.28468400075915734
      ],
      "allowance": 0.018986140117714066,
      "verdict": "INCONCLUSIVE"
    }
  ]
}

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

Three-control replacement: CI timeout, no performance verdict

Run 39521491231 is authoritatively completed / cancelled. Its check annotation states: "The job has exceeded the maximum execution time of 1h30m0s". Job started 2026-10-21 00:28:09 UTC, completed 01:58:32 UTC. It used frozen baseline ee3623f6b7a485754a0a0aa3462d8cba43390e8b, candidate 15bfcba0c3e59aea6b22dd04f91db9d95cd63a0a, exact small/8 and Larson/8 plain/chart rows, 60-to-120 samples/arm, null and sanity enabled.

Both always-run artifact steps executed, but their logs say no files were found for perf-ab-builds or perf-ab-samples.json; the run artifacts API returns an empty list. Logs contain no repetition progress, actual sample counts or measured executable hashes. No PASS, FAIL, CPU identity, hang diagnosis or completed repetition count can be inferred from this run. The three controls remain unverified. The job was not cancelled or restarted by the agent; it reached its configured CI limit.

This exposes a harness gap: raw data is written only at completion/sanity failure and exact builds are copied in finally; forced CI termination can bypass both. The next scoped fix will save incomplete atomic checkpoints after completed child measurements, emit parent progress, and preserve the built executables before starting measurements. Incomplete checkpoints must remain ineligible for pooling or a regression verdict. No allocator code, workload trace, threshold, sampling rule or CI time limit will be changed to make this run pass.

The same three-row group will not be restarted. Future control measurements will use exact individual rows after evidence preservation is fixed and the candidate's separate profiler p99 failure is addressed. That failure remains visible in the current candidate ledger; PR #609 remains draft and unaccepted.

Local-Gate: v1 tree=512cd0a71b7c127b40b0e2ffd9970635c4e40a68 secs=37 lanes=lint:run
Ci-Attestation: {"at":1792548431,"gate":"python/all/lint","host":"linux-x86_64","key":"a41da66c1fe96de5db75bca08c56bace07da7be78b8f505267c7d76be42fb4c5","lane":"lint","parents":["15bfcba0c3e59aea6b22dd04f91db9d95cd63a0a"],"secs":37,"stamp":"bbb8e91cd2ac15392b4353e4a67ec167","tree":"512cd0a71b7c127b40b0e2ffd9970635c4e40a68","v":1,"via":"run"}
@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

Profiler source isolation: byte-threshold step remains INCONCLUSIVE

Run 39526717341 compares old prefix-reuse candidate 9b67e411fc8c8899e07e05d528a4c5b37ab20417 → new byte-threshold candidate 15bfcba0c3e59aea6b22dd04f91db9d95cd63a0a, exact original profiler row, 120 actual samples/arm, on AMD EPYC 7763 64-Core Processor / four logical CPUs. CPU, throughput, faults and peak RSS PASS; p99 and drain RSS remain INCONCLUSIVE, with zero memory credit.

base 9b67e411fc8c head 15bfcba0c3e5 on AMD EPYC 7763 64-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 (profiler on) 0% INCONCLUSIVE +0.2% [-0.1, +0.4] / 0.6% PASS -0.1% [-0.3, +0.2] / 0.5% PASS +0.1% [-0.4, +0.3] / 0.4% PASS +1.0% [-2.0, +6.4] / 6.1% INCONCLUSIVE -0.1% [-0.3, +0.1] / 0.2% PASS +0.2% [-0.0, +1.5] / 0.3% INCONCLUSIVE

All four actual timing/latency binary hashes verify. The attribution baseline matches the preserved old-candidate profiler binary byte for byte; the head matches the actual failed new-candidate profiler binary. Row parameters, balanced arm positions and sampling definition are unchanged. Eight workers on four logical CPUs is 2:1 oversubscription.

This does not establish the cause of the full candidate p99 regression and is not acceptance against frozen ee3623f6. The failed full comparison ran on EPYC 9V45; this source-step comparison ran on EPYC 7763. Results from different hosts/models are not pooled, added into a decomposition, or used to waive the +11.3% p99 FAIL. PR #609 remains draft and unaccepted.

{
  "base/pprof/perf_ab": "9249fe01572f8989a024d0f6ee72b6ded65ea6feff2343e2274ae800b23a5cc3",
  "base/pprof/perf_ab_latency": "43e680d121436d9e4d8f61d66e11a0d162c166a8d7bccaa69209c264355f2022",
  "head/pprof/perf_ab": "ef1f0135f09bec19ff3cba3c26a211359243cbcb40825818f9bddb3c512771ba",
  "head/pprof/perf_ab_latency": "1f432e1f3a4c857bde03dd44859eac39b92a510806cceb4841087d16e98d7a8e"
}
Full-precision cells
{
  "actual_counts": {
    "base": 120,
    "head": 120,
    "null": 120
  },
  "saved_percent": 0.0,
  "cells": [
    {
      "metric": "ops/s",
      "base_median": 4197780.0,
      "head_median": 4204376.449999999,
      "change": [
        0.16305753947899565,
        -0.06370095224841385,
        0.40023848597696116
      ],
      "regression": [
        -0.40023848597696116,
        0.06370095224841385
      ],
      "allowance": 0.5953655172004731,
      "verdict": "PASS"
    },
    {
      "metric": "cpu s",
      "base_median": 2.9408,
      "head_median": 2.9341,
      "change": [
        -0.0878065580051814,
        -0.3093788882536865,
        0.22150281138183509
      ],
      "regression": [
        -0.3093788882536865,
        0.22150281138183509
      ],
      "allowance": 0.4663829787234058,
      "verdict": "PASS"
    },
    {
      "metric": "minor faults",
      "base_median": 1626.0,
      "head_median": 1625.0,
      "change": [
        0.061595324598065375,
        -0.36968576709796674,
        0.30959752321981426
      ],
      "regression": [
        -0.36968576709796674,
        0.30959752321981426
      ],
      "allowance": 0.3708281829419036,
      "verdict": "PASS"
    },
    {
      "metric": "p99 operation ns",
      "base_median": 38362.0,
      "head_median": 39779.0,
      "change": [
        0.956764272117955,
        -2.0434667373442883,
        6.3519620811287485
      ],
      "regression": [
        -2.0434667373442883,
        6.3519620811287485
      ],
      "allowance": 6.073896863370547,
      "verdict": "INCONCLUSIVE"
    },
    {
      "metric": "peak RSS MiB",
      "base_median": 73.64453125,
      "head_median": 73.603515625,
      "change": [
        -0.06267554273962415,
        -0.31027162375483097,
        0.06187799721549012
      ],
      "regression": [
        -0.31027162375483097,
        0.06187799721549012
      ],
      "allowance": 0.23923444976076555,
      "verdict": "PASS"
    },
    {
      "metric": "RSS 0.5 s after drain MiB",
      "base_median": 54.341796875,
      "head_median": 54.75390625,
      "change": [
        0.18931634779456225,
        -0.01415227851684121,
        1.5011132658191482
      ],
      "regression": [
        -0.01415227851684121,
        1.5011132658191482
      ],
      "allowance": 0.3385765676800451,
      "verdict": "INCONCLUSIVE"
    }
  ]
}

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

Forced-termination evidence preservation fixed in draft PR #609

Pushed harness commit d2b31c55c3b5673c307a02d0e848278b4e744f20 after the 90-minute controls timeout lost all raw data and builds.

  • Save an atomic JSON checkpoint before measurements and after every completed timing/latency pair. Checkpoints have complete: false; only normal completion sets complete: true. Existing incomplete-data rejection prevents pooling or a regression verdict.
  • Preserve exact built timing/latency executables and their hashes before the first child measurement. Copy errors stop before measurement. This survives forced termination that skips Python finally.
  • Flush parent progress identifying repetition, workload and arm. Throughput/RSS and latency executables, workload traces, seed/sampling rules, acceptance thresholds and the 90-minute CI limit are unchanged.
  • Four new interruption/pre-copy cases failed before implementation. All 91 focused perf-ab/dispatch/latency tests passed afterward; ruff passed. The same sole primary reviewer returned clean. Required local gate: 45/45 PASS in 37 seconds, attested on the pushed head.

git diff 15bfcba0..d2b31c55 -- src include test CMakeLists.txt rust is empty. The measured allocator candidate remains 15bfcba0, whose full platform CI was 104 successes/five intentional skips. New-head CI is now running; those previous results are not claimed as new-head checks.

Allocator acceptance remains failed on profiler p99. The byte-threshold source step remains INCONCLUSIVE at 120/arm on EPYC 7763 and does not explain or waive the full-candidate EPYC 9V45 failure. The two earlier source steps are now isolated with the fixed harness, exact original profiler row, 60-to-120 samples/arm and unchanged null/sanity gates:

  1. Frozen baseline ee3623f6 → exact-fit-only a9d7d2e3: 39531382092.
  2. Exact-fit-only a9d7d2e3 → mapped-prefix candidate 9b67e411: 39531384408.

These are source attribution, not retries of the failed fixed-baseline candidate for acceptance. No cross-host pooling or addition of per-step deltas is planned. The old three-control group will not be restarted; its controls remain unverified until future exact individual-row measurements after the profiler failure is addressed. Random/8 remains INCONCLUSIVE in both independent runs. The existing short-generation follow-up 39526576845 remains live. No memory policy was changed.

Local-Gate: v1 tree=05076c3b29c48822ea0e6acd6f881407cbe6276d secs=32 lanes=lint:run
Ci-Attestation: {"at":1792549262,"gate":"python/all/lint","host":"linux-x86_64","key":"63d62f4417f1c2377d1ff20ec0a0df7c310ec4da1e1527e337923837406dc40e","lane":"lint","parents":["0331d770f2db9d5815682205fc16ca3173d79b39"],"secs":32,"stamp":"fcf15aa913aa910ac08f28673454ecc5","tree":"05076c3b29c48822ea0e6acd6f881407cbe6276d","v":1,"via":"run"}
@zackees

zackees commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner Author

New word-hash CPU prototype: correctness checked, performance pending

New candidate 6f7be38f43be01f7017a1c7432421e97cd3f30b5 is pushed in draft PR #609. C commit 0331d770 changes only src/profile-stack.c; Rust regeneration is a separate commit. The earlier byte-threshold candidate 15bfcba0 remains rejected for the recorded profiler p99 FAIL. No result for it is reclassified as acceptance of this prototype.

The recorded hypothesis comes from post-processing six existing local cycle recordings, not new local timing: 39.1–55.7% of stack-intern samples lie in the byte-FNV loop under prof_lock. This is a critical-section CPU target, not an established cause of the CI p99 regression.

The prototype uses mimalloc's existing stateless _mi_random_shuffle on each complete PC word. Hash collisions still require equal depth and every PC. Layouts, capture logic, sampling decisions/PRNG state, arena allocation and record accounting are unchanged. Internal hash values and table/export ordering can change; full PC-stack equality and exported counts retain their meaning. There is no added memory structure, discard policy, required dependency or threshold change. The reviewer checked unsigned 32/64-bit behavior and table insertion/growth/deletion/rehashing.

Verification so far:

  • Debug profiler/pprof tests: 10/10 PASS, 22.60 seconds, including race, accumulation, dump and heap destruction.
  • soldr cargo run -p xtask -- amalgamate-c and soldr cargo run -p xtask -- check: PASS; C/Rust commits remain separate.
  • Same sole primary review: clean; one review agent total.
  • Required local gate: 45/45 PASS in 32 seconds, attested on pushed head 6f7be38f.
  • New-head platform CI is running. Earlier-source platform results do not constitute new-head validation.

First performance comparisons use the exact original profiler row, 60-to-120 samples per arm, balanced order, null/sanity enabled and unchanged thresholds:

  1. Fixed-baseline acceptance: ee3623f6 → 6f7be38f, 39532706232.
  2. Hash-step attribution only: 15bfcba0 → 6f7be38f, 39532708955.

No PASS is claimed yet. The first comparison must pass before broadening performance validation. Source-step results on different hosts will not be pooled or added. Older exact-fit/prefix profiler comparisons 39531382092/39531384408 continue on their original refs; the old chart-short follow-up 39526576845 has now finished INCONCLUSIVE on faults at 240/arm (other five metrics PASS). This is evidence for old 15, not new-hash acceptance. The lost-controls timeout and both random/8 INCONCLUSIVE results remain recorded. No new memory policy is accepted, and #585 remains incomplete.

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

Previous-candidate chart-short follow-up: faults still INCONCLUSIVE at maximum budget

Run 39526576845, frozen ee3623f6b7a485754a0a0aa3462d8cba43390e8b → previous byte-threshold candidate 15bfcba0c3e59aea6b22dd04f91db9d95cd63a0a, completed on AMD EPYC 9V45 96-Core Processor / four logical CPUs, with 240 actual samples each for base, head and null. Overall remains INCONCLUSIVE on minor faults. Throughput, CPU, p99, peak RSS and drain RSS PASS. Zero memory credit.

base ee3623f6b7a4 head 15bfcba0c3e5 on AMD EPYC 9V45 96-Core Processor, 120 reps (up to 240); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class-ephemeral/8 short generations (chart build) 0% INCONCLUSIVE +1.0% [+0.7, +1.4] / 0.5% PASS -0.9% [-1.2, -0.4] / 0.7% PASS +0.0% [-0.4, +0.8] / 0.4% INCONCLUSIVE -0.5% [-1.0, +0.0] / 0.5% PASS +0.0% [+0.0, +0.0] / 2.5% PASS +0.0% [+0.0, +0.0] / 2.5% PASS

All four actual timing/latency executable hashes verify and are byte-identical to the original chart batch. Row parameters, sampling and arm-order protocol match. The earlier EPYC 7763 short-generation result at 120/arm remains INCONCLUSIVE on faults/peak/drain; different hosts/models are not pooled or reclassified. Eight workers on four logical CPUs is 2:1 oversubscription.

This is evidence for old 15bfcba, not the newly pushed word-hash prototype 6f7be38. No further acceptance retry of old 15 is planned: its profiler p99 FAIL and unresolved controls remain visible. New prototype acceptance begins with exact profiler run39532706232; no PASS is yet claimed for it. No trace, sampling, threshold or memory-policy change.

{
  "base/chart/perf_ab": "306d5c8a089d29a7ff276df621226fe2d08ffe7013669627f38dc5c0020d33f7",
  "base/chart/perf_ab_latency": "2f883b21f4d48e9e1b404c3110ca058902c68cba23e99a82edd58026c7495090",
  "head/chart/perf_ab": "51b9dde3f2ce5e446eb8443af7ab56136af1b146bbdb61acad221d2a652a0bff",
  "head/chart/perf_ab_latency": "41d91d58314a4ca202ec6b0e4a8a3d62995a39a3aac891edab560f22c0500240"
}
Full-precision cells
{
  "actual_counts": {
    "base": 240,
    "head": 240,
    "null": 240
  },
  "saved_percent": 0.0,
  "cells": [
    {
      "metric": "ops/s",
      "base_median": 6833744.35,
      "head_median": 6919946.55,
      "change": [
        1.035337311654582,
        0.6967528846682861,
        1.443178132619091
      ],
      "regression": [
        -1.443178132619091,
        -0.6967528846682861
      ],
      "allowance": 0.49396029343884473,
      "verdict": "PASS"
    },
    {
      "metric": "cpu s",
      "base_median": 0.4704,
      "head_median": 0.46665,
      "change": [
        -0.8615666670377912,
        -1.1967247533067358,
        -0.42698548249359564
      ],
      "regression": [
        -1.1967247533067358,
        -0.42698548249359564
      ],
      "allowance": 0.7380073800738038,
      "verdict": "PASS"
    },
    {
      "metric": "minor faults",
      "base_median": 251.0,
      "head_median": 252.0,
      "change": [
        0.0,
        -0.39370078740157477,
        0.7905138339920948
      ],
      "regression": [
        -0.39370078740157477,
        0.7905138339920948
      ],
      "allowance": 0.4016064257028112,
      "verdict": "INCONCLUSIVE"
    },
    {
      "metric": "p99 operation ns",
      "base_median": 1943.0,
      "head_median": 1933.0,
      "change": [
        -0.5146816721671118,
        -1.0240655401945724,
        0.0
      ],
      "regression": [
        -1.0240655401945724,
        0.0
      ],
      "allowance": 0.5282620179609087,
      "verdict": "PASS"
    },
    {
      "metric": "peak RSS MiB",
      "base_median": 72.73046875,
      "head_median": 74.7265625,
      "change": [
        0.0,
        0.0,
        0.005371146202599634
      ],
      "regression": [
        0.0,
        0.005371146202599634
      ],
      "allowance": 2.5404386226059343,
      "verdict": "PASS"
    },
    {
      "metric": "RSS 0.5 s after drain MiB",
      "base_median": 72.73046875,
      "head_median": 74.7265625,
      "change": [
        0.0,
        0.0,
        0.005371146202599634
      ],
      "regression": [
        0.0,
        0.005371146202599634
      ],
      "allowance": 2.5404386226059343,
      "verdict": "PASS"
    }
  ]
}

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

New-head correctness gate failure: musl dead-thread page reuse

Head 6f7be38f43be01f7017a1c7432421e97cd3f30b5, c-unit run39532631746 / run-linux, failed the musl bundle. Downloaded JUnit identifies test-purge-holes / dead-thread-page-is-reused: thread B got a different page metadata address (0x36481406f00 vs 0x36481407b00) and created one page in the 400-KiB request class instead of reusing A's. Musl JUnit:57tests/1failure; inside this executable33casespassed/1failed. The new candidate is not correctness-accepted.

This is distinct from the earlier GCC ASan idle-release failure. In this run the idle/reserved/slack checks all passed at222/205/224ms. No retry, timeout relaxation, assertion weakening or RUN_SERIAL change is being made. Cause is not yet established; profiling/hash code being the latest change does not prove causation. The existing four independent performance comparisons continue unchanged.

Next diagnostic: use the actual downloaded musl CI test executable in Alpine3.20, matching the bundle environment, on a declared maximum10-run local plan per source and stop on the first failure. Compare prior15bfcba0's actual CI binary, then frozen baseline source if needed. Local probes are reproduction/attribution only, never gate acceptance.

Actual failed bundle executable SHA256: 588f9d4fedc5ac99b78dc18297a12a142faff74a1ae2db413d7bdcaa0c90f705. Manifest case:

{
  "name": "test-purge-holes",
  "argv": [
    "${BUNDLE}/mimalloc-test-purge-holes"
  ],
  "env": {},
  "cwd": "${BUNDLE}",
  "timeout": 600.0,
  "expect_nonzero": false,
  "expect_text": null,
  "forbid_text": null,
  "labels": [],
  "serial": false
}
Full failed testcase output retained from JUnit
purge_holes is ON, os page size is 4096
test: os-page-arithmetic...  (16586 cases) ok.
test: survivors-16...  ok.
test: survivors-64...  ok.
test: survivors-256...  ok.
test: survivors-1024...  ok.
test: survivors-4096...  ok.
test: survivors-8192...  ok.
test: survivors-16384...  ok.
test: survivors-65536...  ok.
test: purging-actually-happened...  ok.
test: small-blocks-are-eligible...  ok.
test: medium-page-is-eligible...  ok.
test: churn-no-aliasing...  (10199 allocs, 9801 frees, 207 purges) ok.
test: aligned-16k...  ok.
test: page-lifecycle...  ok.
test: abandoned-pages...  ok.
test: large-pages...  (eligible; 0 ineligible pages, 0 bytes, 0 of them free) ok.
test: report-pinned-ospages...  [curve 4K=0 8K=0 16K=0 32K=0 64K=0] (bs=512: 241 pinned OS pages hold 241 live blocks and pin 915968 free bytes; 7x amplification) ok.
test: report-discardable-ospages...  [curve 4K=983040 8K=917504 16K=786432 32K=524288 64K=0] (bs=512: 983040 bytes discarded, 56832 still pinned) ok.
test: report-is-read-only-on-abandoned...  (8 abandoned pages reached) ok.
test: unformed-tail...  ok.
test: unformed-tail-freed...  ok.
test: sweep-skips-unchanged-pages...  (sweep 1: 4092 blocks walked; sweeps 2+3: 0; 70 pages skipped) ok.
test: report-is-read-only...  
holes report: os page = 4096 bytes, 18 pages (0 ineligible, 0 never madvise-able)
  discardable bytes vs hypothetical OS page size (nothing is discarded to measure this):
    @  4096 :       0.88 MB   <-- this machine's OS page size
    @  8192 :       0.80 MB
    @ 16384 :       0.73 MB
    @ 32768 :       0.50 MB
    @ 65536 :       0.00 MB
  live 0.04 MB, free 0.96 MB
  21 pinned OS pages (>= 1 live block): 0.04 MB live + 0.03 MB free trapped in them
  memory partition (walking the arena bitmaps):
    in pages           :       1.53 MB  = live 0.04 + in-page free 0.96 + page overhead 0.52 (header/unformed)
    arena slack        :      14.75 MB  (in NO page, touched at least once; 0.00 MB of it is queued for purge and so certainly still resident)
    arena meta (ROUGH) :       3.06 MB  (arena bitmaps only; excludes the mi_meta heaps)
    ---- ever-touched  :      19.35 MB  (in-pages + slack + meta; the ceiling on what we can be paying for)
    reserved 1024.00 MB, slices_committed 1023.50 MB -- NOTE: on POSIX every slice is marked committed at reserve
      time and a reset-purge never clears it, so slices_committed is address space, NOT residency.
      'arena slack' is an UPPER bound: a slice purged earlier still reads as dirty here.
      'in pages' misses pages owned by OTHER threads' theaps -- this walk cannot read them.
ATTR bin block_size=1024 pages=16 empty=0 extent=1048576 used_live=16384 used_free=86016 used_unformed=0 used_slack=16384 empty_live=0 empty_free=0 empty_unformed=0 empty_slack=0 formed_used=1032192 formed_empty=0 reserved_used=1032192 reserved_empty=0
ATTR bin block_size=8192 pages=1 empty=0 extent=65536 used_live=8192 used_free=0 used_unformed=0 used_slack=0 empty_live=0 empty_free=0 empty_unformed=0 empty_slack=0 formed_used=8192 formed_empty=0 reserved_used=65536 reserved_empty=0
ATTR bin block_size=24576 pages=1 empty=0 extent=524288 used_live=24576 used_free=0 used_unformed=0 used_slack=4096 empty_live=0 empty_free=0 empty_unformed=0 empty_slack=0 formed_used=24576 formed_empty=0 reserved_used=516096 reserved_empty=0
ATTR total pages=18 used_pages=18 empty_pages=0 extent=1638400 used_live=49152 used_free_formed=86016 used_unformed=0 used_slack=20480 empty_live=0 empty_free_formed=0 empty_unformed=0 empty_slack=0
  arena layout (#519): 1 arenas, 32 chunks of 512 slices, 3.06 MB arena meta
    in use 1.68 MB; free: queued 0.00 MB + aged 0.00 MB (resident), dirty not queued 14.75 MB (upper bound), fresh 1004.50 MB
    class    chunks     in_use      fresh free_dirty     queued       aged   (MB per chunk size class)
    none         32       1.68    1004.50      14.75       0.00       0.00
    resident (mincore): in use 0.27 MB, fresh 0.00 MB, free_dirty 0.00 MB, queued 0.00 MB, aged 0.00 MB
    runs per class and kind (a run never crosses a chunk): count, longest, then length:count by power of two
    none    in_use      runs      1  longest   27 | 16:1
    none    fresh       runs     32  longest  512 | 128:1 512:31
    none    free_dirty  runs      1  longest  236 | 128:1
size_class    pages    live_MB    free_MB   undiscardable_MB  discarded_MB avg_live_blocks_per_pinned_ospage
      1024       16       0.01       0.96               0.08          0.88          1.00
      8192        1       0.00       0.00               0.00          0.00          1.00
     24576        1       0.02       0.00               0.00          0.00          1.00
     TOTAL       18       0.04       0.96               0.08          0.88          1.00
  of undiscardable: 0.04 MB lies in a partial OS page (page header / past capacity)
  free and discardable but not discarded: 0.00 MB;  blocks not formed yet: 0.52 MB (of which discarded: 0.52 MB)
  block_size 1024: 0.08 MB undiscardable over 13 pinned OS pages; live blocks per pinned OS page:  1: 13  2: 0  3-4: 0  5-8: 0  9+: 0
  (a live block straddling two pinned OS pages counts in both; abandoned pages are only reached when they are in the arena's abandoned map)
events (#573): large_page_request=6 retired_publish=3 retired_unpublish=3 page_map_register=502 arena_page_alloc=534 arena_page_free=482 large_repurpose_none=6 large_retire_expired=3
arena claims (#517): resident_first=2/32 plain_reused=460/785 plain_fresh=40/263 (claims/slices)
ok.
test: option-off-is-noop...  ok.
test: sweep-does-not-unpurge-on-collect...  ok.
test: sweep-full-every-bounds-a-missed-hole...  (hidden hole found by the full sweep after 4 parks) ok.
test: owner-sweeps-are-paced...  ok.
test: new-thread-does-not-sweep-at-once...  (discards by a short-lived thread: 0) ok.
test: idle-thread-releases-large-page...  (freed block: 0 of 64 OS pages resident after 222 ms; bound 500 ms) ok.
test: reserved-page-is-released...  (reserved page: 0 of 64 OS pages resident after 205 ms; bound 500 ms) ok.
test: retired-page-slack-released...  (slack: 0 of 64 OS pages resident after 224 ms) ok.
test: dead-thread-page-is-reused...  
  thread B's block is not in the page thread A left behind (0x36481406f00 vs 0x36481407b00)

  thread B created 1 page(s) in the 400 KiB class instead of reusing A's

  FAILED: dead-thread-page-is-reused: /__w/mimalloc-pprof/mimalloc-pprof/test/test-purge-holes.c:2305
test: large-holes-without-idle...  ok.
holes: 116224000 bytes discarded in total over 5480 discards / 4118 reuses; 22 pages freed by the sweep; 0 bytes still discarded
holes: the last sweep could not touch 0 pages (0 bytes, of which 0 bytes free)
holes: unformed tail: 14077952 bytes discarded in total over 73 discards / 478 reuses; 0 bytes still discarded
(malloc is overridden: not checking that no hole is outstanding at exit)
subproc 0
 pages           peak       total     current       block      total#   
  touched   :     0           0           0                                ok
  pages     :    97         600           1      
  abandoned :    93           1.2 K       0      
  reclaima  :    33      
  reclaimf  :     1.2 K  
  reabandon :    18      
  waits     :     0      
  extended  :     0      
  retire    :     0      
  searches  :     1.0 avg

 arenas          peak       total     current       block      total#   
  reserved  :     1.0 GiB     1.0 GiB     1.0 GiB                          
  committed :    16.6 MiB    16.6 MiB    16.6 MiB                          
  reset     :     0      
  purged    :   233.0 MiB
  arenas    :     1      
  rollback  :     0      
  mmaps     :    10      
  commits   :    10      
  resets    :     0      
  purges    :     5.7 K  
  guarded   :     0      
  theaps    :     2          15           1      
  heaps     :     1           1           1      
  heap waits:     0      

 process         peak       total     current       block      total#   
  threads   :     2          15           1      
  numa nodes:     1
  elapsed   :     3.035 s
  process   : user: 0.768 s, system: 0.083 s, faults: 0, peak rss: 22.4 MiB, peak commit: 16.6 MiB

holes: 0 bytes / 0 blocks discarded now, 116224000 bytes total, 5480 discards, 4118 reuses, 22 pages freed by the sweep
holes: 0 ineligible pages in the last sweep (0 bytes, of which 0 bytes are free)
holes: the sweeps walked 49996 free blocks and skipped 29964 unchanged pages (4 full sweeps)
holes: unformed tail 0 bytes discarded now, 14077952 bytes total, 73 discards, 478 reuses
events (#573): large_page_request=35 retired_publish=7008589 retired_unpublish=7008589 page_map_register=602 arena_page_alloc=635 arena_page_free=599 large_repurpose_none=33 large_retire_expired=5
arena claims (#517): resident_first=9/144 plain_reused=553/1200 plain_fresh=40/263 (claims/slices)


---------------------------------------------
succeeded: 33
failed   : 1


@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

#585: new word-hash fixed-baseline profiler row PASS; correctness remains unresolved

Run 39532706232 completed. Exact source pair: ee3623f6b7a485754a0a0aa3462d8cba43390e8b → 6f7be38f43be01f7017a1c7432421e97cd3f30b5. Actual counts are 60 base / 60 head / 60 null. Original trace, null/sanity gates, balanced rotating/reversed arm positions, separate latency replay and unchanged thresholds. No cross-CPU pooling.

base ee3623f6b7a4 head 6f7be38f43be on INTEL(R) XEON(R) PLATINUM 8573C, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 (profiler on) 0% PASS +1.1% [-1.1, +2.5] / 2.4% PASS -0.9% [-2.7, +1.4] / 2.7% PASS +0.1% [-0.4, +0.5] / 0.8% PASS -1.4% [-3.7, +0.7] / 2.7% PASS -0.4% [-2.5, +1.4] / 1.6% PASS -0.3% [-1.4, +0.6] / 0.8% PASS

All four actual timing/latency executables were downloaded and SHA256-verified against the artifact manifest:

{
  "base/pprof/perf_ab": "c6784c2c2cb2619d31ce4bc679f840ad7d9a2cf04aba6ce4dda8c374d5c8ba37",
  "base/pprof/perf_ab_latency": "f4e223e4c1489619ef2cd9831bb88a204931fa207d7ec18a17411e2899eef12f",
  "head/pprof/perf_ab": "4dda45628f9468f02b2f3696824e8f9f02bfc9f1407ef242586af6ff8ea88dc7",
  "head/pprof/perf_ab_latency": "4ca3b9a3cff79cfee4e24f5ed292ae8ecd2357166339ec62d701a374c9730c31"
}

Zero memory credit. CPU and p99 intervals contain zero: this is a nonregression PASS for this row, not proof of a CPU or RSS improvement. Previous 15bfcba0 profiler p99 FAIL remains recorded and is not reclassified. The new-head full CI failed the musl page-reuse assertion; that failure is unresolved, and broader acceptance is not being dispatched until it is diagnosed. The isolated 15→6f hash-step comparison is still live.

Full-precision ledger cells:

{
  "workload": "large-class/8 (profiler on)",
  "counts": {
    "base": 60,
    "head": 60,
    "null": 60
  },
  "saved": 0.0,
  "cells": [
    {
      "metric": "ops/s",
      "base_median": 3456017.8,
      "head_median": 3472172.75,
      "change": [
        1.0977398707608812,
        -1.0723401554954406,
        2.4589613632465848
      ],
      "regression": [
        -2.4589613632465848,
        1.0723401554954406
      ],
      "allowance": 2.4038033110313246,
      "verdict": "PASS"
    },
    {
      "metric": "cpu s",
      "base_median": 3.60135,
      "head_median": 3.5604,
      "change": [
        -0.9128394244298823,
        -2.7399764920987386,
        1.3912565301072366
      ],
      "regression": [
        -2.7399764920987386,
        1.3912565301072366
      ],
      "allowance": 2.736381241054142,
      "verdict": "PASS"
    },
    {
      "metric": "minor faults",
      "base_median": 1630.5,
      "head_median": 1632.5,
      "change": [
        0.12302036306259127,
        -0.43023970497848807,
        0.4904966278356836
      ],
      "regression": [
        -0.43023970497848807,
        0.4904966278356836
      ],
      "allowance": 0.8064516129032258,
      "verdict": "PASS"
    },
    {
      "metric": "p99 operation ns",
      "base_median": 39587.5,
      "head_median": 38576.5,
      "change": [
        -1.3605905491732997,
        -3.7010567155035994,
        0.6678667397145913
      ],
      "regression": [
        -3.7010567155035994,
        0.6678667397145913
      ],
      "allowance": 2.6945892072336033,
      "verdict": "PASS"
    },
    {
      "metric": "peak RSS MiB",
      "base_median": 71.103515625,
      "head_median": 71.4921875,
      "change": [
        -0.4040898292187779,
        -2.548122000992775,
        1.3524703284570798
      ],
      "regression": [
        -2.548122000992775,
        1.3524703284570798
      ],
      "allowance": 1.5512006624344465,
      "verdict": "PASS"
    },
    {
      "metric": "RSS 0.5 s after drain MiB",
      "base_median": 68.9453125,
      "head_median": 68.685546875,
      "change": [
        -0.2635640745575398,
        -1.3862386350881133,
        0.6390318384889442
      ],
      "regression": [
        -1.3862386350881133,
        0.6390318384889442
      ],
      "allowance": 0.8086405790545126,
      "verdict": "PASS"
    }
  ]
}

Local-Gate: v1 tree=a6aa7b0bbd4650301c3f628fa62d8a4c0083dfc1 secs=35 lanes=lint:run
Ci-Attestation: {"at":1792551246,"gate":"python/all/lint","host":"linux-x86_64","key":"0a9efb8ac5d221f37cd2f7026a58bcc3655fc7ea641c9d5b78be4a75369ef3a1","lane":"lint","parents":["6f7be38f43be01f7017a1c7432421e97cd3f30b5"],"secs":35,"stamp":"e282c045455acae91bba9af2ddbdb288","tree":"a6aa7b0bbd4650301c3f628fa62d8a4c0083dfc1","v":1,"via":"run"}
@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

#585: deterministic reserved-page scanner interleaving; serial test fixture separated

The failed musl dead-thread-page-is-reused assertion remains recorded in 39532631746. Neither the 20 passing local actual-CI-binary replays (10 old15 + 10 new6f) nor the following probe proves the historical CI cause.

Controlled mechanism evidence: scratch copies of frozen baseline EE and candidate 6f both have byte-identical src/arena.c and original reuse test. The probe injects a callback immediately after mi_arena_page_release_reserved_at clears the abandoned bitmap bit and claims ownership. When it owns A's still-reserved, fully free page, the callback runs and joins B. The scanner then resumes its unchanged code. An independent process without this interleaving is the control.

Both EE and 6f, built via soldr cc, report:

mode=control hook=0 same_page=1 new_pages=0
mode=scanner-owns-page hook=1 same_page=0 new_pages=1

A soldr cc --target x86_64-unknown-linux-musl EE probe gives the same result inside Alpine 3.20. The retention window prevents page release, but does not make a scanner-owned page available to B. This demonstrates a possible baseline interleaving, not a new allocator-source regression and not guaranteed reuse during concurrent scanner ownership.

Fixture correction: #493 explicitly requests a deterministic test with serial threads. The existing test_dead_thread_page_is_reused function and both assertions are unchanged. CMake now invokes it in a fresh process with MIMALLOC_SCAVENGER=0, in both hole-purging ON and OFF configurations. The process verifies both the disabled option and that no scavenger is running before exercising reuse. The normal ON/OFF purge-holes suites retain all background reserved-page release, idle-release, slack, and other integrity tests. No retry, timeout change, RUN_SERIAL workaround, allocator policy change, or relaxed assertion.

Verification: native and musl fixture runs each cover five cases: scavenger-enabled invocation must reject (exit 1), serial reuse ON/OFF pass, original background ON/OFF pass. Actual CMake/CTest registration uses the explicit arguments and startup environment; all four CTest entries passed (7.24 s). All compiler invocations, including CMake's compiler, route through soldr. Same sole primary reviewer clean. Required local gate 45/45 PASS in 35 s. Attested test-only C-core commit da1fdb9819fdaa42a57e96d1d6cd741cf2711d03; no Rust/vendor or allocator C changes. Full platform CI must validate this new head; local passes do not waive it.

Reproducer (diagnostic scratch only, not shipped)

Hook inserted immediately before the original comment a reserved page has no live block... in the callback:

extern void reserve_probe_claimed(mi_page_t* page);
reserve_probe_claimed(page);
#include <assert.h>
#include <pthread.h>
#include <stdbool.h>
#include <stdio.h>
#include <string.h>
#include "mimalloc.h"
#include "mimalloc-stats.h"
#include "mimalloc/internal.h"
#define PROBE_BLOCK_BYTES (400 * 1024)
#define PROBE_BLOCK_COUNT 3
#define PROBE_RESERVE_DELAY_MS 1000
static mi_page_t* page_a;
static mi_page_t* page_b;
static size_t stats_bin;
static bool inject;
static bool hook_fired;
static void* thread_a(void* arg) {
  (void)arg;
  void* blocks[PROBE_BLOCK_COUNT];
  for (size_t i=0;i<PROBE_BLOCK_COUNT;i++) {
    blocks[i]=mi_malloc(PROBE_BLOCK_BYTES);assert(blocks[i]!=NULL);
    memset(blocks[i],1,PROBE_BLOCK_BYTES);
  }
  page_a=_mi_ptr_page(blocks[PROBE_BLOCK_COUNT-1]);
  stats_bin=_mi_page_stats_bin(page_a);
  assert(!mi_page_is_singleton(page_a));
  for (size_t i=0;i<PROBE_BLOCK_COUNT;i++) mi_free(blocks[i]);
  return NULL;
}
static void* thread_b(void* arg) {
  (void)arg;
  void* p=mi_malloc(PROBE_BLOCK_BYTES);assert(p!=NULL);
  page_b=_mi_ptr_page(p);memset(p,2,PROBE_BLOCK_BYTES);mi_free(p);
  return NULL;
}
static void run_thread(void*(*fn)(void*)) {
  pthread_t t;assert(pthread_create(&t,NULL,fn,NULL)==0);
  assert(pthread_join(t,NULL)==0);
}
void reserve_probe_claimed(mi_page_t* page) {
  if (!inject || hook_fired || page!=page_a) return;
  assert(mi_page_is_owned(page));
  assert(page->retired_at!=0);
  assert(mi_page_all_free(page));
  hook_fired=true;
  run_thread(thread_b);
}
static int64_t created(void) {
  mi_stats_t_decl(st);assert(mi_stats_get(&st));
  return st.page_bins[stats_bin].total;
}
int main(int argc,char** argv) {
  (void)argv;
  mi_option_set(mi_option_scavenger,0);
  mi_collect(true);
  mi_option_set(mi_option_purge_delay,PROBE_RESERVE_DELAY_MS);
  mi_option_set(mi_option_page_reserve,1);
  inject=(argc>1);
  run_thread(thread_a);
  const int64_t before=created();
  if (inject) {
    (void)_mi_arenas_release_reserved(mi_heap_main(),false);
    assert(hook_fired);
  } else run_thread(thread_b);
  const int64_t delta=created()-before;
  printf("mode=%s hook=%d same_page=%d new_pages=%lld\n",inject?"scanner-owns-page":"control",hook_fired,page_a==page_b,(long long)delta);
  const bool observed=inject?(page_a!=page_b && delta==1):(page_a==page_b && delta==0);
  mi_collect(true);
  return observed?0:1;
}

Actual fixture results:

[
  {
    "name": "enabled-reject",
    "env": {
      "MIMALLOC_SCAVENGER": "1"
    },
    "args": [
      "--serial-reserve-reuse"
    ],
    "actual": 1,
    "expected": 1
  },
  {
    "name": "reuse-on",
    "env": {
      "MIMALLOC_SCAVENGER": "0",
      "MIMALLOC_PURGE_HOLES": "1"
    },
    "args": [
      "--serial-reserve-reuse"
    ],
    "actual": 0,
    "expected": 0
  },
  {
    "name": "reuse-off",
    "env": {
      "MIMALLOC_SCAVENGER": "0",
      "MIMALLOC_PURGE_HOLES": "0"
    },
    "args": [
      "--serial-reserve-reuse"
    ],
    "actual": 0,
    "expected": 0
  },
  {
    "name": "background",
    "env": {},
    "args": [],
    "actual": 0,
    "expected": 0
  },
  {
    "name": "background-holes-off",
    "env": {
      "MIMALLOC_PURGE_HOLES": "0"
    },
    "args": [],
    "actual": 0,
    "expected": 0
  }
]

CTest:

Test project /tmp/mimalloc-585/reserve-claim-probe/cmake
    Start 41: test-purge-holes
1/4 Test #41: test-purge-holes .................   Passed    3.61 sec
    Start 42: test-purge-holes-off
2/4 Test #42: test-purge-holes-off .............   Passed    3.61 sec
    Start 43: test-reserved-page-reuse
3/4 Test #43: test-reserved-page-reuse .........   Passed    0.01 sec
    Start 44: test-reserved-page-reuse-off
4/4 Test #44: test-reserved-page-reuse-off .....   Passed    0.01 sec

100% tests passed, 0 tests failed out of 4

Total Test time (real) =   7.24 sec

SHA256 evidence manifest:

{
  "probe.c": "e4ed4a5f2249f4ec0cc13d741c5f666f4d4c0cc653366820023b8eb2e3d958d0",
  "baseline-probe": "cecf91e9315efba1cbb42840aff44ea2ce9dff55097bcb858d1d2e9eb0e8e303",
  "candidate-probe": "c8f2914534111545f1f5f4823a8db15216a2b4d2e65486388dffa134340a9561",
  "baseline-musl-probe": "e90f7a1a1a38b1798ca66d40b60be527f13cd802dde08d7c64cf3f8192df15d8",
  "purge-holes-fixture": "34113d7473acd3e0c8e66a56a38356c56a0e76b0fec632f400ac9751395d873f",
  "purge-holes-fixture-musl": "8b0f5e3523827b586dfc88a58bd278887af4d6b9cf4ab35c8ddc45bf39a2e7dd",
  "fixture-results.json": "9c1ed4b68275175a38635d86e63e43d545183158590f8018f25b66720fa2786c",
  "ctest.log": "99148153a95fcc66f21bed28d02315f1106f427dcd21c14ff7df075dfc4b882c",
  "musl-fixture.log": "25f7a5fd694b5a41dcb4edd2aee29c0455953da09379f32a4f047f8bf5b4be23"
}

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

#585: earlier profiler source comparisons completed (attribution only)

These isolate earlier source steps; neither is acceptance of the current full candidate. Original exact profiler trace, null/sanity gates, balanced arm ordering, unchanged 60-to-120 plan. The display header's 60 is the requested starting count; actual counts below come from raw sample-array lengths. Different hosts/source pairs are not pooled and their deltas are not added. Prior full-pair p99 failures are not waived or reclassified.

Frozen EE → exact-fit-only a9d7d2e

Run 39531382092, AMD EPYC 7763 64-Core Processor. Actual samples per arm: {'base': 95, 'head': 95, 'null': 95}. All four actual timing/latency executable hashes verified.

base ee3623f6b7a4 head a9d7d2e372dd on AMD EPYC 7763 64-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 (profiler on) 0% PASS +0.5% [+0.1, +0.7] / 0.4% PASS -0.6% [-0.8, -0.4] / 0.4% PASS -0.1% [-0.4, +0.2] / 0.3% PASS +0.9% [-2.1, +4.5] / 5.1% PASS -0.0% [-2.4, +0.2] / 0.4% PASS -0.5% [-1.7, +0.2] / 0.5% PASS

Verified executable manifest:

{
  "base/pprof/perf_ab": "c6784c2c2cb2619d31ce4bc679f840ad7d9a2cf04aba6ce4dda8c374d5c8ba37",
  "base/pprof/perf_ab_latency": "f4e223e4c1489619ef2cd9831bb88a204931fa207d7ec18a17411e2899eef12f",
  "head/pprof/perf_ab": "63884c20d9d2a222c1983534b9bc18d1d0d22ec844c843aac44fe49009564ca9",
  "head/pprof/perf_ab_latency": "da62ab296e1914727452b2e820282472f716f22c69a61635791b04c3e46c8fd5"
}

Full-precision cells:

[
  {
    "workload": "large-class/8 (profiler on)",
    "counts": {
      "base": 95,
      "head": 95,
      "null": 95
    },
    "saved": 0.0,
    "cells": [
      {
        "metric": "ops/s",
        "base_median": 4202652.8,
        "head_median": 4220480.1,
        "change": [
          0.4905421269055018,
          0.14327076722488194,
          0.708496763044393
        ],
        "regression": [
          -0.708496763044393,
          -0.14327076722488194
        ],
        "allowance": 0.36209909439788956,
        "verdict": "PASS"
      },
      {
        "metric": "cpu s",
        "base_median": 2.9412,
        "head_median": 2.9252,
        "change": [
          -0.6035251354502379,
          -0.7802428930049573,
          -0.3786457444994143
        ],
        "regression": [
          -0.7802428930049573,
          -0.3786457444994143
        ],
        "allowance": 0.3764370739647942,
        "verdict": "PASS"
      },
      {
        "metric": "minor faults",
        "base_median": 1625.0,
        "head_median": 1624.0,
        "change": [
          -0.06172839506172839,
          -0.3703703703703704,
          0.24645717806531117
        ],
        "regression": [
          -0.3703703703703704,
          0.24645717806531117
        ],
        "allowance": 0.3094059405940594,
        "verdict": "PASS"
      },
      {
        "metric": "p99 operation ns",
        "base_median": 38031.0,
        "head_median": 39160.0,
        "change": [
          0.8800679776644816,
          -2.1027695012943877,
          4.457564988667208
        ],
        "regression": [
          -2.1027695012943877,
          4.457564988667208
        ],
        "allowance": 5.077547572678852,
        "verdict": "PASS"
      },
      {
        "metric": "peak RSS MiB",
        "base_median": 75.48046875,
        "head_median": 73.66015625,
        "change": [
          -0.02652942112803099,
          -2.387789028260541,
          0.1750941794450045
        ],
        "regression": [
          -2.387789028260541,
          0.1750941794450045
        ],
        "allowance": 0.3979413169204648,
        "verdict": "PASS"
      },
      {
        "metric": "RSS 0.5 s after drain MiB",
        "base_median": 54.41796875,
        "head_median": 54.37109375,
        "change": [
          -0.4606636435615058,
          -1.6683161317686979,
          0.19435646415202995
        ],
        "regression": [
          -1.6683161317686979,
          0.19435646415202995
        ],
        "allowance": 0.4507994646756357,
        "verdict": "PASS"
      }
    ]
  }
]

Exact-fit-only a9d7d2e → prefix-reuse 9b67e41

Run 39531384408, AMD EPYC 9V45 96-Core Processor. Actual samples per arm: {'base': 120, 'head': 120, 'null': 120}. All four actual timing/latency executable hashes verified.

base a9d7d2e372dd head 9b67e411fc8c on AMD EPYC 9V45 96-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 (profiler on) 0% INCONCLUSIVE +0.4% [-0.2, +0.7] / 0.7% PASS -0.3% [-0.6, +0.1] / 0.5% PASS +0.2% [+0.0, +0.6] / 0.5% INCONCLUSIVE -2.3% [-8.9, +2.9] / 8.4% PASS +0.2% [-0.1, +0.7] / 0.8% PASS -0.5% [-1.0, -0.0] / 0.4% PASS

Verified executable manifest:

{
  "base/pprof/perf_ab": "63884c20d9d2a222c1983534b9bc18d1d0d22ec844c843aac44fe49009564ca9",
  "base/pprof/perf_ab_latency": "da62ab296e1914727452b2e820282472f716f22c69a61635791b04c3e46c8fd5",
  "head/pprof/perf_ab": "9249fe01572f8989a024d0f6ee72b6ded65ea6feff2343e2274ae800b23a5cc3",
  "head/pprof/perf_ab_latency": "43e680d121436d9e4d8f61d66e11a0d162c166a8d7bccaa69209c264355f2022"
}

Full-precision cells:

[
  {
    "workload": "large-class/8 (profiler on)",
    "counts": {
      "base": 120,
      "head": 120,
      "null": 120
    },
    "saved": 0.0,
    "cells": [
      {
        "metric": "ops/s",
        "base_median": 4851513.949999999,
        "head_median": 4863665.4,
        "change": [
          0.35597414933043087,
          -0.15125286464079588,
          0.6580069460234763
        ],
        "regression": [
          -0.6580069460234763,
          0.15125286464079588
        ],
        "allowance": 0.7196587330291655,
        "verdict": "PASS"
      },
      {
        "metric": "cpu s",
        "base_median": 2.5869,
        "head_median": 2.57885,
        "change": [
          -0.30566662877407647,
          -0.631283610007014,
          0.05429302722408633
        ],
        "regression": [
          -0.631283610007014,
          0.05429302722408633
        ],
        "allowance": 0.45851441330089876,
        "verdict": "PASS"
      },
      {
        "metric": "minor faults",
        "base_median": 1630.5,
        "head_median": 1634.0,
        "change": [
          0.24517501874690995,
          0.0,
          0.5528255528255528
        ],
        "regression": [
          0.0,
          0.5528255528255528
        ],
        "allowance": 0.4914004914004914,
        "verdict": "INCONCLUSIVE"
      },
      {
        "metric": "p99 operation ns",
        "base_median": 24206.0,
        "head_median": 23535.5,
        "change": [
          -2.2627565700302705,
          -8.938805338666873,
          2.8635559787783973
        ],
        "regression": [
          -8.938805338666873,
          2.8635559787783973
        ],
        "allowance": 8.434593145059917,
        "verdict": "PASS"
      },
      {
        "metric": "peak RSS MiB",
        "base_median": 73.9453125,
        "head_median": 75.625,
        "change": [
          0.15954175054025288,
          -0.06872488898287164,
          0.7231176860234046
        ],
        "regression": [
          -0.06872488898287164,
          0.7231176860234046
        ],
        "allowance": 0.7875316664830928,
        "verdict": "PASS"
      },
      {
        "metric": "RSS 0.5 s after drain MiB",
        "base_median": 72.041015625,
        "head_median": 71.94140625,
        "change": [
          -0.4801196767773953,
          -1.0019932123040456,
          -0.027114967462039046
        ],
        "regression": [
          -1.0019932123040456,
          -0.027114967462039046
        ],
        "allowance": 0.4405286343612335,
        "verdict": "PASS"
      }
    ]
  }
]

The exact-fit profiler step PASSes all six metrics at 95/arm; prefix-reuse remains INCONCLUSIVE on minor faults at 120/arm, with the other five PASS. Both have zero memory credit. Neither establishes the cause of the old full-candidate p99 failures. The new word-hash EE→6f profiler row separately PASSed at 60/arm; fresh full platform CI on test-only da1fdb9 is pending.

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

#585: capacity-based fit does not rescue tight geometry; rejected before RSS/timing

New diagnostic idea: previous tight geometry kept whole-block capacity but failed the existing large-bin fungibility test (four distinct pages where at most three are allowed). Instead of requiring the complete nominal span in the retired-page search, require its whole-block capacity: (want / block_size) * block_size, still bounded below by the existing overflow-checked MI_LARGE_SPAN_MIN_BLOCKS requirement. This could admit a smaller page with the same usable capacity. No resident bytes are discarded.

This is distinct from repeating the rejected geometry unchanged, but the structural result is still negative. Scratch-only code based on allocator candidate 6f7be38, compiled via soldr with debug invariants, diagnostics and statistics enabled. Both the tight-only control and tight+capacity-fit variant fail the unchanged rotation check: four distinct pages for 160 requests over ten bins (limit three). The capacity-preservation checks pass, but fungibility does not. Therefore rejected, with no RSS screen or timed CI dispatch, and no production source or option changed. There is no claimed memory saving or performance acceptance.

Actual executions (all outcomes retained):

[
  {
    "variant": "tight-only",
    "returncode": 1,
    "binary_sha256": "f7d311999c3a04ef19d5de9c0977f1b5456797f3d534a8e4e012346aef59ba12",
    "stdout": "",
    "stderr": "(a) largest span of a busy 200 KiB bin: 15 slices (cap 16, full 64)\n(b) 4 distinct pages for 160 allocations rotating over 10 bins\nFAILED: (b) rotating over 10 bins with one live block kept 4 pages (want at most 3)\n1 check(s) failed\n"
  },
  {
    "variant": "tight-capacity-fit",
    "returncode": 1,
    "binary_sha256": "1c46c015a0d21a29e340300d2bae6160182df2d3ab53b9acf7f846311a0869e3",
    "stdout": "",
    "stderr": "(a) largest span of a busy 200 KiB bin: 15 slices (cap 16, full 64)\n(b) 4 distinct pages for 160 allocations rotating over 10 bins\nFAILED: (b) rotating over 10 bins with one live block kept 4 pages (want at most 3)\n1 check(s) failed\n"
  }
]

Diagnostic-only patch (not in the PR):

--- a/include/mimalloc/types.h
+++ b/include/mimalloc/types.h
@@ -386,6 +386,14 @@
 // (the #532 behaviour); `-DMI_LARGE_SPAN_MAX_KIB=0` builds that in.
 #ifndef MI_LARGE_SPAN_MAX_KIB
 #define MI_LARGE_SPAN_MAX_KIB             (1024)
+#endif
+// #585 experiment: return whole slices that cannot hold another block without
+// reducing block capacity. Disabled until paired CPU/fault/RSS/latency checks pass.
+#ifndef MI_LARGE_SPAN_TIGHT_GEOMETRY
+#define MI_LARGE_SPAN_TIGHT_GEOMETRY      (0)
+#endif
+#ifndef MI_LARGE_REPURPOSE_CAPACITY_FIT
+#define MI_LARGE_REPURPOSE_CAPACITY_FIT (0)
 #endif
 // each demand step multiplies the span by 2^MI_LARGE_SPAN_GROW_SHIFT (1: 1 -> 2 -> 4 MiB)
 #ifndef MI_LARGE_SPAN_GROW_SHIFT
--- a/src/large-span.c
+++ b/src/large-span.c
@@ -139,6 +139,18 @@
   const size_t min_slices = mi_slice_count_of_size(MI_LARGE_SPAN_MIN_BLOCKS * block_size + overhead);
   if (slices < min_slices) { slices = min_slices; }
   if (slices > full) { slices = full; }
+  #if MI_LARGE_SPAN_TIGHT_GEOMETRY
+  // A capped span need not retain whole slices past its last possible block.
+  // Keep the same block capacity and demand level; no resident byte is discarded.
+  // The uncapped opt-out retains its original exact power-of-two geometry.
+  if (cap < full) {
+    const size_t bytes = mi_size_of_slices(slices);
+    const size_t blocks = (bytes > overhead ? (bytes - overhead) / block_size : 0);
+    if (blocks >= MI_LARGE_SPAN_MIN_BLOCKS) {
+      slices = mi_slice_count_of_size(blocks * block_size + overhead);
+    }
+  }
+  #endif
   return slices;
 }
 
--- a/src/page.c
+++ b/src/page.c
@@ -587,6 +587,10 @@
     MI_EVENT(MI_EVENT_LARGE_REPURPOSE_NONE);
     return NULL;
   }
+  #if MI_LARGE_REPURPOSE_CAPACITY_FIT
+  // A nominal span promises whole blocks, not unusable bytes after its last block.
+  want = (want / block_size) * block_size;
+  #endif
   if (want < min_bytes) { want = min_bytes; }
   mi_page_queue_t* best_pq = NULL;
   size_t best_size = SIZE_MAX;
--- a/test/test-large-fungible.c
+++ b/test/test-large-fungible.c
@@ -136,6 +136,31 @@
   #endif
 }
 
+#if defined(MI_LARGE_SPAN_TIGHT_GEOMETRY) && MI_LARGE_SPAN_TIGHT_GEOMETRY
+static THREAD_RET tight_main(void* arg) {
+  (void)arg;
+  void* const probe = mi_malloc(16);
+  assert(probe != NULL);
+  mi_theap_t* const theap = _mi_ptr_page(probe)->theap;
+  const size_t nominal = mi_size_of_slices(MI_LARGE_SPAN_COMPACT_SLICES);
+  for (size_t i = 0; i < NBINS; i++) {
+    const size_t bsize = mi_good_size(BIN_SIZES[i] * KIB);
+    if (bsize <= MI_MEDIUM_MAX_OBJ_SIZE || bsize > MI_LARGE_MAX_OBJ_SIZE) continue;
+    const size_t blocks = nominal / bsize;
+    const size_t slices = _mi_large_span_peek_slices(theap, bsize, 0);
+    const size_t expected = mi_slice_count_of_size(blocks * bsize);
+    CHECK(slices == expected, "(c) %zu KiB: %zu slices instead of tightly packed %zu", bsize / KIB, slices, expected);
+    CHECK(mi_size_of_slices(slices) / bsize == blocks, "(c) trimming lost block capacity");
+  }
+  mi_free(probe);
+  return THREAD_OK;
+}
+
+static void case_tight(void) {
+  thread_t t; thread_start(&t, &tight_main, NULL); thread_join(t);
+}
+#endif
+
 int main(void) {
   #if !MI_ENABLE_LARGE_PAGES
   fprintf(stderr, "skipped: no large pages in this build\n");
@@ -154,6 +179,9 @@
   mi_option_set(mi_option_page_reserve, 0);
   case_cap();
   case_rotate();
+  #if defined(MI_LARGE_SPAN_TIGHT_GEOMETRY) && MI_LARGE_SPAN_TIGHT_GEOMETRY
+  case_tight();
+  #endif
   #endif
   if (failures > 0) { fprintf(stderr, "%d check(s) failed\n", failures); return 1; }
   fprintf(stderr, "ok\n");

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

#585: current plain executables are byte-identical to the earlier measured candidate

Build-only run 39536056572 freshly built frozen EE and current test-only da1fdb98, with all observability explicitly OFF. No workloads or timings ran. Its complete:false marker intentionally means diagnostic build-only, not an incomplete measured run.

Downloaded actual files: verified all six report SHA256 values (two timing executables, two separate latency executables, two static libraries). Then compared the whole four linked executables byte for byte with the actual timed EE/15 binaries from run39520753532, not merely symbol addresses or text sizes. All four are identical. The static libraries are verified against this report; no assertion of old-library identity is made.

This carries forward measured evidence for those identical GCC13 plain executables; it is not a new measurement on da1fdb9, does not relabel original JSON/source refs, and does not pool different hosts. The original counts, CPU models, traces, null/sanity checks, protocol and verdicts remain attached to their original runs.

There are six distinct already-PASS plain cells: main, plain ephemeral, sparse, bursty, Larson/1 and random/1. The earlier seventh ordinary PASS was standard chart ephemeral, whose changed executable must be measured again. The default-aging trial duplicates main, not a seventh distinct cell. Plain small/8 and Larson/8 remain unverified after their original job timeout; random/8 remains INCONCLUSIVE. Binary identity does not resolve those outcomes.

Chart enables PPROF, so the word-hash change can affect its linked executable even with profiling unarmed. All three chart cells (Larson/8, standard ephemeral, short generations) require current-candidate validation. Current fixed-baseline profiler row separately PASSed at 60/arm. Broader dispatch remains gated on fresh platform CI for the serial fixture correction. No memory win or whole-candidate acceptance is claimed. Same sole primary reviewer checked this evidence scope; its PASS-cell-count correction is reflected above.

Verified build metadata:

{
  "complete": false,
  "diagnostic": "build-only; no measurements or regression verdict",
  "harness_sha": "da1fdb9819fdaa42a57e96d1d6cd741cf2711d03",
  "compiler": "cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0\nCopyright (C) 2023 Free Software Foundation, Inc.\nThis is free software; see the source for copying conditions.  There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n\n",
  "linker": "GNU ld (GNU Binutils for Ubuntu) 2.42\nCopyright (C) 2024 Free Software Foundation, Inc.\nThis program is free software; you may redistribute it under the terms of\nthe GNU General Public License version 3 or (at your option) a later version.\nThis program has absolutely no warranty.\n",
  "cmake": "cmake version 3.31.6\n\nCMake suite maintained and supported by Kitware (kitware.com/cmake).\n",
  "runner": {
    "ImageOS": "ubuntu24",
    "ImageVersion": "20261018.339.1",
    "RUNNER_OS": "Linux",
    "RUNNER_ARCH": "X64"
  },
  "flags": [
    "-DCMAKE_BUILD_TYPE=Release",
    "-DMI_BUILD_SHARED=OFF",
    "-DMI_BUILD_OBJECT=OFF",
    "-DMI_BUILD_TESTS=OFF",
    "-DMI_OVERRIDE=OFF",
    "-DMI_PPROF=OFF",
    "-DMI_MEMEVT=OFF",
    "-DMI_DIAGNOSTICS=OFF",
    "-DMI_DHAT=OFF",
    "-DMI_OWNER_GATE=OFF"
  ],
  "arms": {
    "base": {
      "ref": "ee3623f6b7a485754a0a0aa3462d8cba43390e8b",
      "sha": "ee3623f6b7a485754a0a0aa3462d8cba43390e8b",
      "kind": "plain",
      "files": {
        "perf_ab": {
          "sha256": "7f0c70ce2450dfa09126c1674bddab53187614f12d5bd648b4fda33ccbd3c712",
          "bytes": 336296
        },
        "perf_ab_latency": {
          "sha256": "ef2d5d273681a473e253d1dd2080307f1ae9f09d34977bda2ad4cab9cbfe7823",
          "bytes": 336776
        },
        "libmimalloc.a": {
          "sha256": "0bb97ca951ed5404f754091fa4ec4907949a1973ea6d2f6d559889c66c028d47",
          "bytes": 513378
        }
      }
    },
    "head": {
      "ref": "da1fdb9819fdaa42a57e96d1d6cd741cf2711d03",
      "sha": "da1fdb9819fdaa42a57e96d1d6cd741cf2711d03",
      "kind": "plain",
      "files": {
        "perf_ab": {
          "sha256": "9ee3617e371e93195de1374ef403455e7ae660848d689ce4bf0ae0ce2f8565b1",
          "bytes": 336296
        },
        "perf_ab_latency": {
          "sha256": "c14766d194fd7f3c07652b148d1de18e3e4db90b4414bd563398f5f44ee9b982",
          "bytes": 336776
        },
        "libmimalloc.a": {
          "sha256": "7f5a84746d6ad6520117d81e67f5ae37506d08f51f29ed78b91a27e470f1ddc2",
          "bytes": 513418
        }
      }
    }
  }
}

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

#585: isolated word-hash step improves CPU/p99; drain RSS remains INCONCLUSIVE

Run 39532708955 compares 15bfcba0c3e59aea6b22dd04f91db9d95cd63a0a → 6f7be38f43be01f7017a1c7432421e97cd3f30b5 on EPYC 9V74 / four logical CPUs, exact profiler-on large-class/8 row. 120 actual samples per base/head/null arm, original trace and balanced arm order, unchanged null/sanity/ledger thresholds. The display header's 60 is the starting count, not the actual final count.

This isolates the word-hash source step, not frozen-baseline acceptance. It is INCONCLUSIVE overall on after-drain RSS. CPU -1.84% [-2.11%,-1.55%], throughput +1.91% [1.60%,2.33%], p99 -2.33% [-4.57%,-1.04%]; those three intervals exclude zero in the favorable direction on this host. Peak/faults PASS. Drain +5.87% [1.40%,27.29%] against 1.91% null noise remains INCONCLUSIVE, not PASS. Zero memory credit. No retry or relaxed gate is planned merely to obtain a PASS.

base 15bfcba0c3e5 head 6f7be38f43be on AMD EPYC 9V74 80-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class/8 (profiler on) 0% INCONCLUSIVE +1.9% [+1.6, +2.3] / 0.4% PASS -1.8% [-2.1, -1.6] / 0.5% PASS +0.0% [-0.2, +0.1] / 0.6% PASS -2.3% [-4.6, -1.0] / 3.0% PASS +0.1% [-0.1, +0.4] / 0.5% PASS +5.9% [+1.4, +27.3] / 1.9% INCONCLUSIVE

All four actual executables verified against the SHA256 manifest. The baseline timing/latency files are byte-identical to the original old15 profiler-p99 FAIL run; head files are byte-identical to the new EE→6f profiler PASS run. Those runs used different CPU models, so they are not pooled and their deltas are not added. The old full-pair p99 FAIL is not erased; this step result does not prove its cause. New fixed-baseline acceptance remains the separate Xeon8573C 60/arm all-six PASS; full candidate acceptance additionally requires pending correctness and remaining cells.

Verified manifest:

{
  "base/pprof/perf_ab": "ef1f0135f09bec19ff3cba3c26a211359243cbcb40825818f9bddb3c512771ba",
  "base/pprof/perf_ab_latency": "1f432e1f3a4c857bde03dd44859eac39b92a510806cceb4841087d16e98d7a8e",
  "head/pprof/perf_ab": "4dda45628f9468f02b2f3696824e8f9f02bfc9f1407ef242586af6ff8ea88dc7",
  "head/pprof/perf_ab_latency": "4ca3b9a3cff79cfee4e24f5ed292ae8ecd2357166339ec62d701a374c9730c31"
}

Full-precision cells:

[
  {
    "workload": "large-class/8 (profiler on)",
    "counts": {
      "base": 120,
      "head": 120,
      "null": 120
    },
    "saved": 0.0,
    "cells": [
      {
        "metric": "ops/s",
        "base_median": 4564683.800000001,
        "head_median": 4663804.05,
        "change": [
          1.9106592534741764,
          1.5997172951026735,
          2.3261056701665552
        ],
        "regression": [
          -2.3261056701665552,
          -1.5997172951026735
        ],
        "allowance": 0.3972306579034035,
        "verdict": "PASS"
      },
      {
        "metric": "cpu s",
        "base_median": 2.7489,
        "head_median": 2.69435,
        "change": [
          -1.843709296422775,
          -2.1118825197193862,
          -1.5540988436326064
        ],
        "regression": [
          -2.1118825197193862,
          -1.5540988436326064
        ],
        "allowance": 0.5431068589720495,
        "verdict": "PASS"
      },
      {
        "metric": "minor faults",
        "base_median": 1624.0,
        "head_median": 1625.0,
        "change": [
          0.03076923076923077,
          -0.24706609017912293,
          0.12376237623762376
        ],
        "regression": [
          -0.24706609017912293,
          0.12376237623762376
        ],
        "allowance": 0.5572755417956656,
        "verdict": "PASS"
      },
      {
        "metric": "p99 operation ns",
        "base_median": 38774.0,
        "head_median": 37862.5,
        "change": [
          -2.331617592969682,
          -4.571640958929566,
          -1.0423082389728526
        ],
        "regression": [
          -4.571640958929566,
          -1.0423082389728526
        ],
        "allowance": 3.044764977204255,
        "verdict": "PASS"
      },
      {
        "metric": "peak RSS MiB",
        "base_median": 73.630859375,
        "head_median": 73.603515625,
        "change": [
          0.06809540390977623,
          -0.10833677259595542,
          0.3701083111086921
        ],
        "regression": [
          -0.10833677259595542,
          0.3701083111086921
        ],
        "allowance": 0.4559673400137851,
        "verdict": "PASS"
      },
      {
        "metric": "RSS 0.5 s after drain MiB",
        "base_median": 63.412109375,
        "head_median": 71.775390625,
        "change": [
          5.866936206493472,
          1.4017381553125876,
          27.28913571676501
        ],
        "regression": [
          1.4017381553125876,
          27.28913571676501
        ],
        "allowance": 1.9091229306093693,
        "verdict": "INCONCLUSIVE"
      }
    ]
  }
]

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

#585: six individual remaining measurements dispatched, original numeric plan unchanged

Fixed baseline ee3623f6b7a485754a0a0aa3462d8cba43390e8b → current da1fdb9819fdaa42a57e96d1d6cd741cf2711d03. Each row is its own workflow job/VM with the predeclared 60-to-120 actual-sample plan, exact original workload, same-run null arm, unchanged sanity/ledger gates and balanced arm positions. Checkpoints and actual built timing/latency files are preserved before measurement. No results or PASS are inferred from dispatch.

exact row run
=small/8 (control) 39537461436
=larson/8 39537463373
=random-large/8 39537464950
=larson/8 (chart build) 39537466521
=large-class-ephemeral/8 (chart build) 39537468554
=large-class-ephemeral/8 short generations (chart build) 39537470652

Scheduling adjustment, not a numeric-plan change: the earlier assistant-imposed plan said to wait for every full-CI check before dispatching. Full Linux parallel/serial, musl including both unskipped original same-page assertions, eight memory-gate runs, leak-positive control, diagnostics, native MSVC, both native Macs, Rust, cross and ASan have now passed. The independent Windows bundle runner is still alive, currently in the MSVC debug-full bundle. These Linux measurements can run on separate VMs while Windows finishes. Full candidate acceptance still requires every correctness gate, including that Windows job. No CI job was cancelled or restarted.

The old three-control job timed out at 90 minutes without artifacts; that group is not restarted. Its remaining rows now have individual jobs under the checkpoint-capable harness. The six prior distinct plain PASS cells are carried forward only through verified whole-executable identity, retaining original refs, hosts, actual counts, outcomes and scope. Original failed and INCONCLUSIVE runs remain visible; no cross-CPU pooling, JSON relabeling, changed traces, new discard policy, or loosened thresholds.

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

#585: completed reuse counts and observed free-capacity time answer different questions

Offline analysis of the existing four local arena claim/free traces (old allocator policy 8b412211, no new workloads or source policy changes). It reproduces the previously reported completed per-slice reuse counts and under-1-ms fractions exactly. An independent time-sorted inventory integral equals the sum of completed plus observed censored free-interval durations exactly in every window. No per-slice negative reuse interval; cross-thread record-order/clock inversions (35–175 per trace) are explicitly retained, not assumed away.

For each observed free-slice interval of duration d, time above descriptive age t is max(0,d-t). Sum those durations across slices and divide by total observed free-slice time. Intervals still free at the selected cutoff contribute their observed duration only; their full lifetimes are right-censored. These are virtual arena-free slice-time, not resident-byte-time, retired-page lifetimes, or RSS.

trace completed reuse within 1 ms average known-free virtual MiB share of observed free-slice time older than 1 ms older than 10 ms share contributed by censored intervals
sparse 99.02% 69.31 36.92% 14.14% 11.97%
sparse-2 98.79% 72.47 47.05% 24.15% 24.93%
sparse-3 98.83% 74.35 44.26% 18.05% 16.91%
persistent 98.99% 14.90 50.63% 28.58% 20.28%

Cutoff limit: the table stops at the last recorded arena claim, which is only a provisional diagnostic cutoff. It is NOT an authoritative work-end or peak-RSS marker; allocations can continue using existing pages afterward. Full-trace results are also retained below and include the roughly 2.6-second drain/shutdown window. That full window is 93–97% dominated by censored free-interval time and must not be interpreted as work-phase resident pressure.

Inference: roughly 99% of completed reuse intervals being short does not show that all free capacity is hot. A minority of longer intervals and the still-free tail make a substantial contribution to observed virtual free-slice time. This does not prove those bytes remain resident, quantify a memory win, or justify selecting a discard threshold. The claim/free trace has no purge/residency events or authoritative phase markers. Before a new cold-release design, collect those missing facts together; do not retry the rejected hot-body discard or infer that a count percentile supplies the cold resident bucket. No new memory setting or policy is retained. Same sole primary reviewer checked the arithmetic and inference scope clean.

Full results, limits and raw trace SHA256 identities:

{
  "purpose": "Descriptive existing local traces; virtual arena-free slice-time, not residency or acceptance",
  "limits": [
    "Callbacks instrumented the old8b policy; wall time affected by instrumentation.",
    "Arena claim/free only; no purge/residency events and no page-retirement events.",
    "Full trace includes drain/shutdown. Last recorded arena allocation is only a second diagnostic cutoff, NOT an authoritative work/peak marker.",
    "Intervals still free at cutoff are right-censored; observed time is a lower bound on lifetime."
  ],
  "traces": {
    "sparse": {
      "sha256": "19bd45eec192010986375b3762ba8e27046a73e4689daa8f65c6e722e713dae7",
      "full_trace": {
        "valid": true,
        "events": 120750,
        "window_ms": 2783.352852,
        "global_timestamp_inversions": 175,
        "completed_reuse_slices": 2249519,
        "right_censored_free_slices": 2640,
        "completed_within_1ms_fraction": 0.9901925700560875,
        "known_free_MiB_at_cutoff": 165.0,
        "time_average_known_free_MiB": 133.03359631626307,
        "right_censored_share_of_observed_free_slice_time": 0.9699571379033658,
        "share_of_free_slice_time_with_age_above_ms": {
          "1": 0.9783965215334672,
          "10": 0.9684443110796214,
          "25": 0.9616512222774563,
          "50": 0.9516937710078794,
          "100": 0.9328180829413952
        },
        "known_free_MiB_with_age_at_least_ms_at_cutoff": {
          "1": 137.5,
          "10": 137.5,
          "25": 137.5,
          "50": 137.5,
          "100": 137.5
        },
        "free_slice_time_conservation_exact": true
      },
      "last_recorded_arena_claim_cutoff": {
        "valid": true,
        "events": 120717,
        "window_ms": 182.330442,
        "global_timestamp_inversions": 175,
        "completed_reuse_slices": 2249519,
        "right_censored_free_slices": 2055,
        "completed_within_1ms_fraction": 0.9901925700560875,
        "known_free_MiB_at_cutoff": 128.4375,
        "time_average_known_free_MiB": 69.30854652654492,
        "right_censored_share_of_observed_free_slice_time": 0.11971170719839819,
        "share_of_free_slice_time_with_age_above_ms": {
          "1": 0.36923228362605637,
          "10": 0.1413967486349996,
          "25": 0.08394538990507293,
          "50": 0.05211795702653927,
          "100": 0.023897529158928045
        },
        "known_free_MiB_with_age_at_least_ms_at_cutoff": {
          "1": 97.0,
          "10": 19.4375,
          "25": 15.5,
          "50": 5.1875,
          "100": 4.5625
        },
        "free_slice_time_conservation_exact": true
      }
    },
    "sparse-2": {
      "sha256": "67267e1bafdf3f8c829e69502e15b5b68cb8fa8aadbf52c2a725da5bb5f855e1",
      "full_trace": {
        "valid": true,
        "events": 120758,
        "window_ms": 2829.861223,
        "global_timestamp_inversions": 127,
        "completed_reuse_slices": 2249524,
        "right_censored_free_slices": 2679,
        "completed_within_1ms_fraction": 0.9878818807890025,
        "known_free_MiB_at_cutoff": 167.4375,
        "time_average_known_free_MiB": 134.48932033892532,
        "right_censored_share_of_observed_free_slice_time": 0.9673337360956915,
        "share_of_free_slice_time_with_age_above_ms": {
          "1": 0.9768765425616812,
          "10": 0.9656415940471132,
          "25": 0.9586010737574893,
          "50": 0.9489569464646093,
          "100": 0.9305597977462671
        },
        "known_free_MiB_with_age_at_least_ms_at_cutoff": {
          "1": 140.4375,
          "10": 139.9375,
          "25": 139.9375,
          "50": 139.9375,
          "100": 139.9375
        },
        "free_slice_time_conservation_exact": true
      },
      "last_recorded_arena_claim_cutoff": {
        "valid": true,
        "events": 120727,
        "window_ms": 228.535267,
        "global_timestamp_inversions": 127,
        "completed_reuse_slices": 2249524,
        "right_censored_free_slices": 2229,
        "completed_within_1ms_fraction": 0.9878818807890025,
        "known_free_MiB_at_cutoff": 139.3125,
        "time_average_known_free_MiB": 72.46655080482611,
        "right_censored_share_of_observed_free_slice_time": 0.24930835243074984,
        "share_of_free_slice_time_with_age_above_ms": {
          "1": 0.4705149117375412,
          "10": 0.2414720186460886,
          "25": 0.1538933950421305,
          "50": 0.08156383267192319,
          "100": 0.010497782491972196
        },
        "known_free_MiB_with_age_at_least_ms_at_cutoff": {
          "1": 105.25,
          "10": 69.6875,
          "25": 44.0625,
          "50": 36.875,
          "100": 11.3125
        },
        "free_slice_time_conservation_exact": true
      }
    },
    "sparse-3": {
      "sha256": "cd075a518ab104daec05cb9526ab427a4419850919b69af2b99e9fcca25a6dac",
      "full_trace": {
        "valid": true,
        "events": 120748,
        "window_ms": 2813.780453,
        "global_timestamp_inversions": 174,
        "completed_reuse_slices": 2249443,
        "right_censored_free_slices": 2707,
        "completed_within_1ms_fraction": 0.9883348900149949,
        "known_free_MiB_at_cutoff": 169.1875,
        "time_average_known_free_MiB": 136.6107164314669,
        "right_censored_share_of_observed_free_slice_time": 0.9659005265574301,
        "share_of_free_slice_time_with_age_above_ms": {
          "1": 0.9770499461387435,
          "10": 0.9654998327687769,
          "25": 0.9578835784971109,
          "50": 0.9476435125332179,
          "100": 0.9290353914360091
        },
        "known_free_MiB_with_age_at_least_ms_at_cutoff": {
          "1": 142.1875,
          "10": 141.6875,
          "25": 141.6875,
          "50": 141.6875,
          "100": 141.6875
        },
        "free_slice_time_conservation_exact": true
      },
      "last_recorded_arena_claim_cutoff": {
        "valid": true,
        "events": 120714,
        "window_ms": 212.169042,
        "global_timestamp_inversions": 174,
        "completed_reuse_slices": 2249443,
        "right_censored_free_slices": 2100,
        "completed_within_1ms_fraction": 0.9883348900149949,
        "known_free_MiB_at_cutoff": 131.25,
        "time_average_known_free_MiB": 74.35060220048503,
        "right_censored_share_of_observed_free_slice_time": 0.16908587043965168,
        "share_of_free_slice_time_with_age_above_ms": {
          "1": 0.4425570470824442,
          "10": 0.18053715036602946,
          "25": 0.08718664262036008,
          "50": 0.03304739246110129,
          "100": 0.006376495999629009
        },
        "known_free_MiB_with_age_at_least_ms_at_cutoff": {
          "1": 111.5625,
          "10": 101.6875,
          "25": 18.5,
          "50": 18.25,
          "100": 2.5
        },
        "free_slice_time_conservation_exact": true
      }
    },
    "persistent": {
      "sha256": "abbda7837a018cae3b229912e083e0bd762598012ccb0783ffd9e2d47b4b4ff6",
      "full_trace": {
        "valid": true,
        "events": 85442,
        "window_ms": 2912.37183,
        "global_timestamp_inversions": 35,
        "completed_reuse_slices": 687745,
        "right_censored_free_slices": 1043,
        "completed_within_1ms_fraction": 0.9899105046201717,
        "known_free_MiB_at_cutoff": 65.1875,
        "time_average_known_free_MiB": 19.456344345911525,
        "right_censored_share_of_observed_free_slice_time": 0.9348786434633712,
        "share_of_free_slice_time_with_age_above_ms": {
          "1": 0.9595934305494658,
          "10": 0.9409516759899553,
          "25": 0.9304576425330084,
          "50": 0.9185971665333778,
          "100": 0.8998645878380296
        },
        "known_free_MiB_with_age_at_least_ms_at_cutoff": {
          "1": 20.0,
          "10": 20.0,
          "25": 20.0,
          "50": 20.0,
          "100": 20.0
        },
        "free_slice_time_conservation_exact": true
      },
      "last_recorded_arena_claim_cutoff": {
        "valid": true,
        "events": 85396,
        "window_ms": 310.716953,
        "global_timestamp_inversions": 35,
        "completed_reuse_slices": 687745,
        "right_censored_free_slices": 304,
        "completed_within_1ms_fraction": 0.9899105046201717,
        "known_free_MiB_at_cutoff": 19.0,
        "time_average_known_free_MiB": 14.897483660152911,
        "right_censored_share_of_observed_free_slice_time": 0.20282506100520625,
        "share_of_free_slice_time_with_age_above_ms": {
          "1": 0.5062660823817091,
          "10": 0.2858446232932152,
          "25": 0.18049351279615292,
          "50": 0.08844231107336567,
          "100": 0.003910237776696162
        },
        "known_free_MiB_with_age_at_least_ms_at_cutoff": {
          "1": 17.9375,
          "10": 13.875,
          "25": 11.8125,
          "50": 8.9375,
          "100": 0.0
        },
        "free_slice_time_conservation_exact": true
      }
    }
  }
}

Reproducer (scratch-only standard-library analysis):

"""Offline arena-free slice-time; not RSS, retired-page lifetimes or a discard-policy trial."""
import hashlib,json,struct
from pathlib import Path
ROOT=Path('/tmp/mimalloc-585')
SLICE_BYTES=64*1024
MIB=1024*1024
AGE_THRESHOLDS_NS={str(ms):ms*1_000_000 for ms in (1,10,25,50,100)}
NAMES=('sparse','sparse-2','sparse-3','persistent')

def analyze(events,cutoff):
    selected=events[:cutoff+1]
    start=min(x[1] for x in selected);end=max(x[1] for x in selected)
    live=set();free={};completed=[];changes=[];negative=[]
    for addr,ns,count,freed in selected:
        assert addr%SLICE_BYTES==0
        for index in range(count):
            key=addr//SLICE_BYTES+index
            if freed:
                assert key in live,('free without claim',key)
                live.remove(key);assert key not in free
                free[key]=ns;changes.append((ns,1))
            else:
                assert key not in live,('duplicate claim',key)
                live.add(key)
                if key in free:
                    duration=ns-free.pop(key)
                    if duration<0:negative.append(duration)
                    completed.append(duration);changes.append((ns,-1))
    if negative:
        return {'valid':False,'negative_reuse_intervals':len(negative),'minimum_ns':min(negative)}
    censored=[end-ns for ns in free.values()]
    completed_area=sum(completed);censored_area=sum(censored);area=completed_area+censored_area
    # Independent conservation: integrate the inventory of known free slices over time.
    inventory=0;integral=0;previous=start
    for ns,delta in sorted(changes):
        integral+=inventory*(ns-previous);inventory+=delta;previous=ns
        assert inventory>=0
    integral+=inventory*(end-previous)
    assert integral==area,(integral,area)
    assert inventory==len(free)
    durations=completed+censored
    return {'valid':True,'events':len(selected),'window_ms':(end-start)/1e6,
        'global_timestamp_inversions':sum(a[1]>b[1] for a,b in zip(selected,selected[1:])),
        'completed_reuse_slices':len(completed),'right_censored_free_slices':len(censored),
        'completed_within_1ms_fraction':sum(d<1_000_000 for d in completed)/len(completed),
        'known_free_MiB_at_cutoff':len(free)*SLICE_BYTES/MIB,
        'time_average_known_free_MiB':area/(end-start)*SLICE_BYTES/MIB,
        'right_censored_share_of_observed_free_slice_time':censored_area/area,
        'share_of_free_slice_time_with_age_above_ms':{label:sum(max(0,d-age) for d in durations)/area for label,age in AGE_THRESHOLDS_NS.items()},
        'known_free_MiB_with_age_at_least_ms_at_cutoff':{label:sum(d>=age for d in censored)*SLICE_BYTES/MIB for label,age in AGE_THRESHOLDS_NS.items()},
        'free_slice_time_conservation_exact':True}

out={'purpose':'Descriptive existing local traces; virtual arena-free slice-time, not residency or acceptance',
     'limits':['Callbacks instrumented the old8b policy; wall time affected by instrumentation.',
               'Arena claim/free only; no purge/residency events and no page-retirement events.',
               'Full trace includes drain/shutdown. Last recorded arena allocation is only a second diagnostic cutoff, NOT an authoritative work/peak marker.',
               'Intervals still free at cutoff are right-censored; observed time is a lower bound on lifetime.'],
     'traces':{}}
old=json.loads((ROOT/'reuse-trace-reports.json').read_text())
for name in NAMES:
    path=ROOT/f'reuse-{name}.bin';raw=path.read_bytes();events=list(struct.iter_unpack('<QQII',raw))
    last_claim=max(i for i,e in enumerate(events) if e[3]==0)
    full=analyze(events,len(events)-1);proxy=analyze(events,last_claim)
    assert full['valid'] and proxy['valid'],(name,full,proxy)
    assert full['completed_reuse_slices']==old[name]['reused_slices']
    assert full['completed_within_1ms_fraction']==old[name]['fraction_reused_under_1ms']
    out['traces'][name]={'sha256':hashlib.sha256(raw).hexdigest(),'full_trace':full,'last_recorded_arena_claim_cutoff':proxy}
    print(name,'lastclaim: avgfreeMiB',round(proxy['time_average_known_free_MiB'],2),'censoredshare',round(proxy['right_censored_share_of_observed_free_slice_time'],3),'areaage>1ms',round(proxy['share_of_free_slice_time_with_age_above_ms']['1'],3),'areaage>10ms',round(proxy['share_of_free_slice_time_with_age_above_ms']['10'],3),flush=True)
(ROOT/'reuse-time-weighted.json').write_text(json.dumps(out,indent=2)+'\n')

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

#585: current platform CI complete — 104 SUCCESS, five intentional SKIPs, zero failures

Current head da1fdb9819fdaa42a57e96d1d6cd741cf2711d03, matching the pushed PR source. All 109 current PR checks are terminal. This validates platform correctness, not performance acceptance. Six independent exact-row performance jobs are still live; no new memory policy or RSS win is claimed.

Passed coverage includes:

  • Linux glibc parallel + serial and musl bundles, minimal observability-OFF build and fastpath identity, eight-run memory gate, injected-leak positive control, diagnostics/firewall/ISA and Bun consumer checks.
  • Native Microsoft cl Release build/run hard gate, plus cross-built win-gnu UCRT and MSVC-ABI clang-cl bundles, Rust binaries, pprof text/proto parsing, Windows eight-run memory collections. Both soldr Windows lanes lack committed per-toolchain baselines, so the baseline comparisons and leak-positive controls were skipped with warnings, despite green steps. This is the documented bootstrap path, not proof of Windows memory nonregression or working leak controls. No baseline was created from the candidate to mask this gap. Native compile-compat enumeration and cross bundle coverage agree across all eight Windows configurations (no missing or extra tests).
  • Native Apple Silicon and Intel execution of the Linux-built Mac bundles; Rust-native and the cross target matrix; GCC/Clang ASan, owner-gate/TSan stress, teardown and fuzz gates; vendor consistency, lint and snippets.

The serial reuse fixture is not skipped: actual JUnit has 36 passing purge/reuse entries across nine Linux bundles; musl's two new reuse entries both ran serial-reuse-scavenger-disabled and the original dead-thread-page-is-reused, two assertions passed each. Background release cases remain in the original ON/OFF suites. The old 6f musl failure and controlled baseline interleaving are retained in prior reports; the fixture correction did not relax either same-page or zero-new-pages assertion.

Intentional skipped checks:

  • run-macos-x64-selective (tests labelled macos, in Recovery)
  • run-macos-x64 (macOS Recovery on Linux)
  • memory-gate (minimal lane, linux pprof)
  • benchmark-sentinel (${{ matrix.os }})
  • ab

Recovery execution is manual/selective; native Macs passed in this full lane. The minimal-only memory-gate check skips because the full Linux lane executed its real baseline gate and injected-leak positive control. Windows bootstrap warnings are a separate limitation inside successful steps, as stated above. The automatic perf-ab/sentinel triggers skip independently of the six explicit live diagnostic/acceptance dispatches. None is represented as a performance PASS.

Exact check names, states and links:

[
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119021576073",
    "name": "PR test gate (windows-bundles) / verify",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119016149541",
    "name": "PR test gate (macos-bundles) / verify",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119015779521",
    "name": "PR test gate (c-unit) / verify",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119015711503",
    "name": "coverage",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628377/job/119013565588",
    "name": "PR test gate (rust-native) / verify",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628345/job/119013443953",
    "name": "PR test gate (asan) / verify",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628332/job/119012771747",
    "name": "PR test gate (fuzz) / verify",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628381/job/119012753140",
    "name": "PR test gate (purge-teardown-regression) / verify",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119012598915",
    "name": "ctest (windows-latest)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119012445195",
    "name": "run-windows",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119012258846",
    "name": "PR test gate (cross) / verify",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119012190771",
    "name": "run-macos-native-full (arm64)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119012190686",
    "name": "run-macos-native-full (x64)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119012192604",
    "name": "run-macos-x64-selective (tests labelled macos, in Recovery)",
    "state": "SKIPPED"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119012192003",
    "name": "run-macos-x64 (macOS Recovery on Linux)",
    "state": "SKIPPED"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011940711",
    "name": "test (aarch64-unknown-linux-gnu)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011940670",
    "name": "test (x86_64-pc-windows-msvc)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011940665",
    "name": "test (x86_64-unknown-linux-musl)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011940646",
    "name": "test (x86_64-pc-windows-gnu)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011940726",
    "name": "test (x86_64-unknown-linux-gnu)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011886702",
    "name": "run-linux",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011886669",
    "name": "run-linux-serial",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119011839571",
    "name": "build-windows-gnu (windows-gnu-x64-leak)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119011839584",
    "name": "build-windows-msvc (windows-msvc-x64-leak)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119011839442",
    "name": "build-rust (x86_64-pc-windows-gnu)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119011839377",
    "name": "build-rust (x86_64-pc-windows-msvc)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119011839511",
    "name": "build-windows-msvc (windows-msvc-x64-shared)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119011839494",
    "name": "build-windows-gnu (windows-gnu-x64-gated)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119011839499",
    "name": "build-windows-gnu (windows-gnu-x64-release)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119011839436",
    "name": "build-windows-gnu (windows-gnu-x64-debug-full)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119011839526",
    "name": "build-windows-msvc (windows-msvc-x64-release)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119011839508",
    "name": "build-windows-msvc (windows-msvc-x64-gated)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119011839487",
    "name": "build-windows-msvc (windows-msvc-x64-debug-full)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119011839444",
    "name": "build-windows-gnu (windows-gnu-x64-shared)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119011787833",
    "name": "pr-ci-mode / decide",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628340/job/119011787615",
    "name": "resolve-candidate (exact SHA )",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628381/job/119011745436",
    "name": "reproduce (ON, true)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628381/job/119011745341",
    "name": "reproduce (OFF, false)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628381/job/119011745370",
    "name": "reproduce (ON, false)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545971",
    "name": "build (memory-gate-leak)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119011570767",
    "name": "build-macos (macos-x64-leak)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545931",
    "name": "build (memevt-only)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545912",
    "name": "build (gated)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545899",
    "name": "build (diag-pprof-on)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545896",
    "name": "build (debug-full)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545932",
    "name": "build (guarded)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545738",
    "name": "fastpath-identity",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119011570795",
    "name": "build-macos (macos-arm64-release)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545857",
    "name": "build (shared)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119011570703",
    "name": "build-macos (macos-arm64-debug-full)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119011570764",
    "name": "build-macos (macos-arm64-leak)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545805",
    "name": "build-bun-objects (glibc)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119011570729",
    "name": "build-macos (macos-x64-release)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119011570733",
    "name": "build-macos (macos-x64-debug-full)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011593800",
    "name": "memory-gate (minimal lane, linux pprof)",
    "state": "SKIPPED"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545863",
    "name": "build (release)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545625",
    "name": "isa-baseline (ubuntu-24.04-arm)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011531383",
    "name": "build (x86_64-pc-windows-msvc)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545889",
    "name": "build (debug3-extra)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545856",
    "name": "build (dhat-on)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545858",
    "name": "build (musl-pprof-off)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545824",
    "name": "build (isa-portable)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545868",
    "name": "build (diag-pprof-off)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545675",
    "name": "build-bun-objects (musl)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628289/job/119011544119",
    "name": "lint",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545830",
    "name": "build (musl)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545849",
    "name": "build (pprof-off)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545961",
    "name": "build (isa-arch)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545655",
    "name": "decide (memory gate lane)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011545662",
    "name": "build-windows-native",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011531274",
    "name": "build (x86_64-pc-windows-gnu)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628377/job/119011538138",
    "name": "test-win-gnu",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628377/job/119011538126",
    "name": "test (windows-latest)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628377/job/119011538104",
    "name": "test-no-pprof",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628377/job/119011538011",
    "name": "test (ubuntu-latest)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011531252",
    "name": "build (aarch64-pc-windows-msvc)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628332/job/119011536645",
    "name": "fuzz",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119011529772",
    "name": "build-rust (x86_64-apple-darwin)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119011529711",
    "name": "build-rust (aarch64-apple-darwin)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011531262",
    "name": "build (x86_64-unknown-linux-gnu)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011531354",
    "name": "build (aarch64-unknown-linux-gnu)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011531333",
    "name": "build (aarch64-apple-darwin)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119011529748",
    "name": "decide (selective macOS lane)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011531300",
    "name": "build (x86_64-apple-darwin)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011531330",
    "name": "build (x86_64-unknown-linux-musl)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628345/job/119011527076",
    "name": "address sanitizer (clang, Debug, owner-gate)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628345/job/119011527114",
    "name": "address sanitizer (clang, Debug)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628345/job/119011527687",
    "name": "address sanitizer (gcc, RelWithDebInfo)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628345/job/119011526983",
    "name": "multi-threaded stress (debug-full, tsan)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628345/job/119011527112",
    "name": "address sanitizer (clang, RelWithDebInfo)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628381/job/119011527229",
    "name": "build (ON, false)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628381/job/119011527314",
    "name": "build (OFF, false)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628381/job/119011527415",
    "name": "build (ON, true)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628332/job/119011489896",
    "name": "pr-ci-mode / decide",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119011490396",
    "name": "resolve-candidate (exact SHA)",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628302/job/119011489875",
    "name": "vendored C amalgamation matches src/",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011490911",
    "name": "pr-ci-mode / decide",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011490091",
    "name": "resolve-candidate (exact SHA )",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628377/job/119011490234",
    "name": "resolve-candidate (exact SHA )",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628309/job/119011490106",
    "name": "check",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628337/job/119011490533",
    "name": "pr-ci-mode / decide",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628381/job/119011490224",
    "name": "pr-ci-mode / decide",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628289/job/119011489608",
    "name": "ci-mode",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628377/job/119011490486",
    "name": "pr-ci-mode / decide",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628336/job/119011490428",
    "name": "pr-ci-mode / decide",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628345/job/119011489885",
    "name": "pr-ci-mode / decide",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628470/job/119011490518",
    "name": "resolve-candidate (exact SHA )",
    "state": "SUCCESS"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628292/job/119011490601",
    "name": "benchmark-sentinel (${{ matrix.os }})",
    "state": "SKIPPED"
  },
  {
    "link": "https://github.com/zackees/mimalloc-pprof/actions/runs/39535628288/job/119011490559",
    "name": "ab",
    "state": "SKIPPED"
  }
]

Local-Gate: v1 tree=f7471f8b9f0d0d334508f5208f17de19f724b238 secs=35 lanes=lint:run
Ci-Attestation: {"at":1792553893,"gate":"python/all/lint","host":"linux-x86_64","key":"59477bdcb03af0225aea620651d157f51ae10560f6977c0d0152eee4315bc4f6","lane":"lint","parents":["da1fdb9819fdaa42a57e96d1d6cd741cf2711d03"],"secs":35,"stamp":"1992829c7fbbf4ecd3222dd9111a8f0f","tree":"f7471f8b9f0d0d334508f5208f17de19f724b238","v":1,"via":"run"}
@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

#585: current short-generation chart row PASS at 60 actual samples per arm

Run39537470652, frozen merged ee3623f6b7a485754a0a0aa3462d8cba43390e8b → allocator candidate da1fdb9819fdaa42a57e96d1d6cd741cf2711d03, EPYC7763 / four logical CPUs. Original exact short-generation chart trace, 60 base / 60 head / 60 null, balanced arm positions, unchanged sanity/ledger/latency protocols and thresholds. All six metrics PASS; zero memory credit. No RSS improvement is established.

base ee3623f6b7a4 head da1fdb9819fd on AMD EPYC 7763 64-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class-ephemeral/8 short generations (chart build) 0% PASS +1.0% [+0.5, +1.7] / 0.7% PASS -1.2% [-1.6, -0.7] / 0.6% PASS +0.4% [-0.4, +0.8] / 1.2% PASS -1.1% [-2.0, -0.5] / 1.1% PASS +0.0% [-2.6, +2.6] / 2.7% PASS +0.0% [-2.6, +2.7] / 2.8% PASS

All four actual timing/latency files SHA256-verified. Both baseline chart files match the earlier actual chart baseline byte for byte. Both head chart files differ from old15, as expected with the word-hash source change compiled into chart. The old15 120/240-sample INCONCLUSIVE outcomes remain unchanged and are not pooled into this different-source result. Chart has profiling compiled in but runtime-unarmed; its CPU result must not be attributed to executing the new hash loop. The fixed-baseline ledger reports the whole candidate.

A new config/docs-only commit activates Windows memory baselines from historical merged-policy artifacts; it changes no allocator, header, benchmark or Rust source. This measured allocator remains da1fdb9; no JSON source ref is relabeled. The other five original individual runs remain live and are not cancelled or restarted.

Actual executable manifest:

{
  "base/chart/perf_ab": "306d5c8a089d29a7ff276df621226fe2d08ffe7013669627f38dc5c0020d33f7",
  "base/chart/perf_ab_latency": "2f883b21f4d48e9e1b404c3110ca058902c68cba23e99a82edd58026c7495090",
  "head/chart/perf_ab": "83f28e34da86483f7c43947f64060944f344b460577d03849e2ac952915f8fbc",
  "head/chart/perf_ab_latency": "1f77d5ba8f0e3da542412443fb9569b27fd898dd116f584cee79f4185a9acdad"
}

Full-precision ledger:

[
  {
    "workload": "large-class-ephemeral/8 short generations (chart build)",
    "counts": {
      "base": 60,
      "head": 60,
      "null": 60
    },
    "saved": 0.0,
    "cells": [
      {
        "metric": "ops/s",
        "base_median": 7625855.5,
        "head_median": 7677377.199999999,
        "change": [
          1.017136857954018,
          0.5234892416622049,
          1.7481919919695432
        ],
        "regression": [
          -1.7481919919695432,
          -0.5234892416622049
        ],
        "allowance": 0.7417747291280052,
        "verdict": "PASS"
      },
      {
        "metric": "cpu s",
        "base_median": 0.428,
        "head_median": 0.42315,
        "change": [
          -1.214893414000124,
          -1.6053978594695248,
          -0.6511627906976672
        ],
        "regression": [
          -1.6053978594695248,
          -0.6511627906976672
        ],
        "allowance": 0.6052141527001842,
        "verdict": "PASS"
      },
      {
        "metric": "minor faults",
        "base_median": 250.0,
        "head_median": 250.0,
        "change": [
          0.39526309211348576,
          -0.3952569169960474,
          0.8032128514056224
        ],
        "regression": [
          -0.3952569169960474,
          0.8032128514056224
        ],
        "allowance": 1.2096774193548387,
        "verdict": "PASS"
      },
      {
        "metric": "p99 operation ns",
        "base_median": 1904.0,
        "head_median": 1883.5,
        "change": [
          -1.0676485119880375,
          -1.9969278033794162,
          -0.5252100840336135
        ],
        "regression": [
          -1.9969278033794162,
          -0.5252100840336135
        ],
        "allowance": 1.0638297872340425,
        "verdict": "PASS"
      },
      {
        "metric": "peak RSS MiB",
        "base_median": 74.59765625,
        "head_median": 74.6015625,
        "change": [
          0.0,
          -2.6158788435061955,
          2.6325430470508193
        ],
        "regression": [
          -2.6158788435061955,
          2.6325430470508193
        ],
        "allowance": 2.738620467018186,
        "verdict": "PASS"
      },
      {
        "metric": "RSS 0.5 s after drain MiB",
        "base_median": 74.59765625,
        "head_median": 74.6015625,
        "change": [
          0.0,
          -2.6158788435061955,
          2.6755327504057806
        ],
        "regression": [
          -2.6158788435061955,
          2.6755327504057806
        ],
        "allowance": 2.7546134395007265,
        "verdict": "PASS"
      }
    ]
  }
]

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

#585: activate missing Windows memory comparisons from historical merged-policy data

Current green platform run39535628340 had collected Windows results but skipped both memory baseline comparisons and leak-positive controls because the compiler-specific references were absent. This limitation is retained in the full-CI report. Green steps alone were not evidence that those comparisons ran.

Historical reference, not candidate data: full run36812458448 executed source 80a513dfa6f84a16a07b8612c4041acb7cee4091 on 2026-10-01 (run title and checkout log both verified). It is an ancestor of frozen merged EE; GitHub compare shows six commits ahead/zero behind, and the local ancestor check passes after deepening this shallow checkout. The allocator, headers, tests, CMake/toolchain files, Windows workflow, bundle producer/runner and comparator are byte-identical to EE. The intervening changes concern CI policy, attestation, docs and unrelated benchmark workflows.

memory_gate.py update consumed ONLY each compiler's eight historical, unfiltered, non-leak normal records. Both minimum peak-commit references are71.6MB with0.1%spread. Values have the producer's0.1MB precision; they do not establish byte equality. Distinct keys remain x64/soldr-mingw-gcc-15 and x64/soldr-clang-cl-21; native cl reference is untouched. No tolerance, comparator logic or workflow was changed.

Using these references, each current eight-run normal set PASSes the existing minimum-of-eight comparator, and each eight-run injected-leak set FAILs it as intended (GNU minimum125.7MB; MSVC131.6MB vs75.2MB allowance). Both control calls therefore PASS. This is supplementary historical Windows peak-commit validation, not a same-host rule-12 CPU/RSS ledger or a new memory-saving claim.

Pushed attested config/docs-only 3db601e13aace89ca44b2be9dbd59b2ee8f5f034 adds two historical reference JSONs, input-hash provenance and updated bootstrap docs. No allocator/header/test/CMake/Rust/benchmark/harness source changed. Same sole primary reviewer checked legitimacy and scope; its LOW minimum-of-eight wording correction was fixed. Required local gate45/45PASS35s. Fresh full CI must exercise these newly activated checks. The five live numeric jobs retain their original da source and handles; short-generation chart already PASSed60/arm. No measurement metadata is relabeled and no numeric job is cancelled/restarted.

Comparator output

wrote /tmp/mimalloc-585/windows-memory-baseline/frozen-reference/windows-x64-soldr-mingw-gcc-15-pprof1.json (min peak 71.6 MB of 8 runs)
  runs (MB)              71.6  71.6  71.6  71.6  71.6  71.6  71.6  71.7
  spread across runs     0.1%  (min is used; noise on a shared runner only adds memory)
baseline=windows+x64+soldr-mingw-gcc-15+pprof1 metric=peak_commit  (windows-x64-soldr-mingw-gcc-15-pprof1.json)
  peak_commit                 71.6 MB   baseline      71.6 MB   allowed      75.2 MB
  profiler arena               0.0 MB   (reported so the profiler's own arena is not mistaken for allocator growth)
  runs (MB)              71.6  71.6  71.6  71.6  71.6  71.6  71.6  71.7
  spread across runs     0.1%  (min is used; noise on a shared runner only adds memory)
  scenario               peak MB  delta MB    rss MB baseline MB
  thread_churn              57.8         -       7.6        57.7
  sawtooth                  57.8      +0.0      16.6        57.7
  cross_thread_free         57.8      +0.0      22.0        57.7
  rolling_heaps             57.8      +0.0      22.0        57.7
  huge_churn                71.6     +13.8      22.0        71.6
  counters: threads 1->1  theaps 1->1  pages 0->0

PASS
=== positive control: the gate is expected to FAIL below ===
baseline=windows+x64+soldr-mingw-gcc-15+pprof1 metric=peak_commit  (windows-x64-soldr-mingw-gcc-15-pprof1.json)
  peak_commit                125.7 MB   baseline      71.6 MB   allowed      75.2 MB
  profiler arena               0.0 MB   (reported so the profiler's own arena is not mistaken for allocator growth)
  runs (MB)              125.7  131.6  131.6  131.6  131.6  131.6  131.6  131.7
  spread across runs     4.8%  (min is used; noise on a shared runner only adds memory)
  scenario               peak MB  delta MB    rss MB baseline MB
  thread_churn              93.8         -      42.4        57.7  <- moved
  sawtooth                  93.8      +0.0      51.8        57.7
  cross_thread_free         93.9      +0.1      54.2        57.7
  rolling_heaps             93.9      +0.0      54.2        57.7
  huge_churn               125.7     +31.8      54.2        71.6
  counters: threads 1->1  theaps 1->1  pages 0->41

FAIL
  - peak_commit regressed: 125.7 MB vs baseline 71.6 MB (+75.6%, allowed +5%) -- see the per-scenario table above for which scenario moved (first over its baseline: thread_churn)

If this growth is intentional, re-baseline deliberately:
    python ci/memory_gate.py update --arch x64 --compiler soldr-mingw-gcc-15 /tmp/mimalloc-585/windows-memory-baseline/current-gnu/leak-1.json /tmp/mimalloc-585/windows-memory-baseline/current-gnu/leak-2.json /tmp/mimalloc-585/windows-memory-baseline/current-gnu/leak-3.json /tmp/mimalloc-585/windows-memory-baseline/current-gnu/leak-4.json /tmp/mimalloc-585/windows-memory-baseline/current-gnu/leak-5.json /tmp/mimalloc-585/windows-memory-baseline/current-gnu/leak-6.json /tmp/mimalloc-585/windows-memory-baseline/current-gnu/leak-7.json /tmp/mimalloc-585/windows-memory-baseline/current-gnu/leak-8.json
and say why in the PR. Do not re-baseline to make a red gate green.

PASS (control): the gate detected the injected leak.

wrote /tmp/mimalloc-585/windows-memory-baseline/frozen-reference/windows-x64-soldr-clang-cl-21-pprof1.json (min peak 71.6 MB of 8 runs)
  runs (MB)              71.6  71.6  71.6  71.6  71.6  71.6  71.6  71.7
  spread across runs     0.1%  (min is used; noise on a shared runner only adds memory)
baseline=windows+x64+soldr-clang-cl-21+pprof1 metric=peak_commit  (windows-x64-soldr-clang-cl-21-pprof1.json)
  peak_commit                 71.6 MB   baseline      71.6 MB   allowed      75.2 MB
  profiler arena               0.0 MB   (reported so the profiler's own arena is not mistaken for allocator growth)
  runs (MB)              71.6  71.6  71.6  71.6  71.6  71.6  71.6  71.7
  spread across runs     0.1%  (min is used; noise on a shared runner only adds memory)
  scenario               peak MB  delta MB    rss MB baseline MB
  thread_churn              57.7         -       7.5        57.7
  sawtooth                  57.7      +0.0      16.6        57.7
  cross_thread_free         57.7      +0.0      21.8        57.7
  rolling_heaps             57.7      +0.0      21.8        57.7
  huge_churn                71.6     +13.9      21.8        71.6
  counters: threads 1->1  theaps 1->1  pages 0->0

PASS
=== positive control: the gate is expected to FAIL below ===
baseline=windows+x64+soldr-clang-cl-21+pprof1 metric=peak_commit  (windows-x64-soldr-clang-cl-21-pprof1.json)
  peak_commit                131.6 MB   baseline      71.6 MB   allowed      75.2 MB
  profiler arena               0.0 MB   (reported so the profiler's own arena is not mistaken for allocator growth)
  runs (MB)              131.6  131.7  131.7  131.7  131.7  131.7  131.7  131.7
  spread across runs     0.1%  (min is used; noise on a shared runner only adds memory)
  scenario               peak MB  delta MB    rss MB baseline MB
  thread_churn              97.8         -      42.2        57.7  <- moved
  sawtooth                  99.5      +1.7      51.9        57.7
  cross_thread_free         99.9      +0.4      52.1        57.7
  rolling_heaps             99.9      +0.0      52.1        57.7
  huge_churn               131.6     +31.7      52.2        71.6
  counters: threads 1->1  theaps 1->1  pages 0->43

FAIL
  - peak_commit regressed: 131.6 MB vs baseline 71.6 MB (+83.8%, allowed +5%) -- see the per-scenario table above for which scenario moved (first over its baseline: thread_churn)

If this growth is intentional, re-baseline deliberately:
    python ci/memory_gate.py update --arch x64 --compiler soldr-clang-cl-21 /tmp/mimalloc-585/windows-memory-baseline/current-msvc/msvc-leak-1.json /tmp/mimalloc-585/windows-memory-baseline/current-msvc/msvc-leak-2.json /tmp/mimalloc-585/windows-memory-baseline/current-msvc/msvc-leak-3.json /tmp/mimalloc-585/windows-memory-baseline/current-msvc/msvc-leak-4.json /tmp/mimalloc-585/windows-memory-baseline/current-msvc/msvc-leak-5.json /tmp/mimalloc-585/windows-memory-baseline/current-msvc/msvc-leak-6.json /tmp/mimalloc-585/windows-memory-baseline/current-msvc/msvc-leak-7.json /tmp/mimalloc-585/windows-memory-baseline/current-msvc/msvc-leak-8.json
and say why in the PR. Do not re-baseline to make a red gate green.

PASS (control): the gate detected the injected leak.

Source path identity and all raw input SHA256 values

{
  "source_path_identity_verified": [
    "src",
    "include",
    "test",
    "CMakeLists.txt",
    "cmake",
    ".github/workflows/windows-bundles.yml",
    "ci/bundle_tests.py",
    "ci/run_test_bundle.py",
    "ci/memory_gate.py"
  ],
  "checks": {
    "gnu": {
      "frozen_source": "80a513dfa6f84a16a07b8612c4041acb7cee4091",
      "frozen_run": "36812458448",
      "current_source": "da1fdb9819fdaa42a57e96d1d6cd741cf2711d03",
      "current_run": "39535628340",
      "compiler": "soldr-mingw-gcc-15",
      "normal_old": 8,
      "normal_current": 8,
      "leak_current": 8,
      "baseline_from_old_rc": 0,
      "current_check_rc": 0,
      "current_control_rc": 0
    },
    "msvc": {
      "frozen_source": "80a513dfa6f84a16a07b8612c4041acb7cee4091",
      "frozen_run": "36812458448",
      "current_source": "da1fdb9819fdaa42a57e96d1d6cd741cf2711d03",
      "current_run": "39535628340",
      "compiler": "soldr-clang-cl-21",
      "normal_old": 8,
      "normal_current": 8,
      "leak_current": 8,
      "baseline_from_old_rc": 0,
      "current_check_rc": 0,
      "current_control_rc": 0
    }
  },
  "artifact_sha256": {
    "old-gnu/result-1.json": "20ce9f2f9ea5bd887e952e73bf4c7dc6f015df3dd7479cc6603f48f0db9226c5",
    "old-gnu/result-2.json": "1cd1bbd6ed97a75cf47c7a86a0dcf798c2b372d64f119dc581b3a954a7e9407c",
    "old-gnu/result-3.json": "26b65dad2175b5f6bcccd86af633cb10c93ad5fdcd30721d702327ec266d7abf",
    "old-gnu/result-4.json": "72aa3ad19fb0addd8248e163f34252e1cef0dbe6bcf380df79733e1b7077e13b",
    "old-gnu/result-5.json": "b77896aa12495379fa3882d3168288b22e560699008754f927870c36e39f1559",
    "old-gnu/result-6.json": "7ca80dec1fdeecc8b551aa867708f0b6b35a16433f7b58140025127971d3ceb5",
    "old-gnu/result-7.json": "57d530b510ae17f29ce163219a63ff3bd99552e5dbafb692f9f9c18d7a5f4614",
    "old-gnu/result-8.json": "cb33fc68953d920be2dedac68bd460a11588578ddc4cfe49fec5691ebb50c347",
    "current-gnu/result-1.json": "8994d207d1e8915fa800912d5d17201494b16a2ecf1ce9b54a0742418325c240",
    "current-gnu/result-2.json": "bb21c5bd929630826ff05991f8b8829c3745741702c98885046b0286ca2abc80",
    "current-gnu/result-3.json": "70ac449238c532e06d897141812871ca4cd7c2a1c736f5b194fa7ed83f233150",
    "current-gnu/result-4.json": "72aa3ad19fb0addd8248e163f34252e1cef0dbe6bcf380df79733e1b7077e13b",
    "current-gnu/result-5.json": "9ff362f50571e6b6c4676ebcd64814051ff154d3c65ca37ecc883d08cd4f7b8f",
    "current-gnu/result-6.json": "6550b65fdf64b3d394f592a401ea658b46cf6adbab063a94b01a216c06a3197c",
    "current-gnu/result-7.json": "9e35c2ea0cfe862fd88b2c1b5c6dee0db90e981efd4b089f4ef72f4ae5b32893",
    "current-gnu/result-8.json": "d876eebc4df097f2ec67464aaa08fcfa83b2142d9cd1fd86f99a93d5c5225198",
    "current-gnu/leak-1.json": "726df29832b82e3051a387c68671560bafb20ea7524ad609b7e4697711ebed23",
    "current-gnu/leak-2.json": "2f9ea995cfadb1d67ecbe257a12d4e63f9a4cb2e71ad6dde7be18cb9b40acebe",
    "current-gnu/leak-3.json": "49134c62c527d6783d89a416d951c241c6ba3370c38e530b03ab85b61db11383",
    "current-gnu/leak-4.json": "f710ce8a3acae4aba7d020e3570c2eaa7401be5bd415c7e5d596170ff31a7352",
    "current-gnu/leak-5.json": "cdab56f95f8c39fce18d46c88a12b9eb89de9e5bbcad42b6ebe9e70eb6e8a1af",
    "current-gnu/leak-6.json": "8d94d01971f58878eeef72439ecd64e4ab00d2ad894b502eb85f172a89ac3686",
    "current-gnu/leak-7.json": "c1b9a75c922b27fe57c02d7418e4853d397b13594bf18277d77a240f7fbc8e13",
    "current-gnu/leak-8.json": "1db77d9a7218621cc455b311eb04aff1bea2578421d20ac55b2295fe4c2dbb3b",
    "old-msvc/msvc-result-1.json": "0b6f467ea1176fae063c5b165b31de7570be6dcb0401512a9899ce5c0d8135f7",
    "old-msvc/msvc-result-2.json": "52e64e84746407d291ae2b9ca62c990cee230698c91e0e8db206cd0c410aa8f4",
    "old-msvc/msvc-result-3.json": "0d9706ae80a7148985a9c7e1ba5ac4376d3a55c1ddfdb8d2cdc65ca0338d0bbf",
    "old-msvc/msvc-result-4.json": "7fa2daf1b2f91b627d457b735b7c41354741c14f610e84cc4353a7f378a4d99b",
    "old-msvc/msvc-result-5.json": "99a124e148b9be1c1a3e65555f80e76f8d11b937119ff9fd30e816035d39a4e1",
    "old-msvc/msvc-result-6.json": "b80e9b8798addf85705baaee1f453b1faa84faf8a940ba00aed968f0efa91ee8",
    "old-msvc/msvc-result-7.json": "1e0805e67bc9d8dd6fd8e3687f6cd48f84d87329b7c06c2ce6062ba7e51ab39b",
    "old-msvc/msvc-result-8.json": "1f112edd79aa7ee6cb3b6ecf978e6ac276a1a07c02749c52108d7b7a102bff39",
    "current-msvc/msvc-result-1.json": "72aa3ad19fb0addd8248e163f34252e1cef0dbe6bcf380df79733e1b7077e13b",
    "current-msvc/msvc-result-2.json": "26b65dad2175b5f6bcccd86af633cb10c93ad5fdcd30721d702327ec266d7abf",
    "current-msvc/msvc-result-3.json": "af88bd4dc8e79b323b30efd4122da3d25b8d9a321be54d76c3afbed75e82fdd9",
    "current-msvc/msvc-result-4.json": "bbd2154229bd97979bffd7731f3a6a7a855ef36b36752b882b84400e77f1f2ab",
    "current-msvc/msvc-result-5.json": "3c691487494f64035c81292276c0774392c020e95b2036f103aa07709e19b081",
    "current-msvc/msvc-result-6.json": "af88bd4dc8e79b323b30efd4122da3d25b8d9a321be54d76c3afbed75e82fdd9",
    "current-msvc/msvc-result-7.json": "9ff362f50571e6b6c4676ebcd64814051ff154d3c65ca37ecc883d08cd4f7b8f",
    "current-msvc/msvc-result-8.json": "b80e9b8798addf85705baaee1f453b1faa84faf8a940ba00aed968f0efa91ee8",
    "current-msvc/msvc-leak-1.json": "4b7b0257df3bca46d32801269ae4778fea34d40b7dcce4a5ba1d8ce1d468c4a9",
    "current-msvc/msvc-leak-2.json": "c13ecb965c06f951d8ba06dd9d96319a82c2ff0c3d170ae35b7bb7da6038751b",
    "current-msvc/msvc-leak-3.json": "a43263983405932788ef84e228fb6f0433dce41c8259871b0ba54e0ef8cc5967",
    "current-msvc/msvc-leak-4.json": "ed99d3810925099bfb9890401ddedbff2abf861ab44c86b943cdca20ccaafbdb",
    "current-msvc/msvc-leak-5.json": "4f6f401390615ae60e206c18685f58ee411d757ec3ab260e430868e5cc54b1bc",
    "current-msvc/msvc-leak-6.json": "d81f05b0d7feda227dbe35e50047363a82afca361b67cddb14ad64a40cee6b48",
    "current-msvc/msvc-leak-7.json": "fab9fdadf833cad412045c8c40e5153dd9c2681ed564a2541cf7c766f2a70917",
    "current-msvc/msvc-leak-8.json": "016cbde5c0de98b9db32d3611b2842b6c89efdf416c4f2b63365df345a1bf3c5"
  }
}

Authoritative ancestry

{"ahead_by":6,"behind_by":0,"merge_base_commit":"80a513dfa6f84a16a07b8612c4041acb7cee4091","status":"ahead"}

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

#585: random/8 and standard chart ephemeral PASS at 80 actual samples per arm

Both runs retain frozen EE → allocator candidate da1fdb9, original exact workload parameters, balanced arm order, separate latency replay, same-run null arm and unchanged sanity/ledger/latency gates. 80 actual base/head/null samples per row, not the header's starting count60. Eight workers on four logical CPUs remain2:1 oversubscribed. No pooling across CPU models or source refs; no original failed/INCONCLUSIVE outcome is erased.

39537464950 — AMD EPYC 9V45 96-Core Processor

base ee3623f6b7a4 head da1fdb9819fd on AMD EPYC 9V45 96-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
random-large/8 0% PASS +0.1% [-1.1, +0.8] / 1.5% PASS +0.2% [-0.9, +0.7] / 0.8% PASS +0.4% [-0.7, +0.8] / 1.5% PASS +0.5% [-2.6, +2.6] / 3.3% PASS -0.0% [-1.1, +0.1] / 1.1% PASS +0.0% [-0.0, +0.0] / 0.0% PASS

All six metrics PASS, zero memory credit. CPU +0.244% [-0.874%,+0.723%] fits0.805%nullnoise; this is not a CPU improvement claim. After-drain RSS interval [-0.0189825%,0%] is not identically zero. All four actual files are byte-identical to the earlier old15 plain pair; this is additional independently reported evidence, not a source-change claim or relabeling of its two prior INCONCLUSIVE runs.

Four actual executable SHA256 entries verified:

{
  "base/plain/perf_ab": "7f0c70ce2450dfa09126c1674bddab53187614f12d5bd648b4fda33ccbd3c712",
  "base/plain/perf_ab_latency": "ef2d5d273681a473e253d1dd2080307f1ae9f09d34977bda2ad4cab9cbfe7823",
  "head/plain/perf_ab": "9ee3617e371e93195de1374ef403455e7ae660848d689ce4bf0ae0ce2f8565b1",
  "head/plain/perf_ab_latency": "c14766d194fd7f3c07652b148d1de18e3e4db90b4414bd563398f5f44ee9b982"
}

Full-precision cells:

[
  {
    "workload": "random-large/8",
    "counts": {
      "base": 80,
      "head": 80,
      "null": 80
    },
    "saved": 0.0,
    "cells": [
      {
        "metric": "ops/s",
        "base_median": 987053.05,
        "head_median": 987009.8999999999,
        "change": [
          0.13973691200725502,
          -1.0739905599530075,
          0.8145600147609683
        ],
        "regression": [
          -0.8145600147609683,
          1.0739905599530075
        ],
        "allowance": 1.506251182961233,
        "verdict": "PASS"
      },
      {
        "metric": "cpu s",
        "base_median": 1.28225,
        "head_median": 1.2824499999999999,
        "change": [
          0.2437701956490842,
          -0.8738481093104464,
          0.7234073964715458
        ],
        "regression": [
          -0.8738481093104464,
          0.7234073964715458
        ],
        "allowance": 0.8046593608705613,
        "verdict": "PASS"
      },
      {
        "metric": "minor faults",
        "base_median": 264.0,
        "head_median": 264.0,
        "change": [
          0.37736386420596946,
          -0.7299270072992701,
          0.7782101167315175
        ],
        "regression": [
          -0.7299270072992701,
          0.7782101167315175
        ],
        "allowance": 1.532567049808429,
        "verdict": "PASS"
      },
      {
        "metric": "p99 operation ns",
        "base_median": 15768.5,
        "head_median": 15768.5,
        "change": [
          0.4694225007753404,
          -2.647684081685241,
          2.576347869800176
        ],
        "regression": [
          -2.647684081685241,
          2.576347869800176
        ],
        "allowance": 3.326234026559759,
        "verdict": "PASS"
      },
      {
        "metric": "peak RSS MiB",
        "base_median": 182.375,
        "head_median": 182.3828125,
        "change": [
          -0.007548307659872912,
          -1.113746082832218,
          0.06639537374170058
        ],
        "regression": [
          -1.113746082832218,
          0.06639537374170058
        ],
        "allowance": 1.1474368800227734,
        "verdict": "PASS"
      },
      {
        "metric": "RSS 0.5 s after drain MiB",
        "base_median": 20.578125,
        "head_median": 20.578125,
        "change": [
          0.0,
          -0.018982536066818528,
          0.0
        ],
        "regression": [
          -0.018982536066818528,
          0.0
        ],
        "allowance": 0.018982536066818528,
        "verdict": "PASS"
      }
    ]
  }
]

39537468554 — AMD EPYC 7763 64-Core Processor

base ee3623f6b7a4 head da1fdb9819fd on AMD EPYC 7763 64-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
large-class-ephemeral/8 (chart build) 0% PASS +1.6% [+1.1, +1.8] / 0.4% PASS -1.1% [-1.3, -0.9] / 0.4% PASS +0.4% [-0.8, +1.2] / 1.2% PASS -1.6% [-2.2, -1.1] / 1.1% PASS -0.0% [-0.1, +0.1] / 2.6% PASS +0.0% [-0.0, +10.6] / 10.6% PASS

All six metrics PASS, zero memory credit. CPU -1.060% [-1.258%,-0.855%], p99 -1.606% [-2.176%,-1.115%]. The minor-fault upper bound1.185770751% is below1.190476190%nullnoise; the drain upper bound10.591350397% equals its original null-noise allowance exactly (PASS under the unchanged <= rule). These comparisons use unrounded values. No RSS reduction is established. Both actual chart executables match the new short-generation chart files; chart has profiling compiled in but runtime-unarmed, so the CPU result is not evidence of executing a faster hash loop.

Four actual executable SHA256 entries verified:

{
  "base/chart/perf_ab": "306d5c8a089d29a7ff276df621226fe2d08ffe7013669627f38dc5c0020d33f7",
  "base/chart/perf_ab_latency": "2f883b21f4d48e9e1b404c3110ca058902c68cba23e99a82edd58026c7495090",
  "head/chart/perf_ab": "83f28e34da86483f7c43947f64060944f344b460577d03849e2ac952915f8fbc",
  "head/chart/perf_ab_latency": "1f77d5ba8f0e3da542412443fb9569b27fd898dd116f584cee79f4185a9acdad"
}

Full-precision cells:

[
  {
    "workload": "large-class-ephemeral/8 (chart build)",
    "counts": {
      "base": 80,
      "head": 80,
      "null": 80
    },
    "saved": 0.0,
    "cells": [
      {
        "metric": "ops/s",
        "base_median": 8196132.699999999,
        "head_median": 8311705.65,
        "change": [
          1.603993854553773,
          1.0971011154024806,
          1.8347869001337975
        ],
        "regression": [
          -1.8347869001337975,
          -1.0971011154024806
        ],
        "allowance": 0.43192585753215545,
        "verdict": "PASS"
      },
      {
        "metric": "cpu s",
        "base_median": 1.5466,
        "head_median": 1.5307,
        "change": [
          -1.0597716959638057,
          -1.2584327970939337,
          -0.8548669127647238
        ],
        "regression": [
          -1.2584327970939337,
          -0.8548669127647238
        ],
        "allowance": 0.394362554952159,
        "verdict": "PASS"
      },
      {
        "metric": "minor faults",
        "base_median": 252.0,
        "head_median": 252.0,
        "change": [
          0.3929288250733364,
          -0.7843137254901961,
          1.185770750988142
        ],
        "regression": [
          -0.7843137254901961,
          1.185770750988142
        ],
        "allowance": 1.1904761904761905,
        "verdict": "PASS"
      },
      {
        "metric": "p99 operation ns",
        "base_median": 1853.0,
        "head_median": 1823.0,
        "change": [
          -1.6060072235935183,
          -2.176220806794055,
          -1.1154489682097044
        ],
        "regression": [
          -2.176220806794055,
          -1.1154489682097044
        ],
        "allowance": 1.1031439602868174,
        "verdict": "PASS"
      },
      {
        "metric": "peak RSS MiB",
        "base_median": 74.638671875,
        "head_median": 76.482421875,
        "change": [
          -0.010473124962651806,
          -0.06959634122091866,
          0.08169100377820893
        ],
        "regression": [
          -0.06959634122091866,
          0.08169100377820893
        ],
        "allowance": 2.5929806181246726,
        "verdict": "PASS"
      },
      {
        "metric": "RSS 0.5 s after drain MiB",
        "base_median": 8.7890625,
        "head_median": 8.818359375,
        "change": [
          0.02206531332744925,
          -0.044111160123511246,
          10.59135039717564
        ],
        "regression": [
          -0.044111160123511246,
          10.59135039717564
        ],
        "allowance": 10.59135039717564,
        "verdict": "PASS"
      }
    ]
  }
]

The three remaining original control jobs (small/8, Larson/8 plain/chart) are still live. New head3db601e1 changes only historical Windows memory references and documentation; allocator/header/test/CMake/Rust/benchmark/harness paths are byte-identical to da1fdb9. Fresh platform CI must exercise the newly enabled Windows checks; the prior104PASS5intentionalSKIP result belongs to da. No whole-candidate completion is claimed.

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

#585: final controls — two PASS, chart Larson remains INCONCLUSIVE

Original frozen EE→da1fdb98 candidate source, exact workload params, separate latency replay, balanced arm positions, unchanged null/sanity/ledger gates and requested60-to-120 plan. Actual counts come from raw sample-array lengths below. Four actual timing/latency hashes per row were verified. These results establish12of13 original cells with supported PASS evidence; chart Larson remains unresolved, so whole-candidate acceptance is incomplete.

39537461436 — AMD EPYC 9V45 96-Core Processor

Actual counts: {'base': 110, 'head': 110, 'null': 110}.

base ee3623f6b7a4 head da1fdb9819fd on AMD EPYC 9V45 96-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
small/8 (control) 0% PASS +0.6% [-0.3, +1.1] / 1.2% PASS -0.6% [-1.2, +1.3] / 1.5% PASS +0.0% [-0.6, +0.0] / 0.6% PASS +0.0% [+0.0, +0.0] / 0.0% PASS +0.0% [+0.0, +0.0] / 0.0% PASS +0.0% [+0.0, +0.0] / 0.0% PASS

All six PASS at110/arm, zero credit. P99, peak and drain percent-change intervals are exactly[0,0]; that statement concerns the paired sign-test intervals, not every individual observation. CPU interval crosses0; no CPU improvement claim.

Verified actual executable manifest:

{
  "base/plain/perf_ab": "7f0c70ce2450dfa09126c1674bddab53187614f12d5bd648b4fda33ccbd3c712",
  "base/plain/perf_ab_latency": "ef2d5d273681a473e253d1dd2080307f1ae9f09d34977bda2ad4cab9cbfe7823",
  "head/plain/perf_ab": "9ee3617e371e93195de1374ef403455e7ae660848d689ce4bf0ae0ce2f8565b1",
  "head/plain/perf_ab_latency": "c14766d194fd7f3c07652b148d1de18e3e4db90b4414bd563398f5f44ee9b982"
}

Full-precision cells:

[
  {
    "workload": "small/8 (control)",
    "counts": {
      "base": 110,
      "head": 110,
      "null": 110
    },
    "saved": 0.0,
    "cells": [
      {
        "metric": "ops/s",
        "base_median": 137009922.8,
        "head_median": 136653328.95,
        "change": [
          0.5727066285388089,
          -0.2537313947741482,
          1.0784954237927962
        ],
        "regression": [
          -1.0784954237927962,
          0.2537313947741482
        ],
        "allowance": 1.1835247411663612,
        "verdict": "PASS"
      },
      {
        "metric": "cpu s",
        "base_median": 1.1146500000000001,
        "head_median": 1.108,
        "change": [
          -0.5910932884950753,
          -1.2378026743765853,
          1.3266536784488336
        ],
        "regression": [
          -1.2378026743765853,
          1.3266536784488336
        ],
        "allowance": 1.535068372298179,
        "verdict": "PASS"
      },
      {
        "metric": "minor faults",
        "base_median": 160.0,
        "head_median": 160.0,
        "change": [
          0.0,
          -0.6172839506172839,
          0.0
        ],
        "regression": [
          -0.6172839506172839,
          0.0
        ],
        "allowance": 0.6172839506172839,
        "verdict": "PASS"
      },
      {
        "metric": "p99 operation ns",
        "base_median": 90.0,
        "head_median": 90.0,
        "change": [
          0.0,
          0.0,
          0.0
        ],
        "regression": [
          0.0,
          0.0
        ],
        "allowance": 0.0,
        "verdict": "PASS"
      },
      {
        "metric": "peak RSS MiB",
        "base_median": 16.484375,
        "head_median": 16.484375,
        "change": [
          0.0,
          0.0,
          0.0
        ],
        "regression": [
          0.0,
          0.0
        ],
        "allowance": 0.0,
        "verdict": "PASS"
      },
      {
        "metric": "RSS 0.5 s after drain MiB",
        "base_median": 16.484375,
        "head_median": 16.484375,
        "change": [
          0.0,
          0.0,
          0.0
        ],
        "regression": [
          0.0,
          0.0
        ],
        "allowance": 0.0,
        "verdict": "PASS"
      }
    ]
  }
]

39537463373 — AMD EPYC 7763 64-Core Processor

Actual counts: {'base': 115, 'head': 115, 'null': 115}.

base ee3623f6b7a4 head da1fdb9819fd on AMD EPYC 7763 64-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
larson/8 0% PASS -0.6% [-1.4, +0.7] / 1.7% PASS +0.0% [-0.7, +1.0] / 1.1% PASS -1.6% [-9.8, +6.8] / 10.2% PASS +0.0% [+0.0, +0.4] / 0.4% PASS -0.0% [-0.5, +1.1] / 2.5% PASS +0.0% [-0.4, +0.2] / 0.4% PASS

All six PASS at115/arm, zero credit. P99 upper bound0.43478260869565216% equals the original null-noise allowance exactly, so it passes the unchanged <= comparison. CPU interval crosses0. RSS intervals are not identically zero.

Verified actual executable manifest:

{
  "base/plain/perf_ab": "7f0c70ce2450dfa09126c1674bddab53187614f12d5bd648b4fda33ccbd3c712",
  "base/plain/perf_ab_latency": "ef2d5d273681a473e253d1dd2080307f1ae9f09d34977bda2ad4cab9cbfe7823",
  "head/plain/perf_ab": "9ee3617e371e93195de1374ef403455e7ae660848d689ce4bf0ae0ce2f8565b1",
  "head/plain/perf_ab_latency": "c14766d194fd7f3c07652b148d1de18e3e4db90b4414bd563398f5f44ee9b982"
}

Full-precision cells:

[
  {
    "workload": "larson/8",
    "counts": {
      "base": 115,
      "head": 115,
      "null": 115
    },
    "saved": 0.0,
    "cells": [
      {
        "metric": "ops/s",
        "base_median": 69396089.1,
        "head_median": 69419002.6,
        "change": [
          -0.5763927626097618,
          -1.4295460178576465,
          0.6754231657754093
        ],
        "regression": [
          -0.6754231657754093,
          1.4295460178576465
        ],
        "allowance": 1.6922745482674078,
        "verdict": "PASS"
      },
      {
        "metric": "cpu s",
        "base_median": 1.0922,
        "head_median": 1.0964,
        "change": [
          0.02773925104023939,
          -0.7326355851569972,
          1.0236781199928846
        ],
        "regression": [
          -0.7326355851569972,
          1.0236781199928846
        ],
        "allowance": 1.1007250808071936,
        "verdict": "PASS"
      },
      {
        "metric": "minor faults",
        "base_median": 623.0,
        "head_median": 607.0,
        "change": [
          -1.611047180667434,
          -9.824561403508772,
          6.772334293948126
        ],
        "regression": [
          -9.824561403508772,
          6.772334293948126
        ],
        "allowance": 10.208816705336426,
        "verdict": "PASS"
      },
      {
        "metric": "p99 operation ns",
        "base_median": 230.0,
        "head_median": 230.0,
        "change": [
          0.0,
          0.0,
          0.43478260869565216
        ],
        "regression": [
          0.0,
          0.43478260869565216
        ],
        "allowance": 0.43478260869565216,
        "verdict": "PASS"
      },
      {
        "metric": "peak RSS MiB",
        "base_median": 289.56640625,
        "head_median": 289.7890625,
        "change": [
          -0.026319945254513873,
          -0.5409219190968956,
          1.1169807353675523
        ],
        "regression": [
          -0.5409219190968956,
          1.1169807353675523
        ],
        "allowance": 2.463345528567106,
        "verdict": "PASS"
      },
      {
        "metric": "RSS 0.5 s after drain MiB",
        "base_median": 209.44921875,
        "head_median": 209.7578125,
        "change": [
          0.028017482909335426,
          -0.38910505836575876,
          0.2346456106372677
        ],
        "regression": [
          -0.38910505836575876,
          0.2346456106372677
        ],
        "allowance": 0.3609570944814498,
        "verdict": "PASS"
      }
    ]
  }
]

39537466521 — AMD EPYC 7763 64-Core Processor

Actual counts: {'base': 120, 'head': 120, 'null': 120}.

base ee3623f6b7a4 head da1fdb9819fd on AMD EPYC 7763 64-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
larson/8 (chart build) 0% INCONCLUSIVE +0.7% [-0.3, +1.3] / 1.0% PASS -0.6% [-1.3, +0.5] / 1.4% PASS -1.9% [-13.5, +14.4] / 12.4% INCONCLUSIVE +0.0% [+0.0, +0.4] / 0.4% INCONCLUSIVE +0.7% [-0.8, +2.3] / 3.2% PASS +0.7% [-0.4, +2.3] / 1.5% INCONCLUSIVE

At120/arm, INCONCLUSIVE on minor faults, p99 and drain RSS. CPU, throughput and peak PASS, zero credit. P99 upper0.4166666666666667% exceeds0.398406374501992%nullnoise; rounding both to0.4% cannot turn this into PASS. The absolute head/base p99 delta interval is[0,1]ns and null/base is[-1,0]ns, but the declared percentage-based rule remains authoritative. Fault upper14.4068% exceeds12.4218%noise; drain upper2.3268% exceeds1.4882%noise. There is no FAIL verdict, but this is not accepted.

Verified actual executable manifest:

{
  "base/chart/perf_ab": "306d5c8a089d29a7ff276df621226fe2d08ffe7013669627f38dc5c0020d33f7",
  "base/chart/perf_ab_latency": "2f883b21f4d48e9e1b404c3110ca058902c68cba23e99a82edd58026c7495090",
  "head/chart/perf_ab": "83f28e34da86483f7c43947f64060944f344b460577d03849e2ac952915f8fbc",
  "head/chart/perf_ab_latency": "1f77d5ba8f0e3da542412443fb9569b27fd898dd116f584cee79f4185a9acdad"
}

Full-precision cells:

[
  {
    "workload": "larson/8 (chart build)",
    "counts": {
      "base": 120,
      "head": 120,
      "null": 120
    },
    "saved": 0.0,
    "cells": [
      {
        "metric": "ops/s",
        "base_median": 64240218.6,
        "head_median": 64660126.400000006,
        "change": [
          0.6639348247785972,
          -0.26962345445616737,
          1.3098932733762891
        ],
        "regression": [
          -1.3098932733762891,
          0.26962345445616737
        ],
        "allowance": 0.9956393865420201,
        "verdict": "PASS"
      },
      {
        "metric": "cpu s",
        "base_median": 1.1873,
        "head_median": 1.18275,
        "change": [
          -0.5590210131837745,
          -1.2940575515069046,
          0.5143591941705962
        ],
        "regression": [
          -1.2940575515069046,
          0.5143591941705962
        ],
        "allowance": 1.4303329223181143,
        "verdict": "PASS"
      },
      {
        "metric": "minor faults",
        "base_median": 663.5,
        "head_median": 652.5,
        "change": [
          -1.9171514139261647,
          -13.478260869565217,
          14.40677966101695
        ],
        "regression": [
          -13.478260869565217,
          14.40677966101695
        ],
        "allowance": 12.421805183199284,
        "verdict": "INCONCLUSIVE"
      },
      {
        "metric": "p99 operation ns",
        "base_median": 250.0,
        "head_median": 250.0,
        "change": [
          0.0,
          0.0,
          0.4166666666666667
        ],
        "regression": [
          0.0,
          0.4166666666666667
        ],
        "allowance": 0.398406374501992,
        "verdict": "INCONCLUSIVE"
      },
      {
        "metric": "peak RSS MiB",
        "base_median": 285.2578125,
        "head_median": 285.533203125,
        "change": [
          0.7498337345892028,
          -0.7914719235044394,
          2.3084175480981703
        ],
        "regression": [
          -0.7914719235044394,
          2.3084175480981703
        ],
        "allowance": 3.2188553991113014,
        "verdict": "PASS"
      },
      {
        "metric": "RSS 0.5 s after drain MiB",
        "base_median": 46.3515625,
        "head_median": 46.748046875,
        "change": [
          0.7405989186355686,
          -0.4105831016515383,
          2.3267750213858
        ],
        "regression": [
          -0.4105831016515383,
          2.3267750213858
        ],
        "allowance": 1.4881698854610819,
        "verdict": "INCONCLUSIVE"
      }
    ]
  }
]

The new3db601e1 commit changes only Windows historical reference data and documentation; no measured program source changed. Fresh correctness CI is still running to exercise the newly activated Windows memory comparisons and controls. No source refs are relabeled, no different-CPU pooling, no failed/INCONCLUSIVE results erased, and no acceptance exception is made. A focused larger-budget chart-Larson follow-up is being evaluated; it will retain the original trace, sampling and thresholds.

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

Current 3db Windows memory comparisons and injected-leak controls executed

Run 39539769768 has completed all four memory/comparator/control steps with SUCCESS on current config-only 3db601e13aace89ca44b2be9dbd59b2ee8f5f034. The overall Windows job is still running later checks; this is not a full-platform completion claim.

Both artifacts are already uploaded. Downloading and rejudging their actual eight normal and eight leak records per compiler with the unchanged committed comparator independently reproduces normal PASS and positive-control PASS. A positive-control PASS means the leak set correctly triggers a regression FAIL.

compiler identity actual normal runs normal minimum peak commit unchanged allowance actual leak runs leak minimum peak commit result
soldr-mingw-gcc-15 8 71.6 MB 75.2 MB 8 131.6 MB normal PASS; leak detected
soldr-clang-cl-21 8 71.6 MB 75.2 MB 8 126.8 MB normal PASS; leak detected

These are peak commit, not peak RSS or a new memory saving. The existing gate uses the minimum of eight runs; this table does not assert each individual run is below the allowance. References still contain only historical merged-source normal samples documented in the provenance report; no current candidate record initializes a reference, and no tolerance changed. The prior da run's missing-baseline bootstrap remains disclosed. Terminal job logs will be inspected once available.

Artifacts: memory-gate-windows-gnu-x64-soldr (12319202416), memory-gate-windows-msvc-x64-soldr (12320321080).

SHA256 of the 32 actual CI input records
3427214cd5b5c74a612712a2b953bc5e40d814f9f64e5f3ac54e22532a645469  memory-gate-windows-gnu-x64-soldr/result-1.json
2df2c7697f7716615a63f8709c31082e0a8e6ed213bd301975eb5b512f5a8f0d  memory-gate-windows-gnu-x64-soldr/result-2.json
56c141f20b34aff4bbd6d3e590ef60b4f89fd9009e47d79b9c1e9f370aff9a6c  memory-gate-windows-gnu-x64-soldr/result-3.json
f4d7ac08b5c0f9956f25ba6829465e6283114a9a058c22d8ed1501a816f0d63f  memory-gate-windows-gnu-x64-soldr/result-4.json
3427214cd5b5c74a612712a2b953bc5e40d814f9f64e5f3ac54e22532a645469  memory-gate-windows-gnu-x64-soldr/result-5.json
0d9706ae80a7148985a9c7e1ba5ac4376d3a55c1ddfdb8d2cdc65ca0338d0bbf  memory-gate-windows-gnu-x64-soldr/result-6.json
77cf5c6b0ba016fab3a7d8d55f19392d73ad710e9514aee481b6561317714517  memory-gate-windows-gnu-x64-soldr/result-7.json
558cc45f007c2536b2fd717c8e0d533c600a3d3348fa9285ddf7670d47fc0646  memory-gate-windows-gnu-x64-soldr/result-8.json
d1252acae45beb146ed7c22d15ba8ed46370b5152bcf9fe2e46685c33fccf70a  memory-gate-windows-gnu-x64-soldr/leak-1.json
930c42d3c5f1c1e92dcc4453773a4f15a1cdb99f9d17d8271f81d425fdfbf4e0  memory-gate-windows-gnu-x64-soldr/leak-2.json
45d4e493295a7c93d3e1fdd48bb92891d0e827141d08aec9ab1b2a2ca1f66af9  memory-gate-windows-gnu-x64-soldr/leak-3.json
7c640f4e3e22930a128490e72eda7fd34d71689b24fbfdc59066617158fce9c7  memory-gate-windows-gnu-x64-soldr/leak-4.json
b4a26368b186f959f01cff728aa5f012b6878b2e356eaf4447e2bb5fc92c8da4  memory-gate-windows-gnu-x64-soldr/leak-5.json
3ea69609ea435ab197751ddd320204cdd8e787a9b8a3c8d16e202dded3455725  memory-gate-windows-gnu-x64-soldr/leak-6.json
aed43d82fcba23b3913e6aa710a7be9e163ec4240a5a78bef5646cf3e4f5687e  memory-gate-windows-gnu-x64-soldr/leak-7.json
e05f97206adeb69794ef577621188fbd238d8206b489cf45e2bb2502b86b0d09  memory-gate-windows-gnu-x64-soldr/leak-8.json
94a003125f6116fba0f4d870a03692e4417586db96c8400e7ace47639045356e  memory-gate-windows-msvc-x64-soldr/msvc-result-1.json
18a287d28d5682a4beac672dbef8c7d217e78474f7d273da7c1a4c3617f4ebbb  memory-gate-windows-msvc-x64-soldr/msvc-result-2.json
bfba29353a20ef91ff653c0f9142cd6cd3b890e41bec864702412532f8cebeac  memory-gate-windows-msvc-x64-soldr/msvc-result-3.json
99eb7627f9f14972d0fa738b8ea0b102e6f2475f957b7c43f94f8b6d0afa11e9  memory-gate-windows-msvc-x64-soldr/msvc-result-4.json
0ca2e447b90db55fa3214d9f8a20d1bec5915ca049f17ea2c13ad89ba892887d  memory-gate-windows-msvc-x64-soldr/msvc-result-5.json
99a124e148b9be1c1a3e65555f80e76f8d11b937119ff9fd30e816035d39a4e1  memory-gate-windows-msvc-x64-soldr/msvc-result-6.json
ce60cc30cffb60b15379a2f24cd557220e9179c960c6ccc896f1329f19e43c11  memory-gate-windows-msvc-x64-soldr/msvc-result-7.json
3427214cd5b5c74a612712a2b953bc5e40d814f9f64e5f3ac54e22532a645469  memory-gate-windows-msvc-x64-soldr/msvc-result-8.json
ec24f39eaec4a593f5b7f6f2081ec8c8765836d52380816b0f79c58e5517000b  memory-gate-windows-msvc-x64-soldr/msvc-leak-1.json
6252b28c0b0f7f0689c46d4bcd9307b82d950e3b07b38b74a86773380b8c9cf6  memory-gate-windows-msvc-x64-soldr/msvc-leak-2.json
ce886cf66b962ad33ab383c57807ef0fb22ed39ab2c55db39dc3c1ce8a5bb98b  memory-gate-windows-msvc-x64-soldr/msvc-leak-3.json
22ffd6a2675964c6ec13cd7752795a95f32ff9431fc7f73e56a28f070d26c20e  memory-gate-windows-msvc-x64-soldr/msvc-leak-4.json
8154eec8346bf4f3cbe06f971388c6250da006f668196adf88e281bfff30821c  memory-gate-windows-msvc-x64-soldr/msvc-leak-5.json
e12a2b93142c607d9851eb66c40c42a768496bbb7e539409608a670940a4017a  memory-gate-windows-msvc-x64-soldr/msvc-leak-6.json
60db389960e1193f51233f782a1e9add8fec3fe872e392cc56005a7e4f26d674  memory-gate-windows-msvc-x64-soldr/msvc-leak-7.json
868f492dbae585ef98422c696f72a20364940392ff4411fc0f9fef7b0142e7a6  memory-gate-windows-msvc-x64-soldr/msvc-leak-8.json

@zackees

zackees commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

Current full-platform verification completed (3db601e)

Current PR head 3db601e13aace89ca44b2be9dbd59b2ee8f5f034 now has 104 SUCCESS checks and five intentional SKIPs, with no pending or failing checks. Windows bundles 39539769768 completed SUCCESS; C full DAG 39539769702 is also SUCCESS. These are PR validation runs, not a release exact-SHA dispatch.

Passing gates include native Microsoft cl build/execution (ctest (windows-latest)), soldr win-gnu and clang-cl bundles and their native compatibility checks, Linux parallel/serial configurations including minimal/OFF and musl, native macOS arm64/x64, Rust native/cross, default fast-path identity, sanitizers, TSan and fuzz.

Terminal logs now prove the Windows memory comparators and injected-leak controls actually executed against the two intended compiler-specific references. Both normal minima are 71.6 MB peak commit against 75.2 MB allowed; GNU leak minimum131.6 MB and clang-cl leak minimum126.8 MB trigger expected FAIL and PASS (control). All32 actual input hashes and independent comparator verification. The previous da run's bootstrap gap remains recorded; no candidate data or tolerance change initialized these references.

The actual Linux log likewise executes its memory gate: normal58.1 MB peak RSS against baseline58.0/allowed60.9 MB, and injected-leak154.1 MB correctly rejected with PASS (control). The separate minimal-lane-only memory job skips in this full mode; it is not the source of the full-lane memory result.

Intentional skipped checks:

Performance acceptance remains separate and incomplete: twelve of thirteen original cells have supported PASS evidence; chart Larson is INCONCLUSIVE120/arm, with the single predeclared 120-to-180 follow-up39541307803 still running. No memory policy or RSS saving is newly accepted, and earlier failures are preserved.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-full Opt-in complete platform and release validation on PR SHA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant