Repository navigation
Bound an edited claim's read by a cut its own check refuses, fixing #345's flake (#351) - #353
Open
mattlorimor wants to merge 2 commits into
Open
mattlorimor wants to merge 2 commits into
mattlorimor wants to merge 2 commits into
Conversation
…it (#345) TestAClaimNoCutCanSeeIsRefusedBeforeItIsPricedOrAllocatedWhenEdited, from #345, failed for its SublimeCountMinSketch.CheckShape row in one of four full runs on master, 56 bytes past its bound, and on another branch 8 past it. It passed alone. The cause is the baseline. The row held the edited claim's read to the read of a cut just before the field, its envelope's difference and 1,024 bytes. What a read allocates depends on the code the JIT has made of it so far: optimised code inlines and devirtualises away hundreds of bytes, and with dynamic PGO on, the code in between came out higher than either. A cut before the field is refused on another path than the claim, for the bytes it lacks, so the two are optimised apart, and the difference moves with the run. - The failing figures were 82,904 against a bound of 82,896, and 82,976 against 82,920. - Probed alone, from cold through rounds of reads to hot, under the default runtime, with tiering, dynamic PGO or stack allocation off, and under the segments collector, the claim's read ran from -672 to 1,080 bytes past the old cut's. With tiering off it settled at 432 after one round. - A probe that reads every other structure's cuts and makes the Sublime cut's path hot, then runs the real row, failed it byte for byte (82,976 against 82,920) in 7 of 8 runs on master dde2413, 4 regions and 3 of 4 segments, and in 2 of 3 on 2a4dfbc. - Not the cause: the shared ArrayPool's trim, which after a gen2 GC drops an array a thread has held for 30 seconds of trims, sooner under load (SharedArrayPool.Trim, .NET 10.0.14, read 2026-11-06). The forced failure came in a run of about a second, too short for that. Nor an event listener or first-chance handler: the suite has none. The baseline is now the payload cut just after the field, which the same check refuses at its writer's claim, so it runs the claim's code and differs only in the numbers its messages name. Premises hold that: its price stops at the check, for what it lacks, having priced what the cut before the field does, and each of its reads is refused by the check for what it lacks. It is read under the same budget as the edited read, just before it and just after it, in one no-GC region, and the larger is taken, so neither the JIT nor a pool trim can move one between them unseen. A budget changes a read only by refusing it (its limit is only ever compared with what is charged), and the cut is charged no more than its own price, within every budget the edited read is given. Refusing stays 1,024 bytes. - The forcing probe's row passed 8 of 8 runs, 4 under each collector, on the fix as first written and again on this commit. - In all 30 probed states the claim's read came out 48 bytes past the new cut's, where the old cut's ran from -120 to 1,080 in the same states. - Eight instrumented full runs on 2a4dfbc, alternating the collectors, all passed, and the difference was the same in every read of a claim: HeavyKeeper 32 and 96 (one past, the largest), TopK 32 and 80, Sublime 48. A read of the most the bytes hold came out past the cut's, with the envelope's difference, by 75,928 (HeavyKeeper), 154,560 to 155,024 (TopK) and 4,728 to 5,816 (Sublime); the least in any probed state was 4,632. - Those two were measured on the fix as first written, which read the cut under its own price where this reads it under the edited read's; by the above the two allocate alike. They were not measured again. - On this commit, the whole suite six times, alternating the collectors, passed every time, 2,668 tests each, and a clean -warnaserror build exited 0 with the suite passing after it. Mutation through scripts/mutate.sh, on this commit. The fix's own conditions, against this test alone unless marked: T1 price premise: the cut prices more killed (3) T2 price premise: the cut stops elsewhere killed (3) T3 price premise: refused for another reason killed (3) P1 read premise: dropped survived (a) P2 read premise: refused for another reason killed (3) P3 read premise: null guard dropped survived (b) P4 read premise: the read before only survived (b) F0 baseline: the cut before the field killed (3) T4 baseline: a byte before the field ends killed (3) T5 baseline: a byte after the cut survived (c) W baseline: the payload whole killed (3) T6 beside: the smaller of the two survived (d) T7 beside: the read before only survived (d) T8 Refusing: nothing killed (3) T9 Refusing: 1 << 13, whole suite killed (1) T9b Refusing: 1 << 14 killed (1) T10 bound: the envelope difference dropped killed (3) T11 bound: the envelope difference reversed killed (3) T13 control: dropped survived (a) (a) Guards, which no correct library can fail. Without the read premise T4 survived alone (measured on the commit before it), and with it F0, T4 and W are killed alone. A larger Refusing loosens only the edited reads' bound, so only the control can kill T9 and T9b. (b) Equivalent. A cut that reads fails the test anyway, on its Message; and the second read of the same bytes under the same budget is refused as the first is, since the JIT changes code, not what it does. (c) Any cut the check refuses for what it lacks is as good a baseline, and the read premise holds that one is. The cut at the writer's claim is the coarsest. (d) Not killed: no test can place a change of the JIT between two reads. Kept because the edited read ran from 82,008 to 82,976 bytes across the measured states, 968 bytes against 976 of room, so one change landing between the read before and the edited read could take nearly all of it. The read after sees the edited read's code. Under the forcing probe, on the commit before the read premise, T6 and T7 passed and F0 and T4 failed. The #344 library mutants, against the three #344 tests alone: E1-E3, E5-E10 and E12-E16, R1-R4 killed. E4 and E11, Slice's RequireAhead dropped and a byte laxer, survived, and survive the whole suite here and on master dde2413 (2,668 tests each). Before #349 the nested-envelope test killed them; #349 held it to RequireHeld, which since #347 refuses those claims first. This commit leaves that test alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KK3MspqAcczxT45xCfLgyx
…ted claim (#351) Two passages in the #344 bullet went stale. A nested envelope's length was said to be refused by the outer price's slice of it, with that check's loss showing as an ArgumentOutOfRangeException. Since #347 the outer price, from a held payload, holds the length to what the run holds after the header, with the checksum after it, before it prices it (RequireHeld, PersistenceFormat.PriceNested), and that is what refuses one longer: dropping the slice's check now passes the whole suite, on master dde2413 and on this branch. An edited claim's read was said to be bounded by a cut before the field. It is now bounded by the cut just after the field, which the same check refuses, read beside the edited read under the same budget, as the commit before this one made it. The passage says why: a cut before the field is refused on another path, which the JIT optimises apart from the claim's, and across the states of the JIT probed the difference ran from -672 to 1,080 bytes, past the 1,024 the bound allowed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KK3MspqAcczxT45xCfLgyx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #351.
#345's
TestAClaimNoCutCanSeeIsRefusedBeforeItIsPricedOrAllocatedWhenEditedflaked on master in about 1 of 4 full runs: itsSublimeCountMinSketch.CheckShaperow allocated a few bytes past its bound. This PR finds the cause and fixes it in the test alone. Nothing in the library changes.The cause
The row's baseline was the read of a cut just before the edited field. That cut is refused on a different path from the edited claim:
Takerefuses it, with "Payload ended", while the claim goes throughCheckShape→RequireHolds. What a read allocates depends on what the JIT has made of its code so far. Optimised code inlines and devirtualises away hundreds of bytes, and with dynamic PGO the states in between came out higher than both. The two paths are optimised separately, so the gap between them drifts during a run.dde2413.ArrayPooltrim, which drops a thread's arrays only after 30 seconds, while the forced failure took about one;The fix
Refusing, 1,024 bytes) is unchanged.After the fix:
Mutation: these were all killed:
Refusingat 0, 8,192 and 16,384;The survivors and their reasons are in
39d06bc's message. Two are neither killed nor equivalent: taking the smaller of the two reads, and taking the read before alone. They stay because the edited read ranged over 968 bytes across JIT states, against 976 bytes of room.TESTING.md
The #344 bullet, updated in
3def418:RequireHeldin the outer price that refuses a longer nested length, not the slice. TheArgumentOutOfRangeExceptionclause, which described the slice's check, is gone.Found while doing this
Since #349, no test sees
Slice'sRequireAhead: its drop and its one-byte-laxer mutants survive the whole suite. Filed as #352.Verification
-warnaserror: exit 0, 0 warnings.3def418: 2,668 passed, 0 failed, run by the author and again by the coordinator in a clean clone.🤖 Generated with Claude Code
https://claude.ai/code/session_01KK3MspqAcczxT45xCfLgyx