Repository navigation
Conversation
Split both ring ranges at the union of their physical wrap points and move the (at most 3) chunks with ptr::copy_nonoverlapping; small steals keep an inline scalar path, and loom falls back to the tracked per-element loop. Wraparound is exhaustively covered by new bulk steal/self-steal matrix tests. Signed-off-by: lxl66566 <lxl66566@gmail.com>
Member
|
Thanks @lxl66566 , I just wanted to let you know that we are looking into this. I don't wan't to promise anything about the outcome of the review though. We are mindful about keeping an appropriate balance between performance and code complexity, and this might be one of those cases where we will elect to sacrifice a bit of perf, but let us get back to you on this in a few days so we can evaluate it fully. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace the scalar per-element transfer loop in steal/steal_and_pop with a transfer_items helper that splits the source and destination ring ranges at the union of their physical wrap points and moves the resulting ≤3 chunks with ptr::copy_nonoverlapping. Chunks of ≤8 items keep an inline scalar loop, and ≤1-item steals take a direct fast path, so small steals pay no extra cost.
The inlined
transfer_itemsbecomes:n <= 1fast path: a single bare load+store, no chunk computation;cmp/je), medium chunks via a 4-way unrolled loop;memcpycall (3 call sites, one per chunk), delegating to libc's SIMD implementation;Soundness: the booking step already caps the transfer count by both the number of live items in the source queue and the spare capacity of the destination, so the two slot ranges can never overlap — even when stealing from one's own queue. The transfer is a bitwise move, exactly like the per-element
read_at/write_atpair it replaces. Under loom, the helper falls back to the tracked per-element loop.Benchmark: (
benches/benchmark.rson AMD Zen 4, executor scenario, interleaved A/B, medians of 7 rounds)st3 fifo -7.4%, st3 lifo -5.5%; tokio/crossbeam controls and push/pop paths unchanged.