Repository navigation
perf(websocket): recycle inbound chunk buffers instead of one allocation per read (refs #756) - #1307
Draft
FumingPower3925 wants to merge 3 commits into
Draft
FumingPower3925 wants to merge 3 commits into
FumingPower3925 wants to merge 3 commits into
Conversation
…fault loop count (refs #756) The default loop count (Config.Workers not set, a host of alike CPUs) was n - max(1, n/8): 14 of 16. A cluster comparison on one amd64 host (WebSocket large-echo p99 against the std engine, 5 repetitions per arm, the rated levels 0.5 and 0.75 of saturation) found that the Ps left free decide the tail: 14 loops on 16 Ps read 3.26 and 8.75 times, 12 loops 1.45 and 3.37 times (MEASURED, lane-finding, not closing). The reserve is n - max(1, n/4), none for n <= 2: 12 of 16, 6 of 8, 3 of 4. A Workers set by the user is unchanged and so is the big.LITTLE plan. Updates the defaultLoopReserve comment, the Config.Workers and Resources docs, the README and the pinned figures in the tests. Not to be merged before the A/B with an A/A twin shows no small-message throughput regression.
…one per read (refs #756) The engine hands the WebSocket callback each socket read, and the callback copied every read into a fresh make([]byte, n): a 65550-byte masked frame arriving in 8 KiB reads cost eight allocations on the loop thread, about 64 KiB of garbage per message, charged to the loop as GC assist and as mark work on the few Ps the handlers share with the GC. chanReader.AppendCopy now copies into a pooled buffer (power-of-two classes, 512 B to 256 KiB) and chanReader.Read returns the buffer to the pool once its last byte has been copied out. Ownership is exact: the chunk never leaves chanReader, Read is its only consumer and copies out of it, and a chunk that is never read (the reader closed with chunks buffered) is left to the garbage collector. Only a reader the engine upgrade marks as recycling does this; a reader fed through Append with caller-owned slices never puts them in the pool. BenchmarkChunkDeliver (one 65550-byte frame in 8 KiB reads, delivery and drain): 65559 B/op and 9 allocs/op before, 5 B/op and 0 allocs/op after. Tests: class and foreign-buffer handling, AppendCopy copies, no recycle before a chunk is fully consumed (one-byte reads under pool churn), no recycling of caller-owned chunks, producer/consumer under -race.
…'s smallest class (refs #756) A backed-up reader holds up to two channels of chunks (channel plus spill); a 100-byte read pinning a 512 B buffer would grow that five-fold on the small-message paths. Reads under the smallest class are plain, exactly sized allocations again; putChunk already ignores them. The size-class test now pins that.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #756. Stacked on #1306: the diff against main also contains #1306's commit (f17e18d); the change proper is the last two commits (3680e58, 9d5e134), all in
middleware/websocket.What
The engine hands the WebSocket callback every socket read, and the callback copied each read into a fresh
make([]byte, n). A 65550-byte masked frame arriving in 8 KiB reads cost eight allocations on the loop thread, about 64 KiB of garbage per message, charged to the loop as GC assist and as mark work on the few Ps the handlers share with the GC (D756 FIX-DESIGN, mechanism items 4-5).chanReader.AppendCopynow copies into a pooled buffer (power-of-two classes, 512 B to 256 KiB) andchanReader.Readreturns the buffer to the pool once its last byte has been copied out. Reads under 512 B stay exact, plain allocations (not rounded up).Ownership
The chunk never leaves
chanReader:Readis its only consumer and copies out of it, so the buffer is free the moment the last byte is copied. A chunk that is never read (reader closed with chunks buffered) is left to the GC. Only a reader the engine upgrade marks as recycling does this; a reader fed throughAppendwith caller-owned slices never puts them in the pool. Rejected chunks (never stored) are returned at once.What this does NOT do
This is one pooled copy per read, not per read turn. There is still one
Append, one channel send and onegoreadyper 8 KiB chunk, so 5-8 wakes per message (FIX-DESIGN R2(i), coalescing per turn, is not done: it needs engine-side changes). This change tests the allocation and GC side of the H1-PS mechanism only, not the trapped-wake count; a null A/B result would not rule R2(i) out.Numbers
BenchmarkChunkDeliver*756, one 65550-byte frame in 8 KiB reads, delivery plus drain, single goroutine, soGetandPuthit the same P; the real path puts on the handler's P and gets on the loop's P): 65559 B/op, 9 allocs/op before; 5 B/op, 0 allocs/op after. Not a cluster number.Tests
Size classes and foreign-buffer handling;
AppendCopycopies; no recycle before a chunk is fully consumed (one-byte reads under pool churn from other goroutines; mutation-checked: recycling on everyReadfails it); no recycling of caller-owned chunks; producer/consumer under-race.Do not merge
Not to be merged until the RULE 74 A/B with an A/A twin passes (with #1306), and the guard cells (ws-echo small, HTTP/1, drivers) show no regression. #756 closes only on the closing run on the new FINAL.