Repository navigation
perf(websocket): recycle inbound chunk buffers (alloc cut only, refs #756) - #1308
Draft
FumingPower3925 wants to merge 2 commits into
Draft
FumingPower3925 wants to merge 2 commits into
FumingPower3925 wants to merge 2 commits into
Conversation
…one per read (refs #756) The engine hands the WebSocket callback each socket read, and the callback copied every read into a fresh make([]byte, n): a 65550-byte masked frame arriving in 8 KiB reads cost eight allocations on the loop thread, about 64 KiB of garbage per message, charged to the loop as GC assist and as mark work on the few Ps the handlers share with the GC. chanReader.AppendCopy now copies into a pooled buffer (power-of-two classes, 512 B to 256 KiB) and chanReader.Read returns the buffer to the pool once its last byte has been copied out. Ownership is exact: the chunk never leaves chanReader, Read is its only consumer and copies out of it, and a chunk that is never read (the reader closed with chunks buffered) is left to the garbage collector. Only a reader the engine upgrade marks as recycling does this; a reader fed through Append with caller-owned slices never puts them in the pool. BenchmarkChunkDeliver (one 65550-byte frame in 8 KiB reads, delivery and drain): 65559 B/op and 9 allocs/op before, 5 B/op and 0 allocs/op after. Tests: class and foreign-buffer handling, AppendCopy copies, no recycle before a chunk is fully consumed (one-byte reads under pool churn), no recycling of caller-owned chunks, producer/consumer under -race.
…'s smallest class (refs #756) A backed-up reader holds up to two channels of chunks (channel plus spill); a 100-byte read pinning a 512 B buffer would grow that five-fold on the small-message paths. Reads under the smallest class are plain, exactly sized allocations again; putChunk already ignores them. The size-class test now pins that.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #756. Alloc cut only, independent of #1306 and #1307: this branch is main (8ef98b7) plus the two allocation-cut commits of #1307 (cherry-picked: 3680e58 -> 6569bd7, 9d5e134 -> 80473e5), without #1306's loop-reserve change. The non-test diff against main is byte-identical to the diff #1307 adds on top of #1306 (f17e18d..9d5e134); the test file has the same blob.
What
The engine hands the WebSocket callback every socket read, and the callback copied each read into a fresh
make([]byte, n). A 65550-byte masked frame arriving in 8 KiB reads cost eight allocations on the loop thread, about 64 KiB of garbage per message, charged as GC assist and as mark work on the few Ps the handlers share with the GC.chanReader.AppendCopynow copies into a pooled buffer (power-of-two classes, 512 B to 256 KiB) andchanReader.Readreturns the buffer to the pool once its last byte has been copied out. Reads under 512 B stay exact, plain allocations. The chunk never leaveschanReader; caller-owned slices are never put in the pool; a chunk that is never read is left to the GC.What this does NOT do
One pooled copy per read, not per read turn: still one
Append, one channel send and one wake per 8 KiB chunk. Coalescing per turn needs engine-side changes and is not done.Why a separate branch
L756F1 (MEASURED, lane-finding, not closing; amd64 msa2-server, 5 reps, Opus-verified, class MIXED): p99 ratio vs the std-h1 control at 0.25 / 0.5 / 0.75: main 1.20 / 3.44 / 7.69; #1306 (12 loops) 1.23 / 1.53 / 4.32; main with GOGC=400 1.42 / 1.95 / 1.87. Garbage collection looks like the dominant tail driver at 0.75, and the loop reserve alone does not reach the bar. This branch lets the next A/B (L756H) measure the allocation cut on main without the reserve. #756's closing bar is ratio <= 3.0 at 0.25, 0.5 and 0.75.
Tests
go test -race ./middleware/websocket/passes (host, darwin/arm64, count=1); gofmt clean; go build and go vet clean for linux/amd64 and linux/arm64. Local microbenchmark: 65559 B/op and 9 allocs/op before, 5 B/op and 0 allocs/op after (single goroutine, not a cluster number).Do not merge
Not to be merged until the RULE 74 A/B with an A/A twin passes and the guard cells (ws-echo small, ws-large-echo on the other loop engines, HTTP/1) show no regression. #756 closes only on the closing run on the new FINAL.