Skip to content

perf(websocket): recycle inbound chunk buffers instead of one allocation per read (refs #756) - #1307

Draft
FumingPower3925 wants to merge 3 commits into
mainfrom
fix/756-ws-alloc-cut
Draft

FumingPower3925 wants to merge 3 commits into
mainfrom
fix/756-ws-alloc-cut

Conversation

@FumingPower3925

Copy link
Copy Markdown
Contributor

Refs #756. Stacked on #1306: the diff against main also contains #1306's commit (f17e18d); the change proper is the last two commits (3680e58, 9d5e134), all in middleware/websocket.

What

The engine hands the WebSocket callback every socket read, and the callback copied each read into a fresh make([]byte, n). A 65550-byte masked frame arriving in 8 KiB reads cost eight allocations on the loop thread, about 64 KiB of garbage per message, charged to the loop as GC assist and as mark work on the few Ps the handlers share with the GC (D756 FIX-DESIGN, mechanism items 4-5).

chanReader.AppendCopy now copies into a pooled buffer (power-of-two classes, 512 B to 256 KiB) and chanReader.Read returns the buffer to the pool once its last byte has been copied out. Reads under 512 B stay exact, plain allocations (not rounded up).

Ownership

The chunk never leaves chanReader: Read is its only consumer and copies out of it, so the buffer is free the moment the last byte is copied. A chunk that is never read (reader closed with chunks buffered) is left to the GC. Only a reader the engine upgrade marks as recycling does this; a reader fed through Append with caller-owned slices never puts them in the pool. Rejected chunks (never stored) are returned at once.

What this does NOT do

This is one pooled copy per read, not per read turn. There is still one Append, one channel send and one goready per 8 KiB chunk, so 5-8 wakes per message (FIX-DESIGN R2(i), coalescing per turn, is not done: it needs engine-side changes). This change tests the allocation and GC side of the H1-PS mechanism only, not the trapped-wake count; a null A/B result would not rule R2(i) out.

Numbers

Tests

Size classes and foreign-buffer handling; AppendCopy copies; no recycle before a chunk is fully consumed (one-byte reads under pool churn from other goroutines; mutation-checked: recycling on every Read fails it); no recycling of caller-owned chunks; producer/consumer under -race.

Do not merge

Not to be merged until the RULE 74 A/B with an A/A twin passes (with #1306), and the guard cells (ws-echo small, HTTP/1, drivers) show no regression. #756 closes only on the closing run on the new FINAL.

…fault loop count (refs #756)

The default loop count (Config.Workers not set, a host of alike CPUs) was
n - max(1, n/8): 14 of 16. A cluster comparison on one amd64 host (WebSocket
large-echo p99 against the std engine, 5 repetitions per arm, the rated
levels 0.5 and 0.75 of saturation) found that the Ps left free decide the
tail: 14 loops on 16 Ps read 3.26 and 8.75 times, 12 loops 1.45 and 3.37
times (MEASURED, lane-finding, not closing).

The reserve is n - max(1, n/4), none for n <= 2: 12 of 16, 6 of 8, 3 of 4.
A Workers set by the user is unchanged and so is the big.LITTLE plan.
Updates the defaultLoopReserve comment, the Config.Workers and Resources
docs, the README and the pinned figures in the tests.

Not to be merged before the A/B with an A/A twin shows no small-message
throughput regression.
…one per read (refs #756)

The engine hands the WebSocket callback each socket read, and the callback copied
every read into a fresh make([]byte, n): a 65550-byte masked frame arriving in
8 KiB reads cost eight allocations on the loop thread, about 64 KiB of garbage
per message, charged to the loop as GC assist and as mark work on the few Ps the
handlers share with the GC.

chanReader.AppendCopy now copies into a pooled buffer (power-of-two classes,
512 B to 256 KiB) and chanReader.Read returns the buffer to the pool once its last
byte has been copied out. Ownership is exact: the chunk never leaves chanReader,
Read is its only consumer and copies out of it, and a chunk that is never read
(the reader closed with chunks buffered) is left to the garbage collector. Only a
reader the engine upgrade marks as recycling does this; a reader fed through
Append with caller-owned slices never puts them in the pool.

BenchmarkChunkDeliver (one 65550-byte frame in 8 KiB reads, delivery and drain):
65559 B/op and 9 allocs/op before, 5 B/op and 0 allocs/op after.

Tests: class and foreign-buffer handling, AppendCopy copies, no recycle before a
chunk is fully consumed (one-byte reads under pool churn), no recycling of
caller-owned chunks, producer/consumer under -race.
…'s smallest class (refs #756)

A backed-up reader holds up to two channels of chunks (channel plus spill); a 100-byte
read pinning a 512 B buffer would grow that five-fold on the small-message paths.
Reads under the smallest class are plain, exactly sized allocations again; putChunk
already ignores them. The size-class test now pins that.
@coderabbitai

coderabbitai Bot commented Nov 16, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@codecov

codecov Bot commented Nov 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant