Skip to content

feat: measure benchmarks with and without request pipelining - #20

Merged
assetcorp merged 2 commits into
mainfrom
chore/bechmark-harness
Oct 21, 2026
Merged

assetcorp merged 2 commits into
mainfrom
chore/bechmark-harness

Conversation

@assetcorp

Copy link
Copy Markdown
Owner

Summary

  • The benchmark suite measures every server at two loads: one request at a time on each connection, and 10 pipelined requests on each connection. The --pipelining option selects the loads, and it defaults to 1,10.
  • BENCHMARKS.md shows the pipelined figures under their own headings, and the README headline quotes the pipelined figure. The figures from the 19 October run are unchanged. Its charts now carry a -pipelining-10 suffix.
  • Both READMEs no longer blame the whole gap between uhttp and uWebSockets.js on pipelining. The measurement showed that pipelining does not account for all of it.
  • The uWebSockets.js server in the Speed check builds its status line and content type once at start-up and sends them in one writeStatus call, as uhttp does. Both sides of the check now send their headers the same way.
  • The cloud toolkit passes BENCH_PIPELINING through to the suite. Its published-profile check now requires 6 hours of runtime, because the default run applies about 4 hours of load to 60 server processes.

Testing

  • pnpm check, pnpm lint, pnpm typecheck, pnpm build, and pnpm test all passed in one run. That run came before the final wording edits to the READMEs and CONTRIBUTING.md.
  • Rerun after the final wording edits: pnpm check and generate.py --check in benchmarks/writeup. The output of the last rerun was not captured, so confirm it before merging.
  • The smoke profile (pnpm --dir benchmarks/suite bench:smoke) measured all three servers under both pipelining loads with no errors.
  • Local Speed check (pnpm nx run uhttp:speed) on an unpinned Mac, so the figures are noisy: uhttp served 228,592 requests a second and uWebSockets.js served 261,308, which makes uhttp 12.5% slower. That is over the 5% limit, so the Speed workflow is expected to fail. The CI run is the real measurement.
  • Not run: the published benchmark suite. It rewrites BENCHMARKS.md and the READMEs, so it runs only when a maintainer asks for new figures.

- Add a --pipelining option to the suite, defaulting to 1,10, and pass it through the cloud toolkit as BENCH_PIPELINING
- Publish both loads on BENCHMARKS.md and both READMEs, and name the charts after the pipelining count
- Raise the published profile to 60 server processes, about 4 hours of load, and the GCP deadline check to 6 hours
@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e0eea8e1-03ae-4166-9e89-4bf56a0a719e
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Speed check

h2load sent GET /hello to uWebSockets.js, to uhttp before this PR, and to uhttp after this PR, in 10 rounds of 3 s each over 100 connections. Each figure is the average of two identical copies of a server, and machine noise is the largest gap between two copies in a round.

Does this PR make uhttp slower?

The job never fails on this table.

Round Before this PR After this PR Difference Machine noise
1 130,156 req/s 133,080 req/s 2.2% faster 2.4%
2 129,967 req/s 131,948 req/s 1.5% faster 1.9%
3 127,851 req/s 132,231 req/s 3.4% faster 1.2%
4 129,517 req/s 132,077 req/s 2.0% faster 0.8%
5 129,174 req/s 132,511 req/s 2.6% faster 1.4%
6 128,466 req/s 130,873 req/s 1.9% faster 1.7%
7 128,497 req/s 130,835 req/s 1.8% faster 1.0%
8 128,435 req/s 130,974 req/s 2.0% faster 2.2%
9 129,338 req/s 131,737 req/s 1.9% faster 1.8%
10 128,365 req/s 131,383 req/s 2.4% faster 1.1%
Median 128,835 req/s 131,842 req/s 2.3% faster 1.6%

This PR makes uhttp 2.3% faster. uhttp after this PR was faster in all 10 rounds, by more than the 2.0% that the job treats as machine noise.

How much slower is uhttp than uWebSockets.js?

The job fails when uhttp after this PR is more than 5.0% slower in the median round.

Round uWebSockets.js uhttp after this PR Difference Machine noise
1 133,783 req/s 133,080 req/s 0.5% slower 2.4%
2 133,635 req/s 131,948 req/s 1.3% slower 1.9%
3 134,233 req/s 132,231 req/s 1.5% slower 1.2%
4 133,639 req/s 132,077 req/s 1.2% slower 0.8%
5 133,886 req/s 132,511 req/s 1.0% slower 1.4%
6 133,337 req/s 130,873 req/s 1.8% slower 1.7%
7 133,926 req/s 130,835 req/s 2.3% slower 1.0%
8 133,579 req/s 130,974 req/s 2.0% slower 2.2%
9 133,710 req/s 131,737 req/s 1.5% slower 1.8%
10 134,121 req/s 131,383 req/s 2.0% slower 1.1%
Median 133,746 req/s 131,842 req/s 1.5% slower 1.6%

After this PR, uhttp is 1.5% slower than uWebSockets.js, and before this PR, it was 3.6% slower. The job passes, because the limit is 5.0% slower.

- Copy the request only when a handler first reads `c.req` after an `await`, so routes without middleware skip the copy on each request
- Settle the return value of a route without middleware directly, without building a chain
- Store the exchange flags as bits, and split `exchange.ts` into an `exchange/` directory with connection, header, and event helpers
@assetcorp
assetcorp merged commit a12453b into main Oct 21, 2026
8 checks passed
@assetcorp
assetcorp deleted the chore/bechmark-harness branch October 21, 2026 16:21
@assetcorp assetcorp self-assigned this Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant