Skip to content

Let listeners on several reactors share a port (SO_REUSEPORT) - #16

Merged
pathscale merged 3 commits into
masterfrom
feat/reuseport-listener
Sep 23, 2026
Merged

pathscale merged 3 commits into
masterfrom
feat/reuseport-listener

Conversation

@pathscale

Copy link
Copy Markdown
Owner

A socket accepted by a nagoya listener is registered with that listener's reactor, so a multi-reactor server needs one listening socket per reactor. bind set only SO_REUSEADDR, so a second bind on the port failed. This is why endpoint-libs 3 serves on one thread and warns that WS_SHARDS is ignored (consumers set WS_SHARDS=1 to hide the warning).

  • Listener::bind_shared / TcpListener::bind_shared: bind plus SO_REUSEPORT. Every socket on the port uses it, the first one included.
  • Linux balances connections across the sockets. macOS and the BSDs accept the binds but don't balance.
  • Unix-domain addresses are refused (EINVAL).
  • 0.1.12 to 0.1.13.

Test: shared_listeners_on_two_reactors_hold_one_port. Two reactors bind the same port with bind_shared, and a plain bind on that port fails. reactor_net tests (7/7) pass, clippy is clean.

The endpoint-libs change that uses this, one reactor thread per shard, follows as its own PR.

A socket accepted by a listener is registered with that listener's reactor,
so a server that wants more than one reactor needs a listening socket per
reactor. There was no way to make one: bind set only SO_REUSEADDR, so a
second bind on the port failed. endpoint-libs serves on one thread because of
it and warns that WS_SHARDS is ignored.

Listener::bind_shared and TcpListener::bind_shared set SO_REUSEPORT too.
Linux balances connections across the sockets; macOS accepts the binds
without balancing. Refused for Unix-domain addresses. Release as 0.1.13.
meh added 2 commits September 24, 2026 05:15
The test called raise(SIGHUP) on a helper thread, and raise targets the
calling thread. On Linux that helper inherited the block Signal::new takes, so
SIGHUP stayed pending on the helper, and signalfd, which reports only its
reader's and process-directed signals, never became readable. The waiter
blocked forever, holding the test lock, and the job ran into CI's two-minute
limit with a_parked_waiter queued behind it.

pthread_kill the waiter instead, which is the case the test is named for,
and wrap the wait in a five-second timeout so a regression fails in seconds.
The BSDs refuse them at sigaction. Linux does not: signalfd drops an
uncatchable signal from its mask without an error, so Signal::new(SIGKILL)
succeeded there and returned a waiter that could never fire. The test that
says so had never run on Linux, because the cross-thread test hung first.
Checked in Claim::acquire now, so both platforms answer EINVAL.
@pathscale
pathscale merged commit 20742ff into master Sep 23, 2026
3 checks passed
@pathscale
pathscale deleted the feat/reuseport-listener branch September 23, 2026 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant