Skip to content

iouring: a worker with nothing to send waits only for the requests in the kernel when its drain began - #1243

Merged
FumingPower3925 merged 5 commits into
mainfrom
fix/iouring-hold-snapshot-1229
Nov 3, 2026
Merged

FumingPower3925 merged 5 commits into
mainfrom
fix/iouring-hold-snapshot-1229

Conversation

@FumingPower3925

@FumingPower3925 FumingPower3925 commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Refs #1229

What

#1226 made an io_uring worker with nothing to send wait, inside the 250 ms floor of its send drain, for a request head already in the kernel (#1208). It asked on every pass, with the listener still open, so a connection accepted after the shutdown began, with its request, renewed the wait: under steady arrivals (the churn of TestListenReleasesThePortUnderLoad{Bind,Dial}896) each shutdown could be held to the floor, 400 of them in the Coverage job, where the instrumented worker is slower than its clients. That is the diagnosis in #1229; this PR does not claim the conntrack part of it (see the experiment below).

The look is now a snapshot, taken once:

  • markUnreadAtShutdown runs at the transition where shutdownDrainStart is set (the first pass of the send drain, after the HTTP/2 pool handlers). It marks (cs.shutdownUnread, w.unreadMarked) each HTTP/1 conn that has bytes in its receive queue (TIOCINQ) or a recv CQE with bytes not reaped, with the filters h1UnreadInKernel had (closing, ended detached stream, HTTP/2, detached or busy, fixed-file socket), sockets first and the completion queue second.
  • h1UnreadInKernel asks only the marked conns, with the same filters, inside the same 250 ms floor. A conn accepted after the snapshot is never asked about. The mark is cleared when the connState is released.
  • sendDrainDone returns (done, held). held is true when the look is the only thing the worker waits for (nothing to send, no request in progress): the worker then calls stopAccepting, as net/http closes its listeners first. A worker that has a response going out is untouched: it keeps accepting for the floor, as documented.

The contract of #1226/#1208 is unchanged: a request head already in the kernel when the worker sees the Shutdown is answered, with Connection: close. server.go says what the worker waits for and that its listener is closed meanwhile; the README sentence already says that a request such a connection sends later is dropped.

Behaviour change to know: a connection still in the accept queue (or held back by TCP_DEFER_ACCEPT) when such a hold starts is now reset by the close of the listener, as it is for a worker with nothing to send that exits at once.

Why stop accepting as well

The snapshot alone was not enough, decided by the test: with a head in the kernel at the first pass and a connection accepted in every pass, the worker has a response going out at every look (the documented "keeps accepting for the floor" regime, entered through the wait for the head) and is held to the floor. 30 runs of the new test per arm, -race, docker, 2 CPUs, 8 MiB memlock:

arm hold test (Listen return, from the cancel; 20 ms of it is the test's own sleep) contract test
main (14d38b3) 30/30 RED, 272 to 277 ms 30/30 pass
snapshot only 30/30 RED, 273 to 282 ms 30/30 pass
snapshot plus stop accepting (this PR) 30/30 pass, 23 to 32 ms (-covermode=atomic: 23 to 28 ms) 30/30 pass

Tests

  • New, shutdown_hold_snapshot_1229_linux_test.go, one scenario: two workers requested (config validation needs at least 2; the 8 MiB memlock funds one), a keep-alive conn gets its next request while the worker is held in its first drain pass by the new sendDrainSeam (an atomic.Pointer, as ringWaitSeam is; one load on the shutdown path only), 8 clients start at that moment and go on arriving, every drain pass takes 1 ms more and brings a connection of its own with its request. TestShutdownHoldIsNotRenewedByNewConnections1229 asserts Listen returns within 150 ms of the cancel (floor 250 ms; 23 to 32 ms measured with the fix, about 270 ms without). TestShutdownSnapshotIsTakenOnce1229 pins that the look is made once (a head written to an accepted idle conn after the first pass is not waited for; it fails if the snapshot is taken on every pass). TestShutdownAnswersAHeadInTheKernelUnderChurn1229 asserts the held head is answered, 200 with Connection: close (Shutdown: a conn that sent nothing is not answered, and a request head already in the kernel is reset on std but answered on epoll; the godoc says otherwise #1208 under churn). Both start their engine with start896, so the stop is registered in a cleanup before any assertion.
  • Pure churn (8 real clients from before the cancel, no injected arrival) was not deterministic: 26 of 30 RED on main, and with the fix a worker with a response going out at its first pass keeps accepting by design, so the injected arrival is what makes the scenario deterministic.
  • Existing tests changed, stated here so it is not read as a weakening: TestGateH1UnreadInKernel1208 subtests now take the snapshot and then look (look1229), with the same expectations and an added assertion that the snapshot marks the posted-CQE conn. TestGateSendDrainDoneAsksTheKernelInsideTheFloorOnly1208 now uses two conns: the floor assertions are kept (held at the start, held at the floor, done past it), and it adds that a request arriving after the snapshot does not hold, that held is true only for the look, and that a pending send is not held. TestGateMarkUnreadAtShutdownIsASnapshot1229 is new. shutdown_doc_1208_test.go pins the new sentence of the Server.Shutdown godoc. churn896 became churnN896(addr, stop, n).
  • Run: whole ./internal/engine/iouring package with -race (587 s, no failure), the Shutdown: a conn that sent nothing is not answered, and a request head already in the kernel is reset on std but answered on epoll; the godoc says otherwise #1208/epoll shutdown: an idle conn that is asyncPromoted is held behind another conn's send drain for the whole Shutdown budget #1165/v1.6.0 polish: io_uring engine #884 cells of ./test/server x3 (TestShutdownKernelHead_1208_ShapeB including B-hold-other, B-accept-hold, B-accept-hold-draining; ShapeA; TestShutdownIdleAndInProgressH1ConnsWithoutHTTP2_1165; TestIOUringShutdownGateConns884), TestShutdownDoc*, and the new tests plus the 1208 ones under CELERIS_MAX_IOURING_TIER=base.

CI experiment (not part of this PR)

Throwaway branches exp/1229-hold-ctl (main plus diagnostics) and exp/1229-hold-fix (the same plus these two commits) dispatch the Coverage workflow with conntrack, time-wait and per-test (-v) diagnostics around the iouring step. They are not pull requests.

Later changes and the Coverage evidence

hasPendingSends also counts a SEND_ZC notification outstanding and a send completion held for the dispatch goroutine (connSendInFlight). In every reachable state these come with sending or sendBuf set, so nothing changes today; the predicate just no longer depends on the send buffer being kept. TestGateHasPendingSendsCountsEveryStateOfAResponse1229 pins the six states (two of them RED before this change).

The Coverage failure of TestShutdownSilentConn_1208_ShapeA/io_uring/A-deferred-draining seen on an earlier head of this branch is not caused by this PR. It fails on main as well under CPU starvation (a 50 ms sleep against the worker's 250 ms floor) and is tracked in #1244.

CI experiment (12 test-coverage.yml dispatch runs, 6 per arm, conntrack diagnostics): main with diagnostics failed 2 of 6 with the 896 dial timeouts and the UnderLoad896 tests took 20-39 s / 49-90 s; with the fix 0 of 6 failed and the UnderLoad tests took 3-5 s. At every dial timeout of the main arm the runner's nf_conntrack table was full (262144 of 262144).

…ld by connections that arrive after the shutdown began (celeris#1229)

The send-drain test seam and two tests, failing first: a head is in the kernel
when the workers see the shutdown, connections keep arriving, and Listen is held
to the 250 ms floor on main. The second pins that the head is answered under the
same churn (celeris#1208). The fix follows.
… the kernel when its drain began (celeris#1229)

The look h1UnreadInKernel (celeris#1226, #1208) was made again on every pass of
the send drain, inside the 250 ms floor, while the listener stayed open, so a
connection accepted after the shutdown began, with its request, renewed the
hold: under steady arrivals a worker was held to the floor (400 shutdowns of
the listen-release churn tests in the Coverage job, where the instrumented
worker is slower than its clients).

The look is now a snapshot taken once, at the first pass of the drain
(markUnreadAtShutdown marks each H1 conn with bytes in its receive queue or a
recv CQE with bytes), and the worker waits only for the marked conns, with the
same filters and the same floor. A conn accepted after the snapshot is never
asked about. A worker that waits for them and has nothing else to send also
stops accepting, as net/http closes its listeners first: a connection it
accepted meanwhile brought a response to send, which kept the worker serving
to the floor. A head already in the kernel when the worker sees the shutdown is
answered as before. Refs #1229
@FumingPower3925 FumingPower3925 added this to the v1.6.0 milestone Nov 3, 2026
@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.

@codecov

codecov Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.56098% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/engine/iouring/shutdown_gate_884.go 96.66% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

…ts (celeris#1229)

TestShutdownSnapshotIsTakenOnce1229: a head written to an accepted idle conn
after the first drain pass, when the worker is quiet, is not waited for, while
the head that was in the kernel at the snapshot is answered; it fails when
markUnreadAtShutdown runs on every pass. The hold test's bound is 150 ms (the
floor is 250 ms). Comment and godoc wording: the field is shutdownUnread, the
look is taken when the send drain begins, and a connection still in the accept
queue gets a reset. Refs #1229
…n has a send pending (celeris#1229)

hasPendingSends decides whether a worker is one with nothing to send, which stops
accepting while it waits for the requests in the kernel. It counted a SEND in
flight, queued bytes and a queued resend; a SEND_ZC notification outstanding and
a send completion held for the dispatch goroutine now count on their own, so that
the decision does not depend on the send buffer being kept in those states.
TestGateHasPendingSendsCountsEveryStateOfAResponse1229 lists every state a
response passes through (including the failed ZC first completion before the
fallback resend); the last two fail without the change. Refs #1229
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant