Skip to content

iouring: a worker with nothing to send answers a request already in the kernel before it exits (Refs #1166 #1208) - #1224

Closed
FumingPower3925 wants to merge 10 commits into
mainfrom
iouring-unread-drain-1208
Closed

FumingPower3925 wants to merge 10 commits into
mainfrom
iouring-unread-drain-1208

Conversation

@FumingPower3925

@FumingPower3925 FumingPower3925 commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Refs #1166 #1208

Stacked on #1220 (base branch shutdown-contract-1208; retarget to main once #1220 merges). #1220 documents what main does, including this gap; this PR removes the gap and flips the one line of docs and the two cells that pinned it.

The bug

On its first look at the cancelled context, an io_uring worker with no send pending and no parsed request in progress exits at once (worker.go, the send-drain branch: if !w.hasPendingSends() && !w.h1RequestInProgress() { w.shutdown(); return }). h1RequestInProgress explicitly does not look at the kernel (its comment: those requests are "held at the floor", which such a worker never reaches), so a request head sitting in a conn's receive queue, or as a recv CQE not reaped, is reaped by shutdown() without being served and the conn is closed without an answer. epoll's h1InProgress counts "bytes in the kernel's receive queue of a conn that holds nothing" as in progress, and answers them: this is #1104's own stated contract ("a request that reached the kernel before the gate is answered") and the gap that #1166 closed for a request already parsed. OnConnect is only one way to make a worker's iteration long enough for a head to land unread while it has nothing to send; the axis is "does the worker have anything to drain when it first sees the shutdown" (review r2 of #1220, MAJOR-3).

RED / GREEN

Commit 1 (B-hold-other, and the B-accept-hold want for io_uring flipped to answered), docker arm64 --cpus 2, -count=3:

RED (commit 1): FAIL TestShutdownKernelHead_1208_ShapeB/io_uring/B-hold-other        3/3
                FAIL TestShutdownKernelHead_1208_ShapeB/io_uring/B-accept-hold/GET   3/3
                FAIL TestShutdownKernelHead_1208_ShapeB/io_uring/B-accept-hold/POST  3/3
                (closed-without-answer, unexpected EOF; the conns of the held loop, about half of the 24)
GREEN (commit 2): all cells pass; plain x20 and -race x20 with two busy loops: 720 PASS, 0 FAIL, 0 SKIP

B-hold-other: 24 keep-alive conns the loops are already reading (each served once); one loop is held in the OnConnect of another, silent conn; all 24 send a GET or a 64 KiB POST; Shutdown begins (observed: the FIN of an idle conn of the free loop, never slept for); the loop is released. epoll answers all, with Connection: close; io_uring dropped the held loop's conns. adaptive starts on epoll, so its cell is epoll's (adaptive running on io_uring after a switch is the io_uring worker and is not exercised by a cell: static).

The fix (commit 2, shutdown path only)

  • h1UnreadInKernel (new, shutdown_gate_884.go): the look of the gate's first pass (sweepUnreadAtGate: TIOCINQ on the sockets first, then the unreaped CQEs) without its marks. Skips what the gate skips: closing conns, ended detached streams, detached streams, HTTP/2, a fixed-file conn's socket (its CQE is still seen), and a dispatch goroutine's busy conn (already counted by h1RequestInProgress). holdPostedRecvs is split into eachPostedRecv (walk) plus its marking callback, so the CQ walk is shared.
  • The send-drain exit becomes !hasPendingSends && !h1RequestInProgress && (pastFloor || !h1UnreadInKernel()). The kernel look runs once per iteration only inside the 250 ms floor, while the worker serves as an ordinary iteration, so the look itself can never keep a worker past the floor (see the release note for what a request it finds can do): the floor, stopAccepting, refuseRecv, closeIdlePaced and drainEnd are untouched (the 250 ms semantics are unchanged; the sweep still marks at the floor).
  • h1RequestInProgress now counts a gateRead conn (found by the floor's sweep) as in progress, so a worker that reaches the floor with only unread, marked heads waits for their answers instead of exiting between the mark and the read. Bounded as the drain is (drainEnd).
  • The reviewer's sketch (call sweepUnreadAtGate at drain start) was not taken as is, for its own two caveats: it sets CloseAfter on every HTTP/1 conn ~250 ms earlier than today, and gateRead stays set until the conn is released, so a marked conn whose partial head never completes would hold the worker to drainEnd. The non-marking look has neither.

What was checked

  • test/server (plain and -race) and internal/engine/iouring (plain; -race for the broad filter) with -run 'Shutdown|Gate|Drain|Unread|Close|Sweep|Start|PauseAccept|_884|_885|_896|_1104|_1138|_1163|_1165|_1168|_1208': all pass (plain 171 s and 133 s class, race likewise). Then go test -run 'Shutdown|Gate|Drain|Unread|Sweep' ./internal/engine/iouring once more on the final head: ok, 85 s.
  • Request loop A/A (BenchmarkRequestLoop884, base 4429b9d vs head, 6 alternating rounds, 1 s each, docker --cpus 2): the diff is in the shutdown branch only, and the numbers are noise (the four sub-benchmarks move -29%, -44%, -6%, +16% with no direction; the same binary varies that much between rounds). No hot-path code changed. Raw: scratchpad/b1208/out/bench.txt.
  • After the delta review, on the final head: the whole internal/engine/iouring package under -race (as the base-tier job runs it): ok; test/server shutdown suites under -race: ok; the new unit tests -race x20: ok; _1208_ cells plain x10 (360 PASS) and -race x20 with two busy loops (720 PASS), 0 FAIL, 0 SKIP.
  • Not run locally: x86, SQPOLL, multishot recv; the base-tier job on the runner's kernel, where holdPostedRecvs' CQ walk (reached through eachPostedRecv) is the path that finds a request: the unit tests above exercise it deterministically, but CI's base-tier result is the condition of the review's approval. This PR has no CI until it is retargeted to main (stacked base).
  • golangci-lint v2.14 (CI's pin, GOOS=linux, whole root module): 0 issues.

Behaviour change to know about (delta review)

A request head fragment (or a bare CRLF) that is unread in a conn's receive queue when the worker first looks at the shutdown used to be dropped at about 104 ms (the worker exited and closed the conn). It is now read on the look's say-so and becomes #1166's "request in progress", so it holds Shutdown to its budget (drainEnd, capped by WriteTimeout), exactly as on epoll and as a fragment read before the shutdown already did. Probe, loop held in OnConnect, 1.5 s budget: "GET /ping HTTP/1.1\r\nHo": #1220 epoll 1.501 s, io_uring 104 ms; #1224 epoll 1.501 s, io_uring 1.500 s. Garbage (a TLS hello) is still closed at ~100 ms on both. A bare CRLF (RFC 9112 2.2 says a server SHOULD ignore it) holds Shutdown to the budget on both native engines, also before this PR: a separate, older matter, for its own issue. Godoc: bullet 3 of the Shutdown list says so. A fixed-file conn (development setting) marked by a CQE of a bare CRLF is never closed by the gate, whose TIOCINQ cannot ask its slot, and does the same.

Tests added after the delta review (the three surviving mutants)

Mutants of commit 2 that the end-to-end cells did not kill, and the unit tests (shutdown_unread_look_1208_linux_test.go, on the gate rig, which arms its own recv, so they do not depend on the ring's tier; the base-tier job runs this whole package, so they run there too) that kill each, checked by applying the mutant in a copy:

mutant killed by
h1RequestInProgress without the gateRead count TestGateH1RequestInProgressCountsAMarkedConn1208
h1UnreadInKernel without the completion-queue half (postedRecvs); on this kernel every end-to-end rescue comes from TIOCINQ, only a base-tier ring or one that posts at once reaches the CQ half TestGateH1UnreadInKernel1208/a_recv_CQE_with_bytes,_unreaped,_the_socket_queue_empty_(the_CQ_half) (recv posted by the test itself, socket queue asserted empty)
the floor bound (pastFloor := false) TestGateSendDrainDoneAsksTheKernelInsideTheFloorOnly1208 (the exit decision is now sendDrainDone(now), a method, so the floor is testable)

Also pinned there: EOF CQE, another generation's CQE, HTTP/2 conn, closing conn, ended and running detached streams are not asked, and the look does not mark (gateRead).

Release note

io_uring: a request head that reached a connection's receive queue before Shutdown was first looked at is answered (with Connection: close) even when the worker has no response going out, as epoll does; before, it was closed without an answer. A head fragment or bare CRLF found there is now a request in progress and holds Shutdown until it completes or the budget ends, as on epoll (before, io_uring dropped it at once). Only the shutdown path changed.

…rnel, per engine (#1208)

Refs #1208. Characterization tests: shape A (a conn that sent nothing) and
shape B (a request head in the kernel, unread) on std, epoll, io_uring and
adaptive, with the cells tagged as in the issue's table. The doc test is RED:
the Shutdown godoc still says a silent conn is answered close when its first
request is, and says nothing of the head in the kernel.
…nel get, per engine (#1208)

Refs #1208. The Shutdown godoc said a conn that has sent nothing is answered
close when its first request is; no engine does. It now lists what each state of
a conn gets when Shutdown begins (idle, in progress, head in the kernel, silent),
per engine, as measured by the tests of the previous commit. README and the two
native gate comments say the same; the #1207 test comment no longer calls the
native side unverified.
…; parametrize requireDoc's tag (#1208)

Refs #1208. New cells: A-accepted-draining and A-deferred-draining (a response
going out on every worker: epoll closes at once, io_uring serves on for 250 ms
and answers a first request), B-hold-other (the loop is held in another conn's
OnConnect: epoll answers, io_uring drops), B-accept-hold-draining (io_uring
answers when its worker has a response going out). requireDoc takes the issue tag
for its failures. The doc test now wants the corrected godoc (RED here).
…xis of a head in the kernel (#1208)

Refs #1208. Review r2 of #1220: an io_uring worker that has a response going out
serves on for the first 250 ms of its send drain, so its idle and silent conns
are closed at 250 ms and a first request before then is answered (also for a
conn TCP_DEFER_ACCEPT holds back); a head in the kernel is answered on io_uring
only while the worker is in its drain, a worker with nothing to send exits at
once (an OnConnect callback is one way to the window, not the axis); std's reset
needs unread body bytes (2 KiB gives a FIN). 'Measured on every engine' becomes
what was measured. README and the two native gate comments say the same.
…t conn's FIN (#1208)

Refs #1208. A flake of the previous commit (1 in 20 runs): an epoll loop busy
sending closes its listener a few ms late, and a held-back conn's first request
that arrives before is answered, not reset. The godoc says so.
…n the kernel (RED) (#1208)

Refs #1166 #1208. io_uring closes these conns without an answer (a FIN) when
its worker has no send pending and no parsed request in progress: it exits on
its first look at the shutdown, and h1RequestInProgress does not look at the
kernel. epoll and adaptive answer. The next commit fixes it.
…he kernel before it exits (#1208)

Refs #1166 #1208. On its first look at the shutdown a worker with no send
pending and no parsed request in progress exited at once, and
h1RequestInProgress never looks at the kernel, so a request head in a conn's
receive queue (or as a recv CQE not reaped) was reaped without being served: the
conn was closed without an answer (epoll's h1InProgress counts those bytes and
answers). The worker now asks the kernel once per iteration inside the 250 ms
floor (h1UnreadInKernel: TIOCINQ on the sockets, then the unreaped CQEs, the
look of the gate's first pass without its marks) and stays to serve what it
finds; past the floor the gate's first pass marks such a request (gateRead) and
h1RequestInProgress now counts a marked conn. Shutdown path only: the request
loop is untouched.
…ers a head in the kernel (#1208)

Refs #1166 #1208. With the worker's look at the kernel inside the 250 ms floor,
the head-in-the-kernel line is the same on epoll, io_uring and adaptive; std
still drops it. The OnConnect exception goes.
@FumingPower3925 FumingPower3925 added this to the v1.6.0 milestone Nov 1, 2026
@coderabbitai

coderabbitai Bot commented Nov 1, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

…lf, the gateRead count, the floor (#1208)

Refs #1166 #1208. Three mutants of the previous fix survived the end-to-end
cells: h1RequestInProgress without the gateRead count, h1UnreadInKernel without
the completion-queue look (postedRecvs: on a DEFER_TASKRUN kernel every rescue
there comes from TIOCINQ, so only a base-tier ring or a ring that posts at once
reaches it), and the floor bound. The exit decision becomes sendDrainDone, and
unit tests (the gate rig, which arms its own recv, so independent of the ring's
tier; the base-tier job runs this package) kill each. Notes in the comments: a
head fragment or bare CRLF found in the kernel is a request in progress and holds
the drain to its budget, as on epoll; a fixed-file conn marked by a CQE of a bare
CRLF is never closed by the gate and does the same (dev-only setting).
… holds the drain to its budget (#1208)

Refs #1166 #1208. Review of #1224: with the worker's look at the kernel a head
fragment (or a bare CRLF) that io_uring used to drop at once is read and is a
request in progress, as on epoll: it holds Shutdown until it completes or the
budget ends.
@FumingPower3925

Copy link
Copy Markdown
Contributor Author

Superseded by #1226: #1220 is merged, and this PR's 5 commits are rebased onto main there (same content, targeting main so CI runs, including the io_uring base-tier job).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant