Repository navigation
iouring: the header deadline of a silent connection is a kernel timer armed at the accept; document the bound (Refs #1189) - #1193
Merged
Conversation
…efs #1189) A connection that sends nothing, or one byte, must be closed within ReadHeaderTimeout plus a stated slack of its connect (TCP_DEFER_ACCEPT off), and within the kernel's deferral window plus ReadHeaderTimeout plus the slack with it on. Both native engines run the scenarios (monotest). On io_uring a second test asserts the header timer of a connection whose protocol is not decided yet is in flight when the sweep meets it. RED on main for io_uring: the silent and one-byte connections are closed at 1.7 to 1.9 s for a 1 s ReadHeaderTimeout (9 of 10 runs; the phase of the engine's sweep decides, and one run landed before the deadline), and no sweep finds the timer armed (every run). epoll passes (a 25 ms timerfd sweep): it is the pin.
…cided is a kernel timer armed at the accept, not the sweep's (Refs #1189) A connection that sends nothing, or too little to tell HTTP/1 from HTTP/2, was closed by the sweep, which runs about every 0.83 s on an idle worker (32 iterations of 25 ms waits): a silent connection under ReadHeaderTimeout 1 s was closed at 1.7 to 1.9 s with TCP_DEFER_ACCEPT off (2.0 to 2.9 s with it on, a second of which is the kernel's deferral, which epoll has as well and which this change does not touch). A connection whose protocol is fixed, or already detected, has had an IORING_OP_TIMEOUT of its own since v1.4.11. handleAccept now arms the same timer at detectDeadline for such a connection (armHeaderTimerAt, no SQE more than before for an HTTP/1 connection: initProtocol would arm it at the same instant, finds it in flight and does not). handleHeaderTimer closes an undecided connection at its deadline, or arms again if the timer came early; initProtocol closes a connection whose budget is spent by the time it is detected, which the arm did when it was the first one. An HTTP/2 connection keeps one timer that fires once and does nothing. With no SQE to be had the sweep stays the fallback. Nothing changes per request, and the sweep is untouched, so idle sweeps cost what they did. Docs: ReadHeaderTimeout states the bound of a silent connection (the deferral plus the timeout plus about 30 ms) and ReadTimeout the sweep period the other timeouts are enforced at. The comments that said the sweep was 50 ms apart say 0.8 s. monotest.silentConn sets DisableDeferAccept: the deferral's second made a deadline that fires at once (a forward step, on the wall clock) look like one on time, so the 600 ms lower bound did not bite (the 1.05 s of the #1188 review). BenchmarkAcceptChurnAuto1189 is the instrument for the cost of the arm at the accept.
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…accept is given no recv (Refs #1189) The arm at the accept closes the connection when its budget is already spent: the budget is taken from the CQE batch's clock, which is older than ReadHeaderTimeout when an OnConnect hook or a slow inline handler earlier in the batch ran long. onAcceptedFD goes on and arms a recv on the closed connection, which nothing accounts for (the fd-lifetime and cancel-then-release rules, celeris#685 and celeris#256). The test is the review's probe, made fast: ReadHeaderTimeout 200 ms, OnConnect sleeps 300 ms for the first connection, and after one second no connState of pendingRelease may have an op in flight and no closedOps entry. RED on 421a164: 5 of 5 runs, every look at the released connState finds the orphaned recv.
… given no recv; tighten the bound test, correct the sweep doc (Refs #1189) onAcceptedFD armed a recv on a connection the new header-timer arm had just closed (the batch's clock older than ReadHeaderTimeout: a slow OnConnect or inline handler earlier in the batch). The recv named a released descriptor number and a connState queued for release and nothing accounted for it (celeris#685, celeris#256): the 5 s backstop released it with the "kernel ops unaccounted" warning, and in the worst case a later accept reusing the number would have its first bytes eaten. The accept now returns when the arm closed the connection, as handleRecv does after its own arm. The other outcomes of the arm leave nothing behind: a full SQ or no SQE for the timer returns unarmed with the connection untouched (the sweep is the fallback), and the dirty-list path is only reached by a connection that is still open. Tests: monotest's slack is 150 ms (six times the larger of the engines' measured lateness) and its doc says that the white-box test is the one that catches the sweep regression, since the black-box bound does not tell the sweep from the timer on every run (the sweep's phase decides). The ReadTimeout and ReadHeaderTimeout docs say that the HTTP/2 header-block bound is checked by the sweep and is late by up to a sweep period, and no longer carry the history of the fix.
…1189) TestConnClosedByTheArmAtAcceptGetsNoRecv1189 passed when nothing was closed at the accept (an OnConnect that does not outlast the budget, or a close by a timer or the sweep), because it then had nothing to check. It now records when the slow OnConnect returns and when OnDisconnect runs, and requires the close within 50 ms of the return: the arm's close is 6 to 75 microseconds after it (also under -race), a timer's would be ReadHeaderTimeout (200 ms) after the accept, the sweep's 100 ms or more. RED on 421a164 5 of 5 (the recv left on the closed connection), GREEN on the fix 10 of 10 and under -race 10 of 10. A mutant with an OnConnect of 10 ms, which does not outlast the budget, FAILS 3 of 3 (closed 190 ms after the return) where it passed vacuously before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #1189
What was wrong
The issue's reading (a sweep-cadence problem on every native engine) is half right. The close of a silent connection under
ReadHeaderTimeout= 1 s has two parts, and only one is a defect:TCP_DEFER_ACCEPT: the kernel holds a silent connection out of the accept queue until its first SYN-ACK retransmission, so the engine's "accept" is ~1 s after the SYN. The budget counts from the accept.So 2.5 s = ~1.05 s deferral + 1 s + 0.2 to 0.9 s of io_uring sweep. epoll alone closes at 2.05 s, std at 1.00 s.
On io_uring a connection whose protocol is fixed (
Protocol: HTTP1) or already detected has had a per-connectionIORING_OP_TIMEOUTsince v1.4.11; the defaultProtocol: Autoconnection that has not been detected yet (silent, or a few bytes of the request line / HTTP/2 preface) had none and relied on the sweep.The fix (commit 2)
handleAcceptarms, for such a connection, the same timer atdetectDeadline(armHeaderTimerAt). It is the timerinitProtocolwould arm at detection for the same instant, which now finds it in flight (headerTimerArmed) and does not arm another: an HTTP/1 connection costs no SQE more than before.handleHeaderTimercloses an undecided connection at its deadline (re-arms if the timer came early);initProtocolchecks the clock once when the arm was skipped, so a budget spent by the time of detection still ends the connection there (TestSpentDetectBudgetReturnsItsRingBuffercaught this in my first version). The sweep stays the fallback when no SQE is to be had, and is untouched.Decision between the options in the issue: the per-connection timer (exact, the mechanism the fixed-protocol path already uses) rather than tightening the cadence or a
min(cadence, nearest deadline)sweep: nothing per request changes, idle sweeps cost what they did (10k idle connections unaffected: the sweep code and gate are not touched), no new state. The remaining deadlines that ride on the sweep (below) are documented, not changed.The deferral second is documented, not removed: an engine cannot learn a connection's SYN time without a syscall per accept;
DisableDeferAcceptis the opt-out;config.gonow states the bound.Measurements
docker arm64 (kernel 7.0.12-linuxkit),
golang:1.27.3,--cpus 2 --security-opt seccomp=unconfined --ulimit memlock=8388608, one container at a time, 20 runs per cell through the public API (celeris.New, 2 workers requested,ReadHeaderTimeout: 1s), client close time in ms, min / p50 / max. The memlock limit capped io_uring to 1 worker in all runs.DisableDeferAcceptDisableDeferAcceptDisableDeferAcceptProtocol: HTTP1, silent, no deferral (control: timer path)Server-side accept delay of a silent connection with the deferral on (OnConnect): epoll 1020 to 1060 ms, io_uring 1005 to 1069 ms, adaptive 1034 to 1060 ms. (In the "fresh server" io_uring after-runs 4 of 20 starts failed with ENOMEM on
io_uring_setupunder the 8 MiB memlock after back-to-back restarts, a harness artifact also seen once in the benchmark below; they are excluded, the other 16 are 1000 to 1003.)Sweep cadence per engine, and what rides on it
Measured with the
testHookCheckTimeoutsseam, one worker, 8 s idle, per-worker intervals:On the sweep: the idle, read and write timeouts, the WebSocket idle deadline, the HTTP/2 deadlines and the closing drain (now listed in the
checkTimeoutsdoc). On a per-connection timer in io_uring: the HTTP/1 header deadline and, with this PR, the undecided connection's budget. The idle check of the issue: after one request,IdleTimeout1 s closed at 1.008 to 1.025 s on epoll and 1.000 to 1.002 s on io_uring (the sweeps are dense right after activity).The 1.05 s close in the #1188 review
Reproduced on ce1886a (the tests on the wall clock), forward stair, iouring and epoll: deferral on: closed at 1.04 to 1.10 s;
DisableDeferAccept: 40 to 49 ms. The forward step makes the deadline fire at once; the connection is only accepted ~1.0 s after its SYN (the deferral), so the first sweep after the accept closes it: ~1.0 s + 50 ms. It is "faster than with no step" because with no step the budget starts at that accept (1.0 + 1.0 s + sweep). The 600 ms lower bound ofmonotest.silentConncould not tell, because the deferral supplies the second;silentConnnow setsDisableDeferAccept, so the same scenario fails at 40 to 49 ms on the wall clock.Hot path
io_uring,
-cpu 2 -benchtime 1s, main and head test binaries interleaved (main, head, main, head, ...; A/A twin runs),BenchmarkAcceptChurnAuto1189(new; Protocol Auto, ReadHeaderTimeout 10 s, 8 client goroutines, one op = one connection), 17 to 18 samples per arm:Connection: close, read)The A/A spread is about +/-10%, so no difference is measurable.
BenchmarkRequestLoop884(keep-alive,Protocol: HTTP1, so by construction it does not reach the changed code; pingpong 25 to 29 us, busy 3.5 to 4.0 us) shows the same on both. Nothing changes per request: the diff ishandleAccept(once per undecided connection),handleHeaderTimer(one timer CQE) and one clock read ininitProtocol(once per connection, replacing the one the arm made). No syscall or allocation is added per request. The one new cost: one timer SQE per Auto-detected HTTP/2 connection, which fires once at ReadHeaderTimeout and does nothing.Tests (commit 1 is RED on main)
monotest.SilentConnIsCutAtReadHeaderTimeout(both engines):ReadHeaderTimeout1 s,DisableDeferAccept, engine idle 700 ms, then a silent connection and a one-byte connection must be closed within [0.9 s, 1.15 s] of their connect (the slack was 400 ms in the first push; see the review changes below).monotest.SilentConnIsCutAfterDeferAccept: with the deferral on, within [0.9 s, 1 s + 1.5 s + 0.15 s]: pins the documented bound (1.5 s isdeferlinger.DefaultLinger's budget; 1.07 s was the most measured).TestUndecidedConnHasItsHeaderTimerArmed1189: every sweep that meets an undecided connection finds its header timer in flight (deterministic).-race -count=20on both engines; io_uring 1000 to 1010 ms. Under-racethe tests take 0.7 + 1.0 s, 2.0 to 2.1 s and 1.4 s, each under 3 s, soci.ymlneeds no change (unlike the 80 sTestEngineTimeoutsIgnoreAStepOfTheWallClock1178). Also green: the whole io_uring package (non-race, 547 s),internal/monowith and withoutceleris_monoskewat +/-3600 s, the stepped-clockSilentConnscenarios under the skew,test/integrationandtest/server, golangci-lint.Review r1 changes (commits 3 and 4)
F1 (blocker): a connection closed by the new arm at the accept was then given a recv. The arm closes the connection when its budget is spent already (the batch's clock is older than ReadHeaderTimeout after a slow OnConnect or inline handler);
onAcceptedFDwent on to arm a recv on it, which nothing accounts for (io_uring close paths close the fd while a linked RECV can still resolve its number (the #657 fd-lifetime rule, not yet applied to close/hijack/shutdown) #685, SIGSEGV in iouring Worker.run after ~16h under churn-close+auto+upg load #256). Commit 3 is the test (the review's probe made fast: ReadHeaderTimeout 200 ms, OnConnect sleeps 300 ms, after 1 s no connState ofpendingReleasehas an op in flight without aclosedOpsentry): RED on 421a164 in 5 of 5 runs, every look finds the orphan. Commit 4 returns fromonAcceptedFDwhen the arm closed the connection, ashandleRecvdoes after its arm: 10 of 10 green, no "unaccounted" warning in the test or in the whole io_uring package run. Other outcomes of the arm: no SQE for the timer (full SQ, or ENOMEM on the retry) returns unarmed with the connection untouched and the sweep as fallback; the dirty-list path is now reached only by a connection that is still open.F2: the bound test's slack is 150 ms (six times the larger measured lateness, epoll's 25 ms timer), and its doc says the io_uring package's
TestUndecidedConnHasItsHeaderTimerArmed1189is the regression catcher, since the black-box bound does not tell the sweep from the timer on every run.F3: the ReadTimeout doc says the HTTP/1 header deadline and the undecided budget have a timer on io_uring, and that the HTTP/2 header-block bound of ReadHeaderTimeout is checked by the sweep and late by up to a sweep period; the history of the fix is out of the godoc.
Re-run on 4d4ccd9 (docker arm64,
--cpus 2, seccomp unconfined, memlock 8 MiB):-race -count=20on both engines, the 1189 tests x10 non-race, the whole io_uring package once (552 s),internal/monoand the 1189 +SilentConnstepped-clock tests atCELERIS_MONO_SKEW+/-3600s with-tags celeris_monoskew, golangci-lint: all pass. The new test takes 1.3 s.Delta review D1 (311274a): the close-at-accept test now requires the close within 50 ms of the slow OnConnect's return (measured 6 to 75 microseconds, also under
-race; a timer would be 200 ms after, the sweep 100 ms or more), so it fails when the scenario did not happen. RED on 421a164 5 of 5, GREEN 10 of 10 and under-race10 of 10; a mutant with a 10 ms OnConnect fails 3 of 3 (closed 190 ms after the return) where it passed vacuously.Docs follow-up (goceleris/docs, not touched here)
configuration.md, ReadHeaderTimeout: "On the native engines (epoll, io_uring, adaptive) the budget counts from the engine's accept. With TCP_DEFER_ACCEPT on (the default) a connection that sends nothing is accepted about one second after its SYN (measured 1.00 to 1.07 s), so it is closed about ReadHeaderTimeout + 1 s after it connects; std counts from the handshake. SetDisableDeferAcceptto count from the connect (closed at ReadHeaderTimeout, within about 30 ms)."configuration.md, aDisableDeferAcceptentry: "DisableDeferAcceptturns TCP_DEFER_ACCEPT off on the epoll and io_uring listen sockets (default false). While it is on, the kernel keeps a connection that has completed its handshake but sent no data out of the accept queue until its first byte or about a second, which saves a wakeup per new connection; turning it off gives that up (a cost on connection churn only) and counts ReadHeaderTimeout from the connect. With it on, PauseAccept lingers about 1.5 s; with it off the listen sockets close at once."performance.md, slow-loris paragraph: "ReadTimeout, IdleTimeout and WriteTimeout are enforced at the engine's sweep on the native engines, so a connection is closed up to one sweep period after its deadline: 25 ms on epoll (with ReadHeaderTimeout set, the default) and up to about 0.8 s on io_uring (about 1 s with ReadHeaderTimeout off). The HTTP/1 header deadline on io_uring has a timer of its own and is met within milliseconds. With ReadHeaderTimeout off the epoll sweep runs far less often: an idle, read or write deadline can be seconds late (a 1 s IdleTimeout closed at 3.2 to 5.8 s; tracked in #1194), so do not promise the 25 ms figure for that configuration until #1194 lands."Not in this PR
On epoll with
ReadHeaderTimeout-1, a 1 sIdleTimeoutclosed an idle connection at 3.2 to 5.8 s (the sweep gate is 1024 iterations of up to 500 ms waits, and the 25 ms timerfd is armed only whenReadHeaderTimeoutis set). TheWriteTimeoutdoc already notes the same root cause; theReadTimeoutdoc now says so. A follow-up issue candidate.Release note