Repository navigation
epoll: the 25 ms sweep timer is armed on every loop, whatever the timeouts are (Refs #1194) - #1197
Merged
Merged
Conversation
… close on time with ReadHeaderTimeout off, and that the sweep timer ticks do not restart the idle backoff (Refs #1194) RED on main: with ReadHeaderTimeout -1 the engine's 25 ms timerfd does not exist, and a 1 s IdleTimeout closes an idle keep-alive connection at 3.2 to 5.8 s (docker arm64, 2 CPUs), a stalled upload at ReadTimeout + 4 s, a non-reading peer at 4 to 5 s instead of 2 s.
…eouts are; its tick does not restart the idle backoff (Refs #1194) The timerfd existed only with ReadHeaderTimeout set. With it off, every deadline that rides the sweep (IdleTimeout, ReadTimeout, WriteTimeout, the closing drain, the HTTP/2 deadlines, a detached connection's idle deadline) was seen once per 1024 returns of epoll_wait, 4 to 5 s on an idle loop with its listener open. The timerfd, the sweep gate (every 32 iterations) and the epoll_wait cap (the sweep period) no longer depend on the configuration: a loop sweeps like the default configuration's. A tick of the timerfd as the only event of a round is an idle round for the adaptive backoff of the wait. Counted as activity it restarted the backoff every 25 ms and kept an idle loop at 675 wakeups a second (215 without the timer); with this it settles at the 4 ms wait it has without one. IdleTimeout, ReadTimeout and WriteTimeout document the bound; the tests that used ReadHeaderTimeout -1 as 'no timerfd' disarm the timerfd by hand.
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…cs say what bounds the sweep; the write test's window is 3 s (Refs #1194) Review of the PR: the ceiling of the epoll_wait wait does not keep the bound without the timerfd (the backoff tops out at 4 ms, so it never binds); the loop then sweeps from the gate alone, about 130 ms apart with its listener open. The failure of the timerfd's creation, settime or epoll_ctl is logged once per loop, with a test through a seam. config.go: one paragraph on the sweep (ReadTimeout's, merged with the one of the IdleTimeout doc), one io_uring figure, the detached-stream rule says 25 ms on epoll, WriteTimeout is excluded from 'about 30 ms'. loop.go: the stale 'every 1024 iterations' comment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #1194
With
ReadHeaderTimeoutoff (-1) the epoll engine had no sweep timer, soIdleTimeout,ReadTimeoutandWriteTimeout(and the closing drain, the HTTP/2 deadlines and a detached connection's idle deadline, all of which ride the sweep) were enforced seconds late. This PR arms the 25 ms timerfd on every loop, whatever the timeouts are, and stops its tick from restarting the idle backoff of the wait (a cost the straightforward fix would otherwise add; see "Cost").Root cause (confirmed in the code)
loop.gocreated the timerfd onlyif l.cfg.ReadHeaderTimeout > 0, and the sweep gate was 0x1F only forReadHeaderTimeout > 0or a detached conn, 0x3FF otherwise.epoll_waitsettles at 4 ms (adaptiveTimeoutMs: base 1 ms x 4), so 1024 iterations are a sweep every 4 to 5 s. That is the 3.2 to 5.8 s of the issue. Only with the listener closed (a pause, a drain) is the wait 500 ms and a sweep ~8 minutes apart: that is the "minutes late" case; a quiet loop with its listener open was 4 to 5 s late.Measurements (docker arm64,
--cpus 2, seccomp unconfined, memlock 8 MiB, golang:1.27.3, one container at a time, 20 runs per cell, 4 engines in parallel per cell, each run a fresh 2-worker engine)Closing time in ms after the due point (idle: from the end of the response; read: from the last byte of a stalled upload; write: from the request of a 32 MiB response to a peer that reads nothing, observed with
OnDisconnect):The write rows close at two WriteTimeouts (the stall rule: the bytes the peer's kernel acknowledged in the first window pass it); that is the documented rule, only its lateness changes. Two rows of the first measurement run were dropped by a stale-port bug of the measuring hook (a negative time); the table above is the final run, 20 valid samples per cell. The measuring test is not in the PR.
Adaptive (
internal/adaptive,IdleTimeout1 s, Protocol Auto, 12 runs): main, ReadHeaderTimeout off 4884 to 5099 ms, default 1002 to 1023 ms; this PR, off 1002 to 1023 ms, default 1004 to 1021 ms.Decision, with the numbers
Options: (1) timerfd on every loop; (2) only when one of Idle/Read/Write/ReadHeader is set; (3) a sweep period that follows the nearest pending deadline; plus (4) a coarser fine cadence (100 ms).
ReadHeaderTimeout: -1change.BenchmarkSweep1194, hot cache, 0 allocs), that is 2.6 to 3.1 ms/s = 0.26 to 0.31 % of a core per loop at 40 sweeps/s with 10k idle conns. Not worth a hot-path structure.sweepFloor= 100 ms, an idle loop still wakes 466 to 484 times a second per loop (vs 665 to 690 at 25 ms), because the cost is not the timer's own tick but the backoff it restarts (below). It would also loosen the "about 30 ms" the default configuration has always had forReadHeaderTimeout, for ~30 % fewer wakeups. Kept 25 ms, bound stated in the docs:IdleTimeout/ReadTimeout/WriteTimeoutare enforced within about 30 ms of their time on epoll (measured lateness 0 to 25 ms for idle and read, 1 s and 5 s).Cost (idle loop;
/proc/self/taskdeltas over 15 to 20 s after 3 s of settle; 2 workers; HTTP/1, 10k idle keep-alive conns or none)Context switches per second of each loop thread (the two highest-CPU threads that are the loops; "process" is all threads, the Go runtime's included):
epoll_waitand counted as activity, so it resetsconsecutiveEmptyand the wait goes back to 1 ms x 10 then 2 ms for the next ~17 rounds every 25 ms.timerOnly, 4 lines in the run loop): loop wakeups 216 to 257/s with the timer, within about 15 % of a loop without one, and the default configuration drops from ~675 to ~230/s per loop as a side effect. Closing times are unchanged (table above, "this PR" column is with it).ReadHeaderTimeout: -1only; the default configuration already pays it.Hot path
No per-request code changed: the diff in the run loop is the gate (a constant 0x1F for 0x3FF/0x1F by config), the
epoll_waitcap and one bool per batch.BenchmarkKeepAlive1194(8 keep-alive conns, 11 allocs/op on the client side, 834 B/op, identical on every build), test binaries built once, order main / this PR / main twin / this-PR twin x6,-test.cpu 2 -test.benchtime 3s:The host was noisy (single runs between 10.9 and 26 us), so the result is "inside the A/A spread", and the structural argument above is the actual evidence: no allocation and no syscall per request was added.
Tests
TestIdleTimeoutIsOnTimeWithoutReadHeaderTimeout1194,TestReadTimeoutIsOnTimeWithoutReadHeaderTimeout1194,TestWriteTimeoutIsOnTimeWithoutReadHeaderTimeout1194(window [timeout - 50 ms, timeout + 200 ms]; write [1 s - 50 ms, 2.8 s + 200 ms]; measured jitter on the fixed build 0 to 25 ms, 74 ms at most with 4 engines in parallel; on main: 4.7 to 5.1 s, 4.2 to 5.1 s, 4.7 to 4.9 s),TestEveryLoopHasASweepTimer1194(RHT off, all four off, default) andTestSweepTickAloneIsAnIdleRound1194(the idle counter at a tick, 355 to 367 here and under 20 before). They take 1.0 + 1.0 + 2.0 + 0.05 + 1.5 s = 5.7 s per run and live in the epoll package, which the root-racestep (540 s timeout, ~362 s used) runs; the stepped-clock tests that stay out of the race steps are 80 s, these are not.ReadHeaderTimeout: -1to mean "no timerfd" (TestQueuedZeroTimeoutRoundsDoNotStretchTheIdleWait885,TestResumeAcceptReachesALoopThatCannotPark886,TestBeginPauseAcceptReachesAnIdleLoop886,TestDetachedIdleDeadlineIsReapedPromptlyUnderLoad885) now disarm the timerfd by hand (TimerfdSettimezero), so they still test what they were written for;TestTimeoutSentinelReachesEngineno longer reads the sentinel through the epoll_wait cap (it checks the config fields, as before).Verification run (docker arm64,
--cpus 2, memlock 8 MiB)-race -count=30on the final commit: all pass; idle 1.001 to 1.023 s, read 1.014 to 1.022 s, write 2.114 to 2.179 s (the first-race -count=20run, on the commit before the write window was widened, had write at 2.058 to 2.167 s and one close at 1.242 s: the peer's kernel acknowledged less than the stall rule's floor in the first window, so the rule closed at one WriteTimeout. That is why the write window is[1 s - 50 ms, 2.3 s + 200 ms]; main closes at 4.2 to 5.1 s).TestEveryLoopHasASweepTimer1194(RHT off, all off) andTestSweepTickAloneIsAnIdleRound1194fail.-race -count=1:internal/engine/epollok 489 s,internal/adaptiveok 168 s, root package ok 64 s,middleware/sseok 216 to 218 s (2 of 2 on this PR, 2 of 2 on main).-tags celeris_monoskew,CELERIS_MONO_SKEW=+3600sand-3600s: the whole epoll package passes in both (TestEngineTimeoutsIgnoreAStepOfTheWallClock1178, the monotest scenarios, 75.1 s and 77.3 s, included), andinternal/monopasses in both.TestSlowReadingClientKeepsStream888/epoll/async(middleware/sse) failed once ("the stream of a client that kept reading (75776 bytes taken) was ended before 5s", after 2.3 s) in the first full-package-racerun of this PR, a package run of 211 s on a loaded host. It passed 8 of 8 on this PR and 8 of 8 on main in isolation, and 2 of 2 full-package runs on this PR afterwards. I could not reproduce it. This PR changes the idle wait of every epoll loop (the timer tick no longer restarts the backoff), so the reviewer should look at the async path of that test with that in mind. A search of the repository's issues finds no earlier report of this test flaking.Docs
One paragraph on the sweep, under
ReadTimeout(epoll within about 30 ms for ReadTimeout and IdleTimeout, io_uring about 0.8 s apart on an idle worker, 0.83 s measured, about 1 s with ReadHeaderTimeout off; WriteTimeout is judged in windows that end at a sweep, see its own paragraph);IdleTimeoutpoints to it; the detached-stream rule says 25 ms on epoll; theWriteTimeoutsentence "the times hold with ReadHeaderTimeout set" is gone.Interaction with #1193
#1193 is merged. Main is merged into this branch (no textual conflict); the semantic one is resolved in
config.go: its sentence "With ReadHeaderTimeout off epoll has no such timer..." and the qualifier "when ReadHeaderTimeout is set (the default)" are gone, the two near-identical sweep paragraphs are one (underReadTimeout), and there is one io_uring figure (the 0.8 s / 1 s of #1193, checked againstworker.go's sweep comments).Not in this PR
ReadHeaderTimeout/idle timeouts without this bound.Release note
Review follow-up (r1, APPROVE-WITH-FIXES, no blockers)
Main (f067838) is merged in; two further commits (the merge and
49e12b3):epoll_waitceiling was wrong: with base 1 ms the backoff tops out at 4 ms, so the ceiling never binds and does not keep the bound if the timerfd cannot be created (the loop then sweeps from the gate, ~32 x 4 ms = ~130 ms apart with the listener open, ~16 s with none). The comment says so, and a failed timerfd creation, settime or epoll_ctl is logged once per loop (TestSweepTimerFailureIsLogged1194, through atimerfdCreateseam).-raceagainst a 2.5 s bound; its window is now[1 s - 50 ms, 2.8 s + 200 ms](main closes at 4.2 s and later, so it is still RED there).small-events-no-header-timercell is kept: it now guards the ReadHeaderTimeout -1 configuration that used to have no sweep timer, and its comment says so.--cpus 2: the 5 original tests RED on main f067838 (idle 4.69 s, read 4.65 s, write 4.67 s, the two structural tests), and the 6 tests GREEN with-race -count=20(120 of 120; idle 1.005 to 1.016 s, read 1.006 to 1.017 s, write 2.090 to 2.141 s); epoll package-raceok 492 s, adaptive ok 168 s;CELERIS_MONO_SKEW+3600 s and -3600 s with-tags celeris_monoskew: the epoll package (457 s, the 1178 scenarios included) andinternal/monopass; golangci-lint: 0 issues.CI
CI of the first head (a6f4ca2): all checks pass. The epoll package in the root
-racestep took 393.3 s there; against its own base a4b10df (388.6 s) that is +4.7 s, the new tests; the +29 s I first compared (e449c2c, 364.5 s) was noise between main runs (362 to 389 s). The new head's CI is below.New head 49e12b3: all CI checks pass (31 pass, 2 skipping: CodeRabbit and the arm64 macro benchmark). The epoll package in the root
-racestep took 398.6 s (a6f4ca2: 393.3 s; base a4b10df: 388.6 s; budget 540 s).