Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 19 additions & 15 deletions config.go
Original file line number Diff line number Diff line change
Expand Up @@ -274,16 +274,19 @@ type Config struct {
// take, from a client that keeps sending frames, is not bounded by ReadTimeout.
//
// The native engines look at ReadTimeout, IdleTimeout and WriteTimeout in a
// sweep of their connections, so one is enforced up to a sweep period after
// its time: 25 ms on epoll when ReadHeaderTimeout is set (the default), and on
// io_uring about 0.8 s (0.83 s measured on an idle worker; about 1 s with
// ReadHeaderTimeout off). With ReadHeaderTimeout off epoll has no such timer and
// looks much more rarely (a 1 s IdleTimeout closed an idle connection at 3 to 6 s
// in measurement). A timeout of a few seconds or less is approximate by that
// much on io_uring. The HTTP/1 header deadline, and the budget of a connection
// whose protocol is not decided yet, have a timer of their own on io_uring and
// are not; the HTTP/2 header-block bound of ReadHeaderTimeout is checked by the
// sweep and is late by up to a sweep period.
// sweep of their connections, so ReadTimeout and IdleTimeout are enforced up to
// a sweep period after their time. On epoll the sweep runs every 25 ms, from a
// kernel timer each worker arms whatever ReadHeaderTimeout and the timeouts
// are, so within about 30 ms (a 1 s IdleTimeout closed an idle connection at
// 1.00 to 1.03 s on docker arm64 with 2 CPUs, with ReadHeaderTimeout off as well
// as set). On io_uring the sweep of an idle worker is about 0.8 s apart (0.83 s
// measured; about 1 s with ReadHeaderTimeout off), so a timeout of a few
// seconds or less is approximate by that much there. WriteTimeout is judged in
// windows that end at a sweep too: see WriteTimeout. The HTTP/1 header
// deadline, and the budget of a connection whose protocol is not decided yet,
// have a timer of their own on io_uring and are not late; the HTTP/2
// header-block bound of ReadHeaderTimeout is checked by the sweep and is late
// by up to a sweep period.
ReadTimeout time.Duration
// ReadHeaderTimeout caps the read of just the request line + headers
// (separately from the request body, which ReadTimeout covers). This
Expand Down Expand Up @@ -338,7 +341,7 @@ type Config struct {
// is bounded by the minimum rate described below); -1 sets no such bound. The close comes between about 3/4
// and 5/4 of WriteTimeout after the peer last took a byte (its progress
// is sampled every quarter of WriteTimeout; a SEND completion restarts the
// clock at once), plus the engine's sweep interval (up to about 0.1 s on
// clock at once), plus the engine's sweep interval (25 ms on
// epoll, up to 2 s on io_uring; a WriteTimeout shorter than that can
// double). An h2c stream is not covered yet (celeris#884).
// On the std engine net/http arms WriteTimeout as one deadline for the
Expand Down Expand Up @@ -380,10 +383,9 @@ type Config struct {
// WriteTimeout with that in mind. WriteTimeout -1 is no write timeout at
// any rate. What the client sends is not progress: a pipelined request or
// an HTTP/2 PING does not hold a parked response. The times above hold
// with ReadHeaderTimeout set (the default): the engine's 25 ms sweep timer
// is armed only then, and without it a stalled response can outlast them by
// seconds (measured: WriteTimeout 1s, ReadHeaderTimeout -1, closed at 5 s,
// on main too).
// with the epoll sweep at its 25 ms period, which every worker has whatever
// ReadHeaderTimeout and the other timeouts are; io_uring sweeps more coarsely,
// see ReadTimeout.
//
// While the rule has a baseline for a parked response (from the first sweep
// that finds it parked), it is the only bound on the connection:
Expand Down Expand Up @@ -544,6 +546,8 @@ type Config struct {
// failed getsockopt) the period is always extended. A frame for a stream the server reset moves the period only for a
// request that expired at ReadTimeout, for the 2 x ReadTimeout its late DATA may
// take. On epoll, for a response parked on the socket, see WriteTimeout.
//
// How closely IdleTimeout is enforced is described under ReadTimeout.
IdleTimeout time.Duration
// ShutdownTimeout is the deadline of the graceful shutdown that cancelling
// the context of StartWithContext or StartWithListenerAndContext starts
Expand Down
46 changes: 35 additions & 11 deletions internal/engine/epoll/checktimeouts_floor_885_linux_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ import (
"testing"
"time"

"golang.org/x/sys/unix"

"github.com/goceleris/celeris/internal/conn"
"github.com/goceleris/celeris/internal/engine"
"github.com/goceleris/celeris/internal/mono"
Expand All @@ -22,8 +24,8 @@ import (
)

// celeris#885 item 29: checkTimeouts is driven by the loop's iterations (every
// 32 epoll_wait returns when ReadHeaderTimeout is set, which is the default, or
// a conn is detached), so a busy loop swept thousands of times a second where
// 32 epoll_wait returns, since celeris#1194; before, when ReadHeaderTimeout was
// set, which is the default, or a conn was detached), so a busy loop swept thousands of times a second where
// the 25 ms timerfd promises 40 (measured: 5,400 to 6,200 a second in all on
// two loops with sixteen busy clients, 12.9% of a core on 8192 idle conns).
// The sweep now has a floor between two of them (sweepFloor), outside
Expand All @@ -32,15 +34,21 @@ import (
// Two tests. The first counts the sweeps one busy loop makes in a window, under
// sixteen busy keep-alive clients and the default configuration; it fails on a
// loop whose gate has no floor. The second is the price of the floor and the
// guard against a floor set too high: with the timerfd off (ReadHeaderTimeout
// disabled) the gate is the only source of sweeps on a busy loop, and a
// guard against a floor set too high: with the timerfd disarmed
// (every loop has one since celeris#1194) the gate is the only source of sweeps on a busy loop, and a
// detached conn's idle deadline (a WebSocket's, an SSE stream's) must still be
// reaped about when it expires.

// sweepProbe885 records the time of every sweep of the loop that serves addr.
type sweepProbe885 struct {
addr string

// disarm, when set, disarms the timerfd of every loop at its first sweep,
// so that the gate is the only source of sweeps left (every loop has a
// timerfd since celeris#1194, whatever its ReadHeaderTimeout).
disarm bool
disarmed map[*Loop]bool // guarded by mu

mu sync.Mutex
at map[*Loop][]int64 // UnixNano of each sweep, by loop
live map[*Loop]int // the conns a loop had at its last sweep
Expand All @@ -59,7 +67,14 @@ func (p *sweepProbe885) hook(l *Loop) {
p.at[l] = append(p.at[l], mono.Now())
p.live[l] = len(l.liveConns)
p.timerFD, p.sawLoop = l.timerFD, true
first := p.disarm && !p.disarmed[l]
if first {
p.disarmed[l] = true
}
p.mu.Unlock()
if first && l.timerFD >= 0 {
_ = unix.TimerfdSettime(l.timerFD, 0, &unix.ItimerSpec{}, nil)
}
for _, cs := range l.liveConns {
if h1 := cs.h1State; h1 != nil && h1.Detached.Load() && !p.seen[cs] {
p.seen[cs] = true
Expand Down Expand Up @@ -136,6 +151,12 @@ var _ stream.AsyncRouteResolver = floorHandler885{}
// startFloorEngine885 serves cfg on two loops (the least the engine accepts)
// and returns the address and the probe. The engine is stopped, and Listen waited for, at cleanup.
func startFloorEngine885(t *testing.T, cfg resource.Config) (string, *sweepProbe885) {
return startFloorEngine885x(t, cfg, false)
}

// startFloorEngine885x is startFloorEngine885 with the timerfds disarmed when
// disarm is set (see sweepProbe885).
func startFloorEngine885x(t *testing.T, cfg resource.Config, disarm bool) (string, *sweepProbe885) {
t.Helper()
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
Expand All @@ -150,7 +171,7 @@ func startFloorEngine885(t *testing.T, cfg resource.Config) (string, *sweepProbe
if err != nil {
t.Fatalf("epoll engine: %v", err) // not a skip: a skip would take the witness out of CI silently
}
p := &sweepProbe885{addr: addr, at: map[*Loop][]int64{}, live: map[*Loop]int{},
p := &sweepProbe885{addr: addr, disarm: disarm, disarmed: map[*Loop]bool{}, at: map[*Loop][]int64{}, live: map[*Loop]int{},
detached: make(chan *conn.H1State, 16), seen: map[*connState]bool{}}
hook := p.hook
testHookCheckTimeouts.Store(&hook)
Expand Down Expand Up @@ -292,16 +313,16 @@ func TestBusyLoopSweepsAtMostOnceAFloorPeriod885(t *testing.T) {
}

// TestDetachedIdleDeadlineIsReapedPromptlyUnderLoad885 is the price of the
// floor. ReadHeaderTimeout is disabled, so there is no timerfd and the gate
// (every 32 iterations while a conn is detached) is the only sweep on a busy
// loop. Sixteen clients keep the loop busy; a detached conn (a WebSocket's,
// floor. The timerfd is disarmed (every loop has one since celeris#1194,
// whatever its ReadHeaderTimeout), so the gate (every 32 iterations) is the only
// sweep on a busy loop. Sixteen clients keep the loop busy; a detached conn (a WebSocket's,
// SSE's) gets an idle deadline 200 ms ahead and must be reaped (within 400 ms
// of it, a sanity bound). The median gap between two sweeps is the structural
// bound, and does not depend on where in a gap the deadline falls: a floor of
// 250 ms moves every gap past it (the control of that mutant), where a floor
// of the timerfd's 25 ms leaves the gaps at the time of 32 iterations.
func TestDetachedIdleDeadlineIsReapedPromptlyUnderLoad885(t *testing.T) {
addr, p := startFloorEngine885(t, resource.Config{ReadHeaderTimeout: -1, AsyncHandlers: true})
addr, p := startFloorEngine885x(t, resource.Config{ReadHeaderTimeout: -1, AsyncHandlers: true}, true)
dc, err := net.DialTimeout("tcp", addr, 3*time.Second)
if err != nil {
t.Fatal(err)
Expand Down Expand Up @@ -373,8 +394,11 @@ func TestDetachedIdleDeadlineIsReapedPromptlyUnderLoad885(t *testing.T) {
t.Logf("celeris885 item 29: detached conn reaped %v after its deadline; %d sweeps of its loop in %v, gaps median %v max %v, %d requests in all; timerFD=%d",
late.Round(time.Millisecond), len(w), to.Sub(from).Round(time.Millisecond), medGap.Round(time.Millisecond), maxGap.Round(time.Millisecond), reqs, tfd)

if !saw || tfd >= 0 {
t.Fatalf("PREMISE: the loop has a timerfd (%d, saw %v): ReadHeaderTimeout -1 disables it, and without that the gate is not the only sweep", tfd, saw)
p.mu.Lock()
off := p.disarmed[detLoop]
p.mu.Unlock()
if !saw || !off {
t.Fatalf("PREMISE: the timerfd of the detached conn's loop was not disarmed (timerFD %d, saw %v, disarmed %v): without that the gate is not the only sweep", tfd, saw, off)
}
if reqs < 1000 {
t.Fatalf("PREMISE: the loop was not busy: %d requests in %v", reqs, to.Sub(from))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,19 @@ const sndbuf885x42 = 16 << 20
// gives a few hundred KiB), so the loop reads it over several rounds: a turn is
// driverReadBudget reads of driverReadBufSize (256 KiB), and every round after
// the first is an epoll_wait(0) that collects nothing. The engine has
// ReadHeaderTimeout -1, so no 25 ms timerfd can end the flood's counting between
// two rounds. On the last read the count is 0 (round 1 had an event), and with
// ReadHeaderTimeout -1 and its 25 ms timerfd disarmed, so that nothing can end
// the flood's counting between two rounds. On the last read the count is 0 (round 1 had an event), and with
// more than eleven turns the wait the loop would make next (adaptiveTimeoutMs)
// is the base, 1 ms: the rounds a queue forced to zero are not idle rounds.
func TestQueuedZeroTimeoutRoundsDoNotStretchTheIdleWait885(t *testing.T) {
eng, _ := startE2E885(t, false, newPathHandler885(1), func(c *resource.Config) { c.ReadHeaderTimeout = -1 })
l := eng.WorkerLoop(0).(*Loop)
if l.timerFD >= 0 {
t.Fatalf("apparatus: the loop has a sweep timerfd, which can fire between two rounds of the flood")
if l.timerFD < 0 {
t.Fatalf("apparatus: the loop has no sweep timerfd to disarm (every loop has one, celeris#1194)")
}
// The timerfd can fire between two rounds of the flood: disarm it.
if err := unix.TimerfdSettime(l.timerFD, 0, &unix.ItimerSpec{}, nil); err != nil {
t.Fatalf("apparatus: disarm the sweep timerfd: %v", err)
}
a, peer := socketpairNonblocking(t)
t.Cleanup(func() { _ = unix.Close(a); _ = unix.Close(peer) })
Expand Down
16 changes: 8 additions & 8 deletions internal/engine/epoll/engine.go
Original file line number Diff line number Diff line change
Expand Up @@ -447,14 +447,14 @@ func (e *Engine) ResumeAccept() error {
l.listenFDClosed.Store(false)
l.wakeIfSuspended()
// A loop that has not parked closed its listener at the pause and
// went back to epoll_wait, which is 500 ms long with no listener,
// and a loop without a header deadline has no 25 ms timerfd to end
// it sooner. That is a loop that holds a connection, and also an
// idle one just after the pause (it parks at the iteration after
// the close). wakeIfSuspended does not reach it: the wake eventfd,
// which is in the epoll set, does (celeris#886). A leaf lock and one
// write(2); a loop that is running just reads the counter at its
// next wakeup.
// went back to epoll_wait, which is 500 ms long with no listener;
// its 25 ms timerfd (every loop has one since celeris#1194) ends the
// wait sooner, but only at the next tick. That is a loop that holds a
// connection, and also an idle one just after the pause (it parks at
// the iteration after the close). wakeIfSuspended does not reach it:
// the wake eventfd, which is in the epoll set, does (celeris#886). A
// leaf lock and one write(2); a loop that is running just reads the
// counter at its next wakeup.
l.wakeFD.Signal()
}
return nil
Expand Down
Loading
Loading