diff --git a/README.md b/README.md index 4f2effb96..ce0de0bb6 100644 --- a/README.md +++ b/README.md @@ -421,7 +421,7 @@ For Prometheus exposition and debug endpoints, use [`middleware/metrics`](middle | `Detach` (async keep-alive) | yes | yes | no | | Connection hijack | yes | yes | yes | -**CPU pinning** (epoll and io_uring): loop `i` pins its thread to the `i`-th CPU the thread that starts the server is allowed to use (`taskset`, a cgroup cpuset, `sched_setaffinity`), never to a CPU outside that set; on a host with several NUMA nodes the CPUs are taken from the nodes the set covers in turn, so every allowed CPU holds a loop before any holds a second. With `Workers` unset on a host whose allowed CPUs are alike, the engines start `GOMAXPROCS - max(1, GOMAXPROCS/8)` loops (28 of 32, 14 of 16, 7 of 8, 3 of 4; no reserve for a `GOMAXPROCS` of 2 or less): a loop can hold its thread and its P while it blocks in the kernel, so the Ps without a loop are left to the goroutines that are not loops (async and WebSocket handlers, the HTTP/2 stream pool, the garbage collector's workers) (celeris#756, #757). The mechanism is inferred from traces, not established; the reserve is provisional and its effect on latency is still being measured. To opt out, set `Workers: runtime.GOMAXPROCS(0)`: on such a host it starts one loop per P, as before (on a big.LITTLE host the default is not reduced, and a `Workers` above the non-little CPUs wraps loops over them, see below); a `Workers` you set is never reduced. `Metrics.Workers`, `NumWorkers()` and the server-wide `Config.MaxConns` limit follow the loops started, as on big.LITTLE below. On a host whose allowed CPUs differ in capacity (arm64 big.LITTLE, read from `/sys/devices/system/cpu/cpuN/cpu_capacity`, or the CPU parts in `/proc/cpuinfo` when the kernel exports none), the engines start at most one loop per CPU with at least half the largest capacity (`min(GOMAXPROCS, non-little CPUs)`: 8 loops on a 12-CPU host with 4 little CPUs, fewer than `GOMAXPROCS`) and pin every one of them: none runs on a little CPU and none is left unpinned (celeris#1036; a loop that floated drifted onto the little CPUs and a loop pinned to one stalled its connections). A `Config.Workers` above that count is honoured: the extra loops wrap round-robin over the non-little CPUs, each pinned, so the CPUs that get a second loop (4 of 8 for `Workers: 12`) take about twice the connections of the others; on io_uring the count is still capped by `RLIMIT_MEMLOCK`. Consequences of the smaller count: `Metrics.Workers` and `NumWorkers()` read the loops started (on the adaptive engine, the sum over its two engines), driver pools sized from `NumWorkers()` default smaller (a PostgreSQL pool of `4*NumWorkers`: 32 on that host, not 48), and `Config.MaxConns` is per loop, so the server-wide limit shrinks with it. Measured on the one big.LITTLE host tried (arm64, 8 big and 4 little CPUs, the NIC interrupt pinned to a big CPU; two dispatches, 35 s windows with 5 repetitions per cell, and 90 s windows with 4 repetitions at io_uring with 512 connections), against celeris v1.5.11 (all 12 loops pinned, 4 of them on the little CPUs): requests per second were 1.43 times at io_uring with 256 connections, 1.09 times at adaptive with 128 connections, 1.51 times at io_uring with 512 connections and 1.04 times for epoll with 4 KiB POST bodies, and 0.95 times for epoll with 512 connections (the one cell below v1.5.11). The behaviour before this change, with 4 floating loops, served epoll with 4 KiB POST bodies at 0.47 times v1.5.11 in an earlier run. The tail costs something: for epoll with 4 KiB POST bodies the median p99.99 was 99 ms against 48 ms for v1.5.11, and at io_uring with 512 connections a 90 s run had a median of 61 requests of 1 s or more and its longest took 1.0 to 2.5 s, against 664 and 12 to 21 s for v1.5.11, so stalls of 1 to 2.5 s remain there. A `Config.Workers` above the non-little CPU count is wrapped over the pinned big CPUs; it ran on hardware once, in an earlier run (35 s windows, 3 repetitions), and showed a worse tail than the default at io_uring with 512 connections (0.10% to 0.20% of requests at about 208 ms, maxima of 2.1 to 2.2 s in two of three cells). The engine logs one line saying how many loops it started and why; a pin the kernel refuses is logged once. With `Workers` unset, and a default count of at least one loop for each non-little CPU (8 of 8 on that host; a lower `GOMAXPROCS`, such as the default under a Kubernetes CPU limit, plans as above, so 7 default loops are 0, 1, 6, 7, 8, 9 and 10, more than the 6 loops of 8), the default plan also leaves the loops off the non-little CPUs that service the busy interrupt vectors of the NIC port that received the traffic (celeris#1036 item 1). At start the engine takes the physical ports behind the listen address (through bonds and VLANs); if one of them took at least 60% of the packets received since boot (`/sys/class/net//statistics/rx_packets`, and at least a million) it leaves without a loop the non-little CPUs that service its vectors with at least 1000 interrupts since boot (`/proc/interrupts`, `/proc/irq/N/effective_affinity_list`): CPUs 6 and 8 on that host, so the loops sit on 0, 1, 7, 9, 10 and 11. It frees never more than a quarter of the non-little CPUs (the loops never fall under `max(min(2, n), ceil(3n/4))` for `n` non-little CPUs) and never raises the count. A CPU that also services a busy vector of another port (or one shared with another port) keeps its loop, and at least one busy vector of a port with at least 10% of the packets pooled over the listener's ports (a choice, not measured; msr1's ports carry about 27% and 73%) must stay on a CPU that keeps its loop: with a single port, or with a data NIC beside a management NIC that carries less, the plan is the plan without the rule. The rule also does nothing when a file cannot be read, when a port has a million packets and no busy vector, when the busy vectors sit on more CPUs than the quarter allows (a NIC spread over its queues), when the ports share the packets about evenly, or when a busy vector of another port is serviced by a CPU that runs no loop (the msr1 boot placement, IRQ 167 on little CPU 4: that plan is not measured). A host whose CPUs are alike, and any `Config.Workers` you set, read no NIC file. The Info line of the plan names the CPUs left free, the port and its share; the Debug line names the candidates it passed over or why the rule stepped aside. Validated on one cluster host (msr1, arm64 big.LITTLE; probatorium runs 39355553598, 35 s, and 39367858847, 90 s, same boot): the rule planned exactly the loops 0, 1, 7, 9, 10 and 11 by itself and matched the hand-pinned set within 1% of requests per second on io256, ad128, ep512 and post4k, and at io512 reached 495k against 500k requests per second with a maximum of 34 ms and no request of 1 s or more (v1.5.11: 343k, maximum 14.1 s, 664 requests of 1 s or more; the 8 loops of this paragraph alone: 518k, maximum 1.7 s, 61 requests of 1 s or more). Against v1.5.11 post4k p99.99 was 3.4 ms against 48 ms (v1.5.11) and 99 ms (the 8 loops alone), with throughput within 1% of the hand-pinned set; io256 gained 35%, ad128 5% and io512 45%. Against the 8 loops alone the rule has lower tails at io512 (maximum 34 ms against 1.7 s) and post4k, 495k against 518k requests per second at io512, and its io256 p99.99 is the FAIL cell below. The cost: ep512 throughput is about 0.845 times that of v1.5.11 (15% lower; the 8 loops alone: 0.948 times). The pre-registered test of the rule came out FAIL by its letter, only because the io256 p99.99 of the rule over the hand-pinned set was 5.59 times at the medians (overlapping ranges, 3.2 to 32.8 ms and 3.6 to 28.3 ms). A re-run of that cell on one commit (probatorium run 39396052154: the rule against the same six loops forced by a knob, 8 repetitions per arm) had the rule plan the same six loops by itself, with 0.995 times the requests per second, a maximum-latency ratio of 1.02 and a p99.99 median of 16.8 ms against 6.0 ms (2.83 times; 6 of 8 against 3 of 8 repetitions over 10 ms); the one-sided exact Mann-Whitney test does not reject (p=0.080) and the registered outcome is NOISE-ONLY, but at 8 repetitions its power is low and a modest p99.99 tail cost at io256 in the 10 to 30 ms range is not excluded. Limits: the rule is fitted to that one host after its data was seen (the 60% and the quarter are choices), and its counts are since boot, so a server started at boot steps aside and a wrapped 32-bit interrupt counter can hide a vector (IRQ 135 on msr1 is at about 4.05e9 of 4.29e9); a LAG rehash that moves the traffic to the other port leaves the plan on the old CPUs until the next start; a wildcard listener pools every physical port that is up, related or not (the 10% share above governs a data NIC beside a management NIC); on NICs whose ports share all their vectors (mlx4) the rule never acts; io_uring's `RLIMIT_MEMLOCK` cap keeps the first loops of the plan in CPU order and can drop the CPU the rule meant to keep (a memlock of 4 gives 0, 1, 7 and 9); and the adaptive engine builds its io_uring standby when it first switches, from the counts of that moment, so its loop set can differ from the primary's. A handler that runs inline runs on a pinned thread and a process it starts inherits the pin: start it from a goroutine of its own, or mark the route `.Async()` (see `Config.AsyncHandlers`). Setting `AsyncHandlers` alone is not enough: a route that only inherits it runs inline until it is promoted. +**CPU pinning** (epoll and io_uring): loop `i` pins its thread to the `i`-th CPU the thread that starts the server is allowed to use (`taskset`, a cgroup cpuset, `sched_setaffinity`), never to a CPU outside that set; on a host with several NUMA nodes the CPUs are taken from the nodes the set covers in turn, so every allowed CPU holds a loop before any holds a second. With `Workers` unset on a host whose allowed CPUs are alike, the engines start `GOMAXPROCS - max(1, GOMAXPROCS/4)` loops (24 of 32, 12 of 16, 6 of 8, 3 of 4; no reserve for a `GOMAXPROCS` of 2 or less): a loop can hold its thread and its P while it blocks in the kernel, so the Ps without a loop are left to the goroutines that are not loops (async and WebSocket handlers, the HTTP/2 stream pool, the garbage collector's workers) (celeris#756, #757). The mechanism is inferred from traces, not established; the reserve is provisional and its effect on latency is still being measured. To opt out, set `Workers: runtime.GOMAXPROCS(0)`: on such a host it starts one loop per P, as before (on a big.LITTLE host the default is not reduced, and a `Workers` above the non-little CPUs wraps loops over them, see below); a `Workers` you set is never reduced. `Metrics.Workers`, `NumWorkers()` and the server-wide `Config.MaxConns` limit follow the loops started, as on big.LITTLE below. On a host whose allowed CPUs differ in capacity (arm64 big.LITTLE, read from `/sys/devices/system/cpu/cpuN/cpu_capacity`, or the CPU parts in `/proc/cpuinfo` when the kernel exports none), the engines start at most one loop per CPU with at least half the largest capacity (`min(GOMAXPROCS, non-little CPUs)`: 8 loops on a 12-CPU host with 4 little CPUs, fewer than `GOMAXPROCS`) and pin every one of them: none runs on a little CPU and none is left unpinned (celeris#1036; a loop that floated drifted onto the little CPUs and a loop pinned to one stalled its connections). A `Config.Workers` above that count is honoured: the extra loops wrap round-robin over the non-little CPUs, each pinned, so the CPUs that get a second loop (4 of 8 for `Workers: 12`) take about twice the connections of the others; on io_uring the count is still capped by `RLIMIT_MEMLOCK`. Consequences of the smaller count: `Metrics.Workers` and `NumWorkers()` read the loops started (on the adaptive engine, the sum over its two engines), driver pools sized from `NumWorkers()` default smaller (a PostgreSQL pool of `4*NumWorkers`: 32 on that host, not 48), and `Config.MaxConns` is per loop, so the server-wide limit shrinks with it. Measured on the one big.LITTLE host tried (arm64, 8 big and 4 little CPUs, the NIC interrupt pinned to a big CPU; two dispatches, 35 s windows with 5 repetitions per cell, and 90 s windows with 4 repetitions at io_uring with 512 connections), against celeris v1.5.11 (all 12 loops pinned, 4 of them on the little CPUs): requests per second were 1.43 times at io_uring with 256 connections, 1.09 times at adaptive with 128 connections, 1.51 times at io_uring with 512 connections and 1.04 times for epoll with 4 KiB POST bodies, and 0.95 times for epoll with 512 connections (the one cell below v1.5.11). The behaviour before this change, with 4 floating loops, served epoll with 4 KiB POST bodies at 0.47 times v1.5.11 in an earlier run. The tail costs something: for epoll with 4 KiB POST bodies the median p99.99 was 99 ms against 48 ms for v1.5.11, and at io_uring with 512 connections a 90 s run had a median of 61 requests of 1 s or more and its longest took 1.0 to 2.5 s, against 664 and 12 to 21 s for v1.5.11, so stalls of 1 to 2.5 s remain there. A `Config.Workers` above the non-little CPU count is wrapped over the pinned big CPUs; it ran on hardware once, in an earlier run (35 s windows, 3 repetitions), and showed a worse tail than the default at io_uring with 512 connections (0.10% to 0.20% of requests at about 208 ms, maxima of 2.1 to 2.2 s in two of three cells). The engine logs one line saying how many loops it started and why; a pin the kernel refuses is logged once. With `Workers` unset, and a default count of at least one loop for each non-little CPU (8 of 8 on that host; a lower `GOMAXPROCS`, such as the default under a Kubernetes CPU limit, plans as above, so 7 default loops are 0, 1, 6, 7, 8, 9 and 10, more than the 6 loops of 8), the default plan also leaves the loops off the non-little CPUs that service the busy interrupt vectors of the NIC port that received the traffic (celeris#1036 item 1). At start the engine takes the physical ports behind the listen address (through bonds and VLANs); if one of them took at least 60% of the packets received since boot (`/sys/class/net//statistics/rx_packets`, and at least a million) it leaves without a loop the non-little CPUs that service its vectors with at least 1000 interrupts since boot (`/proc/interrupts`, `/proc/irq/N/effective_affinity_list`): CPUs 6 and 8 on that host, so the loops sit on 0, 1, 7, 9, 10 and 11. It frees never more than a quarter of the non-little CPUs (the loops never fall under `max(min(2, n), ceil(3n/4))` for `n` non-little CPUs) and never raises the count. A CPU that also services a busy vector of another port (or one shared with another port) keeps its loop, and at least one busy vector of a port with at least 10% of the packets pooled over the listener's ports (a choice, not measured; msr1's ports carry about 27% and 73%) must stay on a CPU that keeps its loop: with a single port, or with a data NIC beside a management NIC that carries less, the plan is the plan without the rule. The rule also does nothing when a file cannot be read, when a port has a million packets and no busy vector, when the busy vectors sit on more CPUs than the quarter allows (a NIC spread over its queues), when the ports share the packets about evenly, or when a busy vector of another port is serviced by a CPU that runs no loop (the msr1 boot placement, IRQ 167 on little CPU 4: that plan is not measured). A host whose CPUs are alike, and any `Config.Workers` you set, read no NIC file. The Info line of the plan names the CPUs left free, the port and its share; the Debug line names the candidates it passed over or why the rule stepped aside. Validated on one cluster host (msr1, arm64 big.LITTLE; probatorium runs 39355553598, 35 s, and 39367858847, 90 s, same boot): the rule planned exactly the loops 0, 1, 7, 9, 10 and 11 by itself and matched the hand-pinned set within 1% of requests per second on io256, ad128, ep512 and post4k, and at io512 reached 495k against 500k requests per second with a maximum of 34 ms and no request of 1 s or more (v1.5.11: 343k, maximum 14.1 s, 664 requests of 1 s or more; the 8 loops of this paragraph alone: 518k, maximum 1.7 s, 61 requests of 1 s or more). Against v1.5.11 post4k p99.99 was 3.4 ms against 48 ms (v1.5.11) and 99 ms (the 8 loops alone), with throughput within 1% of the hand-pinned set; io256 gained 35%, ad128 5% and io512 45%. Against the 8 loops alone the rule has lower tails at io512 (maximum 34 ms against 1.7 s) and post4k, 495k against 518k requests per second at io512, and its io256 p99.99 is the FAIL cell below. The cost: ep512 throughput is about 0.845 times that of v1.5.11 (15% lower; the 8 loops alone: 0.948 times). The pre-registered test of the rule came out FAIL by its letter, only because the io256 p99.99 of the rule over the hand-pinned set was 5.59 times at the medians (overlapping ranges, 3.2 to 32.8 ms and 3.6 to 28.3 ms). A re-run of that cell on one commit (probatorium run 39396052154: the rule against the same six loops forced by a knob, 8 repetitions per arm) had the rule plan the same six loops by itself, with 0.995 times the requests per second, a maximum-latency ratio of 1.02 and a p99.99 median of 16.8 ms against 6.0 ms (2.83 times; 6 of 8 against 3 of 8 repetitions over 10 ms); the one-sided exact Mann-Whitney test does not reject (p=0.080) and the registered outcome is NOISE-ONLY, but at 8 repetitions its power is low and a modest p99.99 tail cost at io256 in the 10 to 30 ms range is not excluded. Limits: the rule is fitted to that one host after its data was seen (the 60% and the quarter are choices), and its counts are since boot, so a server started at boot steps aside and a wrapped 32-bit interrupt counter can hide a vector (IRQ 135 on msr1 is at about 4.05e9 of 4.29e9); a LAG rehash that moves the traffic to the other port leaves the plan on the old CPUs until the next start; a wildcard listener pools every physical port that is up, related or not (the 10% share above governs a data NIC beside a management NIC); on NICs whose ports share all their vectors (mlx4) the rule never acts; io_uring's `RLIMIT_MEMLOCK` cap keeps the first loops of the plan in CPU order and can drop the CPU the rule meant to keep (a memlock of 4 gives 0, 1, 7 and 9); and the adaptive engine builds its io_uring standby when it first switches, from the counts of that moment, so its loop set can differ from the primary's. A handler that runs inline runs on a pinned thread and a process it starts inherits the pin: start it from a goroutine of its own, or mark the route `.Async()` (see `Config.AsyncHandlers`). Setting `AsyncHandlers` alone is not enough: a route that only inherits it runs inline until it is promoted. **NIC interrupts on big.LITTLE hosts** (epoll and io_uring, Linux, hosts whose allowed CPUs differ in capacity only; a host whose CPUs are alike reads nothing there). Once the engine has started its loops it finds the CPUs that service the interrupt vectors of the network interface the listener is reachable through (celeris#1036) and logs one Warn when a serviced CPU of the allowed set runs no loop and is either a little CPU, or a big CPU while no vector it found has a loop beside it. It only reports: it adds no loop, moves no loop and moves no interrupt, so the loops, `Metrics.Workers` and `NumWorkers()` are the ones described above on every host. The rule behind the Warn is the premise of the rx-port rule above: a loop stays beside a busy vector. On the host measured, CPUs 6 and 8 service the vectors of enp49s0 and run no loop in the default plan, and that plan is the measured one as long as CPU 1 keeps a loop beside IRQ 167; so a big CPU without a loop is not a reason to warn while another vector has a loop beside it, and with `Workers: 4` or a `GOMAXPROCS` of 4 (loops on 0, 1, 6 and 7, CPU 8 without one) nothing is logged. A little CPU runs no loop by design, so a vector serviced by one is always named. The CPUs the rx-port rule left without a loop on purpose are never named: a vector it listed, on a CPU it freed. What the Warn does on that host: diff --git a/config.go b/config.go index 98fc4b9d7..a91926291 100644 --- a/config.go +++ b/config.go @@ -123,8 +123,8 @@ type Config struct { // Workers is the number of I/O worker goroutines (the loops of the native // engines). Default: GOMAXPROCS, less a reserve on Linux: with Workers unset // the native engines (epoll, io_uring, adaptive) on a host whose allowed - // CPUs are alike start GOMAXPROCS - max(1, GOMAXPROCS/8) loops (no reserve - // for a GOMAXPROCS of 2 or less): 28 on 32, 14 on 16, 7 on 8, 3 on 4, 2 on + // CPUs are alike start GOMAXPROCS - max(1, GOMAXPROCS/4) loops (no reserve + // for a GOMAXPROCS of 2 or less): 24 on 32, 12 on 16, 6 on 8, 3 on 4, 2 on // 3. A loop locks its thread and can block in the kernel while it holds its // P, so the Ps that run no loop are left to the goroutines that are not // loops (async and WebSocket handlers, the HTTP/2 stream pool, the garbage diff --git a/internal/adaptive/started_loops_756_linux_test.go b/internal/adaptive/started_loops_756_linux_test.go index 93914c76c..7b4c88618 100644 --- a/internal/adaptive/started_loops_756_linux_test.go +++ b/internal/adaptive/started_loops_756_linux_test.go @@ -11,7 +11,7 @@ import ( // celeris#756, #757: startedLoops, which the adaptive engine uses to size the // io_uring memlock check, counts the loops the engines start: the default // count on a host of alike CPUs is the request less the reserve of -// max(1, n/8) (none at 2 or less), a user-set count is the request. The real +// max(1, n/4) (none at 2 or less), a user-set count is the request. The real // planner, no stub; the figures are written out. func TestStartedLoopsLeaveAReserve756(t *testing.T) { if platform.PlanWorkerCPUs(12, true).Heterogeneous { @@ -24,7 +24,7 @@ func TestStartedLoopsLeaveAReserve756(t *testing.T) { explicit bool want int }{ - {16, false, 14}, {16, true, 16}, {8, false, 7}, {32, false, 28}, + {16, false, 12}, {16, true, 16}, {8, false, 6}, {32, false, 24}, {4, false, 3}, {3, false, 2}, {2, false, 2}, {1, false, 1}, {5, true, 5}, } { if got := startedLoops(tc.requested, tc.explicit); got != tc.want { diff --git a/internal/engine/epoll/default_loops_756_linux_test.go b/internal/engine/epoll/default_loops_756_linux_test.go index bb9613a5e..ae7dbe4d6 100644 --- a/internal/engine/epoll/default_loops_756_linux_test.go +++ b/internal/engine/epoll/default_loops_756_linux_test.go @@ -14,7 +14,7 @@ import ( ) // celeris#756, #757: an epoll engine whose worker count is not set by the user -// starts GOMAXPROCS - max(1, GOMAXPROCS/8) loops on a host of alike CPUs (none +// starts GOMAXPROCS - max(1, GOMAXPROCS/4) loops on a host of alike CPUs (none // for a GOMAXPROCS of 2 or less; Resolve raises a count to two), and a count // the user sets is started whole. This is the real planner and real engine, no // stub: Metrics().Workers, NumWorkers(), the loops and the "listening" line @@ -37,11 +37,11 @@ func TestDefaultLoopsLeaveAReserve756(t *testing.T) { workers int // Resources.Workers; 0 = the default want int // loops started }{ - {16, 0, 14}, // the default: 16 Ps, 14 loops (rc1: 16) + {16, 0, 12}, // the default: 16 Ps, 12 loops (rc1: 16, first reserve: 14) {16, 16, 16}, // Workers = GOMAXPROCS: one loop per P, as before {16, 5, 5}, // a user-set count is kept whole {16, 20, 20}, // also above GOMAXPROCS - {8, 0, 7}, + {8, 0, 6}, {4, 0, 3}, {3, 0, 2}, {2, 0, 2}, // no reserve at two diff --git a/internal/platform/cpuplan_1036_linux_test.go b/internal/platform/cpuplan_1036_linux_test.go index bd514a3f0..118e179de 100644 --- a/internal/platform/cpuplan_1036_linux_test.go +++ b/internal/platform/cpuplan_1036_linux_test.go @@ -90,11 +90,11 @@ func TestPlanHeterogeneousNeverUnpinnedNeverLittle1036(t *testing.T) { } if !het { // Alike CPUs: one entry per worker for a user-set count; a - // default count leaves the reserve of max(1, n/8) without + // default count leaves the reserve of max(1, n/4) without // a loop, none for n <= 2 (celeris#756, #757). wantLoops := workers if !explicit && workers > 2 { - wantLoops -= max(1, workers/8) + wantLoops -= max(1, workers/4) } if len(p.CPUs) != wantLoops { t.Fatalf("%s: %d entries, want %d (a host of alike CPUs keeps every worker it is asked for, less the default reserve)", tag, len(p.CPUs), wantLoops) diff --git a/internal/platform/cpuplan_linux.go b/internal/platform/cpuplan_linux.go index a0707e533..91a39d23e 100644 --- a/internal/platform/cpuplan_linux.go +++ b/internal/platform/cpuplan_linux.go @@ -221,16 +221,21 @@ func schedAllowedCPUs() ([]int, error) { // loop (celeris#756, #757; a mechanism inferred from traces, not established). // The reserve leaves that many Ps to them. // -// max(1, n/8), none for n <= 2 (a count that small would drop to one loop or -// none): 14 of 16, chosen by the maintainer after a cluster comparison of 16, -// 14 and 12 loops on one amd64 host. The runs on the final commit that would -// confirm the reserve are still owed, so the value may change; changing it is -// this function's one return statement (celeris#756, #757). +// max(1, n/4), none for n <= 2 (a count that small would drop to one loop or +// none): 12 of 16. It was max(1, n/8), 14 of 16, until a cluster comparison on +// one amd64 host (celeris#756: 16 CPUs, WebSocket large-echo p99 against the +// std engine, 5 repetitions per arm, the two rated levels 0.5 and 0.75 of +// saturation) found that the Ps left free decide the tail: 14 loops on 16 Ps +// 3.26 and 8.75 times, 12 loops 1.45 and 3.37 times at the same levels (a +// measurement of one host, lane-finding, not a closing run). The runs on the +// final commit that would confirm the reserve are still owed, so the value may +// change; changing it is this function's one return statement (celeris#756, +// #757). func defaultLoopReserve(n int) int { if n <= 2 { return 0 } - return max(1, n/8) + return max(1, n/4) } func planWorkerCPUs(numWorkers int, explicit bool, src planSource) CPUPlan { diff --git a/internal/platform/cpuplan_linux_test.go b/internal/platform/cpuplan_linux_test.go index ebe9fd804..ff7053f60 100644 --- a/internal/platform/cpuplan_linux_test.go +++ b/internal/platform/cpuplan_linux_test.go @@ -463,7 +463,7 @@ func TestPlanUnrestrictedHomogeneousEqualsTheOldPlan(t *testing.T) { for _, explicit := range []bool{false, true} { wantN := workers if !explicit && workers > 2 { - wantN -= max(1, workers/8) + wantN -= max(1, workers/4) } wantPlan := want[:wantN] got := planWorkerCPUs(workers, explicit, h.src()).CPUs diff --git a/internal/platform/cpuplan_reserve_756_linux_test.go b/internal/platform/cpuplan_reserve_756_linux_test.go index 0dddc349e..e991db4d7 100644 --- a/internal/platform/cpuplan_reserve_756_linux_test.go +++ b/internal/platform/cpuplan_reserve_756_linux_test.go @@ -10,7 +10,7 @@ import ( // celeris#756, #757: a DEFAULT loop count (Config.Workers not set) on a host of // alike CPUs leaves a reserve of Ps to the goroutines that are not loops: the -// plan holds n - max(1, n/8) loops, none for n <= 2. A user-set count is +// plan holds n - max(1, n/4) loops, none for n <= 2. A user-set count is // honoured, and so is the big.LITTLE plan. These tests pin the POLICY (the // loop counts), not any latency: the figures are written out and the tests do // not call defaultLoopReserve, so they compile and fail by assertion on a tree @@ -21,12 +21,12 @@ func TestPlanDefaultLoopReserve756(t *testing.T) { for _, tc := range []struct{ n, want int }{ {0, 0}, {1, 1}, {2, 2}, // no reserve: a count that small would drop to one loop or none {3, 2}, {4, 3}, {7, 6}, - {8, 7}, - {9, 8}, {15, 14}, - {16, 14}, - {17, 15}, {24, 21}, {31, 28}, - {32, 28}, - {48, 42}, {64, 56}, + {8, 6}, + {9, 7}, {15, 12}, + {16, 12}, + {17, 13}, {24, 18}, {31, 24}, + {32, 24}, + {48, 36}, {64, 48}, } { t.Run(fmt.Sprintf("n=%d", tc.n), func(t *testing.T) { h := newFakeHost(t, rng(0, max(tc.n, 1)-1)) @@ -53,12 +53,13 @@ func TestPlanDefaultLoopReserve756(t *testing.T) { } } -// The headline case, alone: 16 identical CPUs, default request 16 -> 14 loops -// (rc1 gave 16). The same request with the count set by the user -> 16. +// The headline case, alone: 16 identical CPUs, default request 16 -> 12 loops +// (rc1 gave 16, the first reserve 14). The same request with the count set by +// the user -> 16. func TestPlanSixteenAlikeCPUs756(t *testing.T) { h := newFakeHost(t, rng(0, 15)) got := planWorkerCPUs(16, false, h.src()).CPUs - eq(t, "default 16 -> 14 loops", got, rng(0, 13)) + eq(t, "default 16 -> 12 loops", got, rng(0, 11)) got = planWorkerCPUs(16, true, h.src()).CPUs eq(t, "explicit 16 -> 16 loops", got, rng(0, 15)) } @@ -82,13 +83,13 @@ func TestPlanExplicitCountIsHonoured756(t *testing.T) { } // 1..64 default workers on a host of that many CPUs, against the rule written -// out inline: loops = n - max(1, n/8) for n > 2, n otherwise. +// out inline: loops = n - max(1, n/4) for n > 2, n otherwise. func TestPlanDefaultReserveTable756(t *testing.T) { for n := 1; n <= 64; n++ { h := newFakeHost(t, rng(0, n-1)) want := n if n > 2 { - want = n - max(1, n/8) + want = n - max(1, n/4) } if got := len(planWorkerCPUs(n, false, h.src()).CPUs); got != want { t.Errorf("default %d workers: %d loops, want %d", n, got, want) @@ -97,14 +98,14 @@ func TestPlanDefaultReserveTable756(t *testing.T) { } // The reserve counts the workers asked for, not the CPUs allowed: a default -// request of 8 (GOMAXPROCS lowered under a CPU limit) on a 16-CPU host runs 7 -// loops on the first 7 CPUs; a request above the CPUs (GOMAXPROCS set above +// request of 8 (GOMAXPROCS lowered under a CPU limit) on a 16-CPU host runs 6 +// loops on the first 6 CPUs; a request above the CPUs (GOMAXPROCS set above // them) wraps. func TestPlanDefaultReserveCountsWorkersNotCPUs756(t *testing.T) { h := newFakeHost(t, rng(0, 15)) - eq(t, "default 8 of 16 CPUs", planWorkerCPUs(8, false, h.src()).CPUs, rng(0, 6)) + eq(t, "default 8 of 16 CPUs", planWorkerCPUs(8, false, h.src()).CPUs, rng(0, 5)) h = newFakeHost(t, rng(0, 3)) - eq(t, "default 8 of 4 CPUs wraps", planWorkerCPUs(8, false, h.src()).CPUs, []int{0, 1, 2, 3, 0, 1, 2}) + eq(t, "default 8 of 4 CPUs wraps", planWorkerCPUs(8, false, h.src()).CPUs, []int{0, 1, 2, 3, 0, 1}) } // A read error means "homogeneous" (no allowed mask, no capacity files): the @@ -113,8 +114,8 @@ func TestPlanDefaultReserveWithoutAMask756(t *testing.T) { h := newFakeHost(t, nil) src := h.src() src.allowed = func() ([]int, error) { return nil, fmt.Errorf("boom") } - if p := planWorkerCPUs(16, false, src); len(p.CPUs) != 14 || p.Heterogeneous { - t.Errorf("default 16 without a mask: %d loops (%+v), want 14", len(p.CPUs), p) + if p := planWorkerCPUs(16, false, src); len(p.CPUs) != 12 || p.Heterogeneous { + t.Errorf("default 16 without a mask: %d loops (%+v), want 12", len(p.CPUs), p) } if p := planWorkerCPUs(16, true, src); len(p.CPUs) != 16 { t.Errorf("explicit 16 without a mask: %d loops, want 16", len(p.CPUs)) diff --git a/internal/resource/resource.go b/internal/resource/resource.go index 99f50f43b..a0a94c6cc 100644 --- a/internal/resource/resource.go +++ b/internal/resource/resource.go @@ -34,7 +34,7 @@ type Resources struct { // count was not user-set: on a host whose CPUs differ in capacity they then // start at most one pinned loop per non-little CPU, min(GOMAXPROCS, // non-little CPUs) (celeris#1036), and on a host whose CPUs are alike (Linux) - // they start the resolved count less a reserve of max(1, count/8) loops + // they start the resolved count less a reserve of max(1, count/4) loops // (none for a count of 2 or less) that leaves Ps to the goroutines that are // not loops (celeris#756, #757): the loops started are what the engines // report, not Resolve's figure. A count set here is honoured, wrapping over