Skip to content

driver/memcached: bound the binary handshake by DialTimeout (v1.6.0, Refs #1213) - #1217

Merged
FumingPower3925 merged 4 commits into
mainfrom
fix/memcached-handshake-dial-timeout-1213
Nov 1, 2026
Merged

FumingPower3925 merged 4 commits into
mainfrom
fix/memcached-handshake-dial-timeout-1213

Conversation

@FumingPower3925

@FumingPower3925 FumingPower3925 commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

driver/memcached: bound the binary handshake by DialTimeout. Refs #1213 (found by the review of #1209; same class as #1205).

The problem

DialTimeout (documented as "the TCP dial timeout") only covered net.Dialer.DialContext. The binary-protocol handshake (handshake: a VERSION round trip that confirms the server speaks binary) ran on the caller's context, so a server that accepts TCP and never answers it held the dial, and any command that needed a new conn, until the caller's context ended: for ever under context.Background(). The text protocol has no handshake and is not affected.

The change

memcachedConn.handshake runs the VERSION round trip under context.WithTimeout(ctx, DialTimeout) (5 s when unset, defaultDialTimeout), exactly the shape of #1209 for redis. All three dial paths end in it (dialDirectMemcachedConn, dialMemcachedConn on unix, dialMemcachedConnNetConn), and the pool's dials use those, so Client commands and the cluster's node clients are bounded alike. Nothing else in driver/memcached runs a handshake (grep 'handshake(' driver/memcached: the three callers).

  • Which timeout is documented to bound it: Config.DialTimeout (WithDialTimeout, ClusterConfig.DialTimeout). Their docs now say it bounds the TCP dial and, separately, the binary handshake; a slow connect followed by a silent server takes up to about 2 x DialTimeout.
  • A caller's shorter deadline still wins and its error is returned unchanged; the driver's own bound returns celeris-memcached: handshake timed out after <d>: context deadline exceeded (on the direct path the socket's own i/o timeout is appended; the socket deadline fires a hair before the context timer, handled as in driver/redis: bound the connection handshake by DialTimeout (v1.6.0, Refs #1205) #1209).
  • Pool acquisition path: Client -> pool -> dial -> the three dials above; TestClientBoundsTheBinaryHandshake1213 goes through NewClient(... WithProtocol(ProtocolBinary), WithDialTimeout(500ms)) + Version(context.Background()).

Commits: 8f5b3c3 the tests (RED on main), 5c5fc6a the fix. handshake_default_1213_test.go (the test of the unset-DialTimeout default, which uses the new testHookDefaultHandshakeTimeout, as redis does) is in the second commit because it does not compile on main.

RED / GREEN

RED, 8f5b3c3 alone (macOS arm64, go test ./driver/memcached -run 1213 -v):

--- FAIL: TestBinaryHandshakeIsBoundedByDialTimeout1213 (direct, netconn, eventloop: "dial still blocked after 2s: the handshake is not bounded by DialTimeout")
--- FAIL: TestClientBoundsTheBinaryHandshake1213        ("Client.Version still blocked after 2s")
--- PASS: TestBinaryHandshakeKeepsTheCallersShorterDeadline1213  (guard, passes on main by design)
--- PASS: TestBinaryHandshakeKeepsTheCallersCancellation1213     (guard)
--- PASS: TestBoundDoesNotTouchTextOrAnAnsweredHandshake1213     (guard)

GREEN, 5c5fc6a: all pass; the three dials return after 500 ms (DialTimeout 500 ms), the caller-deadline ones after 200 ms, the default-timeout one after 300 ms (the hook lowers the 5 s default). Linux 2 CPU -race -count=20: the bounded dials returned after 500-510 ms.

direct:    celeris-memcached: handshake timed out after 500ms: context deadline exceeded: read tcp ...: i/o timeout
netconn:   celeris-memcached: handshake timed out after 500ms: context deadline exceeded
eventloop: celeris-memcached: handshake timed out after 500ms: context deadline exceeded

Each test takes under 1 s (the longest, 0.5 s of bound, runs its three paths in parallel; the "answered" guard sleeps 0.4 s past a 0.3 s DialTimeout to show the conn is still good).

Mutants (each applied alone to 5c5fc6a, go test -run 1213 -count=3)

mutant tests that fail
M1 handshake on the caller's context (no bound) TestBinaryHandshakeIsBoundedByDialTimeout, ...ByTheDefaultDialTimeout, TestClientBoundsTheBinaryHandshake
M2 the caller's earlier deadline relabelled as the driver's TestBinaryHandshakeKeepsTheCallersShorterDeadline
M3 no default bound when DialTimeout is unset TestBinaryHandshakeIsBoundedByTheDefaultDialTimeout
M4 the text protocol runs the binary handshake TestBoundDoesNotTouchTextOrAnAnsweredHandshake (the text dial panics in the binary encoder; the test binary crashes)
M5 the direct path's socket-deadline error not recognised as the bound probabilistic: the socket deadline and the context timer fire at the same instant, so whether the check sees hctx.Err() first is a race; it fails the bounded test (direct) in some runs only (macOS: about 1 in 10 at -count=10, survives -count=3). Same as redis M5 in #1209.
M6 the caller's cancellation relabelled as the driver's TestBinaryHandshakeKeepsTheCallersCancellation
M7 the timeout error does not match context.DeadlineExceeded TestBinaryHandshakeIsBoundedByDialTimeout

Checks

  • Whole driver/memcached package, -race -count=1, Linux docker golang:1.27.3 --cpus 2: ok. -race -run 1213 -count=20: Linux ok, macOS ok (44 s).
  • GOOS=windows go vet ./driver/memcached: clean. golangci-lint v2.15.0 GOOS=linux: nothing in the files of this PR (one errcheck finding in the untouched close_after_teardown_859_linux_test.go is on main too).
  • Not run locally: the windows-latest and macos-latest Portable fallback jobs. The tests use a 200-500 ms bound and a 2 s watchdog, no sleeps for correctness except the guard above, loopback only.

Behaviour notes

  • The text protocol is untouched.

Release note

DialTimeout (default 5 s) now also bounds the binary-protocol handshake of every new memcached conn: a server that accepts the connection and does not answer the VERSION handshake used to hold the dial, and any command that needed a new conn, until the caller's context ended (for ever under context.Background()). A server that takes longer than DialTimeout to answer now fails the dial with an error matching context.DeadlineExceeded, also when the caller's context has a longer deadline or none. Text-protocol clients are not affected.

Review r1 follow-up (commit f69b19c): the direct path, the one the pool uses

The pool always dials direct (pool.go). There a caller's cancellation was ignored until DialTimeout, and the caller then got a bare read tcp ...: i/o timeout that matched neither context.Canceled nor context.DeadlineExceeded. Same approach as #1216: for a direct conn handshake arms context.AfterFunc(hctx, ...) that sets a deadline of "now" on the socket, which wakes the blocked read; the error comes back wrapped with the caller's context error (context.Canceled, or context.DeadlineExceeded for a caller's deadline that fires on the socket a hair before the context reports it). A wake that raced with a handshake that had just succeeded is undone, so the conn is not left with a deadline in the past (a race-only guard; not reachable by a deterministic test). TestBinaryHandshakeKeepsTheCallersCancellation1213 now includes the direct path (RED on 5c5fc6a: "still blocked 2s after the cancel"), and ...KeepsTheCallersShorterDeadline1213 requires errors.Is(err, context.DeadlineExceeded) on all three paths (it failed on the direct path with the bare i/o timeout).

Mutants on f69b19c: no wake (AfterFunc replaced by a no-op) fails the cancellation (direct) and shorter-deadline (direct) tests; no wrapping of the caller's error fails the same two.

Checks: -race -run 1213 -count=20 Linux docker ok, macOS -count=30 ok (a first version raced with Close setting c.tcp = nil: the wake uses a copy of the socket); whole driver/memcached -race Linux ok; GOOS=windows go vet clean; golangci-lint only the pre-existing errcheck finding.

@FumingPower3925 FumingPower3925 added this to the v1.6.0 milestone Nov 1, 2026
@coderabbitai

coderabbitai Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

…ndshake at once and comes back as the caller's context error (Refs #1213)
@codecov

codecov Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.61905% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
driver/memcached/conn.go 97.61% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@FumingPower3925
FumingPower3925 marked this pull request as ready for review November 1, 2026 21:12
@FumingPower3925
FumingPower3925 merged commit c8e574d into main Nov 1, 2026
55 of 56 checks passed
@FumingPower3925
FumingPower3925 deleted the fix/memcached-handshake-dial-timeout-1213 branch November 1, 2026 21:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant