Repository navigation
driver/postgres: bound the startup by ConnectTimeout and end it with the caller's context (v1.6.0, Refs #1212) - #1216
Merged
Conversation
… the caller's context (Refs #1212)
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
…get; arm the startup bound before the first write; reject a negative connect_timeout (Refs #1212)
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
FumingPower3925
marked this pull request as ready for review
November 1, 2026 19:51
This was referenced Nov 1, 2026
Merged
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
driver/postgres: bound the startup by ConnectTimeout and end it with the caller's context. Refs #1212 (found by the review of #1209; same class as #1205).
The problem
Options.ConnectTimeout(the DSN'sconnect_timeout) is documented as bounding "dial + startup", but it only reached thenet.Dialer.doStartupran on the caller's context. The issue says a server that accepts TCP and never completes the startup holdsOpen/acquire "until the caller's context ends". Measured on main, it is worse than that (RED run below): the caller's own context did not end it either.dialConnon unix,dialConnNetConn):waitsends no CancelRequest (the startup has no BackendKeyData yet) and then waits up to 30 s for a reply that is not coming;dialDirectConn, whichPooluses standalone and on async engines):driveDirectchecks the context only between reads, so a read blocked on the silent socket ignores it: never.The change
doStartupis now bounded by two things, whichever comes first: the caller's context (a shorter deadline or a cancellation ends it at once) andConnectTimeout. AstartupBoundfails the startup request and, in direct mode, sets a read deadline of "now" on the socket so the blocked read returns. The dial then closes the conn, as for any startup error. The timeout error isceleris-postgres: startup timed out (connect_timeout 200ms): context deadline exceeded(matcheserrors.Is(err, context.DeadlineExceeded)); the caller's own context error comes back as it is.connect_timeoutmeans): each dial path records its start (pgConn.connectDeadline), the startup ends at that deadline. Redis (driver/redis: bound the connection handshake by DialTimeout (v1.6.0, Refs #1205) #1209) and memcached (driver/memcached: the binary-mode handshake is not bounded by the dial timeout #1213) use two separate budgets (DialTimeouteach); postgres differs because its doc says "dial + startup". A slow connect leaves the startup less time.grep -rn "SSLRequest\|80877103" internal/driver/postgres driver/postgresfinds nothing;sslmode=require|verify-*is refused up front withErrSSLNotSupported,prefer|allowdowngrade to plaintext. So there is nothing to bound there.TestStartupBoundDoesNotOutliveASuccessfulStartup1212pings past the bound).Options.ConnectTimeoutnow says what it bounds, the error, and that zero means no client-side timeout.Unset
ConnectTimeout(decision for the lead)dsn.godocuments zero as "no client-side timeout" and no default is applied anywhere (libpq/pgx behave the same). I kept that. The consequence, to be explicit: withcontext.Background()and noconnect_timeout, a silent server still waits for ever. What changed for that case is that the caller's context now ends the startup at once (30 s / never before). Redis and memcached default to 5 s. If we want the same default here, it is a one-line change in the dial (connectDeadline), but it is a behaviour change for every postgres user, so I did not make it without a decision.The decision the issue asked for (bound vs correct the docs)
Bound it. The doc was right about the intent, the code was wrong; correcting the doc would have left the silent-server hang as designed behaviour, and the caller's-context bug (30 s / never) needed the same mechanism anyway.
Pool acquisition path
Pool.dialonly callsdialDirectConnordialConn; both are covered, andTestPoolAcquireIsBoundedByConnectTimeout1212goes throughOpen(...connect_timeout=1)+Ping(context.Background())(standalone, so direct mode). The engine-mode pool dials throughdialConn, which theeventloopsubtests cover. After the failed dial the pool holds no conn (Stats().Open == 0).Commits:
f6e6583the tests (RED on main),09b3a12the fix.startup_budget_1212_test.gois in the second commit because it references the newconnectDeadline(it does not compile on main).RED / GREEN
RED,
f6e6583alone (macOS arm64,go test ./driver/postgres -run 1212 -v; every failure is the watchdog message, not a test timeout):GREEN,
09b3a12: all pass. The bounded dials return after 200 ms (ConnectTimeout200 ms), the caller-context ones after 200 ms, the pool one after 1.00 s; the budget test after 150 ms against aConnectTimeoutof 10 s (the dial "used" 9.85 s). Linux 2 CPU-race -count=20: the dials returned after 150-210 ms (bound 200 ms) and 1.00-1.01 s (pool).Each test takes at most 1.0 s (the pool one); the other 1212 tests 0.2-0.3 s, parallel subtests.
Mutants (each applied alone to
09b3a12,go test -run 1212 -count=2)ConnectTimeouttimerdoneguard and timer not stopped (bound outlives a successful startup)connectDeadlinecontext.DeadlineExceededAn earlier version passed
context.WithoutCancel(ctx)towaitso it could not race the bound. Mutant "pass the caller's context" survived every test, and the interleaving analysis shows why: whichever of the abort anddriveDirect's ownctx.Err()branch runs first, the abort fails the request before it kicks the read, and the drain loop checks the request's flag before it reads. It was an equivalent mutant, so I removed the wrapper.Checks
driver/postgrespackage,-race -count=1, Linux dockergolang:1.27.3--cpus 2: ok (60 s).-race -run 1212 -count=20: Linux ok (61 s), macOS ok (39 s).GOOS=windows go vet ./driver/postgres: clean.golangci-lintv2.15.0GOOS=linux: nothing in the files of this PR (twoerrcheckfindings in untouched test files are on main too).dialConnNetConnover the portable loop, a 200 ms bound and a 2 s watchdog; loopback only, no reliance on clock resolution (the lower-bound checks are at half the bound).Release note
connect_timeout(Options.ConnectTimeout) now bounds the dial and the startup together, as its doc says: a server that accepts the TCP connection and does not complete the startup (authentication, ParameterStatus, ReadyForQuery) in that time fails the dial with an error matchingcontext.DeadlineExceeded, instead of holdingOpen/acquire. A server that is slow to authenticate (SCRAM with a high iteration count, a loaded server) now needs to answer within the whole budget, which before it did not: users who set a smallconnect_timeoutmay see dial errors where the startup used to take longer. The caller's context also ends a startup in progress at once (it took about 30 s on the event-loop paths and never in direct mode). With noconnect_timeoutthe dial is still unbounded apart from the caller's context.Review r1 follow-ups (commit
70248fb)beginDial, the one place all three dial paths start the clock).TestSlowConnectEatsTheConnectTimeoutBudget1212: connect 400 ms of a 500 msConnectTimeout, the dial ends after about 500 ms (not 900). Mutant "the clock starts after the connect" fails it on the three paths.doStartuparms the bound beforewriteRaw; the abort sets the socket's read and write deadline, so a blocked write ends too. A write error still goes throughfailReqandfinish.resolveAddrruns before the clock and without the context; documented in theConnectTimeoutdoc rather than moved (it needs a context-aware resolver and a change ofpgConn.addr).connect_timeout(NIT, pre-existing): rejected at DSN parse withconnect_timeout = -1: must be >= 0 (0 or absent: no timeout)(key=value and URL forms;TestParseDSNRejectsANegativeConnectTimeout1212, mutant "no rejection" fails it).70248fb:-race -run '1212|ParseDSN' -count=20Linux docker--cpus 2ok (91 s), macOS ok (69 s); wholedriver/postgres-raceLinux ok (60 s);GOOS=windows go vetclean; golangci-lint (own cache) only the two pre-existingerrcheckfindings.