fix(bigquery): scope SSLError non-retryable to streaming inserts - #18202
fix(bigquery): scope SSLError non-retryable to streaming inserts#18202ayam04 wants to merge 2 commits into
Conversation
googleapis#17489 made SSLError globally non-retryable, but _should_retry is the base predicate for every retry surface: a single transient TLS reset (SSLEOFError) now fails jobs.get / result() polling outright, where 3.40.1 retried it as a ConnectionError subclass. Remove SSLError from the global non-retryable set and keep the googleapis#17489 carve-out only on the insertAll path via a scoped predicate (INSERT_ROWS_DEFAULT_RETRY): malformed streaming payloads still fail fast, while job polling keeps retrying transient transport resets. Fixes googleapis#18178
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
There was a problem hiding this comment.
Code Review
This pull request restricts the SSLError retry carve-out to the streaming-insert path, restoring global retries for transient SSLErrors to prevent polling failures. The reviewer correctly identified that the standard insert_rows method must also be updated to use INSERT_ROWS_DEFAULT_RETRY as its default parameter to prevent bypassing the new retry behavior during delegation.
| ignore_unknown_values: Optional[bool] = None, | ||
| template_suffix: Optional[str] = None, | ||
| retry: retries.Retry = DEFAULT_RETRY, | ||
| retry: retries.Retry = INSERT_ROWS_DEFAULT_RETRY, |
There was a problem hiding this comment.
The insert_rows method (defined elsewhere in this file) also has retry: retries.Retry = DEFAULT_RETRY as its default parameter value and delegates to insert_rows_json by passing retry=retry. Because of this, calling client.insert_rows(...) without a retry argument will explicitly pass DEFAULT_RETRY to insert_rows_json, bypassing the new INSERT_ROWS_DEFAULT_RETRY default. Please update the default value of the retry parameter in insert_rows to INSERT_ROWS_DEFAULT_RETRY as well to ensure the SSLError carve-out is correctly applied to the standard insert_rows path.
|
Thanks for the review. I verified this against the code before responding (per AGENTS.md):
The only bypass would be a caller explicitly passing I've added a unit test to lock this in: |
Fixes #18178
Problem
#17489 made
requests.exceptions.SSLErrorglobally non-retryable by adding it to_UNSTRUCTURED_NON_RETRYABLE_TYPES, but_should_retryis the base predicate for every retry surface in the client. A single transient TLS reset (e.g.SSLEOFError/UNEXPECTED_EOF_WHILE_READING on a pooled-connection handshake) now failsjobs.get/result()polling outright, where 3.40.1 retried it as theConnectionErrorsubclass requests deliberately makes it. Production evidence in the issue: 49 polling failures across ~20 DAGs in ~19 h after upgrading to 3.42.2, zero on 3.40.1.Change
retry.py:_UNSTRUCTURED_NON_RETRYABLE_TYPESno longer includes SSLError (empty tuple) — the base predicate retries transient transport resets again.retry.py: new_should_retry_insert_rows+INSERT_ROWS_DEFAULT_RETRYkeep the fix(bigquery): avoid SSLError retry loop #17489 carve-out scoped to the streaming-insert (insertAll) path, where an SSLError typically means the transport rejected a malformed payload (e.g. invalid schema) and would not resolve on retry.client.py:insert_rows_json(andinsert_rows, which delegates to it) now default toINSERT_ROWS_DEFAULT_RETRY.Tests
test_retry.py:test_w_unstructured_requests_sslerrorflipped to expect retryable; newTest_should_retry_insert_rowscovers SSLError -> no retry, ConnectionError -> retry, rateLimitExceeded -> retry.Validation
pytest packages/google-cloud-bigquery/tests/unit -q: 2444 passed, 247 skippedblack --checkon changed files: clean