Skip to content

fix(bigquery): scope SSLError non-retryable to streaming inserts - #18202

Open
ayam04 wants to merge 2 commits into
googleapis:mainfrom
ayam04:fix/bigquery-ssl-retry-scope
Open

fix(bigquery): scope SSLError non-retryable to streaming inserts#18202
ayam04 wants to merge 2 commits into
googleapis:mainfrom
ayam04:fix/bigquery-ssl-retry-scope

Conversation

@ayam04

@ayam04 ayam04 commented Aug 22, 2026

Copy link
Copy Markdown

Fixes #18178

Problem

#17489 made requests.exceptions.SSLError globally non-retryable by adding it to _UNSTRUCTURED_NON_RETRYABLE_TYPES, but _should_retry is the base predicate for every retry surface in the client. A single transient TLS reset (e.g. SSLEOFError/UNEXPECTED_EOF_WHILE_READING on a pooled-connection handshake) now fails jobs.get / result() polling outright, where 3.40.1 retried it as the ConnectionError subclass requests deliberately makes it. Production evidence in the issue: 49 polling failures across ~20 DAGs in ~19 h after upgrading to 3.42.2, zero on 3.40.1.

Change

  • retry.py: _UNSTRUCTURED_NON_RETRYABLE_TYPES no longer includes SSLError (empty tuple) — the base predicate retries transient transport resets again.
  • retry.py: new _should_retry_insert_rows + INSERT_ROWS_DEFAULT_RETRY keep the fix(bigquery): avoid SSLError retry loop #17489 carve-out scoped to the streaming-insert (insertAll) path, where an SSLError typically means the transport rejected a malformed payload (e.g. invalid schema) and would not resolve on retry.
  • client.py: insert_rows_json (and insert_rows, which delegates to it) now default to INSERT_ROWS_DEFAULT_RETRY.

Tests

  • test_retry.py: test_w_unstructured_requests_sslerror flipped to expect retryable; new Test_should_retry_insert_rows covers SSLError -> no retry, ConnectionError -> retry, rateLimitExceeded -> retry.
  • Full unit suite: 2444 passed, 247 skipped.

Validation

  • pytest packages/google-cloud-bigquery/tests/unit -q: 2444 passed, 247 skipped
  • black --check on changed files: clean

googleapis#17489 made SSLError globally non-retryable, but _should_retry is the
base predicate for every retry surface: a single transient TLS reset
(SSLEOFError) now fails jobs.get / result() polling outright, where
3.40.1 retried it as a ConnectionError subclass.

Remove SSLError from the global non-retryable set and keep the
googleapis#17489 carve-out only on the insertAll path via a scoped predicate
(INSERT_ROWS_DEFAULT_RETRY): malformed streaming payloads still fail
fast, while job polling keeps retrying transient transport resets.

Fixes googleapis#18178
@ayam04
ayam04 requested review from a team as code owners August 22, 2026 13:39
@ayam04
ayam04 requested review from sindhuvy and removed request for a team August 22, 2026 13:39
@google-cla

google-cla Bot commented Aug 22, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request restricts the SSLError retry carve-out to the streaming-insert path, restoring global retries for transient SSLErrors to prevent polling failures. The reviewer correctly identified that the standard insert_rows method must also be updated to use INSERT_ROWS_DEFAULT_RETRY as its default parameter to prevent bypassing the new retry behavior during delegation.

ignore_unknown_values: Optional[bool] = None,
template_suffix: Optional[str] = None,
retry: retries.Retry = DEFAULT_RETRY,
retry: retries.Retry = INSERT_ROWS_DEFAULT_RETRY,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The insert_rows method (defined elsewhere in this file) also has retry: retries.Retry = DEFAULT_RETRY as its default parameter value and delegates to insert_rows_json by passing retry=retry. Because of this, calling client.insert_rows(...) without a retry argument will explicitly pass DEFAULT_RETRY to insert_rows_json, bypassing the new INSERT_ROWS_DEFAULT_RETRY default. Please update the default value of the retry parameter in insert_rows to INSERT_ROWS_DEFAULT_RETRY as well to ensure the SSLError carve-out is correctly applied to the standard insert_rows path.

@ayam04

ayam04 commented Aug 22, 2026

Copy link
Copy Markdown
Author

Thanks for the review. I verified this against the code before responding (per AGENTS.md):

insert_rows (client.py:3808) has no explicit retry parameter — its signature is (self, table, rows, selected_fields=None, **kwargs), and it delegates via return self.insert_rows_json(table, json_rows, **kwargs) (client.py:3882). So when a caller invokes insert_rows without a retry argument, nothing is passed in kwargs, and insert_rows_json applies its new INSERT_ROWS_DEFAULT_RETRY default — the SSLError carve-out is correctly in effect on that path.

The only bypass would be a caller explicitly passing retry=DEFAULT_RETRY to either method, which is their deliberate choice to override the default (same as today with any retry customization).

I've added a unit test to lock this in: test_client_retry.py now asserts insert_rows_json uses the scoped predicate by default and insert_rows inherits it through delegation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bigquery: SSLError marked globally non-retryable (#17489) fails jobs.get / result() polling on transient TLS resets

1 participant