Skip to content

gh-160699: Prevent concurrent reuse of cached SQLite statements - #160706

Open
mangod12 wants to merge 4 commits into
python:mainfrom
mangod12:issue-160699-sqlite-threading
Open

mangod12 wants to merge 4 commits into
python:mainfrom
mangod12:issue-160699-sqlite-threading

Conversation

@mangod12

Copy link
Copy Markdown

Summary

Concurrent calls to execute() on a shared SQLite connection can obtain the same cached sqlite3_stmt before SQLite marks it busy. The cursor releases the GIL around SQLite operations, leaving a race where concurrent cursors can reset or use the same statement and crash the process.

Reserve cached statements before releasing the GIL. If a cached statement is already reserved, create an uncached statement for the concurrent execution. Release the reservation when the statement is reset.

Add a regression test that runs concurrent parameterized queries on a shared connection and verifies the returned BLOB contents.

Fixes gh-160699.

Verification

  • Windows debug build succeeded.
  • PCbuild/amd64/python_d.exe -m unittest test.test_sqlite3.test_dbapi: 211 passed, 2 skipped.
  • Tools/patchcheck/patchcheck.py: passed; NEWS fragment recognized.

@python-cla-bot

python-cla-bot Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

@bedevere-app

bedevere-app Bot commented Oct 26, 2026

Copy link
Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@mangod12

Copy link
Copy Markdown
Author

CI for PR #160706 fails on all three macOS jobs in the new test_concurrent_queries_on_shared_connection: it segfaults inside sqlite3_prepare_v2 while threads share the connection. I pushed cf13208 to explicitly request SQLITE_OPEN_FULLMUTEX whenever the linked SQLite build has mutex support, while keeping the statement reservation fix. SQLite documents that flag as selecting serialized mode and notes sqlite3_threadsafe() alone does not report runtime mode: https://sqlite.org/threadsafe.html

The new CI run will tell us whether that addresses the macOS failure. I could not build CPython locally here because the Windows C toolchain is unavailable.

@mangod12

Copy link
Copy Markdown
Author

@erlend-aasland The latest CI run for 76eb2d3 failed on macOS Intel with a hosted-runner communication error; GitHub’s annotation reports that the runner lost communication with the server, and the detailed job log is unavailable. Could you rerun that job when convenient? The other macOS jobs passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sqlite3: concurrent execute() on a shared Connection crashes the process with a native access violation (threadsafety == 3)

1 participant