Skip to content

Add Cloudflare Hyperdrive support for Python Workers - #28

Merged
CollierKing merged 1 commit into
mainfrom
feature/hyperdrive-support
Sep 19, 2026
Merged

CollierKing merged 1 commit into
mainfrom
feature/hyperdrive-support

Conversation

@CollierKing

Copy link
Copy Markdown
Owner

Summary

  • Adds create_engine_from_hyperdrive() and hyperdrive_connection() for using SQLAlchemy against PostgreSQL through a Cloudflare Hyperdrive binding inside a Python Worker
  • No custom dialect is involved: Hyperdrive speaks the PostgreSQL wire protocol, so SQLAlchemy's own postgresql+pg8000 does the work and this package supplies only the binding-to-engine glue
  • Engines use NullPool, since a Worker cannot reuse sockets across requests and Hyperdrive pools server-side
  • hyperdrive_connection() holds a per-event-loop asyncio.Lock, because the Workers socket layer does not tolerate concurrent synchronous driver I/O
  • Driver is auto-detected from the binding's scheme, falling back to connectionString, and can be overridden with driver=
  • Adds a [hyperdrive] extra and an examples/workers-hyperdrive/ example Worker
  • Adds 22 unit tests and 6 Worker integration tests; bumps to 0.4.0

Driver support

Cloudflare documents five drivers for Python Workers. Only two can back a SQLAlchemy engine, verified by running each from inside a Worker:

Driver Status Notes
pg8000 Supported PostgreSQL, covered by integration tests
pymysql Best effort MySQL, verified manually against MySQL 8, no automated coverage
psycopg Unsupported Needs libpq, absent from the Workers Pyodide build
asyncpg Unsupported Async-only
aiomysql Unsupported Async-only

Only synchronous SQLAlchemy works. asyncpg and aiomysql run fine in Workers on their own, but driving them from SQLAlchemy requires create_async_engine() and therefore greenlet, which Workers lack — so the two drivers Cloudflare recommends are precisely the two that cannot be used here.

Both supported drivers need TLS explicitly disabled, spelled differently for each, since Hyperdrive terminates TLS itself.

Notes for reviewers

  • Hyperdrive has no local emulation. wrangler dev always connects to the Postgres named by CLOUDFLARE_HYPERDRIVE_LOCAL_CONNECTION_STRING_HYPERDRIVE, and "remote": true is not valid on a hyperdrive binding. Integration tests skip unless that variable is set.
  • Behaviour was additionally confirmed against a real deployed Worker, which is the only way to exercise the actual Hyperdrive config rather than the local proxy.
  • The integration fixture creates its table before the run and drops it afterwards, since a Hyperdrive config often points at a shared database.
  • The example Worker's schema-changing endpoints are gated behind an ADMIN_TOKEN when one is set, because a deployed Worker is publicly reachable.
  • __version__ had drifted to 0.3.1 while the package shipped 0.3.11. The version is now derived from __version__ via hatchling, with a test asserting the two agree.

Testing

  • 43 unit tests pass on Python 3.9, 3.10, 3.11 and 3.12
  • 6 new Hyperdrive Worker integration tests pass against PostgreSQL 17
  • 66 existing D1 Worker integration tests still pass
  • ruff check, ruff format --diff, codespell, mypy src/, pre-commit --all-files and uv build all clean

@CollierKing
CollierKing force-pushed the feature/hyperdrive-support branch from 1b0b110 to 6a7f942 Compare September 19, 2026 04:06
Adds create_engine_from_hyperdrive() and hyperdrive_connection() for using
SQLAlchemy against a PostgreSQL database through a Hyperdrive binding inside a
Python Worker. No dialect is involved: Hyperdrive speaks the real wire
protocol, so SQLAlchemy's own postgresql+pg8000 does the work and this module
supplies only the binding-to-engine glue.

Engines use NullPool, since a Worker cannot reuse sockets across requests and
Hyperdrive pools server-side. hyperdrive_connection() holds a per-event-loop
asyncio lock, because the Workers socket layer does not tolerate concurrent
synchronous driver I/O. The driver is chosen from the binding's scheme and can
be overridden.

Of the five drivers Cloudflare documents, only pg8000 and pymysql can back a
SQLAlchemy engine. psycopg needs libpq, which the Workers Pyodide build lacks,
and asyncpg and aiomysql are async-only while SQLAlchemy's async engine
requires greenlet. Both usable drivers need TLS explicitly disabled, spelled
differently for each, since Hyperdrive terminates TLS itself.

PostgreSQL is the supported path and is covered by Worker integration tests.
MySQL is best effort: verified by hand against MySQL 8, but with no automated
coverage. The two drivers ship as separate extras so a PostgreSQL deployment
does not carry a MySQL driver, and a missing driver reports which extra
provides it.

Also derives the package version from __version__, which had drifted to 0.3.1
while the package shipped 0.3.11, and releases 0.4.0.
@CollierKing
CollierKing force-pushed the feature/hyperdrive-support branch from 6a7f942 to 0a7e005 Compare September 19, 2026 04:18
@CollierKing
CollierKing merged commit dc99af2 into main Sep 19, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant