Skip to content

feat: use rustls for RCSB access and ship RCSB in Linux wheels - #27

Merged
HFooladi merged 4 commits into
mainfrom
feature/8-rustls-linux-rcsb
Sep 16, 2026
Merged

HFooladi merged 4 commits into
mainfrom
feature/8-rustls-linux-rcsb

Conversation

@HFooladi

@HFooladi HFooladi commented Sep 16, 2026

Copy link
Copy Markdown
Owner

Closes #8.

Problem

Linux Python wheels were built with --no-default-features --features core, without RCSB search and download. reqwest's default TLS backend (native-tls) needs OpenSSL, and OpenSSL failed to cross-compile in the manylinux containers (#8). macOS and Windows wheels had RCSB; Linux, the most common platform for scientific Python, did not.

Change

  • Cargo.toml: reqwest is built without default features and with rustls:
    • rustls-tls-native-roots trusts the operating system's certificate store, as native-tls did, so custom or corporate CAs keep working;
    • rustls-tls-webpki-roots adds Mozilla's bundled roots, so minimal containers without system CA certificates also work;
    • reqwest's other defaults stay enabled: charset, http2, system-proxy.
  • Cargo.lock: adds the rustls, webpki-roots and rustls-native-certs crates. Versions are resolved against the 1.85 MSRV.
  • Release wheels (python-publish.yml):
    • Linux wheels are built with --features full, like macOS and Windows.
    • aarch64 wheels are built natively on GitHub's ubuntu-24.04-arm runner. The manylinux2014 cross-compiler cannot assemble ring's ARM code (#error "ARM assembler must define __ARM_ARCH"), which is the blocker Enable RCSB feature for Linux Python wheels #8 described. The wheel tag is unchanged: manylinux_2_17_aarch64.manylinux2014_aarch64.
    • The two Linux builds run with fail-fast: false.
  • CI Python job (rust.yml): builds with --features full to match the release wheels.
  • Tests: pdbrust-python/tests/test_rcsb.py:
    • every wheel must expose the full RCSB API (failed with the old Linux features);
    • download, download-to-file, search and bulk download over HTTPS, which run only with PDBRUST_NETWORK_TESTS=1 so CI is not flaky.
  • CHANGELOG.md: Added (Linux RCSB) and Changed (rustls).

Verification

Dependencies (cargo tree -e normal): OpenSSL, native-tls and quinn are not built for pdbrust --features rcsb,rcsb-async or for pdbrust-python --all-features. The TLS stack is rustls 0.23 + ring 0.17. OpenSSL remains in Cargo.lock only through the unused codecov dev-dependency.

Cross-platform HTTPS: a temporary commit (feefd4b, since reverted) ran the RCSB network tests in CI:

Platform Result
macOS (Rust) 24/24 pass
Windows (Rust) 24/24 pass
Linux, Python 3.10 and 3.14 30/30 Python tests pass, including the network tests
Ubuntu (Rust) 23/24

The one Ubuntu failure is not TLS-related. test_download_to_file_async read the downloaded file right after download_to_file_async returned and did not find ATOM. That function writes with tokio's write_all but never flushes, so the file can still be incomplete when it returns. The same test also failed once locally with native-tls. The bug is pre-existing and will be fixed in a separate PR.

Release artifacts from this PR's CI:

  • 25 wheels (cp310–cp314 × Linux x86_64/aarch64, macOS x86_64/arm64, Windows x64) plus the sdist; 0 stray files.
  • The actual Linux x86_64 release wheel, installed locally: all 5 RCSB tests pass, including 4 real HTTPS operations.
  • The aarch64 wheel contains the RCSB functions and does not link OpenSSL.

Local, Linux, Rust 1.98.1:

  • The 24 Rust RCSB network tests pass with rustls, as with native-tls.
  • cargo test: 174 passed (default features), 831 passed (all features).
  • Clippy, rustfmt and rustdoc are clean.
  • The full-features wheel, which previously failed on this machine because of OpenSSL linking, builds and links only against libc, libm and libgcc.

CI: all Rust jobs pass, including MSRV 1.85 on all three operating systems, and all wheel and sdist builds pass.

🤖 Generated with Claude Code

HFooladi and others added 2 commits September 16, 2026 14:01
Linux Python wheels were built without the rcsb feature because
reqwest's default TLS backend (native-tls) needs OpenSSL, which failed
to cross-compile in the manylinux containers (#8).

- Build reqwest without default features and with rustls: trust the
  operating system's certificate store (rustls-native-certs) plus
  Mozilla's bundled roots (webpki-roots). reqwest's other default
  features (charset, http2, system-proxy) stay enabled.
- OpenSSL/native-tls are no longer compiled for pdbrust's rcsb features
  or for the Python bindings; the extension links only against libc,
  libm and libgcc.
- Build Linux release wheels, and the CI Python job, with the full
  feature set like macOS and Windows.
- Python tests: every wheel must expose the RCSB API; download/search
  tests over HTTPS run with PDBRUST_NETWORK_TESTS=1.

Verified locally: the 24 Rust RCSB network tests pass with rustls, as
they do with native-tls, and all 30 Python tests pass including the
network tests.

Closes #8

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.83%. Comparing base (9f146c3) to head (19cdb21).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #27   +/-   ##
=======================================
  Coverage   90.83%   90.83%           
=======================================
  Files          60       60           
  Lines       12270    12270           
=======================================
  Hits        11145    11145           
  Misses       1125     1125           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

HFooladi and others added 2 commits September 16, 2026 14:09
The manylinux2014 aarch64 cross-compiler cannot assemble ring's ARM code
("ARM assembler must define __ARM_ARCH"), which rustls needs. Build the
aarch64 wheels natively on GitHub's ubuntu-24.04-arm runner instead; the
wheel tag (manylinux2014) is unchanged. Also let the two Linux builds
finish independently (fail-fast: false).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e reverted)"

The cross-platform check is done: the RCSB network tests passed over
rustls on macOS and Windows (24/24) and in the Python tests on Linux.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@HFooladi
HFooladi merged commit 24eef42 into main Sep 16, 2026
23 checks passed
@HFooladi
HFooladi deleted the feature/8-rustls-linux-rcsb branch September 16, 2026 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enable RCSB feature for Linux Python wheels

1 participant