feat: use rustls for RCSB access and ship RCSB in Linux wheels - #27
Merged
Merged
Conversation
Linux Python wheels were built without the rcsb feature because reqwest's default TLS backend (native-tls) needs OpenSSL, which failed to cross-compile in the manylinux containers (#8). - Build reqwest without default features and with rustls: trust the operating system's certificate store (rustls-native-certs) plus Mozilla's bundled roots (webpki-roots). reqwest's other default features (charset, http2, system-proxy) stay enabled. - OpenSSL/native-tls are no longer compiled for pdbrust's rcsb features or for the Python bindings; the extension links only against libc, libm and libgcc. - Build Linux release wheels, and the CI Python job, with the full feature set like macOS and Windows. - Python tests: every wheel must expose the RCSB API; download/search tests over HTTPS run with PDBRUST_NETWORK_TESTS=1. Verified locally: the 24 Rust RCSB network tests pass with rustls, as they do with native-tls, and all 30 Python tests pass including the network tests. Closes #8 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #27 +/- ##
=======================================
Coverage 90.83% 90.83%
=======================================
Files 60 60
Lines 12270 12270
=======================================
Hits 11145 11145
Misses 1125 1125 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
The manylinux2014 aarch64 cross-compiler cannot assemble ring's ARM code
("ARM assembler must define __ARM_ARCH"), which rustls needs. Build the
aarch64 wheels natively on GitHub's ubuntu-24.04-arm runner instead; the
wheel tag (manylinux2014) is unchanged. Also let the two Linux builds
finish independently (fail-fast: false).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e reverted)" The cross-platform check is done: the RCSB network tests passed over rustls on macOS and Windows (24/24) and in the Python tests on Linux. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #8.
Problem
Linux Python wheels were built with
--no-default-features --features core, without RCSB search and download. reqwest's default TLS backend (native-tls) needs OpenSSL, and OpenSSL failed to cross-compile in the manylinux containers (#8). macOS and Windows wheels had RCSB; Linux, the most common platform for scientific Python, did not.Change
Cargo.toml: reqwest is built without default features and with rustls:rustls-tls-native-rootstrusts the operating system's certificate store, as native-tls did, so custom or corporate CAs keep working;rustls-tls-webpki-rootsadds Mozilla's bundled roots, so minimal containers without system CA certificates also work;charset,http2,system-proxy.Cargo.lock: adds the rustls, webpki-roots and rustls-native-certs crates. Versions are resolved against the 1.85 MSRV.python-publish.yml):--features full, like macOS and Windows.ubuntu-24.04-armrunner. The manylinux2014 cross-compiler cannot assemble ring's ARM code (#error "ARM assembler must define __ARM_ARCH"), which is the blocker Enable RCSB feature for Linux Python wheels #8 described. The wheel tag is unchanged:manylinux_2_17_aarch64.manylinux2014_aarch64.fail-fast: false.rust.yml): builds with--features fullto match the release wheels.pdbrust-python/tests/test_rcsb.py:PDBRUST_NETWORK_TESTS=1so CI is not flaky.CHANGELOG.md: Added (Linux RCSB) and Changed (rustls).Verification
Dependencies (
cargo tree -e normal): OpenSSL, native-tls and quinn are not built forpdbrust --features rcsb,rcsb-asyncor forpdbrust-python --all-features. The TLS stack is rustls 0.23 + ring 0.17. OpenSSL remains inCargo.lockonly through the unusedcodecovdev-dependency.Cross-platform HTTPS: a temporary commit (
feefd4b, since reverted) ran the RCSB network tests in CI:The one Ubuntu failure is not TLS-related.
test_download_to_file_asyncread the downloaded file right afterdownload_to_file_asyncreturned and did not findATOM. That function writes with tokio'swrite_allbut never flushes, so the file can still be incomplete when it returns. The same test also failed once locally with native-tls. The bug is pre-existing and will be fixed in a separate PR.Release artifacts from this PR's CI:
Local, Linux, Rust 1.98.1:
cargo test: 174 passed (default features), 831 passed (all features).libc,libmandlibgcc.CI: all Rust jobs pass, including MSRV 1.85 on all three operating systems, and all wheel and sdist builds pass.
🤖 Generated with Claude Code