[AURON #2477] Add AES_GCM_CTR_V1 encryption support for Parquet - #2478
[AURON #2477] Add AES_GCM_CTR_V1 encryption support for Parquet#2478Jack1007 wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Adds support for reading/writing Parquet files encrypted with the AES_GCM_CTR_V1 algorithm by introducing an upstream arrow-rs patch that uses AES-GCM for metadata modules and AES-CTR for page data (via aws-lc-rs), enabling Auron to interoperate with CTR-encrypted Parquet tables (Issue #2477).
Changes:
- Extends Parquet modular encryption to support
AES_GCM_CTR_V1, separating page data vs metadata encryption/decryption paths. - Introduces AES-CTR block encryptor/decryptor implemented with
aws-lc-rsand wires it through file/page encryption plumbing. - Updates Parquet encryption tests to validate successful reads instead of asserting
NYI.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| +#[derive(Debug, Clone)] | ||
| +pub(crate) struct CtrBlockEncryptor { | ||
| + key: Arc<aws_lc_rs::cipher::EncryptingKey>, | ||
| + nonce_sequence: CounterNonce, | ||
| +} |
…e root Cargo.lock
| # Enable Parquet modular encryption support | ||
| -encryption = ["dep:ring"] | ||
| +encryption = ["dep:ring", "dep:aws-lc-rs"] |
Jack1007
changed the title
| } | ||
|
|
||
| + /// Use the AES_GCM_CTR_V1 algorithm: metadata modules use AES-GCM, page data uses AES-CTR. | ||
| + pub fn with_aes_gcm_ctr_v1(mut self) -> Self { |
There was a problem hiding this comment.
The new writer mode is not covered by any test. The updated tests only read the pre-generated CTR fixture, and nothing calls with_aes_gcm_ctr_v1(). Also, the root workspace excludes vendored arrow-rs, so cargo test --workspace does not execute these patched tests. Please add a full Parquet write/read round-trip using this option and ensure the targeted vendored Parquet tests run in CI.
Which issue does this PR close?
Closes #2477