feat(encryption/kms): Add AWS KMS client - #3173
Conversation
There was a problem hiding this comment.
I don't know if this should be a whole new crate or just a feature flag, opendal backends we just feature flag WDYT?
There was a problem hiding this comment.
Yeah I'm not sure either, I considered both but for opendal the backends all share the same adapter while the KMS implementations wouldn't have the same. Java Iceberg also has separate modules for AWS, etc. Having them in separate crates also keeps SDK deps isolated. I can see three potential ways to organise this:
- Keep it in the core crate and use feature flags
- Have separate crates for iceberg-kms-aws, etc.
- Have one iceberg-kms crate with feature flags for AWS, etc.
I'm happy with any of these approaches honestly
There was a problem hiding this comment.
One KMS crate with KMS impls makes sense to me rather than a crate per provider. @blackmwk might have a preference here
It implements the same but the previous PR has been inactive for a few months now. This also addresses some of @xanderbailey's comments. |
| iceberg-property-macro = { workspace = true } | ||
|
|
||
| [dev-dependencies] | ||
| aws-sdk-kms = { workspace = true, features = ["test-util"] } |
There was a problem hiding this comment.
Security audit is failing because of h2 0.3.27 (RUSTSEC-2026-0258) introduced through aws-sdk-kms/test-util. It's a low-severity issue but still fails the checks.
We could do a temporary audit exception for it? Especially since this only affects tests.
|
@zakariya-s Thanks for working on it, I'll review your PR. @mbutrovich Thanks for the reminder! |
4 participants
Which issue does this PR close?
KeyManagementClientfor AWS KMS #2466.What changes are included in this PR?
Adds a new
iceberg-kms-awscrate implementing Iceberg client-side encryption key management with AWS KMS.The KMS client is created once from the application-supplied catalog properties and shared across the catalog, matching the existing
KmsClientFactorylifecycle.Are these changes tested?
Yes.
The AWS SDK mock framework was also used to test AWS calls.
AI Disclosure
LLMs were used to assist, but the code has been validated by hand.