[shunping]

Descriptipon

Adds AwsSecret to retrieve secrets from AWS Secrets Manager via Secret.from_json(spec, 'AwsSecretsManager') or Secret.parse_secret_option('type:awssecret;name:...').

Changes

• AwsSecret: accepts secret_id/name/arn, version_id/version, version_stage/stage, plus any non-credential AwsOptions setting (e.g. region_name, endpoint_url) as a per-secret override.
• Secret.apply_pipeline_options(options): passed from from_json() / parse_secret_option() (and --gbek). AwsSecret uses it to pull credentials and defaults from AwsOptions lazily on workers.
• client_options: rejects AwsSecretsManager as --aws_credentials_secret_manager.

Part of #40528. Builds on #40547.

[codecov]

Codecov Report

❌ Patch coverage is 98.64865% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 79.90%. Comparing base (76f6e6d) to head (b329853).
⚠️ Report is 194 commits behind head on master.

Files with missing lines	Patch %	Lines
sdks/python/apache_beam/utils/secret.py	98.52%	1 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             master   #40558       +/-   ##
=============================================
+ Coverage     56.16%   79.90%   +23.74%     
=============================================
  Files          1124      488      -636     
  Lines        178336    82502    -95834     
  Branches       1489        0     -1489     
=============================================
- Hits         100170    65927    -34243     
+ Misses        75645    16575    -59070     
+ Partials       2521        0     -2521     

Flag	Coverage Δ
go	?
java	?
python	79.90% <98.64%> (+0.35%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Assigning reviewers:

R: @damccorm for label python.
Backup reviewer: claudevdm (if the main reviewer is unresponsive, you can escalate to them by tagging their username)

This pull request likely touches a core component ("core" label). Please review with scrutiny.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[github-actions]

📋 Review Summary

This pull request adds AwsSecret to Apache Beam's Python Secret utility to enable retrieving secrets from AWS Secrets Manager (AwsSecretsManager / type:awssecret). It also introduces Secret.apply_pipeline_options() for lazy resolution of AWS credentials and defaults from AwsOptions on workers, and prevents circular configuration where AwsSecretsManager is used as --aws_credentials_secret_manager.

🔍 General Feedback

• The design cleanly separates secret construction from pipeline option resolution (apply_pipeline_options), ensuring that secret objects remain picklable and resolve credentials lazily on workers.
• Per-secret connection overrides and validation against inline credentials integrate well with client_options.
• Unit test coverage is thorough, covering string/binary secret retrieval, caching, pickling, option overrides, and invalid configurations.

---

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

[damccorm]

Thanks! Generally LGTM - had 2 suggestions, but feel free to merge after addressing