Skip to content

Redact S3 credentials from logs - #10811

Draft
jerome079 wants to merge 3 commits into
apache:mainfrom
jerome079:fix/s3-credential-log-leak
Draft

jerome079 wants to merge 3 commits into
apache:mainfrom
jerome079:fix/s3-credential-log-leak

Conversation

@jerome079

Copy link
Copy Markdown

Description

This PR addresses a security issue where S3 credentials used for Secondary Storage were being logged in plain text in CloudStack logs (access.log and management-server.log). Even when debug logging is enabled, secret credentials such as accessKey and secretKey should never appear in logs.

Fix details:

  • Redacts the accessKey and secretKey from the S3TO object before logging DownloadCommand in NfsSecondaryStorageResource.java.
  • Adds a unit test in NfsSecondaryStorageResourceTest.java to verify that credentials are redacted.

Steps to reproduce the issue:

  1. Deploy CloudStack 4.20.0.0 with KVM and Ceph RGW S3 as Secondary Storage.
  2. Create a Secondary Storage using S3 credentials.
  3. Observe logs in /var/log/cloudstack/management/access.log or management-server.log — credentials will be printed.

Fixes: #10339

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • test (unit or integration test code)

Bug Severity

  • Major

How Has This Been Tested?

  • Added a unit test that mocks S3TO and verifies that setAccessKey("***REDACTED***") and setSecretKey("***REDACTED***") are called during executeRequest.

@boring-cyborg

boring-cyborg Bot commented May 4, 2025

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache CloudStack community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md)
Here are some useful points:

Comment on lines +288 to +293
DataStoreTO store = safeCmd.getDataStore();
if (store instanceof S3TO) {
((S3TO) store).setAccessKey("***REDACTED***");
((S3TO) store).setSecretKey("***REDACTED***");
}
logger.debug(LogUtils.logGsonWithoutException("Executing command %s [%s].", safeCmd.getClass().getSimpleName(), safeCmd));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code looks good, but it seems this should be in LogUtils. There is other obfuscation code also scattered across the code base, so definately not a 👎 but a mere suggestion.

@sureshanaparti sureshanaparti added this to the 4.20.2 milestone Jun 5, 2025
@DaanHoogland
DaanHoogland force-pushed the fix/s3-credential-log-leak branch from 0dedb70 to a43d34c Compare July 10, 2025 10:27
…udstack/storage/resource/NfsSecondaryStorageResourceTest.java
@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan LLpackage

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [LL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@codecov

codecov Bot commented Jul 10, 2025 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 3.88%. Comparing base (4d2e712) to head (3b23fb9).

❗ There is a different number of reports uploaded between BASE (4d2e712) and HEAD (3b23fb9). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (4d2e712) HEAD (3b23fb9)
unittests 1 0
Additional details and impacted files
@@              Coverage Diff              @@
##               main   #10811       +/-   ##
=============================================
- Coverage     20.76%    3.88%   -16.89%     
=============================================
  Files          6496      493     -6003     
  Lines        591612    42571   -549041     
  Branches      72585     8069    -64516     
=============================================
- Hits         122859     1653   -121206     
+ Misses       455416    40687   -414729     
+ Partials      13337      231    -13106     
Flag Coverage Δ
uitests 3.88% <ø> (ø)
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@blueorangutan

Copy link
Copy Markdown

Packaging result [LL]: ✖️ el7 ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 6219

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 14149

@DaanHoogland

Copy link
Copy Markdown
Contributor

@jerome079 can you have a look at this PR?

20:43:58 [ERROR] /jenkins/workspace/acs-centos8-pkg-builder/dist/rpmbuild/BUILD/cloudstack-4.21.0.0-SNAPSHOT/services/secondary-storage/server/src/main/java/org/apache/cloudstack/storage/resource/NfsSecondaryStorageResource.java:299: Line has trailing spaces. [RegexpSingleline]

@github-actions

Copy link
Copy Markdown

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

@rajujith

Copy link
Copy Markdown
Contributor

@jerome079 can you resolve the conflicts and retarget the PR to the 4.22 branch?

@DaanHoogland

Copy link
Copy Markdown
Contributor

@jerome079 can you resolve the conflicts and retarget the PR to the 4.22 branch?

@jerome079 @rajujith , I think if any retargeting for this one is done it should be for 4.20 (20.3 release)

@sureshanaparti

Copy link
Copy Markdown
Contributor

Hi @jerome079 is this PR ready for review? can you check the build errors, and outstanding comments. thanks.

@sureshanaparti sureshanaparti modified the milestones: 4.22.1, 4.23.0 Apr 24, 2026
@weizhouapache weizhouapache modified the milestones: 4.23.0, 4.24.0 Jun 29, 2026
@nvazquez

Copy link
Copy Markdown
Contributor

Hi @jerome079 can you please address the conflicts?

@DaanHoogland DaanHoogland moved this from Backlog to conflict/waiting for author in CloudStack Testing Aug 31, 2026
@nvazquez nvazquez modified the milestones: 24.0, 25.0 Oct 27, 2026
@DaanHoogland DaanHoogland modified the milestones: 25.0, 24.0 Nov 3, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: conflict/waiting

Development

Successfully merging this pull request may close these issues.

S3 credentials leak in log files

8 participants