Skip to content

[worker] Mask secrets in every field of uploaded log records - #4698

Merged
brentvatne merged 2 commits into
mainfrom
@brent/worker-mask-err-logs
Oct 20, 2026
Merged

brentvatne merged 2 commits into
mainfrom
@brent/worker-mask-err-logs

Conversation

@brentvatne

@brentvatne brentvatne commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Why

The worker masks job secrets in uploaded logs, but only in each record's msg (createTransformStream in packages/worker/src/logger.ts). Every other field was uploaded as is, for example:

  • A logged error's err (message, stack, …). Unknown failures inside a build phase are logged as this.logger.error({ err }, '') (packages/build-tools/src/context.ts:310), so their cause, secrets included, is only in err. A comment there says the website displays err.stack for these records.
  • Other structured fields: { error } in reportMaestroTestResults.ts, { code, data } in restoreCache.ts, and the job dump at build.ts:48, which can contain a copy of a secret's value.

Found while making eas simulator:start / simulator:get show why a session failed (#4693, #4694). The CLI prints only the masked msg for this reason.

How

  • Which fields: mask every field of a record except bunyan's own fields and the ones that identify a log line (name, pid, level, time, v, phase, source, marker, result, buildStepId, buildStepInternalId).
  • Masking:
    • each value is round-tripped through JSON first, which is what the upload streams send, so a toJSON method can't bring back the original value, and a Date becomes its string form;
    • strings, object keys, and numbers equal to a secret are then masked;
    • values nested more than 20 levels deep become [Truncated] instead of being uploaded unmasked;
    • values that can't be serialized become [Unserializable].
  • The copy: the record is copied before masking. Bunyan passes the same record object to every raw stream, so the WorkerLogBuffer and the caller's objects are left alone.
  • No secrets: nothing changes when no secrets are registered.
  • Unchanged: context.ts:310. Filling msg for phase failures would change what the website shows, which is a separate decision.

Test Plan

  • New tests in src/__unit__/logger.test.ts. For every record they assert that JSON.stringify(record) contains no secret. Cases:
    • errors logged as error({ err }, '') and error({ err }) have err.message and err.stack masked;
    • another field with a secret as a value, as a key, and as a number is masked, while metadata fields are unchanged and the caller's object is not modified;
    • a toJSON method that returns a secret is masked;
    • a deeply nested secret becomes [Truncated].
  • Each fix is load-bearing: reverting any one (letting deep values through, skipping the JSON round-trip, or masking only err) fails one of these tests.
  • packages/worker unit tests: 85 pass.
  • yarn typecheck (worker), oxlint and oxfmt --check pass.

🤖 Generated with Claude Code

The upload transform masked only each record's msg, so a serialized err
(message, stack and other fields) was uploaded with secrets intact. That
includes unknown build phase failures, which log { err } with an empty msg.
Mask the strings inside err on a copy, since bunyan shares the record with
other streams.
@codecov

codecov Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.29412% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.30%. Comparing base (d0cb5a2) to head (bf44cd8).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
packages/worker/src/logger.ts 85.30% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4698      +/-   ##
==========================================
+ Coverage   75.25%   75.30%   +0.05%     
==========================================
  Files        1053     1053              
  Lines       52045    52174     +129     
  Branches    11357    11395      +38     
==========================================
+ Hits        39162    39283     +121     
- Misses      11895    11903       +8     
  Partials      988      988              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Mask all fields except bunyan's metadata, not only err: other fields such as
error, data and the job dump also carried secrets. Round-trip each value
through JSON first so a toJSON method can't bring back the original, mask keys
and numbers, and replace values nested too deeply instead of uploading them
unmasked.
@brentvatne brentvatne changed the title [worker] Mask secrets in logged errors [worker] Mask secrets in every field of uploaded log records Oct 20, 2026
@github-actions

Copy link
Copy Markdown

✅ Thank you for adding a changeset!

@brentvatne
brentvatne requested a review from gwdp October 20, 2026 04:27

@gwdp gwdp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥

@brentvatne
brentvatne merged commit c756c5d into main Oct 20, 2026
10 checks passed
@brentvatne
brentvatne deleted the @brent/worker-mask-err-logs branch October 20, 2026 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants