feat(sbom): read build secrets in packages env values

[alexey-igrychev]

Summary

There is no way to pass a mounted build secret to a package manager: packages[].env values are literal text, and the packages stage is deliberately declarative, so a shell substitution is not an option. A value can now reference a secret declared in the secrets section, and the reference is resolved while the package manager runs.

secrets:
  - id: GOPROXY
    env: GOPROXY
  - id: CI_JOB_TOKEN
    env: CI_JOB_TOKEN
packages:
  - type: go-mod
    workdir: /app
    env:
      GOPROXY: "%secret:GOPROXY%"
      GOPRIVATE: git.example.com/*
      GIT_CONFIG_KEY_0: 'url.https://gitlab-ci-token:%secret:CI_JOB_TOKEN%@git.example.com/.insteadOf'
      GIT_CONFIG_VALUE_0: 'https://git.example.com/'
      GIT_CONFIG_COUNT: "1"

What

• BREAKING: a packages[].env value that literally contains %secret:<id>% or %secret_path:<id>% is now expanded instead of passed through. Any other %...% sequence — %2F, %H:%M, %image% — stays literal, so no other value changes.
• %secret:<id>% in a packages[].env value expands to the secret's contents, without trailing newlines, and may sit inside a larger value.
• %secret_path:<id>% expands to /run/secrets/<id>, for tools that take a path (DOCKER_CONFIG, AWS_SHARED_CREDENTIALS_FILE).
• A reference to a secret missing from the secrets section fails the build with packages[0].env["GOPROXY"] references secret "MISSING", which is not declared in the secrets section, and a malformed id with invalid secret id "my token" — both during configuration parsing, before any stage runs. VERIFIED: real build, see the comment.
• The generated command contains the mount path only, never the secret value, so the secret does not enter the stage digest, the build log or the image. VERIFIED: real build, see the comment.
• Deliberately not included: referencing another packages[].env key or a build-container variable. werf.yaml is rendered as a Go template, so composing ordinary values is already possible without a runtime mechanism.

Why

The declarative packages section is the only stapel stage with network access, which is exactly why it must not accept shell. Reading a secret is the one capability that a template cannot provide, because the value does not exist at render time — so it needs a syntax of its own rather than a hole in the escaping.

The %name% form is the placeholder syntax werf already has: %image%, %image_slug%, %image_safe_slug%, %image_content_based_tag% and %project% in --add-custom-tag, --use-custom-tag, werf export --tag and imageSpec labels. Only the two known prefixes expand and everything else stays literal, exactly as those placeholders behave, so no escape mechanism is needed and existing values keep working.

${...} and $(...) forms were rejected: they read as shell in a place where shell does not run, and a value that resembles a mounted path (/run/secrets/<id> treated as a reference) cannot express "pass this path as text", which %secret_path:<id>% does explicitly.

[alexey-igrychev]

Verification

• Real build, Docker backend on macOS, registry.werf.io/base/golang:1.12-alpine3.9 base, secrets: [{env: TOKEN}] and env: {GOPROXY: "%secret:TOKEN%"} with TOKEN=http://127.0.0.1:1/: go dialed the secret's value — Get http://127.0.0.1:1/example.com/missing/@v/v1.0.0.info: dial tcp 127.0.0.1:1: connect: connection refused.
• Real build with %secret:MISSING%: Error: unable to load werf config: packages[0].env["GOPROXY"] references secret "MISSING", which is not declared in the secrets section, before any stage ran.
• Mutation: dropped the validatePackageEnvValues call from toStapelImageBaseDirective → the undeclared secret and malformed secret id entries of validates packages env secret references against the secrets section failed; reverted, suite green. That table drives the real entry point (toStapelImageDirective), so it pins the call site, not just the function.
• Mutation: dropped the declared-id check inside validatePackageEnvValues → undeclared secret failed. Mutation: made %secret:...% expand to the path instead of the contents → the content entries and the runtime entries failed. Reverted, suite green.
• The runtime entries run the generated assignment through a real bash with the secret directory redirected to a temp dir, so they cover the resolution rather than the string shape.

Review focus

• formatPackageEnvValue falls back to quoting the raw value when parsing fails, because malformed references are already rejected by validatePackageEnvValues during config parsing. The mutation above is what keeps that assumption honest.
• A typo in the namespace (%secrets:TOKEN%) stays literal and reaches the package manager as text — the price of expanding only known prefixes. A typo in the id still fails, because the id has to be declared.

Follow-up

• Stacked on fix(sbom): make packages env variables reach the package manager #316, which is stacked on fix(sbom): stop executing shell constructs from packages env values #315; rebase onto main as they merge.
• Close feat(sbom): generilize package secret references #313: it gives the same capability with a literal /run/secrets/<id> value treated as a reference, and this PR supersedes it.