Add workflow to CLI to consume agent-proxy - #556
Closed
amoses12 wants to merge 29 commits into
Closed
Conversation
Remove initial envoy implementation
deny is the default so the operator, not the agent, picks where a credential goes
dopplerSource fetched once, so a rotated value would be served until restart
…ess-v2 Wire bindings, secret refresh, and allow-private-egress into proxy start
…check-v2 doctor: check the agent cannot read the credential source
`doppler-proxy.yaml` can now declare a non-static credential method per secret under a `methods: block` entry, static (default), oauth2_client_credentials, or aws_sigv4. This mirrors the existing bindings: block. A CredentialMethod DTO (snake_case yaml) maps to agentproxy.MethodConfig, threaded through Options -> agentproxy.Config alongside Binding. The starter config documents each method with a commented example. This wires the agent-proxy credential-method framework (OAuth2 client-credentials + AWS SigV4) so it's usable end to end. The operator declares token_url/client_id/scopes for OAuth2 or service/region/access_key_id for SigV4, and the agent still only holds the mask. Tests: the methods block parses (oauth2 + sigv4) and maps to the registry; the setting reaches the engine (control: dropping the wiring fails the test); binding/static behavior unchanged. Builds + tests locally against the sibling agent-proxy (replace => ../agent-proxy); CI goes green once agent-proxy is merged and go.mod is pinned. Release Note: (internal) Parse credential methods from doppler-proxy.yaml
The shared-box egress lock writes only iptables rules, so the agent's IPv6 traffic is unrestricted wherever the container has an IPv6 route — external routes and `::1` services (e.g. a local Postgres) alike. The doctor probed only IPv4 literals, so it reported a clean, contained result on a box whose IPv6 egress was wide open. Extract the clause-1 egress targets into egressProbeTargets and add two IPv6 probes: an external literal (2606:4700:4700::1111) and an IPv6 loopback service port ([::1]:5432). EgressBlockedTCP already dials "tcp", so bracketed IPv6 literals work with no change to the verify package. Test asserts the list covers both IP families and that every entry is an IP literal (a hostname would fail at resolution and falsely look contained) — wiring only, no dialing, so it can't go flaky; control: dropping the IPv6 entries fails it. The dial behavior itself is covered by EgressBlockedTCP's own tests. Buildable against the current agent-proxy verify API (local replace); the go.mod pin lands with the other agent-proxy follow-ups once that module is in main.
Sentry (a third-party, world-writable error sink) and statsig telemetry were blind-tunneled — unexamined holes that never reach the audit log. Both work fine intercepted, so drop them and document why each survivor stays. Removing a host from the list only causes it to be intercepted, never blocked, so the list should hold just the agent's control plane and the auth endpoints that break under interception.
Parse credential methods from doppler-proxy.yaml
doctor: probe IPv6 egress, not just IPv4 (ENG-9747)
agent-proxy dropped enforce.Config.CACertPath (it no longer installs a system-trust CA), so setting the field no longer compiles. The agent env's CA vars carry the trust the preflight doctor already verifies.
Adapts the CLI to two hardening changes in the agent-proxy `enforce`/`verify` packages. This PR makes it so agent-proxy's `enforce` no longer installs the proxy CA into the system trust store. That method trusted an arbitrary CA box-wide and let a hostile data dir have root copy a symlinked `ca.crt` into a world-readable path. The `Config.CACertPath` field is gone with it, so the CLI's `enforce.Config` literal no longer compiles until it stops settingit. Nothing is lost: the proxy already writes `NODE_EXTRA_CA_CERTS`, `CURL_CA_BUNDLE` and `SSL_CERT_FILE` (all → the data-dir `ca.crt`) into the agent env, so Node, curl and Go trust the proxy without a system-store install, and the preflight doctor still verifies that trust via `CATrustEnv` + `CACertValid(caPath)`. `caPath` stays for exactly that check. `TestPrivilegeChecksCoverBoundingSet` asserts the list covers both the effective and bounding sets (a check's `Name` is set regardless of platform/result, so it reads names without dialing or touching `/proc`. control: dropping the check fails it). The checks' own pass/fail logic is covered by agent-proxy's cap tests. ### CI note Red until agent-proxy lands in main and `go.mod` is pinned (the module currently builds against a local `replace => ../agent-proxy`). Tracked as ENG-9769; not part of this diff. Release Note: (internal) Drop system-trust CA in favor of enforce script
…orce Adapt CLI to hardened enforce and drop system-trust CA
Pare down the scaffolded passthrough list
An explicit secret-to-upstream mapping is where most of these vulnerabilities get closed, but an operator faced with a blank example rarely writes one. On first run the scaffolded config now documents the binding shape (host/paths/methods) and lists one commented stub per secret in the user's own Doppler config, so they edit real names instead of transcribing a generic example. Every stub stays commented, so a fresh scaffold still injects nothing until a host is filled in; with no names reachable it falls back to the provider example. The existing --proxy-config flag already lets the file live anywhere.
Pre-seed scaffolded bindings from secret names
names the secrets the agent has to hold for real, typically its own model provider token whose host is passed through
the agent's token now comes from the Doppler config via pass_by_value, so the CLI no longer needs to know any agent's variable names
Read pass_by_value from proxy config and drop agent auth forwarding
A demo distribution of the fork (the agent-proxy build) must not collide with a customer's production doppler CLI: it needs a distinct name on PATH and its own config dir so it can't read or clobber their real credentials. Rather than fork the code, add build-time-injectable identity in pkg/version (ProgramName, ConfigDirName, ConfigFileName, already the -ldflags -X home alongside ProgramVersion): the cobra root Use and the config path derive from these, and a renamed build (IsRenamed) drops the self-update command and the startup update check so it never overwrites itself with the official binary. The default build is unchanged (doppler / ~/.doppler); the demo build sets doppler-agent / ~/.doppler-agent via ldflags in the demo goreleaser config (next).
A separate config from the production .goreleaser.yml so the pilot-customer build can't disturb the real release. It builds a doppler-agent binary for the big four targets (darwin/linux x amd64/arm64), flips the identity with the pkg/version ldflags, ships GCS-only (no GitHub release, brew, Docker, or apt/rpm), and uploads archives + checksums to a placeholder bucket the release workflow authenticates to. Won't build in CI until the agent-proxy stacks land in main and go.mod is pinned (ENG-9769).
install-demo.sh is the one-line customer installer for the doppler-agent preview: detect OS/arch, fetch the matching archive from the demo GCS bucket, verify its sha256, and drop doppler-agent on PATH (never doppler). Bucket is a placeholder until infra provisions it.
Point the 8 import sites and go.mod at github.com/DopplerTest/agent-proxy — the module's new real home — and require v0.1.0 instead of the local replace => ../agent-proxy. This is what lets CI build the demo binary without a checkout of the private agent-proxy repo sitting next to the CLI. The agent-proxy transitive deps (aws-sdk-go-v2, smithy-go, x/net) are tidied into go.mod at their 1.25-safe versions. go.sum gets the v0.1.0 module hash once that tag is pushed; until then, local dev adds an uncommitted replace => ../agent-proxy.
release-demo.yml (workflow_dispatch) builds via .goreleaser.demo.yml and uploads the doppler-agent archives to GCS, plus a latest marker and the install script. It fetches the private agent-proxy module with a read token (GOPRIVATE + a git insteadOf rewrite) and reads the Go version from go.mod. GCP_KEY_DEMO, AGENT_PROXY_READ_TOKEN, and the bucket are placeholders until infra provisions them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a CI workflow to the CLI to consume the agent-proxy as a library and pull it from the DopplerTest organizaiton