diff --git a/configuration/secrets.md b/configuration/secrets.md new file mode 100644 index 0000000..cc8669e --- /dev/null +++ b/configuration/secrets.md @@ -0,0 +1,210 @@ +# Secrets Providers + +Development projects often need a handful of secrets that should not sit in plaintext in the project's `.env` file: a `COMPOSER_AUTH` token, a licence key, or a sandbox API key. Warden can read those values from a secrets manager and inject them into the PHP containers on `warden env up`. + +Secrets injection is opt-in. Set `WARDEN_SECRETS` to a provider name, and that provider is asked for the values; when `WARDEN_SECRETS` is unset, nothing changes and the environment behaves exactly as before. + +## How it works + +```{mermaid} +sequenceDiagram + autonumber + participant Dev as Developer + participant Core as Warden core + participant CLI as Provider CLI (op / vault) + participant Compose as Docker Compose + participant PHP as PHP containers + + Dev->>Core: warden env up + Core->>CLI: read secrets + CLI-->>Core: KEY=VALUE lines + Core->>Core: filter reserved names + Core->>Compose: compose up with a names-only override,
values exported to this process only + Compose->>PHP: container environment + Note over Core: override removed on exit +``` + +- Injection and re-reading happen only on `warden env up`, and never for an environment of type `local`. `warden env start` and `warden env restart` reuse the existing containers, and a container keeps the environment it was created with, so the values are still there; they are just not read from the provider again. +- The `php-fpm` and `php-debug` containers always receive the variables. When `WARDEN_PHP_SPX`, `WARDEN_BLACKFIRE` or `WARDEN_MAGENTO2_GRAPHQL_SERVER` is enabled, the matching `php-spx`, `php-blackfire` or `php-graphql` container receives them too. +- `warden shell` inherits the variables from the running container, so they are available in a shell without extra setup. +- Every `warden env up` reads the secrets again, so a rotated value arrives on the next `warden env up`. +- The values are exported into the `docker compose` process, so they are also available for `${VAR}` interpolation in the compose files, including the project's `.warden/warden-env.yml`, on the first and every later `warden env up`. + +## Choosing a provider + +| Provider | `WARDEN_SECRETS` value | Required variables | Optional variables | CLI needed | +|----------|------------------------|--------------------|--------------------|------------| +| [1Password](https://developer.1password.com/docs/cli/) | `1password` | `WARDEN_OP_ENVIRONMENT_ID` or a `.env.op` file | `WARDEN_OP_ACCOUNT` | `op` (2.33.0-beta.02 or later for Environments) | +| [HashiCorp Vault](https://developer.hashicorp.com/vault) | `vault` | `WARDEN_VAULT_PATH` | `WARDEN_VAULT_MOUNT` (default `secret`), `WARDEN_VAULT_ADDR`, `WARDEN_VAULT_NAMESPACE` | `vault` and `jq` | + +## 1Password + +The 1Password provider has two modes. Pick one per project: a 1Password Environment, or secret references listed in a `.env.op` file. + +```{mermaid} +flowchart TD + B{"WARDEN_OP_ENVIRONMENT_ID set?"} -->|yes| C{".env.op present?"} + C -->|yes| D["warning: use one of them"] + C -->|no| E["Environments mode"] + B -->|no| F{".env.op present?"} + F -->|yes| G["Items mode"] + F -->|no| H["warning: set one"] +``` + +| Mode | Configured by | 1Password side | CLI call | Best for | +|------|---------------|----------------|----------|----------| +| Environments | `WARDEN_OP_ENVIRONMENT_ID` | A variable set managed in 1Password | `op environment read --no-masking` | A variable set a team manages in one place | +| Items | a `.env.op` file | Individual fields of existing vault items | `op read` | Picking specific fields from existing items, with the references reviewed in git | + +### Environments + +Add the following to the project's `.env` file: + +```bash +WARDEN_SECRETS=1password +WARDEN_OP_ENVIRONMENT_ID= +``` + +Values are injected as 1Password returns them. + +Warden does not read a 1Password credential from the project `.env`. The `op` CLI authenticates on its own, using the signed-in desktop app integration or a session token from `op signin`. On a host signed in to more than one 1Password account, set `WARDEN_OP_ACCOUNT` to the account to read from: + +```bash +WARDEN_OP_ACCOUNT= +``` + +1Password Environments require 1Password CLI 2.33.0-beta.02 or later. See the [1Password Environments documentation](https://developer.1password.com/docs/environments/) for creating an environment and adding variables to it. + +### Items (secret references) + +Add the following to the project's `.env` file: + +```bash +WARDEN_SECRETS=1password +``` + +and a `.env.op` file in the project root next to `.env` with the references: + +``` +COMPOSER_AUTH=op://project-x/composer/composer-auth +NEW_RELIC_KEY="op://project-x/New Relic/license key" +``` + +- The `.env.op` file holds references only, so it is meant to be committed. +- Each reference is resolved with `op read` on `warden env up`, and the field's content becomes the variable's value. See the [secret reference syntax](https://developer.1password.com/docs/cli/secret-reference-syntax/). +- If one reference cannot be read, the whole injection is skipped. +- Lines that are not `KEY=op://…` are ignored and reported by line number. +- Values with line breaks are skipped with a warning, so a multi-line field such as a private key cannot be injected this way. +- `WARDEN_OP_ACCOUNT` applies to every reference. +- `WARDEN_OP_ENVIRONMENT_ID` must be unset to use this mode. + +## HashiCorp Vault + +Add the following to the project's `.env` file: + +```bash +WARDEN_SECRETS=vault +WARDEN_VAULT_PATH=myproject/development +``` + +The remaining settings are optional: + +```bash +WARDEN_VAULT_MOUNT=secret +WARDEN_VAULT_ADDR=https://vault.example.com:8200 +WARDEN_VAULT_NAMESPACE=admin +``` + +- `WARDEN_VAULT_MOUNT` defaults to `secret`. +- `WARDEN_VAULT_ADDR` and `WARDEN_VAULT_NAMESPACE` default to whatever the Vault CLI is already configured with. +- The secret path is relative to the mount, may not contain `..`, and may not start with `/`. + +Warden does not read a Vault credential from the project `.env`. The `vault` CLI authenticates on its own, using the host's `VAULT_TOKEN` or the token helper configured after `vault login`. + +Both KV v1 and KV v2 secrets engines are supported. Warden reads the pairs from `.data.data` for KV v2 and from `.data` for KV v1. + +Values of type object, array or null, and strings that contain a line break, cannot be injected. They are skipped and the skipped names are listed in a warning. Keys that are not valid shell identifiers (letters, digits and underscores, not starting with a digit) are dropped silently, because they cannot be used as environment variable names. + +The `jq` command is required alongside the `vault` CLI. + +## Reserved variables + +Warden refuses to inject some names, so a secrets source cannot change how the stack is built. There are two groups. + +Managed by Warden: the variables Warden uses to build the stack: `CHOWN_DIR_LIST`, `COMPOSER_MEMORY_LIMIT`, `COMPOSER_VERSION`, `HISTFILE`, `NODE_VERSION`, `SSH_AUTH_SOCK`, `SSH_AUTH_SOCK_PATH_ENV`, `XDEBUG_VERSION`, plus every name under the `WARDEN_`, `TRAEFIK_` or `PHP_` prefixes. + +Would change the host process that runs `docker compose`: `PATH`, `HOME`, `USER`, `LOGNAME`, `SHELL`, `PWD`, `OLDPWD`, `IFS`, `TMPDIR`, `ENV`, `BASH_ENV`, `SHELLOPTS`, `BASHOPTS`, `PS4`, `CDPATH`, `GLOBIGNORE`, `TERM`, `LANG`, plus every name under the `DOCKER_`, `COMPOSE_`, `BUILDKIT_`, `BUILDX_`, `LD_`, `DYLD_`, `BASH_FUNC_`, `LC_`, `OP_`, `VAULT_` or `MUTAGEN_` prefixes. + +When a provider returns a reserved name, Warden warns `Variables reserved by Warden were not injected: ` and starts the environment without it. + +## Security notes + +:::{important} +- Warden writes no secret values to disk. The temporary compose override holds variable names only, and it is removed when `warden env up` exits. +- The values are exported only to the `docker compose` process that starts the environment. Once the containers run, the values live in the container environment and are visible to anyone who can run `docker inspect` on the host, exactly like any other compose `environment:` entry, so treat this as central storage and rotation of development secrets, not as container isolation. +- Names that would steer that `docker compose` process itself, such as `PATH` and the Docker client and loader variables, are refused outright. +- Provider output is parsed, never evaluated, so a value cannot execute a command on the host. +- The secrets manager credentials themselves never come from the project `.env`; the provider CLI authenticates on its own. +- Keep development secrets separate from production ones. +::: + +## Troubleshooting + +Every failure below is a warning. `warden env up` still starts the environment; it just starts without the secrets. + +| Warning | Cause | Fix | +|---------|-------|-----| +| `1Password CLI (op) could not be found; …` / `Vault CLI (vault) could not be found; …` / `jq could not be found; …` | The provider CLI, or `jq` for Vault, is not on the `PATH`. | Install the missing tool and re-run `warden env up`. | +| `Unknown secrets provider '' in WARDEN_SECRETS. Available providers: …; skipping secret injection.` | `WARDEN_SECRETS` names a provider that has no `utils/secrets/.sh` file. | Use one of the listed provider names. | +| `WARDEN_SECRETS must be a provider name (lowercase letters, digits, dashes). …` | `WARDEN_SECRETS` contains characters outside lowercase letters, digits and dashes. | Correct the value. | +| `WARDEN_OP_ENVIRONMENT_ID is not a valid 1Password Environment ID; …` / `WARDEN_VAULT_PATH is not a valid Vault secret path; …` | The ID or path fails the provider's validation. The same pattern applies to `WARDEN_VAULT_MOUNT`, `WARDEN_VAULT_NAMESPACE` and `WARDEN_VAULT_ADDR`. | Fix the value in the project `.env`. | +| `Both WARDEN_OP_ENVIRONMENT_ID and .env.op are configured; use one of them. Skipping secret injection.` | A 1Password Environment ID and a `.env.op` file are both present. | Keep one mode and remove the other. | +| `Set WARDEN_OP_ENVIRONMENT_ID or add a .env.op file with secret references; skipping secret injection.` | Neither 1Password mode is configured. | Set the environment ID, or add `.env.op`. | +| `Could not read 1Password Environment '' (Environments require 1Password CLI 2.33.0-beta.02 or later); …` / `Could not read 1Password secret reference for ; …` / `Could not read Vault secret '/'; …` | The provider CLI is not signed in, lacks permission, or cannot reach the service. | Sign in with the provider CLI, check access, then re-run. | +| `1Password returned masked values; skipping secret injection.` | The `op` CLI ignored `--no-masking` and returned concealed values, for example an older beta. | Update the 1Password CLI to 2.33.0-beta.02 or later. | +| `Ignored .env.op lines that are not KEY=op://… references: ` | `.env.op` contains lines that are not `KEY=op://…` references. | Fix or remove those lines; they are not injected. | +| `1Password values with line breaks cannot be injected and were skipped: ` | A referenced field's value contains a line break. | Store a single-line value in that field, or inject it another way. | +| `Variables reserved by Warden were not injected: ` | The provider returns one or more names Warden reserves, because they build the stack or steer the `docker compose` process. | Rename the variable in the secrets manager. | +| `Secrets provider '' returned no variables; …` / `Secrets provider '' defines only variables reserved by Warden; …` | The provider returned an empty result, or only reserved names. | Check the environment ID or secret path, and that it holds injectable variables. | + +## Writing a provider + +A provider is a shell file at `utils/secrets/.sh` that implements the two functions from the core contract: + +```{mermaid} +flowchart LR + A["core sources utils/secrets/PROVIDER.sh"] --> B["secretsProviderRequireConfig
validate configuration"] + B --> C["secretsProviderRead
fetch secrets"] + C --> D["KEY=VALUE lines on stdout"] + D --> E["core filters and injects"] +``` + +A minimal provider looks like this: + +```bash +#!/usr/bin/env bash +[[ ! ${WARDEN_DIR} ]] && >&2 echo -e "\033[31mThis script is not intended to be run directly!\033[0m" && exit 1 + +function secretsProviderRequireConfig { + if [[ ! "${WARDEN_EXAMPLE_PATH:-}" =~ ^[A-Za-z0-9_./-]+$ ]]; then + warning "WARDEN_EXAMPLE_PATH is not a valid path; skipping secret injection." + return 1 + fi +} + +function secretsProviderRead { + if ! command -v example >/dev/null 2>&1; then + warning "Example CLI (example) could not be found; skipping secret injection." + return 1 + fi + + example read "${WARDEN_EXAMPLE_PATH}" +} +``` + +Rules for a provider: + +- Prefix the provider's own settings with `WARDEN__` so they cannot collide with Warden's own variables. +- Never read the secrets manager credentials from the project `.env`; let the provider CLI authenticate on its own. +- On failure, print a warning that names the problem and return non-zero. The core then skips injection and the environment starts without secrets. +- Never print secret values to stderr or write them to disk; the only output that carries values is the `KEY=VALUE` lines on stdout. diff --git a/index.md b/index.md index a43ba4f..7ea5ee8 100644 --- a/index.md +++ b/index.md @@ -10,6 +10,7 @@ Under the hood `docker-compose` is used to control everything which Warden runs * Dnsmasq to serve DNS responses for `.test` domains eliminating manual editing of `/etc/hosts` * An SSH tunnel for connecting from Sequel Pro or TablePlus into any one of multiple running database containers. * Warden issued wildcard SSL certificates for running https on all local development domains. +* Optional injection of per-project secrets from a secrets manager such as 1Password or HashiCorp Vault. * Full support for Magento 1, Magento 2, Laravel, Symfony 4, Shopware 6 on both macOS and Linux. * Ability to override, extend, or setup completely custom environment definitions on a per-project basis.