diff --git a/README.md b/README.md index 9feb6ad..d27550c 100644 --- a/README.md +++ b/README.md @@ -226,37 +226,43 @@ GitHub environment variables. The exact Brio inventory is: | Canonical Proton Pass item | Protected GitHub environment | Exact mirrored fields | | --- | --- | --- | -| `Hetzner Database Server makepad` | `canary`, `production`, `staging-brio-identity-db`, and `keycloak-cohort-restore` | canonical SSH fields `DEPLOY_SSH_HOST`, `DEPLOY_SSH_PORT`, `DEPLOY_SSH_USER`, `DEPLOY_SSH_PRIVATE_KEY`, `DEPLOY_SSH_KNOWN_HOSTS`; mirror the same reviewed values under the workflow aliases `BRIO_IDENTITY_DB_DEPLOY_SSH_HOST`, `BRIO_IDENTITY_DB_DEPLOY_SSH_PORT`, `BRIO_IDENTITY_DB_DEPLOY_SSH_USER`, `BRIO_IDENTITY_DB_DEPLOY_SSH_PRIVATE_KEY`, `BRIO_IDENTITY_DB_DEPLOY_SSH_KNOWN_HOSTS`, `KEYCLOAK_COHORT_DB_SSH_HOST`, `KEYCLOAK_COHORT_DB_SSH_PORT`, `KEYCLOAK_COHORT_DB_SSH_USER`, `KEYCLOAK_COHORT_DB_SSH_PRIVATE_KEY`, and `KEYCLOAK_COHORT_DB_SSH_KNOWN_HOSTS` only in their named environments | +| `Hetzner App Server makepad` | `canary` and `production` | native fields `host`, `port`, `user`, `private_key`, and `known_hosts` map to the existing shared `DEPLOY_SSH_*` destinations | +| `Hetzner Database Server makepad` | `staging-brio-identity-db` and `keycloak-cohort-restore` | canonical fields `DEPLOY_SSH_HOST`, `DEPLOY_SSH_PORT`, `DEPLOY_SSH_USER`, `DEPLOY_SSH_PRIVATE_KEY`, and `DEPLOY_SSH_KNOWN_HOSTS` map only to the `BRIO_IDENTITY_DB_DEPLOY_SSH_*` and `KEYCLOAK_COHORT_DB_SSH_*` aliases in their named environments | | `Brio Staging - PostgreSQL` | `canary` and `staging-brio-identity-db` | secrets `POSTGRES_CANARY_SUPERUSER_PASSWORD`, `BRIO_STAGING_DB_PASSWORD`, `BRIO_STAGING_BACKUP_DB_PASSWORD`, `KEYCLOAK_BRIO_STAGING_DB_PASSWORD`, and `KEYCLOAK_BRIO_STAGING_BACKUP_DB_PASSWORD` | | `Brio Staging - PKI and Backup Keys` | `canary` and `staging-brio-identity-db` | secrets `POSTGRES_CA_PEM`, `POSTGRES_SERVER_CERT_PEM`, `POSTGRES_SERVER_KEY_PEM`, and public recipient certificate `BRIO_BACKUP_RECIPIENT_CERT_PEM` | | `PostgreSQL · Brio identity release orchestrator` | `release-brio-identity-db` | secret `KEYCLOAK_RELEASE_ORCHESTRATOR_TOKEN` | | `PostgreSQL · Keycloak cohort source reader` | `keycloak-cohort-restore` | secret `KEYCLOAK_COHORT_SOURCE_TOKEN` | | `Makepad Docker Hardened Images` | `keycloak-cohort-restore` | canonical fields `DOCKERHUB_USERNAME` and `DOCKERHUB_PRO_PAT`, mirrored as secrets `DHI_REGISTRY_USERNAME` and `DHI_REGISTRY_PASSWORD` | -The `canary`, `production`, `staging-brio-identity-db`, and -`keycloak-cohort-restore` environments also hold reviewed non-secret constants -such as `DEPLOY_REMOTE_DIR`, stack/network names, -`BRIO_IDENTITY_DB_HOSTNAME`, and `BRIO_KEYCLOAK_DB_SOURCE_CIDR`. Store them as -environment variables, not duplicated password-vault secrets. +The shared workflows retain their existing GitHub classification for remote +directory and stack/network destinations. Only the two public standalone-host +inputs `BRIO_IDENTITY_DB_HOSTNAME` and `BRIO_KEYCLOAK_DB_SOURCE_CIDR` are +environment variables; the other reviewed destinations are environment +secrets because their workflows consume `secrets.*`. -Use `pass-cli` from an approved administrator workstation and stream secret -values over standard input: +Audit the reviewed policy and destination names without materializing fields: ```bash -pass-cli item view --item-title '' --field '' \ - | gh secret set '' --env '' --repo 'Makepad-fr/postgres' +./scripts/sync-github-environments.sh --check ``` -Never place values in command arguments, temporary files, shell history, -Actions logs, or issue text. Mirror non-secret variables with the same reviewed -reconciliation session, compare their GitHub read-back, and record only item -IDs, field names, timestamps, and non-secret fingerprints in the deployment -change record. Every listed environment, including `production`, must have +After reviewing that result and receiving action-time approval, sync exactly +one environment with the matching confirmation string: + +```bash +./scripts/sync-github-environments.sh --sync \ + --environment staging-brio-identity-db \ + --confirm Makepad-fr/postgres:staging-brio-identity-db +``` + +The helper streams values over standard input and never changes repository or +environment policy. Every listed environment, including `production`, has exactly one custom branch deployment policy whose type is `branch` and whose name is exactly `main`; GitHub's generic "protected branches" option is not an equivalent restriction. A release is blocked if an item or field is missing, -if that exact policy or required reviewers are absent, or if GitHub differs -from the reviewed Proton version. +if the pinned policy identity or required reviewers drift, or if GitHub differs +from the reviewed Proton version. See `docs/credential-sync.md` for the exact +read/write boundary and adversarial validation. Audit all five policies without changing provider state: diff --git a/deploy/credential-inventory.json b/deploy/credential-inventory.json new file mode 100644 index 0000000..c0ae1b9 --- /dev/null +++ b/deploy/credential-inventory.json @@ -0,0 +1,128 @@ +{ + "schemaVersion": 3, + "repository": "Makepad-fr/postgres", + "vault": "Makepad", + "repositoryPolicy": { + "repositoryId": 1200300784, + "private": false, + "visibility": "public", + "allowForking": true, + "defaultBranch": "main", + "requiredChecks": ["policy-and-integration"], + "requiredCheckAppId": 15368, + "mainProtection": { + "strictStatusChecks": true, + "enforceAdmins": true, + "dismissStaleReviews": true, + "requireCodeOwnerReviews": true, + "requiredApprovingReviewCount": 1, + "requireLastPushApproval": true, + "requiredSignatures": true, + "requiredLinearHistory": true, + "requiredConversationResolution": true, + "allowForcePushes": false, + "allowDeletions": false, + "blockCreations": false, + "lockBranch": false, + "allowForkSyncing": false + }, + "actionsPolicy": { + "defaultWorkflowPermissions": "read", + "canApprovePullRequestReviews": false + }, + "environments": { + "canary": {"id": 21262761188, "branchPolicyId": 59128365, "reviewerId": null, "reviewerLogin": null}, + "production": {"id": 15050761884, "branchPolicyId": 59156955, "reviewerId": null, "reviewerLogin": null}, + "staging-brio-identity-db": {"id": 21278291993, "branchPolicyId": 59143916, "reviewerId": 39597780, "reviewerLogin": "idilsaglam"}, + "release-brio-identity-db": {"id": 21284627193, "branchPolicyId": 59149936, "reviewerId": 39597780, "reviewerLogin": "idilsaglam"}, + "keycloak-cohort-restore": {"id": 21284627918, "branchPolicyId": 59149937, "reviewerId": 39597780, "reviewerLogin": "idilsaglam"} + } + }, + "operatorEntries": [], + "preservedDestinations": { + "canary": { + "secret": [ + "KEYCLOAK_BRIO_STAGING_BACKUP_DB_PASSWORD", + "KEYCLOAK_BRIO_STAGING_DB_PASSWORD" + ], + "variable": [] + }, + "production": { + "secret": [ + "DEPLOY_FASHION_DB_NAME", + "DEPLOY_FASHION_DB_NETWORK", + "DEPLOY_FASHION_DB_PASSWORD", + "DEPLOY_FASHION_DB_USER", + "DEPLOY_SCRAPING_DB_NAME", + "DEPLOY_SCRAPING_DB_NETWORK", + "DEPLOY_SCRAPING_DB_PASSWORD", + "DEPLOY_SCRAPING_DB_USER" + ], + "variable": [] + }, + "staging-brio-identity-db": { + "secret": [ + "BRIO_STAGING_BACKUP_DB_PASSWORD", + "BRIO_STAGING_DB_PASSWORD" + ], + "variable": ["POSTGRES_HOST_COMPOSE_PROJECT"] + }, + "release-brio-identity-db": {"secret": [], "variable": []}, + "keycloak-cohort-restore": {"secret": [], "variable": []} + }, + "entries": [ + {"environment":"canary","kind":"secret","requirement":"required","destination":"DEPLOY_SSH_HOST","item":"Hetzner App Server makepad","field":"host"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"DEPLOY_SSH_PORT","item":"Hetzner App Server makepad","field":"port"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"DEPLOY_SSH_USER","item":"Hetzner App Server makepad","field":"user"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"DEPLOY_SSH_PRIVATE_KEY","item":"Hetzner App Server makepad","field":"private_key"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"DEPLOY_SSH_KNOWN_HOSTS","item":"Hetzner App Server makepad","field":"known_hosts"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"DEPLOY_REMOTE_DIR","item":"PostgreSQL · shared Swarm deployment","field":"DEPLOY_REMOTE_DIR"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"DEPLOY_STACK_NAME","item":"PostgreSQL · shared Swarm deployment","field":"DEPLOY_STACK_NAME"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"DEPLOY_CATWLK_DB_NETWORK","item":"PostgreSQL · shared Swarm deployment","field":"DEPLOY_CATWLK_DB_NETWORK"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"DEPLOY_LE_PETIT_COIN_DB_NETWORK","item":"Le Petit Coin GitHub Deploy Secrets","field":"DEPLOY_DB_NETWORK"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"DEPLOY_BRIO_STAGING_DB_NETWORK","item":"Brio Staging - PostgreSQL","field":"DEPLOY_BRIO_STAGING_DB_NETWORK"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"POSTGRES_CANARY_SUPERUSER_PASSWORD","item":"Brio Staging - PostgreSQL","field":"POSTGRES_CANARY_SUPERUSER_PASSWORD"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"BRIO_STAGING_DB_PASSWORD","item":"Brio Staging - PostgreSQL","field":"BRIO_STAGING_DB_PASSWORD"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"BRIO_STAGING_BACKUP_DB_PASSWORD","item":"Brio Staging - PostgreSQL","field":"BRIO_STAGING_BACKUP_DB_PASSWORD"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"POSTGRES_CA_PEM","item":"Brio Staging - PKI and Backup Keys","field":"POSTGRES_CA_PEM"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"POSTGRES_SERVER_CERT_PEM","item":"Brio Staging - PKI and Backup Keys","field":"POSTGRES_SERVER_CERT_PEM"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"POSTGRES_SERVER_KEY_PEM","item":"Brio Staging - PKI and Backup Keys","field":"POSTGRES_SERVER_KEY_PEM"}, + {"environment":"canary","kind":"secret","requirement":"required","destination":"BRIO_BACKUP_RECIPIENT_CERT_PEM","item":"Brio Staging - PKI and Backup Keys","field":"BRIO_BACKUP_RECIPIENT_CERT_PEM"}, + + {"environment":"production","kind":"secret","requirement":"required","destination":"DEPLOY_SSH_HOST","item":"Hetzner App Server makepad","field":"host"}, + {"environment":"production","kind":"secret","requirement":"required","destination":"DEPLOY_SSH_PORT","item":"Hetzner App Server makepad","field":"port"}, + {"environment":"production","kind":"secret","requirement":"required","destination":"DEPLOY_SSH_USER","item":"Hetzner App Server makepad","field":"user"}, + {"environment":"production","kind":"secret","requirement":"required","destination":"DEPLOY_SSH_PRIVATE_KEY","item":"Hetzner App Server makepad","field":"private_key"}, + {"environment":"production","kind":"secret","requirement":"required","destination":"DEPLOY_SSH_KNOWN_HOSTS","item":"Hetzner App Server makepad","field":"known_hosts"}, + {"environment":"production","kind":"secret","requirement":"required","destination":"DEPLOY_REMOTE_DIR","item":"PostgreSQL · shared Swarm deployment","field":"DEPLOY_REMOTE_DIR"}, + {"environment":"production","kind":"secret","requirement":"required","destination":"DEPLOY_STACK_NAME","item":"PostgreSQL · shared Swarm deployment","field":"DEPLOY_STACK_NAME"}, + {"environment":"production","kind":"secret","requirement":"required","destination":"DEPLOY_CATWLK_DB_NETWORK","item":"PostgreSQL · shared Swarm deployment","field":"DEPLOY_CATWLK_DB_NETWORK"}, + {"environment":"production","kind":"secret","requirement":"required","destination":"DEPLOY_LE_PETIT_COIN_DB_NETWORK","item":"Le Petit Coin GitHub Deploy Secrets","field":"DEPLOY_DB_NETWORK"}, + {"environment":"production","kind":"secret","requirement":"required","destination":"DEPLOY_VIF_DB_NETWORK","item":"PostgreSQL · shared Swarm deployment","field":"DEPLOY_VIF_DB_NETWORK"}, + {"environment":"production","kind":"secret","requirement":"required","destination":"DEPLOY_VIF_DB_NAME","item":"PostgreSQL · shared Swarm deployment","field":"DEPLOY_VIF_DB_NAME"}, + {"environment":"production","kind":"secret","requirement":"required","destination":"DEPLOY_VIF_DB_USER","item":"PostgreSQL · shared Swarm deployment","field":"DEPLOY_VIF_DB_USER"}, + {"environment":"production","kind":"secret","requirement":"required","destination":"DEPLOY_VIF_DB_PASSWORD","item":"PostgreSQL · shared Swarm deployment","field":"DEPLOY_VIF_DB_PASSWORD"}, + + {"environment":"staging-brio-identity-db","kind":"secret","requirement":"required","destination":"BRIO_IDENTITY_DB_DEPLOY_SSH_HOST","item":"Hetzner Database Server makepad","field":"DEPLOY_SSH_HOST"}, + {"environment":"staging-brio-identity-db","kind":"secret","requirement":"required","destination":"BRIO_IDENTITY_DB_DEPLOY_SSH_PORT","item":"Hetzner Database Server makepad","field":"DEPLOY_SSH_PORT"}, + {"environment":"staging-brio-identity-db","kind":"secret","requirement":"required","destination":"BRIO_IDENTITY_DB_DEPLOY_SSH_USER","item":"Hetzner Database Server makepad","field":"DEPLOY_SSH_USER"}, + {"environment":"staging-brio-identity-db","kind":"secret","requirement":"required","destination":"BRIO_IDENTITY_DB_DEPLOY_SSH_PRIVATE_KEY","item":"Hetzner Database Server makepad","field":"DEPLOY_SSH_PRIVATE_KEY"}, + {"environment":"staging-brio-identity-db","kind":"secret","requirement":"required","destination":"BRIO_IDENTITY_DB_DEPLOY_SSH_KNOWN_HOSTS","item":"Hetzner Database Server makepad","field":"DEPLOY_SSH_KNOWN_HOSTS"}, + {"environment":"staging-brio-identity-db","kind":"secret","requirement":"required","destination":"KEYCLOAK_BRIO_STAGING_DB_PASSWORD","item":"Brio Staging - PostgreSQL","field":"KEYCLOAK_BRIO_STAGING_DB_PASSWORD"}, + {"environment":"staging-brio-identity-db","kind":"secret","requirement":"required","destination":"KEYCLOAK_BRIO_STAGING_BACKUP_DB_PASSWORD","item":"Brio Staging - PostgreSQL","field":"KEYCLOAK_BRIO_STAGING_BACKUP_DB_PASSWORD"}, + {"environment":"staging-brio-identity-db","kind":"secret","requirement":"required","destination":"BRIO_BACKUP_RECIPIENT_CERT_PEM","item":"Brio Staging - PKI and Backup Keys","field":"BRIO_BACKUP_RECIPIENT_CERT_PEM"}, + {"environment":"staging-brio-identity-db","kind":"variable","requirement":"required","destination":"BRIO_IDENTITY_DB_HOSTNAME","item":"Brio Staging - PKI and Backup Keys","field":"BRIO_IDENTITY_DB_HOSTNAME"}, + {"environment":"staging-brio-identity-db","kind":"variable","requirement":"required","destination":"BRIO_KEYCLOAK_DB_SOURCE_CIDR","item":"Brio Staging - PKI and Backup Keys","field":"BRIO_KEYCLOAK_DB_SOURCE_CIDR"}, + + {"environment":"release-brio-identity-db","kind":"secret","requirement":"required","destination":"KEYCLOAK_RELEASE_ORCHESTRATOR_TOKEN","item":"PostgreSQL · Brio identity release orchestrator","field":"KEYCLOAK_RELEASE_ORCHESTRATOR_TOKEN"}, + + {"environment":"keycloak-cohort-restore","kind":"secret","requirement":"required","destination":"KEYCLOAK_COHORT_SOURCE_TOKEN","item":"PostgreSQL · Keycloak cohort source reader","field":"KEYCLOAK_COHORT_SOURCE_TOKEN"}, + {"environment":"keycloak-cohort-restore","kind":"secret","requirement":"required","destination":"KEYCLOAK_COHORT_DB_SSH_HOST","item":"Hetzner Database Server makepad","field":"DEPLOY_SSH_HOST"}, + {"environment":"keycloak-cohort-restore","kind":"secret","requirement":"required","destination":"KEYCLOAK_COHORT_DB_SSH_PORT","item":"Hetzner Database Server makepad","field":"DEPLOY_SSH_PORT"}, + {"environment":"keycloak-cohort-restore","kind":"secret","requirement":"required","destination":"KEYCLOAK_COHORT_DB_SSH_USER","item":"Hetzner Database Server makepad","field":"DEPLOY_SSH_USER"}, + {"environment":"keycloak-cohort-restore","kind":"secret","requirement":"required","destination":"KEYCLOAK_COHORT_DB_SSH_PRIVATE_KEY","item":"Hetzner Database Server makepad","field":"DEPLOY_SSH_PRIVATE_KEY"}, + {"environment":"keycloak-cohort-restore","kind":"secret","requirement":"required","destination":"KEYCLOAK_COHORT_DB_SSH_KNOWN_HOSTS","item":"Hetzner Database Server makepad","field":"DEPLOY_SSH_KNOWN_HOSTS"}, + {"environment":"keycloak-cohort-restore","kind":"secret","requirement":"required","destination":"DHI_REGISTRY_USERNAME","item":"Makepad Docker Hardened Images","field":"DOCKERHUB_USERNAME"}, + {"environment":"keycloak-cohort-restore","kind":"secret","requirement":"required","destination":"DHI_REGISTRY_PASSWORD","item":"Makepad Docker Hardened Images","field":"DOCKERHUB_PRO_PAT"} + ] +} diff --git a/docs/credential-sync.md b/docs/credential-sync.md new file mode 100644 index 0000000..b3fb7b9 --- /dev/null +++ b/docs/credential-sync.md @@ -0,0 +1,104 @@ +# PostgreSQL Proton-to-GitHub credential sync + +`deploy/credential-inventory.json` maps the shared Proton Pass vault `Makepad` +to the five existing GitHub Environments used by the reviewed PostgreSQL +release path: + +- `canary` +- `production` +- `staging-brio-identity-db` +- `release-brio-identity-db` +- `keycloak-cohort-restore` + +The inventory contains names and immutable provider identities, never values. +It also records existing shared-environment names that this helper must preserve +without reading, changing, or deleting them. Its `operatorEntries` list is +empty: it grants no authority over runners, GitHub Apps, OAuth Apps, repository +values, host files, branch protection, or environment configuration. + +## Read-only audit + +Run the name-and-policy audit first: + +```sh +./scripts/sync-github-environments.sh --check +./scripts/sync-github-environments.sh --check --environment staging-brio-identity-db +``` + +Check mode pins the public repository identity, native `policy-and-integration` +required check, protected-main settings, read-only Actions token policy, exact +environment IDs, reviewer boundary, and exact `main` deployment branch-policy +IDs. It reads active Proton item titles and GitHub destination names only. It +does not call `pass-cli item view` or perform a provider write. + +A missing required source or destination returns status 1. An unmanaged GitHub +destination returns status 2. Preserved names are reported separately and do +not enter the helper's write set. + +## Explicit one-environment sync + +After reviewing the read-only result and receiving action-time approval, sync +one existing environment with its exact confirmation string: + +```sh +./scripts/sync-github-environments.sh --sync \ + --environment staging-brio-identity-db \ + --confirm Makepad-fr/postgres:staging-brio-identity-db +``` + +Replace the environment in both places with one of the other four reviewed +names when that exact scope is approved. There is no all-environment write +mode. + +Before the first write the helper: + +1. rejects repository, main-policy, environment, reviewer, or branch-policy + identity drift; +2. rejects repository-level or unmanaged environment destinations; +3. reads and bounds every selected Proton field in memory; +4. repeats all provider policy/name checks; and +5. rereads every source field to detect rotation during preflight. + +Values are streamed to `gh` over standard input. They are not placed in +arguments, exported child environments, logs, or workspace files. Public +environment variables receive exact-value readback. GitHub does not expose +secret values, so secrets receive name/metadata readback. The helper repeats +source and destination checks before reporting `SYNC_COMPLETE`. + +The operation is idempotent. A failed partial sync can be inspected and rerun +because each write sets the same reviewed destination. The helper cannot create +or delete an environment, modify Proton Pass, alter a provider policy, delete a +legacy name, or write repository-level values. + +## Reviewed source records + +- `Hetzner App Server makepad` supplies the native SSH fields for the existing + shared canary and production workflows. +- `Hetzner Database Server makepad` supplies the five standalone database-host + SSH fields for Brio identity deployment and cohort restore. +- `PostgreSQL · shared Swarm deployment` and + `Le Petit Coin GitHub Deploy Secrets` supply existing shared deployment + constants. +- `Brio Staging - PostgreSQL` supplies Brio role passwords and canary inputs. +- `Brio Staging - PKI and Backup Keys` supplies certificate, recovery, hostname, + and source-CIDR inputs. +- `PostgreSQL · Brio identity release orchestrator` supplies the dedicated + Keycloak release-dispatch token. +- `PostgreSQL · Keycloak cohort source reader` supplies the read-only source + token used by restore evidence. +- `Makepad Docker Hardened Images` supplies the cohort verifier pull identity. + +The JSON inventory is authoritative for every destination-to-field mapping. +Changing a workflow credential requires a reviewed inventory and adversarial +test update in the same PR. + +## Validation + +```sh +bash scripts/test-sync-github-environments.sh +``` + +The test uses isolated fake providers to prove check-mode non-disclosure, exact +write confirmation, preflight-before-write behavior, source and provider drift +rejection, no value output, variable readback, preserved-name isolation, and +the absence of runner/App authority. diff --git a/scripts/run-ci.sh b/scripts/run-ci.sh index 85e70af..a5f6788 100755 --- a/scripts/run-ci.sh +++ b/scripts/run-ci.sh @@ -32,6 +32,8 @@ shellcheck_paths=( \ scripts/test-keycloak-cohort-hardening.sh \ scripts/capture-keycloak-cohort-backups.sh \ scripts/restore-keycloak-cohort-backups.sh \ + scripts/sync-github-environments.sh \ + scripts/test-sync-github-environments.sh \ scripts/fixtures/brio-deployment-failure-fixture.sh \ scripts/fixtures/keycloak-cohort-cleaner-fixture.sh \ scripts/fixtures/keycloak-cohort-dispatch-fixture.sh \ @@ -53,6 +55,7 @@ for source in ( ast.parse(Path(source).read_text(), filename=source) PY PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-github-environment-main-policy.py +bash scripts/test-sync-github-environments.sh actionlint git diff --check ./scripts/test-brio-deploy-guards.sh diff --git a/scripts/sync-github-environments.sh b/scripts/sync-github-environments.sh new file mode 100755 index 0000000..d55b108 --- /dev/null +++ b/scripts/sync-github-environments.sh @@ -0,0 +1,610 @@ +#!/usr/bin/env bash + +# Reconcile reviewed Proton Pass fields into existing GitHub Environments. +# Repository/environment policy is a read-only invariant of this helper. +set +x +set -Eeuo pipefail +umask 077 +IFS=$' \t\n' +export LANG=C +export LC_ALL=C +unset BASH_XTRACEFD DEBUG GH_DEBUG PASS_CLI_DEBUG + +readonly repository=Makepad-fr/postgres +readonly vault=Makepad +readonly github_api_version=2022-11-28 +readonly allowed_environments='canary production staging-brio-identity-db release-brio-identity-db keycloak-cohort-restore' +repo_root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P) +readonly repo_root +readonly inventory=${repo_root}/deploy/credential-inventory.json +readonly max_value_bytes=49152 + +usage() { + printf '%s\n' \ + 'usage: sync-github-environments.sh [--check] [--environment NAME]' \ + ' sync-github-environments.sh --sync --environment NAME --confirm Makepad-fr/postgres:NAME' \ + '' \ + ' --check Inspect policy and names only; never read Proton field values (default).' \ + ' --sync Preflight and reconcile exactly one existing environment.' +} + +die() { + printf 'credential sync: %s\n' "$*" >&2 + exit 1 +} + +mode=check +mode_selected=0 +selected_environment= +provided_confirmation= +while (( $# > 0 )); do + case "$1" in + --check|--sync) + (( mode_selected == 0 )) || die 'select exactly one mode' + mode=${1#--} + mode_selected=1 + ;; + --environment) + (( $# >= 2 )) || die '--environment requires a value' + [[ -z "${selected_environment}" ]] || die '--environment may be supplied only once' + selected_environment=$2 + shift + ;; + --confirm) + (( $# >= 2 )) || die '--confirm requires a value' + [[ -z "${provided_confirmation}" ]] || die '--confirm may be supplied only once' + provided_confirmation=$2 + shift + ;; + --help|-h) + usage + exit 0 + ;; + *) + usage >&2 + die "unsupported argument: $1" + ;; + esac + shift +done + +case "${selected_environment:-all}" in + all|canary|production|staging-brio-identity-db|release-brio-identity-db|keycloak-cohort-restore) ;; + *) die 'environment is not in the reviewed PostgreSQL inventory' ;; +esac +if [[ "${mode}" == sync ]]; then + [[ -n "${selected_environment}" ]] || die '--sync requires one explicit --environment' + [[ "${provided_confirmation}" == "${repository}:${selected_environment}" ]] || + die '--sync requires the exact repository and environment confirmation' +elif [[ -n "${provided_confirmation}" ]]; then + die '--confirm is accepted only with --sync' +fi + +for command_name in pass-cli gh jq python3 sort grep awk mktemp find wc; do + command -v "${command_name}" >/dev/null || die "${command_name} is required" +done +[[ -f "${inventory}" && ! -L "${inventory}" ]] || die 'credential inventory is missing or is a symbolic link' + +tmp_base=${TMPDIR:-/tmp} +[[ -d "${tmp_base}" && ! -L "${tmp_base}" ]] || die 'temporary directory base is unsafe' +tmp_base=$(cd "${tmp_base}" && pwd -P) +readonly tmp_base +work_root=$(mktemp -d "${tmp_base}/postgres-credential-sync.XXXXXXXX") +[[ -d "${work_root}" && ! -L "${work_root}" ]] || die 'could not create a private work directory' +chmod 0700 "${work_root}" +readonly work_root +readonly entries_file=${work_root}/entries.tsv +readonly policy_file=${work_root}/policy.json +readonly preserved_file=${work_root}/preserved.tsv +readonly proton_items_file=${work_root}/proton-items.txt +readonly repository_secrets_file=${work_root}/repository-secrets.txt +readonly repository_variables_file=${work_root}/repository-variables.txt + +declare -a entry_environment=() +declare -a entry_kind=() +declare -a entry_requirement=() +declare -a entry_destination=() +declare -a entry_item=() +declare -a entry_field=() +declare -a source_values=() +declare -a baseline_environment_id=() +declare -a baseline_branch_policy_id=() + +cleanup() { + local index + for index in "${!source_values[@]}"; do + unset 'source_values[index]' + done + if [[ "${work_root:-}" == "${tmp_base}/postgres-credential-sync."* && + -d "${work_root}" && ! -L "${work_root}" ]]; then + find "${work_root}" -depth -mindepth 1 -delete + rmdir -- "${work_root}" + fi +} +trap cleanup EXIT +trap 'exit 130' HUP INT TERM + +python3 - "${inventory}" "${repository}" "${vault}" "${selected_environment}" \ + "${policy_file}" "${preserved_file}" >"${entries_file}" <<'PY' +import hashlib +import json +import pathlib +import re +import sys + +path = pathlib.Path(sys.argv[1]) +repository = sys.argv[2] +vault = sys.argv[3] +selected = sys.argv[4] +policy_path = pathlib.Path(sys.argv[5]) +preserved_path = pathlib.Path(sys.argv[6]) +raw_payload = path.read_text(encoding="utf-8") +payload = json.loads(raw_payload) +inventory_digest = hashlib.sha256(json.dumps( + payload, sort_keys=True, separators=(",", ":"), ensure_ascii=False +).encode()).hexdigest() +if inventory_digest != "912ae075075aa0843909af003619081b5d183e4e20c72e8e260dcd067cd665bd": + raise SystemExit("credential inventory changed without review") + +if set(payload) != { + "schemaVersion", "repository", "vault", "repositoryPolicy", + "operatorEntries", "preservedDestinations", "entries", +}: + raise SystemExit("credential inventory has unexpected top-level keys") +if payload["schemaVersion"] != 3 or payload["repository"] != repository or payload["vault"] != vault: + raise SystemExit("credential inventory identity is invalid") +if payload["operatorEntries"] != []: + raise SystemExit("credential inventory must not manage operator, runner, or App authority") + +expected_preserved = { + "canary": { + "secret": ["KEYCLOAK_BRIO_STAGING_BACKUP_DB_PASSWORD", "KEYCLOAK_BRIO_STAGING_DB_PASSWORD"], + "variable": [], + }, + "production": { + "secret": [ + "DEPLOY_FASHION_DB_NAME", "DEPLOY_FASHION_DB_NETWORK", + "DEPLOY_FASHION_DB_PASSWORD", "DEPLOY_FASHION_DB_USER", + "DEPLOY_SCRAPING_DB_NAME", "DEPLOY_SCRAPING_DB_NETWORK", + "DEPLOY_SCRAPING_DB_PASSWORD", "DEPLOY_SCRAPING_DB_USER", + ], + "variable": [], + }, + "staging-brio-identity-db": { + "secret": ["BRIO_STAGING_BACKUP_DB_PASSWORD", "BRIO_STAGING_DB_PASSWORD"], + "variable": ["POSTGRES_HOST_COMPOSE_PROJECT"], + }, + "release-brio-identity-db": {"secret": [], "variable": []}, + "keycloak-cohort-restore": {"secret": [], "variable": []}, +} +if payload["preservedDestinations"] != expected_preserved: + raise SystemExit("preserved destination boundary is invalid") + +policy = payload["repositoryPolicy"] +if not isinstance(policy, dict) or set(policy) != { + "repositoryId", "private", "visibility", "allowForking", "defaultBranch", + "requiredChecks", "requiredCheckAppId", "mainProtection", "actionsPolicy", "environments", +}: + raise SystemExit("repository policy has unexpected keys") +if ( + policy["repositoryId"] != 1200300784 + or policy["private"] is not False + or policy["visibility"] != "public" + or policy["allowForking"] is not True + or policy["defaultBranch"] != "main" + or policy["requiredChecks"] != ["policy-and-integration"] + or policy["requiredCheckAppId"] != 15368 +): + raise SystemExit("repository identity or native status policy is invalid") +if policy["mainProtection"] != { + "strictStatusChecks": True, + "enforceAdmins": True, + "dismissStaleReviews": True, + "requireCodeOwnerReviews": True, + "requiredApprovingReviewCount": 1, + "requireLastPushApproval": True, + "requiredSignatures": True, + "requiredLinearHistory": True, + "requiredConversationResolution": True, + "allowForcePushes": False, + "allowDeletions": False, + "blockCreations": False, + "lockBranch": False, + "allowForkSyncing": False, +}: + raise SystemExit("reviewed main protection is invalid") +if policy["actionsPolicy"] != { + "defaultWorkflowPermissions": "read", + "canApprovePullRequestReviews": False, +}: + raise SystemExit("reviewed Actions token policy is invalid") +if policy["environments"] != { + "canary": {"id": 21262761188, "branchPolicyId": 59128365, "reviewerId": None, "reviewerLogin": None}, + "production": {"id": 15050761884, "branchPolicyId": 59156955, "reviewerId": None, "reviewerLogin": None}, + "staging-brio-identity-db": {"id": 21278291993, "branchPolicyId": 59143916, "reviewerId": 39597780, "reviewerLogin": "idilsaglam"}, + "release-brio-identity-db": {"id": 21284627193, "branchPolicyId": 59149936, "reviewerId": 39597780, "reviewerLogin": "idilsaglam"}, + "keycloak-cohort-restore": {"id": 21284627918, "branchPolicyId": 59149937, "reviewerId": 39597780, "reviewerLogin": "idilsaglam"}, +}: + raise SystemExit("reviewed environment identities are invalid") + +allowed_environments = set(policy["environments"]) +entry_keys = {"environment", "kind", "requirement", "destination", "item", "field"} +destination_pattern = re.compile(r"^[A-Z][A-Z0-9_]{1,127}$") +field_pattern = re.compile(r"^[A-Za-z][A-Za-z0-9_ -]{0,127}$") +seen = set() +counts = {environment: 0 for environment in allowed_environments} +entries = payload["entries"] +if not isinstance(entries, list) or not entries: + raise SystemExit("credential inventory entries are missing") +for offset, entry in enumerate(entries): + if not isinstance(entry, dict) or set(entry) != entry_keys: + raise SystemExit(f"credential inventory entry {offset} has unexpected keys") + environment = entry["environment"] + kind = entry["kind"] + requirement = entry["requirement"] + destination = entry["destination"] + item = entry["item"] + field = entry["field"] + if environment not in allowed_environments or kind not in {"secret", "variable"}: + raise SystemExit(f"credential inventory entry {offset} has an invalid scope") + if requirement not in {"required", "optional"}: + raise SystemExit(f"credential inventory entry {offset} has an invalid requirement") + if not isinstance(destination, str) or not destination_pattern.fullmatch(destination): + raise SystemExit(f"credential inventory entry {offset} has an invalid destination") + if not isinstance(item, str) or not item or len(item) > 128 or any(c in item for c in "\t\r\n"): + raise SystemExit(f"credential inventory entry {offset} has an invalid Proton item") + if not isinstance(field, str) or not field_pattern.fullmatch(field): + raise SystemExit(f"credential inventory entry {offset} has an invalid Proton field") + identity = (environment, kind, destination) + if identity in seen: + raise SystemExit(f"duplicate GitHub destination: {environment}/{kind}/{destination}") + seen.add(identity) + counts[environment] += 1 + if not selected or selected == environment: + print("\t".join((environment, kind, requirement, destination, item, field))) +if any(count == 0 for count in counts.values()): + raise SystemExit("every approved environment must have at least one entry") +for environment, kinds in expected_preserved.items(): + for kind, destinations in kinds.items(): + for destination in destinations: + if (environment, kind, destination) in seen: + raise SystemExit("preserved destination overlaps a managed destination") + +policy_path.write_text(json.dumps(policy, separators=(",", ":")), encoding="utf-8") +preserved_path.write_text("".join( + f"{environment}\t{kind}\t{destination}\n" + for environment in sorted(expected_preserved) + if not selected or selected == environment + for kind in ("secret", "variable") + for destination in expected_preserved[environment][kind] +), encoding="utf-8") +PY + +[[ -s "${entries_file}" ]] || die 'the selected inventory is empty' +while IFS=$'\t' read -r environment kind requirement destination item field; do + index=${#entry_environment[@]} + entry_environment[index]=${environment} + entry_kind[index]=${kind} + entry_requirement[index]=${requirement} + entry_destination[index]=${destination} + entry_item[index]=${item} + entry_field[index]=${field} +done <"${entries_file}" + +pass-cli test >/dev/null || die 'Proton Pass is not authenticated' +GH_PROMPT_DISABLED=1 gh auth status >/dev/null 2>&1 || die 'GitHub CLI is not authenticated' + +environment_selected() { + [[ -z "${selected_environment}" || "${selected_environment}" == "$1" ]] +} + +destination_expected() { + local environment=$1 kind=$2 destination=$3 + if awk -F '\t' -v environment="${environment}" -v kind="${kind}" -v destination="${destination}" \ + '$1 == environment && $2 == kind && $4 == destination { found = 1 } END { exit !found }' "${entries_file}"; then + return 0 + fi + awk -F '\t' -v environment="${environment}" -v kind="${kind}" -v destination="${destination}" \ + '$1 == environment && $2 == kind && $3 == destination { found = 1 } END { exit !found }' "${preserved_file}" +} + +protection_errors=0 +missing_required_sources=0 +missing_required_destinations=0 +missing_optional_sources=0 +missing_optional_destinations=0 +unexpected_destinations=0 + +load_names_and_policy() { + local repository_json protection_json actions_json environment environment_json policies_json + local expected_id expected_policy_id expected_reviewer_id expected_reviewer_login actual_id actual_policy_id + local identity_index kind names_file + + protection_errors=0 + find "${work_root}" -maxdepth 1 -type f -name 'github-*.txt' -delete + if ! pass-cli item list --vault-name "${vault}" --filter-state active --output json | + jq -er '.items | if type == "array" then . else error("invalid item list") end | .[] | .title' | + sort >"${proton_items_file}"; then + die 'could not read Proton Pass item names' + fi + + if ! repository_json=$(GH_PROMPT_DISABLED=1 gh api --header "X-GitHub-Api-Version: ${github_api_version}" "repos/${repository}") || + ! jq -e --arg repository "${repository}" --argjson policy "$(<"${policy_file}")" ' + .full_name == $repository and .id == $policy.repositoryId and + .private == $policy.private and .visibility == $policy.visibility and + .allow_forking == $policy.allowForking and .default_branch == $policy.defaultBranch and + .archived == false and .disabled == false + ' >/dev/null <<<"${repository_json}"; then + printf 'REPOSITORY name=%s policy=identity-or-visibility-invalid\n' "${repository}" + ((protection_errors += 1)) + elif ! protection_json=$(GH_PROMPT_DISABLED=1 gh api --header "X-GitHub-Api-Version: ${github_api_version}" "repos/${repository}/branches/main/protection") || + ! jq -e --argjson policy "$(<"${policy_file}")" ' + .required_status_checks.strict == $policy.mainProtection.strictStatusChecks and + ([.required_status_checks.checks[]? | {context,app_id}] == + [{context:$policy.requiredChecks[0],app_id:$policy.requiredCheckAppId}]) and + .enforce_admins.enabled == $policy.mainProtection.enforceAdmins and + .required_pull_request_reviews.dismiss_stale_reviews == $policy.mainProtection.dismissStaleReviews and + .required_pull_request_reviews.require_code_owner_reviews == $policy.mainProtection.requireCodeOwnerReviews and + .required_pull_request_reviews.required_approving_review_count == $policy.mainProtection.requiredApprovingReviewCount and + .required_pull_request_reviews.require_last_push_approval == $policy.mainProtection.requireLastPushApproval and + .required_signatures.enabled == $policy.mainProtection.requiredSignatures and + .required_linear_history.enabled == $policy.mainProtection.requiredLinearHistory and + .required_conversation_resolution.enabled == $policy.mainProtection.requiredConversationResolution and + .allow_force_pushes.enabled == $policy.mainProtection.allowForcePushes and + .allow_deletions.enabled == $policy.mainProtection.allowDeletions and + .block_creations.enabled == $policy.mainProtection.blockCreations and + .lock_branch.enabled == $policy.mainProtection.lockBranch and + .allow_fork_syncing.enabled == $policy.mainProtection.allowForkSyncing + ' >/dev/null <<<"${protection_json}"; then + printf 'REPOSITORY name=%s policy=main-protection-invalid\n' "${repository}" + ((protection_errors += 1)) + elif ! actions_json=$(GH_PROMPT_DISABLED=1 gh api --header "X-GitHub-Api-Version: ${github_api_version}" "repos/${repository}/actions/permissions/workflow") || + ! jq -e --argjson policy "$(<"${policy_file}")" ' + .default_workflow_permissions == $policy.actionsPolicy.defaultWorkflowPermissions and + .can_approve_pull_request_reviews == $policy.actionsPolicy.canApprovePullRequestReviews + ' >/dev/null <<<"${actions_json}"; then + printf 'REPOSITORY name=%s policy=actions-token-policy-invalid\n' "${repository}" + ((protection_errors += 1)) + else + printf 'REPOSITORY name=%s policy=reviewed-native-main-only identity=stable\n' "${repository}" + fi + + GH_PROMPT_DISABLED=1 gh secret list --repo "${repository}" --json name --jq '.[].name' | + sort >"${repository_secrets_file}" || die 'could not list repository secret names' + GH_PROMPT_DISABLED=1 gh variable list --repo "${repository}" --json name --jq '.[].name' | + sort >"${repository_variables_file}" || die 'could not list repository variable names' + + identity_index=0 + for environment in ${allowed_environments}; do + environment_selected "${environment}" || continue + expected_id=$(jq -er --arg environment "${environment}" '.environments[$environment].id' "${policy_file}") + expected_policy_id=$(jq -er --arg environment "${environment}" '.environments[$environment].branchPolicyId' "${policy_file}") + expected_reviewer_id=$(jq -r --arg environment "${environment}" '.environments[$environment].reviewerId // empty' "${policy_file}") + expected_reviewer_login=$(jq -r --arg environment "${environment}" '.environments[$environment].reviewerLogin // empty' "${policy_file}") + if ! environment_json=$(GH_PROMPT_DISABLED=1 gh api --header "X-GitHub-Api-Version: ${github_api_version}" "repos/${repository}/environments/${environment}") || + ! jq -e --arg environment "${environment}" --argjson expected_id "${expected_id}" \ + --arg reviewer_id "${expected_reviewer_id}" --arg reviewer_login "${expected_reviewer_login}" ' + .name == $environment and .id == $expected_id and .can_admins_bypass == true and + .deployment_branch_policy.protected_branches == false and + .deployment_branch_policy.custom_branch_policies == true and + ([.protection_rules[] | select(.type == "branch_policy")] | length) == 1 and + if $reviewer_id == "" then + ([.protection_rules[] | select(.type != "branch_policy")] | length) == 0 + else + ([.protection_rules[] | select(.type == "required_reviewers")] | length) == 1 and + ([.protection_rules[] | select(.type == "required_reviewers")][0] | + .prevent_self_review == true and (.reviewers | length) == 1 and + .reviewers[0].type == "User" and + (.reviewers[0].reviewer.id | tostring) == $reviewer_id and + .reviewers[0].reviewer.login == $reviewer_login) + end + ' >/dev/null <<<"${environment_json}"; then + printf 'ENVIRONMENT name=%s protection=invalid\n' "${environment}" + ((protection_errors += 1)) + continue + fi + if ! policies_json=$(GH_PROMPT_DISABLED=1 gh api --header "X-GitHub-Api-Version: ${github_api_version}" \ + "repos/${repository}/environments/${environment}/deployment-branch-policies?per_page=100") || + ! jq -e --argjson expected_policy_id "${expected_policy_id}" ' + .total_count == 1 and (.branch_policies | length) == 1 and + .branch_policies[0].id == $expected_policy_id and + .branch_policies[0].name == "main" and .branch_policies[0].type == "branch" + ' >/dev/null <<<"${policies_json}"; then + printf 'ENVIRONMENT name=%s protection=invalid-branch-policy\n' "${environment}" + ((protection_errors += 1)) + continue + fi + actual_id=$(jq -er '.id' <<<"${environment_json}") + actual_policy_id=$(jq -er '.branch_policies[0].id' <<<"${policies_json}") + if [[ "${mode}" == sync && ${#baseline_environment_id[@]} -gt identity_index ]]; then + [[ "${baseline_environment_id[identity_index]}" == "${actual_id}" && + "${baseline_branch_policy_id[identity_index]}" == "${actual_policy_id}" ]] || { + printf 'ENVIRONMENT name=%s protection=identity-changed\n' "${environment}" + ((protection_errors += 1)) + continue + } + elif [[ "${mode}" == sync ]]; then + baseline_environment_id[identity_index]=${actual_id} + baseline_branch_policy_id[identity_index]=${actual_policy_id} + fi + ((identity_index += 1)) + printf 'ENVIRONMENT name=%s protection=exact-reviewed-matrix identity=stable\n' "${environment}" + + for kind in secret variable; do + names_file=${work_root}/github-${environment}-${kind}.txt + if [[ "${kind}" == secret ]]; then + GH_PROMPT_DISABLED=1 gh secret list --repo "${repository}" --env "${environment}" \ + --json name --jq '.[].name' | sort >"${names_file}" || die "could not list ${environment} secret names" + else + GH_PROMPT_DISABLED=1 gh variable list --repo "${repository}" --env "${environment}" \ + --json name --jq '.[].name' | sort >"${names_file}" || die "could not list ${environment} variable names" + fi + done + done +} + +report_status() { + local environment kind requirement destination item item_count destination_file status actual_name + missing_required_sources=0 + missing_required_destinations=0 + missing_optional_sources=0 + missing_optional_destinations=0 + unexpected_destinations=0 + + while IFS=$'\t' read -r item requirement; do + item_count=$(grep -Fxc -- "${item}" "${proton_items_file}" || true) + case "${item_count}" in + 1) status=present ;; + 0) status=missing ;; + *) status=ambiguous ;; + esac + printf 'SOURCE_ITEM title=%s requirement=%s status=%s\n' "${item}" "${requirement}" "${status}" + if [[ "${status}" != present ]]; then + if [[ "${requirement}" == required ]]; then + ((missing_required_sources += 1)) + else + ((missing_optional_sources += 1)) + fi + fi + done < <(awk -F '\t' '{key=$5 FS $3; seen[key]=1} END {for (key in seen) print key}' "${entries_file}" | sort) + + for index in "${!entry_environment[@]}"; do + environment=${entry_environment[index]} + kind=${entry_kind[index]} + requirement=${entry_requirement[index]} + destination=${entry_destination[index]} + destination_file=${work_root}/github-${environment}-${kind}.txt + if [[ -f "${destination_file}" ]] && grep -Fqx -- "${destination}" "${destination_file}"; then + status=present + else + status=missing + if [[ "${requirement}" == required ]]; then + ((missing_required_destinations += 1)) + else + ((missing_optional_destinations += 1)) + fi + fi + printf 'DESTINATION environment=%s kind=%s name=%s requirement=%s status=%s\n' \ + "${environment}" "${kind}" "${destination}" "${requirement}" "${status}" + done + + while IFS=$'\t' read -r environment kind destination; do + destination_file=${work_root}/github-${environment}-${kind}.txt + if [[ -f "${destination_file}" ]] && grep -Fqx -- "${destination}" "${destination_file}"; then + status=preserved-existing + else + status=preserved-absent + fi + printf 'PRESERVED_DESTINATION environment=%s kind=%s name=%s status=%s\n' \ + "${environment}" "${kind}" "${destination}" "${status}" + done <"${preserved_file}" + + for environment in ${allowed_environments}; do + environment_selected "${environment}" || continue + for kind in secret variable; do + destination_file=${work_root}/github-${environment}-${kind}.txt + [[ -f "${destination_file}" ]] || continue + while IFS= read -r actual_name; do + [[ -n "${actual_name}" ]] || continue + if ! destination_expected "${environment}" "${kind}" "${actual_name}"; then + printf 'UNEXPECTED_DESTINATION scope=environment environment=%s kind=%s name=%s status=legacy-or-unmanaged\n' \ + "${environment}" "${kind}" "${actual_name}" + ((unexpected_destinations += 1)) + fi + done <"${destination_file}" + done + done + + for kind in secret variable; do + destination_file=${repository_secrets_file} + [[ "${kind}" == secret ]] || destination_file=${repository_variables_file} + while IFS= read -r actual_name; do + [[ -n "${actual_name}" ]] || continue + printf 'UNEXPECTED_DESTINATION scope=repository kind=%s name=%s status=forbidden\n' "${kind}" "${actual_name}" + ((unexpected_destinations += 1)) + done <"${destination_file}" + done + + printf 'SUMMARY required_source_issues=%d required_destination_missing=%d optional_source_issues=%d optional_destination_missing=%d unexpected_destinations=%d protection_errors=%d\n' \ + "${missing_required_sources}" "${missing_required_destinations}" \ + "${missing_optional_sources}" "${missing_optional_destinations}" \ + "${unexpected_destinations}" "${protection_errors}" +} + +load_names_and_policy +report_status +if [[ "${mode}" == check ]]; then + if (( protection_errors > 0 || missing_required_sources > 0 || missing_required_destinations > 0 )); then + exit 1 + fi + (( unexpected_destinations == 0 )) || exit 2 + exit 0 +fi + +(( protection_errors == 0 )) || die 'refusing to sync into an invalid repository or environment policy' +(( unexpected_destinations == 0 )) || die 'refusing to sync while forbidden or unmanaged GitHub names remain' +(( missing_required_sources == 0 )) || die 'required Proton Pass source items are missing or ambiguous' +ulimit -c 0 || die 'could not disable process core dumps before handling credential values' + +for index in "${!entry_environment[@]}"; do + item=${entry_item[index]} + field=${entry_field[index]} + if ! value=$(pass-cli item view --vault-name "${vault}" --item-title "${item}" --field "${field}" 2>/dev/null); then + die "required Proton field is missing or unreadable: ${item}/${field}" + fi + [[ -n "${value}" ]] || die "required Proton field is empty: ${item}/${field}" + (( ${#value} <= max_value_bytes )) || die "Proton field exceeds the bounded value size: ${item}/${field}" + source_values[index]=${value} + unset value +done + +load_names_and_policy +report_status +(( protection_errors == 0 )) || die 'repository or environment protection changed during source preflight' +(( unexpected_destinations == 0 )) || die 'a forbidden or unmanaged GitHub name appeared during source preflight' +(( missing_required_sources == 0 )) || die 'a required Proton item disappeared during source preflight' + +for index in "${!entry_environment[@]}"; do + item=${entry_item[index]} + field=${entry_field[index]} + if ! value=$(pass-cli item view --vault-name "${vault}" --item-title "${item}" --field "${field}" 2>/dev/null); then + die "required Proton field disappeared during preflight: ${item}/${field}" + fi + [[ "${value}" == "${source_values[index]}" ]] || die "Proton source changed during preflight: ${item}/${field}" + unset value +done + +for index in "${!entry_environment[@]}"; do + environment=${entry_environment[index]} + kind=${entry_kind[index]} + destination=${entry_destination[index]} + if [[ "${kind}" == secret ]]; then + printf '%s' "${source_values[index]}" | + GH_PROMPT_DISABLED=1 gh secret set "${destination}" --repo "${repository}" --env "${environment}" >/dev/null 2>&1 || + die "GitHub rejected ${environment}/${kind}/${destination}" + else + printf '%s' "${source_values[index]}" | + GH_PROMPT_DISABLED=1 gh variable set "${destination}" --repo "${repository}" --env "${environment}" >/dev/null 2>&1 || + die "GitHub rejected ${environment}/${kind}/${destination}" + readback=$(GH_PROMPT_DISABLED=1 gh variable get "${destination}" --repo "${repository}" --env "${environment}" \ + --json value --jq '.value' 2>/dev/null) || die "GitHub variable readback failed: ${destination}" + [[ "${readback}" == "${source_values[index]}" ]] || die "GitHub variable readback differs from Proton: ${destination}" + unset readback + fi + printf 'SYNCED environment=%s kind=%s name=%s\n' "${environment}" "${kind}" "${destination}" +done + +load_names_and_policy +report_status +(( protection_errors == 0 && missing_required_destinations == 0 && unexpected_destinations == 0 )) || + die 'GitHub destination or policy readback did not match the reviewed inventory' +for index in "${!entry_environment[@]}"; do + item=${entry_item[index]} + field=${entry_field[index]} + if ! value=$(pass-cli item view --vault-name "${vault}" --item-title "${item}" --field "${field}" 2>/dev/null); then + die "required Proton field disappeared after write: ${item}/${field}" + fi + [[ "${value}" == "${source_values[index]}" ]] || die "Proton source changed during write: ${item}/${field}" + unset value 'source_values[index]' +done +printf 'SYNC_COMPLETE repository=%s vault=%s environment=%s\n' "${repository}" "${vault}" "${selected_environment}" diff --git a/scripts/test-sync-github-environments.sh b/scripts/test-sync-github-environments.sh new file mode 100755 index 0000000..d07af74 --- /dev/null +++ b/scripts/test-sync-github-environments.sh @@ -0,0 +1,367 @@ +#!/usr/bin/env bash +set -Eeuo pipefail +umask 077 + +repo_root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P) +helper=${repo_root}/scripts/sync-github-environments.sh +inventory=${repo_root}/deploy/credential-inventory.json +test_root=$(mktemp -d "${TMPDIR:-/tmp}/postgres-credential-sync-test.XXXXXXXX") +fake_bin=${test_root}/bin +audit_log=${test_root}/audit.log +provider_state=${test_root}/provider +mkdir -m 0700 "${fake_bin}" "${provider_state}" + +cleanup() { + find "${test_root}" -depth -mindepth 1 -delete + rmdir -- "${test_root}" +} +trap cleanup EXIT + +cat >"${fake_bin}/pass-cli" <<'FAKE_PASS' +#!/usr/bin/env bash +set -Eeuo pipefail +printf 'pass-cli' >>"${FAKE_AUDIT_LOG}" +printf ' %q' "$@" >>"${FAKE_AUDIT_LOG}" +printf '\n' >>"${FAKE_AUDIT_LOG}" + +if [[ "${1:-}" == test ]]; then + exit 0 +fi +if [[ "${1:-} ${2:-}" == 'item list' ]]; then + jq -c --arg missing "${FAKE_MISSING_ITEM:-}" --arg duplicate "${FAKE_DUPLICATE_ITEM:-}" ' + ([.entries[].item] | unique | map(select(. != $missing) | {title:.})) as $items | + {items: ($items + (if $duplicate == "" then [] else [{title:$duplicate}] end))} + ' "${FAKE_INVENTORY}" + exit 0 +fi +if [[ "${1:-} ${2:-}" == 'item view' ]]; then + field= + while (( $# > 0 )); do + if [[ "$1" == --field ]]; then + field=$2 + break + fi + shift + done + [[ -n "${field}" && "${field}" != "${FAKE_MISSING_FIELD:-}" ]] || exit 1 + count_file=${FAKE_PROVIDER_STATE}/field-${field}.count + count=0 + [[ ! -f "${count_file}" ]] || read -r count <"${count_file}" + ((count += 1)) + printf '%s\n' "${count}" >"${count_file}" + if [[ "${field}" == "${FAKE_DRIFT_FIELD:-}" && "${count}" -ge "${FAKE_DRIFT_ON_CALL:-2}" ]]; then + printf 'ROTATED_%s' "${field}" + elif [[ "${field}" == DEPLOY_SSH_PRIVATE_KEY || "${field}" == private_key ]]; then + printf '%s' 'HIGHLY_SECRET_PRIVATE_KEY' + else + printf 'HIGHLY_SECRET_%s' "${field}" + fi + exit 0 +fi +exit 97 +FAKE_PASS + +cat >"${fake_bin}/gh" <<'FAKE_GH' +#!/usr/bin/env bash +set -Eeuo pipefail +printf 'gh' >>"${FAKE_AUDIT_LOG}" +printf ' %q' "$@" >>"${FAKE_AUDIT_LOG}" +printf '\n' >>"${FAKE_AUDIT_LOG}" +raw_arguments=$* +if env | grep -Eq 'HIGHLY_SECRET_|ROTATED_'; then + echo 'credential leaked through the process environment' >&2 + exit 95 +fi + +if [[ "${1:-} ${2:-}" == 'auth status' ]]; then + exit 0 +fi + +api_count() { + local label=$1 + local file=${FAKE_PROVIDER_STATE}/api-${label}.count + local count=0 + [[ ! -f "${file}" ]] || read -r count <"${file}" + ((count += 1)) + printf '%s\n' "${count}" >"${file}" + printf '%s' "${count}" +} + +if [[ "${1:-}" == api ]]; then + path=${*: -1} + if [[ "${path}" == repos/Makepad-fr/postgres ]]; then + count=$(api_count repository) + repository_id=1200300784 + if [[ "${FAKE_REPOSITORY_INVALID:-0}" == 1 || "${FAKE_REPOSITORY_DRIFT_ON_CALL:-0}" == "${count}" ]]; then + ((repository_id += 1)) + fi + printf '{"id":%s,"full_name":"Makepad-fr/postgres","private":false,"visibility":"public","allow_forking":true,"default_branch":"main","archived":false,"disabled":false}\n' "${repository_id}" + exit 0 + fi + if [[ "${path}" == repos/Makepad-fr/postgres/branches/main/protection ]]; then + strict=true + [[ "${FAKE_MAIN_POLICY_INVALID:-0}" != 1 ]] || strict=false + printf '{"required_status_checks":{"strict":%s,"checks":[{"context":"policy-and-integration","app_id":15368}]},"enforce_admins":{"enabled":true},"required_pull_request_reviews":{"dismiss_stale_reviews":true,"require_code_owner_reviews":true,"required_approving_review_count":1,"require_last_push_approval":true},"required_signatures":{"enabled":true},"required_linear_history":{"enabled":true},"required_conversation_resolution":{"enabled":true},"allow_force_pushes":{"enabled":false},"allow_deletions":{"enabled":false},"block_creations":{"enabled":false},"lock_branch":{"enabled":false},"allow_fork_syncing":{"enabled":false}}\n' "${strict}" + exit 0 + fi + if [[ "${path}" == repos/Makepad-fr/postgres/actions/permissions/workflow ]]; then + if [[ "${FAKE_ACTIONS_POLICY_INVALID:-0}" == 1 ]]; then + printf '%s\n' '{"default_workflow_permissions":"write","can_approve_pull_request_reviews":true}' + else + printf '%s\n' '{"default_workflow_permissions":"read","can_approve_pull_request_reviews":false}' + fi + exit 0 + fi + if [[ "${path}" == */deployment-branch-policies* ]]; then + environment_path=${path%/deployment-branch-policies*} + environment=${environment_path##*/} + policy_id=$(jq -er --arg environment "${environment}" '.repositoryPolicy.environments[$environment].branchPolicyId' "${FAKE_INVENTORY}") + if [[ "${FAKE_BRANCH_POLICY_INVALID:-0}" == 1 ]]; then + ((policy_id += 1)) + fi + printf '{"total_count":1,"branch_policies":[{"id":%s,"name":"main","type":"branch"}]}\n' "${policy_id}" + exit 0 + fi + environment=${path##*/} + environment_id=$(jq -er --arg environment "${environment}" '.repositoryPolicy.environments[$environment].id' "${FAKE_INVENTORY}") + reviewer_id=$(jq -r --arg environment "${environment}" '.repositoryPolicy.environments[$environment].reviewerId // empty' "${FAKE_INVENTORY}") + reviewer_login=$(jq -r --arg environment "${environment}" '.repositoryPolicy.environments[$environment].reviewerLogin // empty' "${FAKE_INVENTORY}") + count=$(api_count environment-${environment}) + if [[ "${FAKE_ENVIRONMENT_ID_DRIFT_ON_CALL:-0}" == "${count}" ]]; then + ((environment_id += 1)) + fi + if [[ "${FAKE_ENVIRONMENT_POLICY_INVALID:-0}" == reviewer ]]; then + reviewer_id=77 + elif [[ "${FAKE_ENVIRONMENT_POLICY_INVALID:-0}" == bypass ]]; then + can_bypass=false + else + can_bypass=true + fi + if [[ -n "${reviewer_id}" ]]; then + rules='[{"id":1,"type":"required_reviewers","prevent_self_review":true,"reviewers":[{"type":"User","reviewer":{"type":"User","id":'"${reviewer_id}"',"login":"'"${reviewer_login}"'"}}]},{"id":2,"type":"branch_policy"}]' + else + rules='[{"id":2,"type":"branch_policy"}]' + fi + printf '{"id":%s,"name":"%s","can_admins_bypass":%s,"protection_rules":%s,"deployment_branch_policy":{"protected_branches":false,"custom_branch_policies":true}}\n' \ + "${environment_id}" "${environment}" "${can_bypass}" "${rules}" + exit 0 +fi + +kind=${1:-} +operation=${2:-} +[[ "${kind}" == secret || "${kind}" == variable ]] || exit 96 +shift 2 +destination= +if [[ "${operation}" == set || "${operation}" == get ]]; then + destination=${1:-} + shift +fi +environment= +while (( $# > 0 )); do + case "$1" in + --env) environment=$2; shift 2 ;; + *) shift ;; + esac +done + +if [[ -z "${environment}" ]]; then + if [[ "${operation}" == list && -n "${FAKE_REPOSITORY_DESTINATION:-}" && "${FAKE_REPOSITORY_KIND:-secret}" == "${kind}" ]]; then + printf '%s\n' "${FAKE_REPOSITORY_DESTINATION}" + fi + exit 0 +fi + +marker_for() { + printf '%s/%s-%s-%s' "${FAKE_PROVIDER_STATE}" "${environment}" "${kind}" "$1" +} + +names() { + while IFS= read -r name; do + marker=$(marker_for "${name}") + if [[ "${name}" == "${FAKE_MISSING_DESTINATION:-}" && ! -f "${marker}" ]]; then + continue + fi + printf '%s\n' "${name}" + done < <(jq -r --arg environment "${environment}" --arg kind "${kind}" ' + (.entries[] | select(.environment == $environment and .kind == $kind) | .destination), + (.preservedDestinations[$environment][$kind][]?) + ' "${FAKE_INVENTORY}") + if [[ -n "${FAKE_UNEXPECTED_DESTINATION:-}" && + "${FAKE_UNEXPECTED_ENVIRONMENT:-staging-brio-identity-db}" == "${environment}" && + "${FAKE_UNEXPECTED_KIND:-secret}" == "${kind}" ]]; then + printf '%s\n' "${FAKE_UNEXPECTED_DESTINATION}" + fi +} + +if [[ "${operation}" == list ]]; then + if printf '%s\n' "${raw_arguments}" | grep -Fq -- '--jq'; then + names + elif [[ "${kind}" == secret ]]; then + names | jq -Rn '[inputs | {name:.,updatedAt:"2026-09-05T00:00:00Z"}]' + else + names | jq -Rn '[inputs | {name:.,value:"not-disclosed"}]' + fi + exit 0 +fi + +if [[ "${operation}" == get && "${kind}" == variable ]]; then + field=$(jq -er --arg environment "${environment}" --arg destination "${destination}" ' + .entries[] | select(.environment == $environment and .kind == "variable" and .destination == $destination) | .field + ' "${FAKE_INVENTORY}") + if [[ "${destination}" == "${FAKE_STALE_VARIABLE:-}" ]]; then + printf 'STALE_%s' "${field}" + else + printf 'HIGHLY_SECRET_%s' "${field}" + fi + exit 0 +fi + +[[ "${operation}" == set && -n "${destination}" ]] +bytes=$(wc -c | tr -d '[:space:]') +[[ "${bytes}" =~ ^[1-9][0-9]*$ ]] +printf 'gh-set environment=%s kind=%s name=%s bytes=%s\n' "${environment}" "${kind}" "${destination}" "${bytes}" >>"${FAKE_AUDIT_LOG}" +touch "$(marker_for "${destination}")" +FAKE_GH + +chmod 0755 "${fake_bin}/pass-cli" "${fake_bin}/gh" + +inventory_digest=$(python3 - "${inventory}" <<'PY' +import hashlib +import json +import pathlib +import sys + +payload = json.loads(pathlib.Path(sys.argv[1]).read_text(encoding="utf-8")) +print(hashlib.sha256(json.dumps(payload, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()).hexdigest()) +PY +) +expected_inventory_digest=912ae075075aa0843909af003619081b5d183e4e20c72e8e260dcd067cd665bd +[[ "${inventory_digest}" == "${expected_inventory_digest}" ]] || { + echo "credential inventory changed without review: ${inventory_digest}" >&2 + exit 1 +} + +run_helper() { + local expected_exit=$1 + shift + find "${provider_state}" -mindepth 1 -maxdepth 1 -delete + : >"${audit_log}" + set +e + output=$(PATH="${fake_bin}:${PATH}" FAKE_INVENTORY="${inventory}" FAKE_AUDIT_LOG="${audit_log}" \ + FAKE_PROVIDER_STATE="${provider_state}" "$@" 2>&1) + helper_exit=$? + set -e + if (( helper_exit != expected_exit )); then + printf 'unexpected exit %d, wanted %d:\n%s\n' "${helper_exit}" "${expected_exit}" "${output}" >&2 + exit 1 + fi + if grep -Eq 'HIGHLY_SECRET_|ROTATED_' <<<"${output}"; then + echo 'credential value leaked to helper output' >&2 + exit 1 + fi +} + +run_helper 0 "${helper}" --check +grep -Fq 'SUMMARY required_source_issues=0 required_destination_missing=0' <<<"${output}" +grep -Fq 'name=DEPLOY_FASHION_DB_NAME status=preserved-existing' <<<"${output}" +if grep -Fq 'pass-cli item view' "${audit_log}"; then + echo 'check mode read Proton field values' >&2 + exit 1 +fi + +run_helper 1 "${helper}" --check --sync +grep -Fq 'select exactly one mode' <<<"${output}" +run_helper 1 "${helper}" --sync +grep -Fq -- '--sync requires one explicit --environment' <<<"${output}" +run_helper 1 "${helper}" --sync --environment staging-brio-identity-db +grep -Fq -- '--sync requires the exact repository and environment confirmation' <<<"${output}" +run_helper 1 "${helper}" --check --confirm Makepad-fr/postgres:staging-brio-identity-db +grep -Fq -- '--confirm is accepted only with --sync' <<<"${output}" + +run_helper 1 env FAKE_MISSING_ITEM='Brio Staging - PostgreSQL' "${helper}" --check --environment staging-brio-identity-db +grep -Fq 'status=missing' <<<"${output}" +run_helper 1 env FAKE_MISSING_DESTINATION=KEYCLOAK_BRIO_STAGING_DB_PASSWORD \ + "${helper}" --check --environment staging-brio-identity-db +grep -Fq 'name=KEYCLOAK_BRIO_STAGING_DB_PASSWORD requirement=required status=missing' <<<"${output}" +run_helper 2 env FAKE_UNEXPECTED_DESTINATION=LEGACY_POSTGRES_SECRET \ + "${helper}" --check --environment staging-brio-identity-db +grep -Fq 'status=legacy-or-unmanaged' <<<"${output}" +run_helper 2 env FAKE_REPOSITORY_DESTINATION=LEGACY_RUNNER_SECRET \ + "${helper}" --check --environment staging-brio-identity-db +grep -Fq 'scope=repository kind=secret' <<<"${output}" + +run_helper 1 env FAKE_REPOSITORY_INVALID=1 "${helper}" --check --environment canary +grep -Fq 'policy=identity-or-visibility-invalid' <<<"${output}" +run_helper 1 env FAKE_MAIN_POLICY_INVALID=1 "${helper}" --check --environment canary +grep -Fq 'policy=main-protection-invalid' <<<"${output}" +run_helper 1 env FAKE_ACTIONS_POLICY_INVALID=1 "${helper}" --check --environment canary +grep -Fq 'policy=actions-token-policy-invalid' <<<"${output}" +run_helper 1 env FAKE_ENVIRONMENT_POLICY_INVALID=reviewer \ + "${helper}" --check --environment staging-brio-identity-db +grep -Fq 'protection=invalid' <<<"${output}" +run_helper 1 env FAKE_BRANCH_POLICY_INVALID=1 "${helper}" --check --environment canary +grep -Fq 'protection=invalid-branch-policy' <<<"${output}" + +run_helper 1 env FAKE_MISSING_FIELD=KEYCLOAK_BRIO_STAGING_DB_PASSWORD "${helper}" --sync \ + --environment staging-brio-identity-db --confirm Makepad-fr/postgres:staging-brio-identity-db +grep -Fq 'required Proton field is missing or unreadable' <<<"${output}" +if grep -Fq 'gh-set ' "${audit_log}"; then + echo 'sync wrote before source preflight completed' >&2 + exit 1 +fi + +run_helper 1 env FAKE_DRIFT_FIELD=KEYCLOAK_BRIO_STAGING_DB_PASSWORD FAKE_DRIFT_ON_CALL=2 \ + "${helper}" --sync --environment staging-brio-identity-db \ + --confirm Makepad-fr/postgres:staging-brio-identity-db +grep -Fq 'Proton source changed during preflight' <<<"${output}" +if grep -Fq 'gh-set ' "${audit_log}"; then + echo 'sync wrote after source drift' >&2 + exit 1 +fi + +run_helper 1 env FAKE_ENVIRONMENT_ID_DRIFT_ON_CALL=2 "${helper}" --sync \ + --environment staging-brio-identity-db --confirm Makepad-fr/postgres:staging-brio-identity-db +grep -Fq 'protection=invalid' <<<"${output}" +if grep -Fq 'gh-set ' "${audit_log}"; then + echo 'sync wrote after environment identity drift' >&2 + exit 1 +fi + +run_helper 0 env FAKE_MISSING_DESTINATION=KEYCLOAK_BRIO_STAGING_DB_PASSWORD "${helper}" --sync \ + --environment staging-brio-identity-db --confirm Makepad-fr/postgres:staging-brio-identity-db +grep -Fq 'SYNC_COMPLETE repository=Makepad-fr/postgres vault=Makepad environment=staging-brio-identity-db' <<<"${output}" +expected_writes=$(jq '[.entries[] | select(.environment == "staging-brio-identity-db")] | length' "${inventory}") +actual_writes=$(grep -Fc 'gh-set ' "${audit_log}") +[[ "${actual_writes}" == "${expected_writes}" ]] + +run_helper 1 env FAKE_STALE_VARIABLE=BRIO_IDENTITY_DB_HOSTNAME "${helper}" --sync \ + --environment staging-brio-identity-db --confirm Makepad-fr/postgres:staging-brio-identity-db +grep -Fq 'GitHub variable readback differs from Proton' <<<"${output}" +if grep -Fq 'SYNC_COMPLETE' <<<"${output}"; then + echo 'failed variable readback claimed completion' >&2 + exit 1 +fi + +candidate_root=${test_root}/candidate +mkdir -m 0700 "${candidate_root}" +mkdir -m 0700 "${candidate_root}/deploy" "${candidate_root}/scripts" +cp "${inventory}" "${candidate_root}/deploy/credential-inventory.json" +cp "${helper}" "${candidate_root}/scripts/sync-github-environments.sh" +chmod 0755 "${candidate_root}/scripts/sync-github-environments.sh" +jq '.operatorEntries = [{"boundary":"runner"}]' "${inventory}" >"${candidate_root}/deploy/credential-inventory.json" +run_helper 1 "${candidate_root}/scripts/sync-github-environments.sh" --check --environment canary +grep -Fq 'credential inventory changed without review' <<<"${output}" +jq '(.repositoryPolicy.repositoryId) += 1' "${inventory}" >"${candidate_root}/deploy/credential-inventory.json" +run_helper 1 "${candidate_root}/scripts/sync-github-environments.sh" --check --environment canary +grep -Fq 'credential inventory changed without review' <<<"${output}" +jq '(.entries[0].destination) = "ADVERSARIAL_SECRET"' "${inventory}" >"${candidate_root}/deploy/credential-inventory.json" +run_helper 1 "${candidate_root}/scripts/sync-github-environments.sh" --check --environment canary +grep -Fq 'credential inventory changed without review' <<<"${output}" + +if rg -n 'JIT|just-in-time|ephemeral runner|runner group|GitHub App|OAuth App|POSTGRES_PR_CHECK|POSTGRES_CI_LAUNCHER' \ + "${inventory}" "${helper}"; then + echo 'credential sync regained runner or App authority' >&2 + exit 1 +fi + +echo 'PostgreSQL Proton-to-GitHub credential sync tests passed.' diff --git a/scripts/validate-postgres-config.sh b/scripts/validate-postgres-config.sh index bd5513d..255e92d 100755 --- a/scripts/validate-postgres-config.sh +++ b/scripts/validate-postgres-config.sh @@ -10,6 +10,7 @@ script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) repo_root=$(cd "${script_dir}/.." && pwd) REPO_ROOT="${repo_root}" python3 - <<'PY' +import json import os import re from pathlib import Path @@ -123,6 +124,18 @@ deployment_failure_fixture = read_required_text(deployment_failure_fixture_path, manual_deploy = manual_deploy_workflow + "\n" + remote_deploy + "\n" + canary_deploy ci_workflow = read_required_text(repo_root / ".github/workflows/ci.yml", "CI workflow") ci_runner = read_required_text(repo_root / "scripts/run-ci.sh", "CI suite runner") +credential_inventory = json.loads(read_required_text( + repo_root / "deploy/credential-inventory.json", "credential inventory" +)) +credential_sync = read_required_text( + repo_root / "scripts/sync-github-environments.sh", "credential sync helper" +) +credential_sync_test = read_required_text( + repo_root / "scripts/test-sync-github-environments.sh", "credential sync test" +) +credential_sync_docs = read_required_text( + repo_root / "docs/credential-sync.md", "credential sync documentation" +) normalized_readme = re.sub(r"\s+", " ", readme) for environment in ( @@ -642,60 +655,50 @@ for workflow_path in workflow_paths: f"Workflow {workflow_path.name} must not use a GitHub-hosted runner image.", ) -# Credential material is canonical in Proton Pass and may be mirrored only to -# the protected environment that consumes it. Validate each complete table row -# so a field cannot silently drift into a different environment or item. -credential_inventory = { - "Hetzner Database Server makepad": ( - ("canary", "production", "staging-brio-identity-db", "keycloak-cohort-restore"), - ( - "DEPLOY_SSH_HOST", "DEPLOY_SSH_PORT", "DEPLOY_SSH_USER", - "DEPLOY_SSH_PRIVATE_KEY", "DEPLOY_SSH_KNOWN_HOSTS", - "BRIO_IDENTITY_DB_DEPLOY_SSH_HOST", "BRIO_IDENTITY_DB_DEPLOY_SSH_PORT", - "BRIO_IDENTITY_DB_DEPLOY_SSH_USER", "BRIO_IDENTITY_DB_DEPLOY_SSH_PRIVATE_KEY", - "BRIO_IDENTITY_DB_DEPLOY_SSH_KNOWN_HOSTS", "KEYCLOAK_COHORT_DB_SSH_PRIVATE_KEY", - "KEYCLOAK_COHORT_DB_SSH_KNOWN_HOSTS", "KEYCLOAK_COHORT_DB_SSH_HOST", - "KEYCLOAK_COHORT_DB_SSH_PORT", "KEYCLOAK_COHORT_DB_SSH_USER", - ), - ), - "Brio Staging - PostgreSQL": ( - ("canary", "staging-brio-identity-db"), - ( - "POSTGRES_CANARY_SUPERUSER_PASSWORD", "BRIO_STAGING_DB_PASSWORD", - "BRIO_STAGING_BACKUP_DB_PASSWORD", "KEYCLOAK_BRIO_STAGING_DB_PASSWORD", - "KEYCLOAK_BRIO_STAGING_BACKUP_DB_PASSWORD", - ), - ), - "Brio Staging - PKI and Backup Keys": ( - ("canary", "staging-brio-identity-db"), - ( - "POSTGRES_CA_PEM", "POSTGRES_SERVER_CERT_PEM", "POSTGRES_SERVER_KEY_PEM", - "BRIO_BACKUP_RECIPIENT_CERT_PEM", - ), - ), - "PostgreSQL · Brio identity release orchestrator": ( - ("release-brio-identity-db",), - ("KEYCLOAK_RELEASE_ORCHESTRATOR_TOKEN",), - ), - "PostgreSQL · Keycloak cohort source reader": ( - ("keycloak-cohort-restore",), - ("KEYCLOAK_COHORT_SOURCE_TOKEN",), - ), - "Makepad Docker Hardened Images": ( - ("keycloak-cohort-restore",), - ("DOCKERHUB_USERNAME", "DOCKERHUB_PRO_PAT", "DHI_REGISTRY_USERNAME", "DHI_REGISTRY_PASSWORD"), - ), +# Credential reconciliation may manage only the reviewed five existing +# environments. Its JSON mapping and adversarial test are the authoritative +# destination contract; runner/App/repository authority is intentionally empty. +require(set(credential_inventory) == { + "schemaVersion", "repository", "vault", "repositoryPolicy", + "operatorEntries", "preservedDestinations", "entries", +}, "Credential inventory has an unexpected top-level shape.") +require( + credential_inventory["schemaVersion"] == 3 + and credential_inventory["repository"] == "Makepad-fr/postgres" + and credential_inventory["vault"] == "Makepad", + "Credential inventory identity is invalid.", +) +require(credential_inventory["operatorEntries"] == [], "Credential inventory must not manage operator authority.") +expected_credential_environments = { + "canary", "production", "staging-brio-identity-db", + "release-brio-identity-db", "keycloak-cohort-restore", } -readme_lines = readme.splitlines() -for item, (environments, fields) in credential_inventory.items(): - candidate_rows = [line for line in readme_lines if line.startswith("|") and f"`{item}`" in line] - require(candidate_rows, f"README credential inventory is missing canonical Proton item {item}.") - require( - any(all(value in row for value in (*environments, *fields)) for row in candidate_rows), - f"README must map every field for {item} to its exact protected GitHub environment.", - ) -require("pass-cli item view --item-title '' --field ''" in readme, "README must document stdin-only pass-cli credential synchronization.") -require("| gh secret set '' --env '' --repo 'Makepad-fr/postgres'" in normalized_readme, "README must mirror workflow secrets only into protected GitHub environments.") +require( + set(credential_inventory["repositoryPolicy"]["environments"]) == expected_credential_environments, + "Credential inventory environment boundary changed.", +) +entries = credential_inventory["entries"] +require(len(entries) == 49, "Credential inventory entry count changed without review.") +managed_destinations = {(entry["environment"], entry["kind"], entry["destination"]) for entry in entries} +require(len(managed_destinations) == len(entries), "Credential inventory contains duplicate destinations.") +for environment in expected_credential_environments: + require(any(entry["environment"] == environment for entry in entries), f"Credential inventory omits {environment}.") +for required in ( + "--sync --environment NAME --confirm Makepad-fr/postgres:NAME", + "credential inventory must not manage operator, runner, or App authority", + "Proton source changed during preflight", + "GitHub variable readback differs from Proton", +): + require(required in credential_sync, f"Credential helper is missing boundary: {required}") +for forbidden in ( + "--sync-repository-variables", "configure-postgres-ci-runner-group", + "POSTGRES_PR_CHECK_APP", "POSTGRES_CI_LAUNCHER", +): + require(forbidden not in credential_sync, f"Credential helper regained forbidden authority: {forbidden}") +require("PostgreSQL Proton-to-GitHub credential sync tests passed." in credential_sync_test, "Credential sync adversarial suite is missing.") +require("./scripts/sync-github-environments.sh --check" in readme, "README must document read-only credential audit.") +require("--confirm Makepad-fr/postgres:staging-brio-identity-db" in readme, "README must document exact credential write confirmation.") +require("There is no all-environment write" in credential_sync_docs, "Credential docs must prohibit broad writes.") for workflow_path in workflow_paths: workflow_text = read_required_text(workflow_path, f"workflow {workflow_path.name}") for field in set(re.findall(r"(?:secrets|vars)\.([A-Z][A-Z0-9_]*)", workflow_text)):