From ff11bf5b9ac12d827f2223c64d44c8236aa6e070 Mon Sep 17 00:00:00 2001 From: bea-claude Date: Wed, 7 Oct 2026 05:21:41 +0200 Subject: [PATCH] =?UTF-8?q?feat(db):=20the=20estates'=20database=20gates?= =?UTF-8?q?=20=E2=80=94=20the=20migrations=20PR=20check=20and=20the=20nigh?= =?UTF-8?q?tly=20drift=20=E2=80=94=20as=20two=20reusable=20workflows=20(BE?= =?UTF-8?q?-232)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit mrkithq/monorepo and ironmountain-cz/monorepo each carried db-migrations-pr.yml (146 / 137 lines) and db-drift-env.yml (105 / 91); bepluscloud/ai had neither. Both now live here once and run the repository's own `beplus db` commands — @beplus/database-tools, resolved from the repository's install, never bundled in the CLI (D3) — over beplus.estate.json: - db-migrations-pr.yml: a pgvector service pinned to pgvector/pgvector:0.8.7-pg16 (the copies floated on pg16), the pinned CLI, `beplus npm auth` with the stub-200 registry check, rush install / build --to the database project, then `beplus db check`, `db check pending`, `db check chains`, `db check rebuild --compare` — each the command the copies' `npm run db:check:*` ran or now resolves to. BE_POSTGRES_URL is its own localhost service; BE_NPM_TOKEN is its only secret. - db-drift-env.yml: `beplus db diff migrations ` for each environment, then each environment against the next, aggregated so one drift does not hide another; scheduled + on demand stay in the caller. Connection strings stay secrets (BE_POSTGRES_URL_DEV / _STAGE / _PROD): no account has a database-read OIDC role yet, and `db diff` does not reach a private instance through the tunnel, so it assumes no AWS role — said in the header, with what it takes. A plan job derives what each copy wrote out: the rush project (database.package → its package.json name; `project` wins), the test database (naming.prefix + _test: mrkit_test, mcl_test, be_test), the environments (`environments`; the input wins — MRKIT passes `dev prod` until `db diff` knows stage). All validated before a step pastes them. `github-environment` says which GitHub environment holds the secrets when they are environment secrets (MRKIT: dev); empty claims none. The dump image is no longer set in the job: the repository's database.dumpImage, else the tools' pin — pgvector/pgvector:pg16, the value the copies set. The registry check is the `cli` action's, opt-in: BE_NPM_CHECK names package directories whose @beplus dependencies must come back as real packages after the auth. npm.beplus.cloud answers an unauthenticated read with HTTP 200 and {"status":200,"message":"OK"}; `npm view` prints nothing and exits 0, and pnpm fails minutes later with ERR_PNPM_TARBALL_INTEGRITY. The check fails the step first, naming the stub or npm's error (a 404). Locally against npm.beplus.cloud: with the token every dependency of the three estates' P3 database packages resolves but @beplus/database (unpublished → 404, as it must until 0.1.0 is out); anonymous → the stub, red. test-cli.yml gains the anonymous leg. scripts/verify-callers.mjs gains `db`: the shared jobs, resolved for an estate, against the copy's jobs — every key, every step — with the derivation not re-implemented but RUN (the plan step, on the estate's own manifest and package.json), and the drift job's guard and comparison, which are loops over the environments now, run side by side with the copies' (stub npm / beplus, every scenario that changes their output: 6 for the guard, 5 for the comparison) — same stdout, stderr, exit. Callers for MRKIT (dev), Mountaineer (origin/dev) and bepluscloud (origin/dev, gains both, with the database section BE-234 adds): 0 divergent — 12 identical, 6 identical once derived, 11 intended (CLI pin, the auth step's name in Mountaineer, `npm run db:check:*` → `beplus db check …`), 4 run side by side, 1 added (MRKIT's chains check, which its CI never ran), 1 removed (MRKIT's drift build: `db diff` reads no compiled schema). Eighteen mutations (an edited or dropped step, a remote BE_POSTGRES_URL, a connection secret in the PR gate, a migrate in the drift job, id-token, a wrong test database, a floating service image, a caller that drops a trigger, passes an undeclared input, renames its job, forgets its environment or lets stage in) all go red. infra-diff mode unchanged (30 steps, 0 divergent). The library mode no longer crashes: HEAD~1 compared each caller with itself once the callers had landed; it now takes the parent of the commit that made each file a caller ($BEFORE_REF still wins). ci: 12/12 identical; publish: 12 report library-publish's deliberate later changes (#8–#10). Co-Authored-By: Igor Lamos --- .github/workflows/db-drift-env.yml | 321 ++++++++++++ .github/workflows/db-migrations-pr.yml | 273 +++++++++++ .github/workflows/test-cli.yml | 20 +- README.md | 77 +++ cli/README.md | 36 ++ cli/action.yml | 79 +++ scripts/verify-callers.mjs | 650 ++++++++++++++++++++++++- test/cli/npm-check/package.json | 10 + 8 files changed, 1454 insertions(+), 12 deletions(-) create mode 100644 .github/workflows/db-drift-env.yml create mode 100644 .github/workflows/db-migrations-pr.yml create mode 100644 test/cli/npm-check/package.json diff --git a/.github/workflows/db-drift-env.yml b/.github/workflows/db-drift-env.yml new file mode 100644 index 0000000..9ea8edd --- /dev/null +++ b/.github/workflows/db-drift-env.yml @@ -0,0 +1,321 @@ +# Do the live databases still match the migrations? — once. +# --------------------------------------------------------- +# ironmountain-cz/monorepo (91 lines) and mrkithq/monorepo (105) each carried a +# copy of this file; bepluscloud/ai had none. It verifies that each environment's +# database structurally matches the committed migrations (drizzle/schema.sql) and +# the next environment in the promotion order. SEPARATE from the pull-request +# gate (db-migrations-pr.yml) because it needs connection secrets — it never +# runs on a pull request. +# +# The comparison is the repository's own `beplus db diff` (@beplus/database-tools, +# resolved from the repository's install), where the copies ran `npm run db:diff` +# over a `bin/db-diff.ts` copy of their own; it prints what that printed, line +# for line (BE-231's golden harness is the proof). What the copies wrote out by +# hand comes from beplus.estate.json: +# +# the rush project → `database.package` (default packages/modules/database) +# and its package.json `name`; the `project` input wins +# the environments dev, prod (each copy's list) +# → beplus.estate.json `environments`; the `environments` +# input wins (MRKIT passes `dev prod` until its stage +# database exists) +# +# and the guard and the drift step loop over that list. For the copies' two +# environments they print exactly what the copies printed and exit as they did — +# `scripts/verify-callers.mjs db` runs both side by side to prove it. Every other +# step body is byte-identical, bar the CLI version — the copies installed +# `latest`; empty now installs the repository's pin (beplus.estate.json → +# cli.version, `be auto`; BE-213), and `vars.BE_CLI_VERSION` still wins when it +# is set — and the registry check after `beplus npm auth` (`BE_NPM_CHECK`). The +# copy's `rush build` is gone: `beplus db diff` builds nothing it reads. +# +# Connection strings (read-capable): BE_POSTGRES_URL_DEV, BE_POSTGRES_URL_STAGE, +# BE_POSTGRES_URL_PROD — secrets, as in the copies. Repository secrets, or +# environment secrets of the environment named in `github-environment` (MRKIT's +# live on `dev`, beside BE_NPM_TOKEN). The diff prints only masked URLs; raw +# connection strings are never logged. A missing one fails the guard loudly +# rather than reporting "no drift". +# +# The trigger stays in the repository — the nightly schedule and the manual run: +# +# name: DB Drift (dev/prod) +# on: +# workflow_dispatch: +# schedule: +# - cron: "0 6 * * *" # 06:00 UTC nightly +# permissions: +# contents: read +# concurrency: +# group: db-drift-env-${{ github.workflow }} +# cancel-in-progress: false +# jobs: +# drift: +# uses: beplus/setup-beplus/.github/workflows/db-drift-env.yml@v2 +# secrets: inherit +# +# It only reads: `beplus db diff` dumps the schemas (pg_dump, structure only) and +# compares them. Nothing here migrates, writes or connects with more than the +# secrets' read-capable roles. +# +# NO AWS ROLE, YET. The connection strings are secrets because nothing else +# reaches a database from GitHub today: no account has a database-read OIDC +# role (GitHubOIDCStack trusts cdk-diff, cdk-deploy and provisioning only), and +# `beplus db diff` reads BE_POSTGRES_URL_ — it does not open the ECS-Exec +# tunnel `beplus db migrate --env` uses to reach a private instance. Once both +# exist (the GitHub OIDC construct in @beplus/cdk-estate, and `db diff` through +# the tunnel), this job assumes that read role per environment instead, and the +# secrets go. +# +# See cli_v2 EXTRACTION-PLAN.md, slice R2-4, and Linear BE-232. + +name: DB Drift + +on: + workflow_call: + inputs: + environments: + description: >- + The environments whose databases are compared — with the committed + migrations, and each with the next — space- or comma-separated, in + promotion order (e.g. `dev prod`). Each reads BE_POSTGRES_URL_; dev, + stage and prod are mapped. Default: beplus.estate.json → `environments`. + required: false + type: string + default: "" + project: + description: >- + The Rush project of the database package — what `rush install --to` + names. Default: the package.json `name` of beplus.estate.json → + `database.package` (default packages/modules/database). + required: false + type: string + default: "" + github-environment: + description: >- + The GitHub environment the drift job claims, when the connection + strings and BE_NPM_TOKEN are environment secrets there (MRKIT: `dev`). + Empty (the default): none — they are repository secrets. + required: false + type: string + default: "" + secrets: + BE_NPM_TOKEN: + description: npm.beplus.cloud token for the @beplus scope. + required: false + BE_POSTGRES_URL_DEV: + description: A read-capable connection string for dev's database. + required: false + BE_POSTGRES_URL_STAGE: + description: A read-capable connection string for stage's database. + required: false + BE_POSTGRES_URL_PROD: + description: A read-capable connection string for prod's database. + required: false + +permissions: + contents: read + +jobs: + # What the copies wrote out by hand, read from the manifest that already + # holds it — validated here, because it comes from the repository's tree. + plan: + name: plan + runs-on: ubuntu-latest + outputs: + package: ${{ steps.database.outputs.package }} + project: ${{ steps.database.outputs.project }} + environments: ${{ steps.database.outputs.environments }} + label: ${{ steps.database.outputs.label }} + summary: ${{ steps.database.outputs.summary }} + steps: + - uses: actions/checkout@v4 + + - name: Read the database package and the environments from the manifest + id: database + env: + INPUT_PROJECT: ${{ inputs.project }} + INPUT_ENVIRONMENTS: ${{ inputs.environments }} + run: | + set -euo pipefail + manifest=beplus.estate.json + if [ ! -f "$manifest" ]; then + echo "::error::No beplus.estate.json at the repository root: its \`database\` section says which package the comparison runs in." + exit 1 + fi + + # The same default `beplus db` applies (cli-plugin-database's section). + package="$(jq -r '.database.package // "packages/modules/database"' "$manifest")" + if ! [[ "$package" =~ ^[A-Za-z0-9_][A-Za-z0-9._-]*(/[A-Za-z0-9_][A-Za-z0-9._-]*)*$ ]] || [[ "/$package/" == */../* ]]; then + echo "::error file=$manifest::database.package \"$package\" is not a relative directory in this repository." + exit 1 + fi + if [ ! -f "$package/package.json" ]; then + echo "::error file=$manifest::database.package is \"$package\", which has no package.json — set it to the directory of the repository's database package." + exit 1 + fi + + project="${INPUT_PROJECT:-$(jq -r '.name // empty' "$package/package.json")}" + if ! [[ "$project" =~ ^(@[a-z0-9][a-z0-9._-]*/)?[a-z0-9][a-z0-9._-]*$ ]]; then + if [ -n "$INPUT_PROJECT" ]; then + echo "::error::the \`project\` input: \"$project\" is not a Rush project name." + else + echo "::error file=$package/package.json::\"$project\" is not a Rush project name." + fi + exit 1 + fi + + if [ -n "$INPUT_ENVIRONMENTS" ]; then + list="$(printf '%s' "$INPUT_ENVIRONMENTS" | tr ',' ' ')" + source="::error::the \`environments\` input:" + else + list="$(jq -r '(.environments // []) | map(tostring) | join(" ")' "$manifest")" + source="::error file=$manifest::environments:" + fi + environments="" + for env in $list; do + # The job maps these three connection strings and no others. + case "$env" in + dev | stage | prod) ;; + *) + echo "$source \"$env\" — the drift job reads BE_POSTGRES_URL_DEV, _STAGE and _PROD, and no connection string for \"$env\"." + exit 1 + ;; + esac + case " $environments " in *" $env "*) continue ;; esac + environments="${environments:+$environments }$env" + done + if [ -z "$environments" ]; then + echo "::error file=$manifest::No environment to compare: pass \`environments\`, or list them in beplus.estate.json → environments." + exit 1 + fi + + # "dev vs prod" and "migrations, dev, and prod are structurally 1:1:1", + # the copies' words, for any number of environments. + read -r -a envs <<< "$environments" + label="${environments// / vs }" + ratio="1" + for env in "${envs[@]}"; do ratio="$ratio:1"; done + if [ "${#envs[@]}" -eq 1 ]; then + summary="migrations and ${envs[0]} are structurally $ratio" + else + summary="migrations" + for env in "${envs[@]:0:${#envs[@]}-1}"; do summary="$summary, $env"; done + summary="$summary, and ${envs[-1]} are structurally $ratio" + fi + + echo "database package $package ($project); environments: $environments" + { + echo "package=$package" + echo "project=$project" + echo "environments=$environments" + echo "label=$label" + echo "summary=$summary" + } >> "$GITHUB_OUTPUT" + + drift: + name: structural drift (migrations vs ${{ needs.plan.outputs.label }}) + needs: plan + runs-on: ubuntu-latest + # Where the connection strings and BE_NPM_TOKEN are environment secrets + # (MRKIT: `dev`), the job claims that environment to see them. Empty: none. + environment: ${{ inputs.github-environment }} + timeout-minutes: 20 + + env: + BE_POSTGRES_URL_DEV: ${{ secrets.BE_POSTGRES_URL_DEV }} + BE_POSTGRES_URL_STAGE: ${{ secrets.BE_POSTGRES_URL_STAGE }} + BE_POSTGRES_URL_PROD: ${{ secrets.BE_POSTGRES_URL_PROD }} + # A pinned pg16 client via docker. WHICH image is the repository's: + # beplus.estate.json → database.dumpImage, else @beplus/database-tools' pin + # (pgvector/pgvector:pg16) — so it is not written here. + BE_PGDUMP_ENGINE: docker + + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-node@v6 + with: + node-version: 22.14 + + - name: Cache Rush and pnpm + uses: actions/cache@v4 + with: + path: | + common/temp + key: ${{ runner.os }}-${{ runner.arch }}-rush-${{ hashFiles('common/config/rush/pnpm-lock.yaml', 'rush.json') }} + restore-keys: | + ${{ runner.os }}-${{ runner.arch }}-rush- + + # @beplus/* lives on npm.beplus.cloud; see db-migrations-pr.yml for why + # the scope has to be authenticated before install. The same step installs + # the beplus CLI the repository pins (beplus.estate.json → cli.version). + - name: Authenticate the @beplus scope + uses: beplus/setup-beplus/cli@v2 + env: + BE_NPM_TARGET: ${{ vars.BE_NPM_TARGET }} + BE_NPM_TOKEN: ${{ secrets.BE_NPM_TOKEN }} + with: + BE_CLI_VERSION: ${{ vars.BE_CLI_VERSION }} + BE_NPM_AUTH: true + # ⚠️ AND PROVE IT WORKED. npm.beplus.cloud answers a read it cannot + # authenticate with HTTP 200 and a stub body, never 401, so a token + # that did not reach this job fails nothing until pnpm, minutes later, + # reports ERR_PNPM_TARBALL_INTEGRITY, then ERR_PNPM_NO_VERSIONS. This + # asks the registry for every @beplus package the database package + # depends on, and fails here, naming the cause. + BE_NPM_CHECK: ${{ needs.plan.outputs.package }} + + # Every environment's connection string, or nothing runs: a missing one is + # a failure, never "no drift". The message names all of them, as the + # copies' did. + - name: Guard — secrets present + id: guard + env: + ENVIRONMENTS: ${{ needs.plan.outputs.environments }} + run: | + missing=0 + names="" + for env in $ENVIRONMENTS; do + variable="BE_POSTGRES_URL_$(printf '%s' "$env" | tr '[:lower:]' '[:upper:]')" + names="${names:+$names / }$variable" + [ -n "${!variable:-}" ] || missing=1 + done + if [ "$missing" -ne 0 ]; then + echo "::error::$names secrets are not set" + exit 1 + fi + + # `beplus db diff` needs the repository's @beplus/database-tools installed — + # and nothing built: it reads the committed schema file and the databases, + # no compiled schema (MRKIT's copy built anyway; Mountaineer's did not). + - name: Rush install + run: node common/scripts/install-run-rush.js install --to ${{ needs.plan.outputs.project }} + + # Run every comparison and aggregate, so one failure doesn't hide the others. + # migrations = committed drizzle/schema.sql (canonical "what migrations produce"); + # then each environment against the next one in the promotion order. + - name: Structural drift (migrations vs ${{ needs.plan.outputs.label }}) + id: drift + working-directory: ${{ needs.plan.outputs.package }} + env: + ENVIRONMENTS: ${{ needs.plan.outputs.environments }} + SUMMARY: ${{ needs.plan.outputs.summary }} + run: | + set +e + rc=0 + for env in $ENVIRONMENTS; do + echo "::group::migrations vs $env"; beplus db diff migrations "$env"; [ $? -ne 0 ] && rc=1; echo "::endgroup::" + done + previous="" + for env in $ENVIRONMENTS; do + if [ -n "$previous" ]; then + echo "::group::$previous vs $env"; beplus db diff "$previous" "$env"; [ $? -ne 0 ] && rc=1; echo "::endgroup::" + fi + previous="$env" + done + if [ $rc -ne 0 ]; then + echo "::error::Structural drift detected — see the grouped diffs above" + else + echo "✓ $SUMMARY" + fi + exit $rc diff --git a/.github/workflows/db-migrations-pr.yml b/.github/workflows/db-migrations-pr.yml new file mode 100644 index 0000000..410d674 --- /dev/null +++ b/.github/workflows/db-migrations-pr.yml @@ -0,0 +1,273 @@ +# The pull-request gate for a database's migrations — once. +# --------------------------------------------------------- +# mrkithq/monorepo (146 lines) and ironmountain-cz/monorepo (137) each carried +# a copy of this file; bepluscloud/ai had none. It catches the classes of +# breakage a migrations-only, never-push database is exposed to: +# 1. migration metadata corruption / out-of-order `when` (the silent-skip hazard) +# 2. a TypeScript schema change committed WITHOUT its migration +# 3. an embedded chain (@beplus/ai-db's beplus_ai, @beplus/analytics-db's +# analytics) that no longer resolves, or whose journal is unsound +# 4. a clean database that can no longer be rebuilt from the committed +# migrations, or whose structure no longer matches drizzle/schema.sql +# +# The checks are the repository's own `beplus db` commands (@beplus/database-tools, +# resolved from the repository's install, never bundled in the CLI), where the +# copies ran `npm run db:check:*` scripts over `bin/db-*.ts` copies of their own. +# They print what those scripts printed, line for line (@beplus/database-tools +# holds that parity; BE-231's golden harness proves it), and they read what used +# to be written out per repository from beplus.estate.json: +# +# the rush project @mrkit/database / @mountaineer/database / @bepluscloud/database +# → `database.package` (default packages/modules/database) +# and its package.json `name`; the `project` input wins +# the test database mrkit_test / mcl_test / be_test +# → `naming.prefix`, lower-cased, + `_test` +# the chains → the product's own migrator declaration (`db check chains`) +# +# Every other step body is byte-identical to the copies, bar three deliberate +# changes: the CLI version — `vars.BE_CLI_VERSION || '2'` floated on the newest +# 2.x; empty now installs the repository's pin (beplus.estate.json → +# cli.version, `be auto`; BE-213), and the variable still wins when it is set — +# the registry check after `beplus npm auth` (`BE_NPM_CHECK`), and the Postgres +# service pinned to pgvector 0.8.7 / pg16 instead of the floating `pg16`. The pin +# must be a CLI that bundles @beplus/cli-plugin-database with +# `db check pending | rebuild | chains`. `scripts/verify-callers.mjs db` proves +# all of it, step for step. +# +# What stays in the repository is what is the repository's: when it runs (the +# paths), the permissions it grants, the concurrency group — and, when its +# BE_NPM_TOKEN is an environment secret, which environment holds it: +# +# name: DB Migrations (PR-safe) +# on: +# pull_request: +# paths: ["packages/modules/database/drizzle/**", …, ".github/workflows/db-migrations-pr.yml"] +# permissions: +# contents: read +# concurrency: +# group: db-migrations-pr-${{ github.workflow }}-${{ github.head_ref || github.ref }} +# cancel-in-progress: true +# jobs: +# checks: +# uses: beplus/setup-beplus/.github/workflows/db-migrations-pr.yml@v2 +# with: +# github-environment: dev # only where BE_NPM_TOKEN is an environment secret +# secrets: inherit +# +# It reaches NO database but the Postgres service it starts: BE_POSTGRES_URL is +# localhost, no connection secret is read, and the only secret it uses is +# BE_NPM_TOKEN, for the private registry. It needs no AWS credentials. +# +# See cli_v2 EXTRACTION-PLAN.md, slice R2-4, and Linear BE-232. + +name: DB Migrations (PR-safe) + +on: + workflow_call: + inputs: + project: + description: >- + The Rush project of the database package — what `rush install --to` and + `rush build --to` name. Default: the package.json `name` of + beplus.estate.json → `database.package` (default + packages/modules/database), the package every `beplus db` command reads. + required: false + type: string + default: "" + github-environment: + description: >- + The GitHub environment the checks job claims, when BE_NPM_TOKEN is an + environment secret there (MRKIT: `dev`). Its branch policy must then + admit every branch this gate runs on. Empty (the default): none — the + token is a repository or organization secret. + required: false + type: string + default: "" + secrets: + BE_NPM_TOKEN: + description: >- + npm.beplus.cloud token for the @beplus scope. `secrets: inherit` (or, for + an environment secret, `github-environment`) is all a caller passes. + required: false + +permissions: + contents: read + +jobs: + # What the copies wrote out by hand, read from the manifest that already + # holds it. Both values come from the pull request's own tree, so both are + # validated here before any step pastes them. + plan: + name: plan + runs-on: ubuntu-latest + outputs: + package: ${{ steps.database.outputs.package }} + project: ${{ steps.database.outputs.project }} + database: ${{ steps.database.outputs.database }} + steps: + - uses: actions/checkout@v4 + + - name: Read the database package from the manifest + id: database + env: + INPUT_PROJECT: ${{ inputs.project }} + run: | + set -euo pipefail + manifest=beplus.estate.json + if [ ! -f "$manifest" ]; then + echo "::error::No beplus.estate.json at the repository root: its \`database\` section says which package the checks run over." + exit 1 + fi + + # The same default `beplus db` applies (cli-plugin-database's section). + package="$(jq -r '.database.package // "packages/modules/database"' "$manifest")" + if ! [[ "$package" =~ ^[A-Za-z0-9_][A-Za-z0-9._-]*(/[A-Za-z0-9_][A-Za-z0-9._-]*)*$ ]] || [[ "/$package/" == */../* ]]; then + echo "::error file=$manifest::database.package \"$package\" is not a relative directory in this repository." + exit 1 + fi + if [ ! -f "$package/package.json" ]; then + echo "::error file=$manifest::database.package is \"$package\", which has no package.json — set it to the directory of the repository's database package." + exit 1 + fi + + project="${INPUT_PROJECT:-$(jq -r '.name // empty' "$package/package.json")}" + if ! [[ "$project" =~ ^(@[a-z0-9][a-z0-9._-]*/)?[a-z0-9][a-z0-9._-]*$ ]]; then + if [ -n "$INPUT_PROJECT" ]; then + echo "::error::the \`project\` input: \"$project\" is not a Rush project name." + else + echo "::error file=$package/package.json::\"$project\" is not a Rush project name." + fi + exit 1 + fi + + # mrkit_test, mcl_test, be_test — the names the copies gave it. + prefix="$(jq -r '.naming.prefix // empty' "$manifest")" + database="$(printf '%s' "${prefix:-${GITHUB_REPOSITORY#*/}}" | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9_' '_')_test" + if ! [[ "$database" =~ ^[a-z_][a-z0-9_]{0,62}$ ]]; then + echo "::error file=$manifest::naming.prefix gives the test database \"$database\", which is not a Postgres identifier." + exit 1 + fi + + echo "database package $package ($project), test database $database" + { + echo "package=$package" + echo "project=$project" + echo "database=$database" + } >> "$GITHUB_OUTPUT" + + checks: + name: migration integrity + rebuild + needs: plan + runs-on: ubuntu-latest + # Where BE_NPM_TOKEN is an environment secret (MRKIT: `dev`) the job has to + # claim the environment to see it, and the environment's branch policy must + # then admit every branch this gate runs on — it has no `branches` filter + # by design. Empty: no environment. + environment: ${{ inputs.github-environment }} + timeout-minutes: 25 + + # pgvector image is required — the baseline migration runs CREATE EXTENSION + # vector. Pinned, not `pg16`: the server a rebuild runs on must not change + # under a pull request (the test kit, @beplus/database/testing, pins the same). + services: + postgres: + image: pgvector/pgvector:0.8.7-pg16 + env: + POSTGRES_PASSWORD: postgres + POSTGRES_DB: ${{ needs.plan.outputs.database }} + ports: + - 5432:5432 + options: >- + --health-cmd "pg_isready -U postgres" + --health-interval 5s + --health-timeout 5s + --health-retries 20 + + env: + BE_POSTGRES_URL: postgres://postgres:postgres@localhost:5432/${{ needs.plan.outputs.database }} + # A pinned pg16 client via docker, so dumps match the committed + # drizzle/schema.sql byte-for-byte regardless of the runner's installed + # pg_dump. WHICH image is the repository's: beplus.estate.json → + # database.dumpImage, else @beplus/database-tools' pin + # (pgvector/pgvector:pg16) — the same choice a laptop's `beplus db` makes, + # so it is not written here (BE_PGDUMP_IMAGE would outrank the manifest). + BE_PGDUMP_ENGINE: docker + + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-node@v6 + with: + node-version: 22.14 + + - name: Cache Rush and pnpm + uses: actions/cache@v4 + with: + path: | + common/temp + key: ${{ runner.os }}-${{ runner.arch }}-rush-${{ hashFiles('common/config/rush/pnpm-lock.yaml', 'rush.json') }} + restore-keys: | + ${{ runner.os }}-${{ runner.arch }}-rush- + + # @beplus/* lives on npm.beplus.cloud; this writes the token into + # ~/.npmrc. The same step installs the beplus CLI the repository pins + # (beplus.estate.json → cli.version) — a v2 CLI, which `beplus db check` + # needs; `vars.BE_CLI_VERSION` still wins when it is set. + - name: Authenticate the @beplus scope + uses: beplus/setup-beplus/cli@v2 + env: + BE_NPM_TARGET: ${{ vars.BE_NPM_TARGET }} + BE_NPM_TOKEN: ${{ secrets.BE_NPM_TOKEN }} + with: + BE_CLI_VERSION: ${{ vars.BE_CLI_VERSION }} + BE_NPM_AUTH: true + # ⚠️ AND PROVE IT WORKED. npm.beplus.cloud answers a read it cannot + # authenticate with HTTP 200 and a stub body, never 401, so a token + # that did not reach this job fails nothing until pnpm, minutes later, + # reports ERR_PNPM_TARBALL_INTEGRITY, then ERR_PNPM_NO_VERSIONS. This + # asks the registry for every @beplus package the database package + # depends on, and fails here, naming the cause. + BE_NPM_CHECK: ${{ needs.plan.outputs.package }} + + - name: Rush install + run: node common/scripts/install-run-rush.js install --to ${{ needs.plan.outputs.project }} + + # ⚠️ INSTALL IS NOT ENOUGH. drizzle-kit reads dist/, including workspace + # dependencies' — Mountaineer's @beplus/postmark contributes tables. + # Without a build its dist/db is absent, drizzle-kit prints + # MODULE_NOT_FOUND, exits 0 anyway, and `db check pending` passes over a + # schema it never finished reading. MRKIT's copy always built; Mountaineer's + # did not, until it learnt this. + - name: Rush build + run: node common/scripts/install-run-rush.js build --to ${{ needs.plan.outputs.project }} + + # 1) Pure file check — journal/snapshot integrity + the `when`-ordering guard + # that catches a freshly-merged migration whose `when` slipped below an + # earlier one (the silent-skip hazard). No database needed. + # (The copies' `npm run db:check:collisions` was exactly this command.) + - name: Migration collision + ordering check + working-directory: ${{ needs.plan.outputs.package }} + run: beplus db check + + # 2) The TS schema must not have changes that lack a committed migration. + - name: No pending (uncommitted) migration + working-directory: ${{ needs.plan.outputs.package }} + run: beplus db check pending + + # 3) Every chain the product's migrator declaration embeds — @beplus/ai-db's + # beplus_ai, @beplus/analytics-db's analytics — resolves, and its + # journal is sound. Mountaineer's copy checked beplus_ai here; MRKIT's + # checked neither of its two (only its local `db:check` did). + - name: Embedded migration chains check + id: chains + working-directory: ${{ needs.plan.outputs.package }} + run: beplus db check chains + + # 4) A clean DB must rebuild from the migrations (every chain), AND the + # rebuilt structure must match the committed drizzle/schema.sql. + # If this fails only because the pinned pg16 image bumped minor versions, + # regenerate it: `beplus db migrate --url `, then + # `beplus db schema dump --url --write`. + - name: Rebuild from migrations + compare schema.sql + working-directory: ${{ needs.plan.outputs.package }} + run: beplus db check rebuild --compare diff --git a/.github/workflows/test-cli.yml b/.github/workflows/test-cli.yml index f380483..dd33260 100644 --- a/.github/workflows/test-cli.yml +++ b/.github/workflows/test-cli.yml @@ -12,6 +12,10 @@ # # `be` itself comes from beplus/be@main, so the pinned legs go green once # `be auto` reads beplus.estate.json there. +# +# BE_NPM_CHECK: with no token the auth is skipped and npm.beplus.cloud answers +# the @beplus read with HTTP 200 and its stub body, so the check must fail the +# run — before any install would have. name: Test cli on: @@ -60,6 +64,10 @@ jobs: - case: an inexact pin fails the run dir: test/cli/inexact expect: failure + - case: the registry check fails on the anonymous stub + dir: test/cli/pinned + npm-check: test/cli/npm-check + expect: failure steps: - uses: actions/checkout@v4 @@ -69,6 +77,11 @@ jobs: - name: Make a package directory below the pinned manifest run: mkdir -p test/cli/pinned/packages/app + # The scope mapped, no token: what a job that BE_NPM_TOKEN never reached sees. + - name: Map @beplus without a token + if: ${{ matrix.npm-check }} + run: echo "@beplus:registry=https://npm.beplus.cloud/" >> ~/.npmrc + - name: Setup beplus CLI id: setup uses: ./cli @@ -77,10 +90,13 @@ jobs: BE_CLI_VERSION: ${{ matrix.version }} # Empty `dir` = a directory with no beplus.estate.json at or above it. BE_WORKING_DIRECTORY: ${{ matrix.dir || runner.temp }} + BE_NPM_AUTH: ${{ matrix.npm-check && 'true' || 'false' }} + BE_NPM_CHECK: ${{ matrix.npm-check }} - name: Assert the installed version shell: bash env: + CASE: ${{ matrix.case }} EXPECT: ${{ matrix.expect }} OUTCOME: ${{ steps.setup.outcome }} run: | @@ -88,10 +104,10 @@ jobs: if [[ "$EXPECT" == failure ]]; then if [[ "$OUTCOME" != failure ]]; then - echo "::error::The action accepted a pin that is not an exact version (outcome: $OUTCOME)." + echo "::error::The action did not fail — $CASE (outcome: $OUTCOME)." exit 1 fi - echo "The action refused the inexact pin, as it must." + echo "The action failed, as it must: $CASE." exit 0 fi diff --git a/README.md b/README.md index abbba20..6e0d6c0 100644 --- a/README.md +++ b/README.md @@ -133,6 +133,83 @@ CLI the repository pins (`cli.version`). `node scripts/verify-callers.mjs infra- --repo ` proves a caller plus this workflow reproduce the file it replaced, step for step. +An estate's database gates are two more. MRKIT and Mountaineer each carried +`db-migrations-pr.yml` (146 / 137 lines) and `db-drift-env.yml` (105 / 91); +bepluscloud had neither. Both now run the repository's own `beplus db` commands — +`@beplus/database-tools`, resolved from the repository's install, never bundled +in the CLI — over `beplus.estate.json`'s `database` section: + +- **`db-migrations-pr.yml`** — on a pull request: a pgvector Postgres service + (`pgvector/pgvector:0.8.7-pg16`), the pinned CLI, `beplus npm auth` with the + stub-200 registry check (`BE_NPM_CHECK`, below), `rush install` / `rush build + --to `, then `beplus db check`, `db check pending`, + `db check chains` and `db check rebuild --compare`. It reaches no database but + its own service. +- **`db-drift-env.yml`** — nightly and on demand: `beplus db diff migrations + ` for each environment, then each environment against the next, all + aggregated so one drift does not hide another. Connection strings are the + secrets `BE_POSTGRES_URL_DEV` / `_STAGE` / `_PROD`, as before; there is no + database-read OIDC role in any account yet, nor a `db diff` through the tunnel, + so it assumes no AWS role. + +The rush project (`database.package` → its package.json `name`), the test +database (`naming.prefix` + `_test`) and the environments (`environments`) come +from the manifest; the inputs are for what cannot: + +| Input | Workflow | Default | | +|---|---|---|---| +| `github-environment` | both | none | The GitHub environment the job claims, when `BE_NPM_TOKEN` (and the connection strings) are environment secrets there — MRKIT: `dev` | +| `environments` | drift | `beplus.estate.json` → `environments` | e.g. `dev prod`; dev, stage and prod are mapped | +| `project` | both | `database.package`'s package.json `name` | The Rush project to install (and build) | + +```yaml +# .github/workflows/db-migrations-pr.yml — keep your own triggers +name: DB Migrations (PR-safe) +on: + pull_request: + paths: + - "packages/modules/database/drizzle/**" + - ".github/workflows/db-migrations-pr.yml" + # … the schema sources, drizzle.config.ts, package.json +permissions: + contents: read +concurrency: + group: db-migrations-pr-${{ github.workflow }}-${{ github.head_ref || github.ref }} + cancel-in-progress: true +jobs: + checks: + uses: beplus/setup-beplus/.github/workflows/db-migrations-pr.yml@v2 + secrets: inherit +``` + +```yaml +# .github/workflows/db-drift-env.yml +name: DB Drift (dev/prod) +on: + workflow_dispatch: + schedule: + - cron: "0 6 * * *" +permissions: + contents: read +concurrency: + group: db-drift-env-${{ github.workflow }} + cancel-in-progress: false +jobs: + drift: + uses: beplus/setup-beplus/.github/workflows/db-drift-env.yml@v2 + secrets: inherit +``` + +Keep the job ids (`checks`, `drift`): the check is then named +`checks / migration integrity + rebuild` — a required status check that named +`migration integrity + rebuild` has to be renamed once. The pinned CLI must bundle +`@beplus/cli-plugin-database` with `db check pending | rebuild | chains` and the +repository must have installed `@beplus/database-tools`; `beplus db` refuses +cleanly otherwise. `node scripts/verify-callers.mjs db --repo ` proves a +caller plus these workflows reproduce the files it replaced: it runs the shared +plan step on the estate's own manifest, compares every job and step, and runs the +drift job's rewritten guard and comparison side by side with the copies'. + A repository's docs site — `@beplus/docs-site` from beplus/docs — is built and published to GitHub Pages by one more: diff --git a/cli/README.md b/cli/README.md index 9a87a56..bcb8e6c 100644 --- a/cli/README.md +++ b/cli/README.md @@ -13,6 +13,7 @@ Composite GitHub Action that installs [@beplus/be](https://github.com/beplus/be) - otherwise → `be latest`, as before, with a `::warning::` that the repository floats 3. Verifies `beplus --version` — and, for a pinned run, fails unless it is exactly the pinned version 4. Optionally runs `beplus npm auth` if enabled and required environment variables are present +5. Optionally (`BE_NPM_CHECK`) proves the registry then serves real `@beplus` packages — see [The registry never says no](#the-registry-never-says-no) --- @@ -22,6 +23,7 @@ Composite GitHub Action that installs [@beplus/be](https://github.com/beplus/be) |------|----------|---------|-------------| | `BE_CLI_VERSION` | No | empty | Version or channel of beplus CLI to install (passed to `be`). Empty installs the repository's pin (`be auto`), or `latest` with a warning when there is none. A value here wins over the pin. | | `BE_NPM_AUTH` | No | `false` | If `true`, runs `beplus npm auth` | +| `BE_NPM_CHECK` | No | empty | Package directories (space- or newline-separated, relative to the workspace) whose `@beplus` dependencies must come back from the registry as real packages. Needs `npm` and `jq`. Empty: no check. | | `BE_PREFIX` | No | empty | Install prefix for beplus binaries. If empty, defaults to `$HOME/.beplus`. Must be writable. | | `BE_WORKING_DIRECTORY` | No | the workspace | Where the pin is looked up: the nearest `beplus.estate.json` at or above this directory, exactly as `be auto` finds it. Relative to the workspace. | @@ -82,6 +84,40 @@ Required only when `BE_NPM_AUTH` is set to `"true"`: - `BE_NPM_AUTH == "true"` - both `BE_NPM_TARGET` and `BE_NPM_TOKEN` are present +Otherwise it is skipped, and says so in the log — which is not an error, and +that is the trap below. + +### The registry never says no + +npm.beplus.cloud answers a read it cannot authenticate with **HTTP 200 and a stub +body** (`{"status":200,"message":"OK"}`), never 401. So a token that did not +reach the job — an environment secret in a job that claims no environment, a +fork's pull request — fails nothing here: `npm view` prints nothing and exits 0, +and pnpm fails minutes later with `ERR_PNPM_TARBALL_INTEGRITY`, then +`ERR_PNPM_NO_VERSIONS`. + +`BE_NPM_CHECK` closes that gap. After the auth, it asks the registry for every +`@beplus` dependency of the packages you name (`dependencies`, +`devDependencies`, `optionalDependencies`; `workspace:` ones are skipped) — +through the registry Rush hands pnpm (`common/config/rush/.npmrc`), else npm's +own — and fails the step unless each comes back as a real package, naming the +cause: the stub (no token, or a refused one), or npm's own error (a 404 for a +package that is not published). + +```yaml +- uses: actions/setup-node@v6 + with: + node-version: 22.14 + +- uses: beplus/setup-beplus/cli@v2 + env: + BE_NPM_TARGET: ${{ vars.BE_NPM_TARGET }} + BE_NPM_TOKEN: ${{ secrets.BE_NPM_TOKEN }} + with: + BE_NPM_AUTH: true + BE_NPM_CHECK: packages/modules/database +``` + --- ## Usage diff --git a/cli/action.yml b/cli/action.yml index 42f5680..91a1089 100644 --- a/cli/action.yml +++ b/cli/action.yml @@ -13,6 +13,17 @@ inputs: description: "Authenticate beplus.cloud NPM? (requires BE_NPM_TARGET and BE_NPM_TOKEN env vars)" required: false default: "false" + BE_NPM_CHECK: + description: >- + Package directories (space- or newline-separated, relative to the workspace) + whose @beplus dependencies must come back from the registry as real packages + after `beplus npm auth`. npm.beplus.cloud answers a read it cannot + authenticate with HTTP 200 and a stub body — never 401 — so without this a + token that did not reach the job fails nothing until pnpm, minutes later, + reports ERR_PNPM_TARBALL_INTEGRITY. Needs npm and jq. Empty (the default): + no check. + required: false + default: "" BE_PREFIX: description: "Install prefix for beplus binaries (must be writable)." required: false @@ -163,3 +174,71 @@ runs: else echo "Skipping beplus npm auth (BE_NPM_AUTH=${{ inputs.BE_NPM_AUTH }} because one of (BE_NPM_TARGET,BE_NPM_TOKEN) is missing." fi + + # ⚠️ THE REGISTRY NEVER SAYS NO. npm.beplus.cloud answers a read it cannot + # authenticate with HTTP 200 and a stub body — {"status":200,"message":"OK"} + # — never 401, and the step above skips the auth (saying so in its log) when + # BE_NPM_TARGET or BE_NPM_TOKEN did not reach the job. `npm view` then prints + # nothing and exits 0; pnpm fails minutes later with ERR_PNPM_TARBALL_INTEGRITY, + # then ERR_PNPM_NO_VERSIONS. So ask, through the registry the install will use, + # for every @beplus package the named packages depend on, and fail here, + # naming the cause. + - name: Check the @beplus registry serves real packages + if: ${{ inputs.BE_NPM_CHECK != '' }} + shell: bash + working-directory: ${{ github.workspace }} + env: + CHECK: ${{ inputs.BE_NPM_CHECK }} + run: | + set -euo pipefail + command -v npm > /dev/null || { echo "::error::BE_NPM_CHECK needs npm on PATH (actions/setup-node first)."; exit 1; } + command -v jq > /dev/null || { echo "::error::BE_NPM_CHECK needs jq on PATH."; exit 1; } + + # The registry Rush hands pnpm for the scope; else npm's own (~/.npmrc, + # which `beplus npm auth` wrote). + registry="$(sed -n 's/^[[:space:]]*@beplus:registry[[:space:]]*=[[:space:]]*//p' common/config/rush/.npmrc 2> /dev/null | tail -n 1 | tr -d '[:space:]')" || registry="" + scope=() + if [[ "$registry" =~ ^https?://[^[:space:]\$]+$ ]]; then + scope=("--@beplus:registry=$registry") + else + registry="$(npm config get @beplus:registry)" + [ "$registry" != undefined ] || registry="npm's default registry (nothing maps @beplus)" + fi + + # Each package's direct @beplus dependencies. A workspace project is + # built, not fetched. + deps="" + for dir in $CHECK; do + if [ ! -f "$dir/package.json" ]; then + echo "::error::BE_NPM_CHECK: $dir has no package.json." + exit 1 + fi + deps+="$(jq -r ' + [.dependencies, .devDependencies, .optionalDependencies] + | map(objects | to_entries[]) | .[] + | select((.key | startswith("@beplus/")) and ((.value | tostring) | startswith("workspace:") | not)) + | .key' "$dir/package.json")"$'\n' + done + mapfile -t wanted < <(printf '%s' "$deps" | sed '/^$/d' | sort -u) + if [ "${#wanted[@]}" -eq 0 ]; then + echo "Nothing in @beplus to check ($CHECK)." + exit 0 + fi + + failed=0 + for pkg in "${wanted[@]}"; do + err="$(mktemp)" + rc=0 + name="$(npm view "$pkg" name "${scope[@]}" 2> "$err")" || rc=$? + if [ "$rc" -eq 0 ] && [ "$name" = "$pkg" ]; then + echo "✓ $pkg — a real package on $registry" + elif [ "$rc" -ne 0 ]; then + failed=1 + echo "::error::$pkg: $(grep -m 1 -E '^npm (error|ERR!) [0-9]{3} ' "$err" || grep -m 1 -E '^npm (error|ERR!)' "$err" || head -n 1 "$err")" + else + failed=1 + echo "::error::$pkg: $registry answered with its anonymous stub (HTTP 200, no package). BE_NPM_TOKEN did not reach this job, or the registry refused it — a repository or organization secret reaches it; an environment secret only reaches a job that claims that environment." + fi + rm -f "$err" + done + exit "$failed" diff --git a/scripts/verify-callers.mjs b/scripts/verify-callers.mjs index 4e5a174..517c0ec 100755 --- a/scripts/verify-callers.mjs +++ b/scripts/verify-callers.mjs @@ -1,13 +1,14 @@ #!/usr/bin/env node /** - * The caller verification gate (R2-1, R2-3). - * ------------------------------------------ + * The caller verification gate (R2-1, R2-3, R2-4). + * ------------------------------------------------- * A reusable workflow cannot be run locally, so this proves the next best thing: * that resolving the shared workflow WITH EACH CALLER'S INPUTS yields the same * enabled steps, in the same order, as the file that caller replaced. * - * The originals come from git — `HEAD~1` by default, or $BEFORE_REF — so this - * keeps working as a regression check across the commit that matters. + * The originals come from git — $BEFORE_REF, else (libraries) the parent of the + * commit that made each file a caller, else `HEAD~1` — so this keeps working as a + * regression check across the commit that matters. * * Usage: * @@ -26,21 +27,39 @@ * e.g. node scripts/verify-callers.mjs infra-diff \ * --repo ~/Git/github.com/bepluscloud/ai --ref origin/dev --caller /tmp/bepluscloud.yml * + * node scripts/verify-callers.mjs db --repo [--ref ] [--callers ] + * [--manifest ] [--repository ] [--repo …] + * Each estate's `.github/workflows/db-migrations-pr.yml` and `db-drift-env.yml` + * callers of the shared workflows of the same names (R2-4, BE-232). `--ref` + * is the commit whose copies the callers replaced (default $BEFORE_REF, else + * HEAD~1); a repository with no copy there (bepluscloud) GAINS the workflow, + * and only its caller and the manifest's derivation are checked. `--callers` + * is a directory holding callers not committed yet; the manifest and the + * database package are then read at `--ref` too, otherwise from the + * checkout. `--manifest` replaces beplus.estate.json (a section an adoption + * will add). Needs bash ≥ 4 and jq: the shared plan step and the rewritten + * guard / drift steps are RUN, side by side with the copies' steps. + * e.g. node scripts/verify-callers.mjs db \ + * --repo ~/Git/github.com/ironmountain-cz/monorepo --ref origin/dev --callers /tmp/db-callers/mountaineer + * * What it CANNOT prove: that the shared workflow runs green on GitHub. That * needs setup-beplus pushed and tagged, and one repository taken through a real * run. For the libraries do that with beplus/docs before the other eleven; for - * infra-diff, with one real pull request per estate, bepluscloud first. + * infra-diff and the db workflows, with one real pull request (one manual drift + * run) per estate, bepluscloud first. */ -import { execFileSync } from 'node:child_process'; +import { execFileSync, spawnSync } from 'node:child_process'; import { createRequire } from 'node:module'; -import { existsSync, readFileSync } from 'node:fs'; -import { join, resolve } from 'node:path'; +import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join, resolve } from 'node:path'; const HERE = new URL('..', import.meta.url).pathname; const usage = () => { console.error('usage: node scripts/verify-callers.mjs '); console.error(' node scripts/verify-callers.mjs infra-diff --repo [--ref ] [--caller ] [--repository ] [--repo …]'); + console.error(' node scripts/verify-callers.mjs db --repo [--ref ] [--callers ] [--manifest ] [--repository ] [--repo …]'); process.exit(2); }; @@ -100,14 +119,34 @@ const libraries = (ORG) => { .map((step) => step.name ?? `uses:${step.uses}`); }; + /** + * The commit whose file the caller replaced. `HEAD~1` was right only on the day + * each caller landed; every later commit made it compare a caller with a caller + * (and crash on its missing `steps`). So, unless $BEFORE_REF says otherwise: the + * parent of the oldest commit that wrote the `uses:` line in. + */ + const beforeRef = (dir, path, workflow) => { + if (process.env.BEFORE_REF) return process.env.BEFORE_REF; + const introduced = git(dir, 'log', '--format=%H', '-S', `setup-beplus/.github/workflows/${workflow}.yml`, '--', path) + .trim().split('\n').filter(Boolean).at(-1); + return introduced ? `${introduced}~1` : 'HEAD~1'; + }; + let failures = 0; for (const repo of REPOS) { for (const [file, workflow] of [['ci', 'library-ci'], ['publish', 'library-publish']]) { const path = `.github/workflows/${file}.yml`; - const before = YAML.parse(git(join(ORG, repo), 'show', `${process.env.BEFORE_REF ?? 'HEAD~1'}:${path}`)); + const ref = beforeRef(join(ORG, repo), path, workflow); + const before = YAML.parse(git(join(ORG, repo), 'show', `${ref}:${path}`)); const caller = YAML.parse(readFileSync(join(ORG, repo, path), 'utf8')); - const wanted = Object.values(before.jobs)[0].steps.map((s) => s.name ?? `uses:${s.uses}`); + const beforeSteps = Object.values(before.jobs)[0].steps; + if (!Array.isArray(beforeSteps)) { + failures += 1; + console.log(` ✗ ${repo}/${file} at ${ref} it is already a caller — set BEFORE_REF to the commit before the switch`); + continue; + } + const wanted = beforeSteps.map((s) => s.name ?? `uses:${s.uses}`); const got = enabledSteps(shared[workflow], Object.values(caller.jobs)[0].with ?? {}); if (JSON.stringify(wanted) === JSON.stringify(got)) { @@ -445,9 +484,600 @@ const infraDiff = (argv) => { process.exit(divergent === 0 ? 0 : 1); }; +// ═════════════════════════════════════════════════════════════════════════════ +// R2-4 — db-migrations-pr.yml and db-drift-env.yml in the estates (BE-232) +// ═════════════════════════════════════════════════════════════════════════════ +// +// The infra-diff method — the shared jobs, resolved for an estate, against the +// copy's jobs, whole: every job key, every step, every byte — with two additions: +// +// 1. The values the shared workflow derives are not re-implemented here. Its +// `plan` step is RUN, on the estate's own beplus.estate.json and database +// package.json, and its outputs are what `${{ needs.plan.outputs.* }}` +// resolves to. A derivation that is wrong for an estate shows up as a job or +// step that no longer matches the copy. +// 2. Where a step body became a loop over the manifest's environments (the +// drift job's guard and comparison), text cannot match. Those steps are run +// side by side with the copy's — stub `npm` / `beplus`, every scenario that +// changes their output — and must print the same and exit the same. +// +// Every other difference is declared below with its reason, and reported; an +// undeclared one is divergent. + +const DB_WORKFLOWS = ['db-migrations-pr', 'db-drift-env']; +const CLI_ACTION = /^beplus\/setup-beplus\/cli@/; +const AUTH_STEP = 'Authenticate the @beplus scope'; +const PINNED_CLI = '${{ vars.BE_CLI_VERSION }}'; +const TOOLS_DUMP_IMAGE = 'pgvector/pgvector:pg16'; +const PINNED_SERVER = 'pgvector/pgvector:0.8.7-pg16'; + +/** Rewrites of the COPY's step: what the shared workflow does differently on purpose. `null`: not this step. */ +const COMMON_STEP_INTENDED = [ + { + what: "CLI: the repository's pin (`be auto`), not `vars.BE_CLI_VERSION || '2'` / `latest` — BE-213", + apply: (s) => CLI_ACTION.test(s.uses ?? '') && ["${{ vars.BE_CLI_VERSION || '2' }}", 'latest'].includes(s.with?.BE_CLI_VERSION) + ? { ...s, with: { ...s.with, BE_CLI_VERSION: PINNED_CLI } } : null, + }, + { + what: `named "${AUTH_STEP}", as MRKIT's copy names it`, + apply: (s) => CLI_ACTION.test(s.uses ?? '') && s.name === undefined ? { ...s, name: AUTH_STEP } : null, + }, +]; + +/** Additions on the SHARED side, stripped before comparing. `null`: not there. */ +const COMMON_STEP_ADDED = [ + { + what: 'BE_NPM_CHECK: the stub-200 registry check (the @beplus packages the database package needs come back as real packages)', + strip: (s) => { + if (!CLI_ACTION.test(s.uses ?? '') || s.with?.BE_NPM_CHECK === undefined) return null; + const { BE_NPM_CHECK: _check, ...rest } = s.with; + return { ...s, with: rest }; + }, + }, +]; + +const runRewrite = (from, to, what) => ({ what, apply: (s) => (s.run === from ? { ...s, run: to } : null) }); + +const DB = { + 'db-migrations-pr': { + jobs: { checks: 'checks' }, + jobIntended: [ + { + what: `Postgres service ${PINNED_SERVER}, not the floating pg16 tag — a rebuild's server cannot change under a pull request`, + apply: (j) => (j.services?.postgres?.image === TOOLS_DUMP_IMAGE + ? { ...j, services: { ...j.services, postgres: { ...j.services.postgres, image: PINNED_SERVER } } } : null), + }, + { + what: `BE_PGDUMP_IMAGE unset: the repository's database.dumpImage, else @beplus/database-tools' pin — ${TOOLS_DUMP_IMAGE}, this value`, + apply: (j) => (j.env?.BE_PGDUMP_IMAGE === TOOLS_DUMP_IMAGE + ? { ...j, env: Object.fromEntries(Object.entries(j.env).filter(([k]) => k !== 'BE_PGDUMP_IMAGE')) } : null), + }, + ], + stepIntended: [ + ...COMMON_STEP_INTENDED, + runRewrite('npm run db:check:collisions', 'beplus db check', 'the npm script\'s own command — `db:check:collisions` was `beplus db check`'), + runRewrite('npm run db:check:pending', 'beplus db check pending', '`beplus db check pending` — @beplus/database-tools, the copies\' bin/db-check-pending.ts (BE-228; parity BE-231)'), + runRewrite('npm run db:check:rebuild -- --compare', 'beplus db check rebuild --compare', '`beplus db check rebuild --compare` — @beplus/database-tools, the copies\' bin/db-check-rebuild.ts (BE-228; parity BE-231)'), + { + what: '`beplus db check chains` — every chain the declaration embeds, the copy\'s bin/db-check-ai-chain.ts among them (BE-228)', + apply: (s) => (s.run === 'npm run db:check:ai-chain' + ? { ...s, name: 'Embedded migration chains check', run: 'beplus db check chains' } : null), + }, + ], + stepAdded: COMMON_STEP_ADDED, + addedIfAbsent: { + chains: { + why: 'the declaration\'s chains, which this copy never checked in CI (MRKIT: beplus_ai + analytics, only in its local `db:check`)', + present: (s) => /^npm run db:check:[a-z]+-chain$/.test(String(s.run ?? '')), + }, + }, + removedIfPresent: [], + behaviour: {}, + }, + 'db-drift-env': { + jobs: { drift: 'drift' }, + jobIntended: [ + { + what: `BE_PGDUMP_IMAGE unset: the repository's database.dumpImage, else @beplus/database-tools' pin — ${TOOLS_DUMP_IMAGE}, this value`, + apply: (j) => (j.env?.BE_PGDUMP_IMAGE === TOOLS_DUMP_IMAGE + ? { ...j, env: Object.fromEntries(Object.entries(j.env).filter(([k]) => k !== 'BE_PGDUMP_IMAGE')) } : null), + }, + ], + jobAdded: [ + { + what: 'BE_POSTGRES_URL_STAGE mapped: the stage leg of an estate that lists stage (MRKIT, bepluscloud)', + strip: (j) => (j.env?.BE_POSTGRES_URL_STAGE === '${{ secrets.BE_POSTGRES_URL_STAGE }}' + ? { ...j, env: Object.fromEntries(Object.entries(j.env).filter(([k]) => k !== 'BE_POSTGRES_URL_STAGE')) } : null), + }, + ], + stepIntended: COMMON_STEP_INTENDED, + stepAdded: COMMON_STEP_ADDED, + addedIfAbsent: {}, + removedIfPresent: [ + { + what: 'no build: `beplus db diff` reads the committed schema file and the databases, no compiled schema (Mountaineer\'s copy never built)', + match: (s) => s.name === 'Rush build' && /^node common\/scripts\/install-run-rush\.js build --to \S+$/.test(String(s.run ?? '')), + }, + ], + // Steps whose body became a loop over the environments: run side by side. + behaviour: { + guard: { added: ['ENVIRONMENTS'], scenarios: 'guard' }, + drift: { added: ['ENVIRONMENTS', 'SUMMARY'], scenarios: 'drift' }, + }, + }, +}; + +/** What the shared workflow may do at all, whoever calls it. */ +const dbSafety = (name, shared, raw) => { + const failures = []; + const triggers = Object.keys(shared.on ?? {}); + if (triggers.join() !== 'workflow_call') failures.push(`triggers are [${triggers}], not just workflow_call`); + if (canon(shared.permissions) !== canon({ contents: 'read' })) failures.push(`permissions ${canon(shared.permissions)} — contents: read is all it needs`); + for (const [jobName, job] of Object.entries(shared.jobs ?? {})) { + if (job.permissions !== undefined) failures.push(`job "${jobName}" sets its own permissions`); + for (const step of job.steps ?? []) { + if (/configure-aws-credentials/.test(String(step.uses ?? ''))) failures.push(`job "${jobName}" assumes an AWS role`); + } + } + const secrets = [...new Set([...raw.matchAll(/secrets\.([A-Z0-9_]+)/g)].map((m) => m[1]))]; + const allowed = name === 'db-migrations-pr' + ? ['BE_NPM_TOKEN'] + : ['BE_NPM_TOKEN', 'BE_POSTGRES_URL_DEV', 'BE_POSTGRES_URL_STAGE', 'BE_POSTGRES_URL_PROD']; + for (const secret of secrets.filter((x) => !allowed.includes(x))) failures.push(`reads secrets.${secret}`); + // What runs, not what the comments say about other commands. + const code = raw.split('\n').filter((line) => !/^\s*#/.test(line)).join('\n'); + const commands = [...code.matchAll(/\bbeplus\s+db\s+([a-z-]+(?:\s+[a-z-]+)?)/g)].map((m) => m[1]); + for (const command of commands) { + const ok = name === 'db-migrations-pr' ? /^check(\s|$)/.test(command) : /^diff(\s|$)/.test(command); + if (!ok) failures.push(`runs \`beplus db ${command}\` — ${name === 'db-migrations-pr' ? 'only `db check …`' : 'only `db diff`'} belongs here`); + } + if (/--write\b/.test(code)) failures.push('writes a schema file (--write)'); + if (name === 'db-migrations-pr') { + const url = String(shared.jobs?.checks?.env?.BE_POSTGRES_URL ?? ''); + if (!/^postgres:\/\/[^@]+@localhost:5432\//.test(url)) failures.push(`BE_POSTGRES_URL is "${url}", not the job's own localhost service`); + } + return failures; +}; + +const parseDbArgs = (argv) => { + const entries = []; + for (let i = 0; i < argv.length; i += 2) { + const [flag, value] = [argv[i], argv[i + 1]]; + if (value === undefined) usage(); + if (flag === '--repo') { entries.push({ dir: resolve(value) }); continue; } + const current = entries.at(-1); + if (!current) usage(); + if (flag === '--ref') current.ref = value; + else if (flag === '--callers') current.callers = resolve(value); + else if (flag === '--manifest') current.manifest = resolve(value); + else if (flag === '--repository') current.repository = value; + else usage(); + } + if (entries.length === 0) usage(); + return entries; +}; + +/** A file at `ref` (or in the checkout when `ref` is undefined); undefined when it is not there. */ +const readAt = (dir, ref, path) => { + try { + return ref + ? execFileSync('git', ['-C', dir, 'show', `${ref}:${path}`], { encoding: 'utf8', maxBuffer: 1e8, stdio: ['ignore', 'pipe', 'ignore'] }) + : readFileSync(join(dir, path), 'utf8'); + } catch { + return undefined; + } +}; + +/** Replace `${{ needs.plan.outputs.x }}` and `${{ inputs.x }}` in every string; collect what was used. */ +const EXPRESSION = /\$\{\{\s*(needs\.plan\.outputs|inputs)\.([A-Za-z0-9_-]+)\s*\}\}/g; +const resolveExpressions = (value, scope, used) => { + if (typeof value === 'string') { + return value.replace(EXPRESSION, (whole, where, key) => { + const table = where === 'inputs' ? scope.inputs : scope.outputs; + if (!Object.hasOwn(table, key)) { used.unknown.add(whole); return whole; } + used.keys.add(`${where === 'inputs' ? 'inputs' : 'plan'}.${key}`); + return table[key]; + }); + } + if (Array.isArray(value)) return value.map((v) => resolveExpressions(v, scope, used)); + if (value && typeof value === 'object') { + return Object.fromEntries(Object.entries(value).map(([k, v]) => [k, resolveExpressions(v, scope, used)])); + } + return value; +}; + +const bashOk = () => { + const r = spawnSync('bash', ['-c', 'echo "${BASH_VERSINFO[0]}"'], { encoding: 'utf8' }); + return Number(r.stdout.trim()) >= 4 && spawnSync('jq', ['--version']).status === 0; +}; + +/** Run a `run:` body as GitHub does when no shell is given (`bash -e {0}`). */ +const runBody = (body, { cwd, env }) => { + const file = join(cwd, `.step-${Math.random().toString(36).slice(2)}.sh`); + writeFileSync(file, body); + const r = spawnSync('bash', ['--noprofile', '--norc', '-e', file], { cwd, env, encoding: 'utf8' }); + rmSync(file, { force: true }); + return { status: r.status, stdout: r.stdout, stderr: r.stderr }; +}; + +/** + * The shared workflow's `plan` step, run as GitHub would run it on the estate's tree: + * beplus.estate.json and the database package's package.json in a scratch directory. + */ +const runPlan = (shared, inputs, files, repository) => { + const plan = shared.jobs?.plan; + const step = plan?.steps?.find((s) => s.id === 'database'); + if (!step) return { error: 'the shared workflow has no plan job with a `database` step' }; + const sandbox = mkdtempSync(join(tmpdir(), 'verify-callers-plan-')); + try { + for (const [path, text] of Object.entries(files)) { + mkdirSync(dirname(join(sandbox, path)), { recursive: true }); + writeFileSync(join(sandbox, path), text); + } + const output = join(sandbox, '.github-output'); + writeFileSync(output, ''); + const used = { keys: new Set(), unknown: new Set() }; + const stepEnv = resolveExpressions(step.env ?? {}, { inputs, outputs: {} }, used); + const r = runBody(step.run, { + cwd: sandbox, + env: { PATH: process.env.PATH, HOME: sandbox, GITHUB_OUTPUT: output, GITHUB_REPOSITORY: repository, ...stepEnv }, + }); + if (r.status !== 0) return { error: `the plan step failed (exit ${r.status}): ${(r.stdout + r.stderr).trim().split('\n').slice(-2).join(' / ')}` }; + const stepOutputs = Object.fromEntries(readFileSync(output, 'utf8').split('\n').filter(Boolean).map((line) => { + const at = line.indexOf('='); + return [line.slice(0, at), line.slice(at + 1)]; + })); + const outputs = Object.fromEntries(Object.entries(plan.outputs ?? {}).map(([key, expr]) => { + const m = String(expr).match(/^\$\{\{\s*steps\.database\.outputs\.([A-Za-z0-9_-]+)\s*\}\}$/); + return [key, m ? stepOutputs[m[1]] ?? '' : String(expr)]; + })); + return { outputs, log: r.stdout.trim() }; + } finally { + rmSync(sandbox, { recursive: true, force: true }); + } +}; + +/** The stubs a rewritten step runs against: the copy's `npm run db:diff -- a b` and the shared `beplus db diff a b`. */ +const STUBS = { + npm: '#!/usr/bin/env bash\nif [ "$1" = run ] && [ "$2" = db:diff ] && [ "$3" = -- ]; then exec "$STUB_DIFF" npm "$4" "$5"; fi\necho "unexpected: npm $*" >&2; exit 97\n', + beplus: '#!/usr/bin/env bash\nif [ "$1" = db ] && [ "$2" = diff ]; then exec "$STUB_DIFF" beplus "$3" "$4"; fi\necho "unexpected: beplus $*" >&2; exit 97\n', + // The bins exit 1 on drift; the CLI turns a failed program into exit 2. The aggregation must not care. + diff: '#!/usr/bin/env bash\necho "[db:diff] $2 vs $3"\necho "[db:diff] (stderr) $2 vs $3" >&2\ncase ",$STUB_FAIL," in *",$2:$3,"*) if [ "$1" = beplus ]; then exit 2; else exit 1; fi ;; esac\nexit 0\n', +}; + +/** Every scenario that changes what a guard or drift step prints, for these environments. */ +const scenarios = (kind, envs) => { + const url = (env) => `BE_POSTGRES_URL_${env.toUpperCase()}`; + const all = Object.fromEntries(envs.map((e) => [url(e), `postgres://reader:secret@${e}.example:5432/db`])); + if (kind === 'guard') { + return [ + { name: 'every connection string set', env: all }, + ...envs.map((e) => ({ name: `${url(e)} unset`, env: Object.fromEntries(Object.entries(all).filter(([k]) => k !== url(e))) })), + ...envs.map((e) => ({ name: `${url(e)} empty`, env: { ...all, [url(e)]: '' } })), + { name: 'none set', env: {} }, + ]; + } + const pairs = [...envs.map((e) => `migrations:${e}`), ...envs.slice(1).map((e, i) => `${envs[i]}:${e}`)]; + return [ + { name: 'no drift', env: { ...all, STUB_FAIL: '' } }, + ...pairs.map((p) => ({ name: `drift in ${p.replace(':', ' vs ')}`, env: { ...all, STUB_FAIL: p } })), + { name: 'drift everywhere', env: { ...all, STUB_FAIL: pairs.join(',') } }, + ]; +}; + +/** Run the copy's step and the shared step under each scenario; the transcripts must be equal. */ +const behaves = (kind, envs, original, shared, jobEnvOriginal, jobEnvShared) => { + const sandbox = mkdtempSync(join(tmpdir(), 'verify-callers-steps-')); + try { + const bin = join(sandbox, 'bin'); + mkdirSync(bin); + for (const [name, text] of Object.entries(STUBS)) { + writeFileSync(join(bin, name), text); + chmodSync(join(bin, name), 0o755); + } + const base = { PATH: `${bin}:/usr/bin:/bin:${process.env.PATH}`, HOME: sandbox, STUB_DIFF: join(bin, 'diff') }; + // A job's connection strings come from its `env`; the scenario decides which are set. + const strip = (env) => Object.fromEntries(Object.entries(env ?? {}).filter(([k]) => !/^BE_POSTGRES_URL_/.test(k))); + const results = []; + for (const scenario of scenarios(kind, envs)) { + const before = runBody(original.run, { cwd: sandbox, env: { ...base, ...strip(jobEnvOriginal), ...(original.env ?? {}), ...scenario.env } }); + const after = runBody(shared.run, { cwd: sandbox, env: { ...base, ...strip(jobEnvShared), ...(shared.env ?? {}), ...scenario.env } }); + const same = before.status === after.status && before.stdout === after.stdout && before.stderr === after.stderr; + results.push({ scenario: scenario.name, same, before, after }); + } + return results; + } finally { + rmSync(sandbox, { recursive: true, force: true }); + } +}; + +const without = (object, keys) => Object.fromEntries(Object.entries(object ?? {}).filter(([k]) => !keys.includes(k))); + +/** Apply each declared rewrite that matches; return the result and what applied. */ +const rewrite = (value, declarations, key) => { + const applied = []; + let current = value; + for (const d of declarations) { + const next = d[key](current); + if (next) { current = next; applied.push(d.what); } + } + return { value: current, applied }; +}; + +const databaseWorkflows = (argv) => { + const entries = parseDbArgs(argv); + if (!bashOk()) { + console.error('verify-callers db needs bash ≥ 4 and jq on PATH (GitHub\'s runners have both): it runs the shared plan step and the rewritten steps.'); + process.exit(2); + } + const YAML = loadYaml([ + ...entries.map((e) => e.dir), + join(HERE, '..', '..', 'mrkithq', 'monorepo'), + join(HERE, '..', '..', 'bepluscloud', 'ai'), + ]); + + const shared = {}; + const sharedRaw = {}; + let problemsTotal = 0; + const totals = { identical: 0, derived: 0, intended: 0, added: 0, removed: 0, behaviour: 0, divergent: 0 }; + + for (const name of DB_WORKFLOWS) { + sharedRaw[name] = readFileSync(join(HERE, '.github', 'workflows', `${name}.yml`), 'utf8'); + shared[name] = YAML.parse(sharedRaw[name]); + console.log(`${name}.yml (shared)`); + const failures = dbSafety(name, shared[name], sharedRaw[name]); + for (const failure of failures) console.log(` ✗ ${failure}`); + if (failures.length === 0) { + console.log(name === 'db-migrations-pr' + ? ' ✓ workflow_call only; contents: read; no AWS; BE_NPM_TOKEN its only secret; `beplus db check …` only, against its own localhost service' + : ' ✓ workflow_call only; contents: read; no AWS; BE_NPM_TOKEN + BE_POSTGRES_URL_DEV/STAGE/PROD its only secrets; `beplus db diff` only, nothing written'); + } + problemsTotal += failures.length; + } + + for (const entry of entries) { + const ref = entry.ref ?? process.env.BEFORE_REF ?? 'HEAD~1'; + const repository = entry.repository ?? repositoryOf(entry.dir); + // Callers not committed yet run against the tree their originals came from. + const treeRef = entry.callers ? ref : undefined; + const manifestText = entry.manifest ? readFileSync(entry.manifest, 'utf8') : readAt(entry.dir, treeRef, 'beplus.estate.json'); + console.log(); + console.log(`${repository}`); + if (manifestText === undefined) { + console.log(` ✗ no beplus.estate.json${treeRef ? ` at ${treeRef}` : ''}`); + problemsTotal += 1; + continue; + } + const manifest = JSON.parse(manifestText); + // Which package.json the plan step will look for — the step itself decides, and fails if this guess is wrong. + const packageDir = manifest.database?.package ?? 'packages/modules/database'; + const packageJson = readAt(entry.dir, treeRef, `${packageDir}/package.json`); + const files = { 'beplus.estate.json': manifestText, ...(packageJson === undefined ? {} : { [`${packageDir}/package.json`]: packageJson }) }; + console.log(` manifest ${entry.manifest ?? `beplus.estate.json${treeRef ? ` at ${treeRef}` : ''}`}; copies at ${ref}`); + + for (const name of DB_WORKFLOWS) { + const decl = DB[name]; + const path = join('.github', 'workflows', `${name}.yml`); + const callerPath = entry.callers ? join(entry.callers, `${name}.yml`) : join(entry.dir, path); + const originalText = readAt(entry.dir, ref, path); + const problems = []; + const counts = { identical: 0, derived: 0, intended: 0, added: 0, removed: 0, behaviour: 0, divergent: 0 }; + const sharedWorkflow = shared[name]; + const declaredInputs = sharedWorkflow.on?.workflow_call?.inputs ?? {}; + + console.log(); + console.log(` ${name}.yml — ${originalText === undefined ? `GAINED (no copy at ${ref})` : `replaces the copy at ${ref}`}; caller ${callerPath}`); + if (!existsSync(callerPath)) { + console.log(' ✗ no caller'); + problemsTotal += 1; + continue; + } + const caller = YAML.parse(readFileSync(callerPath, 'utf8')); + const original = originalText === undefined ? undefined : YAML.parse(originalText); + + // ── The caller: what stays in the repository, exactly as it was ─────── + const KEPT = ['name', 'on', 'permissions', 'concurrency']; + for (const key of Object.keys(caller)) { + if (![...KEPT, 'jobs'].includes(key)) problems.push(`caller: top-level \`${key}\` — nothing but ${KEPT.join(', ')} and the call belongs here`); + } + if (original) { + for (const key of Object.keys(original)) { + if (![...KEPT, 'jobs'].includes(key)) problems.push(`original: top-level \`${key}\` has no home in caller or shared workflow`); + } + for (const key of KEPT) { + if (canon(caller[key]) !== canon(original[key])) { + problems.push(`caller: \`${key}\` differs from the original\n had: ${canon(original[key])}\n gets: ${canon(caller[key])}`); + } + } + } else { + for (const key of KEPT) if (caller[key] === undefined) problems.push(`caller: no \`${key}\``); + } + // The jobs run with what the caller grants; a called workflow can only narrow it. + if (canon(caller.permissions) !== canon(sharedWorkflow.permissions)) { + problems.push(`caller: permissions ${canon(caller.permissions)} ≠ the shared workflow's ${canon(sharedWorkflow.permissions)} — grant exactly that`); + } + const callerJobs = Object.entries(caller.jobs ?? {}); + if (callerJobs.length !== 1) problems.push(`caller: ${callerJobs.length} jobs — exactly one, the call, belongs here`); + const [callerJobName, callerJob = {}] = callerJobs[0] ?? []; + const callPattern = new RegExp(`^beplus/setup-beplus/\\.github/workflows/${name}\\.yml@\\S+$`); + if (!callPattern.test(String(callerJob.uses ?? ''))) problems.push(`caller: job "${callerJobName}" does not call beplus/setup-beplus/.github/workflows/${name}.yml`); + const originalJobName = original ? Object.keys(original.jobs ?? {})[0] : Object.keys(decl.jobs)[0]; + if (callerJobName !== originalJobName) { + problems.push(`caller: job id "${callerJobName}" — keep "${originalJobName}", so the check is still "${originalJobName} / …"`); + } + for (const key of Object.keys(callerJob.with ?? {})) { + if (!Object.hasOwn(declaredInputs, key)) problems.push(`caller: passes \`${key}\`, which the shared workflow does not declare — GitHub refuses the run`); + } + if (callerJob.secrets !== 'inherit') problems.push('caller: `secrets: inherit` — the token (and the connection strings) are the repository\'s'); + for (const key of Object.keys(callerJob)) { + if (!['uses', 'with', 'secrets'].includes(key)) problems.push(`caller: job "${callerJobName}" sets \`${key}\` — the shared jobs own it`); + } + const inputs = Object.fromEntries(Object.entries(declaredInputs).map(([key, spec]) => [key, String(callerJob.with?.[key] ?? spec.default ?? '')])); + const passed = Object.entries(callerJob.with ?? {}).map(([k, v]) => `${k}: ${v}`).join(', ') || 'no inputs'; + if (problems.length === 0) { + console.log(` ✓ caller ${original ? `${KEPT.join(', ')} identical to the original` : 'its own triggers'}; one job "${callerJobName}", ${callerJob.uses}, ${passed}, secrets: inherit`); + } + + // ── The derivation: the plan step, run on this estate's tree ────────── + const plan = runPlan(sharedWorkflow, inputs, files, repository); + if (plan.error) { + problems.push(`plan: ${plan.error}`); + for (const problem of problems) console.log(` ✗ ${problem}`); + problemsTotal += problems.length; + continue; + } + console.log(` ✓ plan ${Object.entries(plan.outputs).map(([k, v]) => `${k}=${v}`).join(' · ')}`); + const scope = { inputs, outputs: plan.outputs }; + + if (!original) { + // Nothing to compare with: the caller, the derivation, and that every expression resolves. + const used = { keys: new Set(), unknown: new Set() }; + resolveExpressions(sharedWorkflow.jobs, scope, used); + for (const unknown of used.unknown) problems.push(`shared: ${unknown} resolves to nothing`); + for (const problem of problems) console.log(` ✗ ${problem}`); + console.log(` gained: ${Object.keys(sharedWorkflow.jobs).length} jobs, ${Object.values(sharedWorkflow.jobs).reduce((n, j) => n + (j.steps?.length ?? 0), 0)} steps${problems.length ? ` — ${problems.length} problem(s)` : ''}`); + problemsTotal += problems.length; + continue; + } + + // ── The jobs: the shared workflow, resolved for this estate, vs the copy ─ + const sharedJobs = Object.keys(sharedWorkflow.jobs ?? {}); + const originalJobs = Object.keys(original.jobs ?? {}); + for (const job of sharedJobs.filter((j) => j !== 'plan' && !Object.values(decl.jobs).includes(j))) problems.push(`jobs: "${job}" is in the shared workflow and in no declaration`); + for (const job of originalJobs.filter((j) => !Object.hasOwn(decl.jobs, j))) problems.push(`jobs: the original's "${job}" has no shared counterpart`); + console.log(` + job plan (reads the manifest; ${Object.keys(plan.outputs).join(', ')})`); + + for (const [originalJobName_, sharedJobName] of Object.entries(decl.jobs)) { + if (!original.jobs?.[originalJobName_] || !sharedWorkflow.jobs?.[sharedJobName]) continue; + const used = { keys: new Set(), unknown: new Set() }; + const { steps: rawSteps = [], ...rawJob } = sharedWorkflow.jobs[sharedJobName]; + const sharedJobRaw = resolveExpressions(rawJob, scope, used); + const { steps: originalSteps = [], ...originalJobRaw } = original.jobs[originalJobName_]; + for (const unknown of used.unknown) problems.push(`job "${sharedJobName}": ${unknown} resolves to nothing`); + + // Job keys. + const notes = []; + let sharedJob = sharedJobRaw; + if (sharedJob.needs === 'plan') { sharedJob = without(sharedJob, ['needs']); notes.push('needs: plan'); } + if (sharedJob.environment === '') { sharedJob = without(sharedJob, ['environment']); notes.push('no github-environment: no environment, as the copy'); } + const added = rewrite(sharedJob, decl.jobAdded ?? [], 'strip'); + const intended = rewrite(originalJobRaw, decl.jobIntended ?? [], 'apply'); + if (canon(added.value) !== canon(intended.value)) { + problems.push(`job "${sharedJobName}": job keys differ\n had: ${canon(intended.value)}\n gets: ${canon(added.value)}`); + } + console.log(` job ${sharedJobName} (${[...notes, ...added.applied.map((w) => `+ ${w}`), ...intended.applied.map((w) => `~ ${w}`)].join('; ') || 'keys identical'})`); + + // Steps, in order. + const behaviour = decl.behaviour ?? {}; + let next = 0; + const report = (mark, label, why) => console.log(` ${mark} ${label.padEnd(58)} ${why}`); + const skipRemoved = () => { + while (next < originalSteps.length) { + const removal = (decl.removedIfPresent ?? []).find((r) => r.match(originalSteps[next])); + if (!removal) break; + counts.removed += 1; + report('−', stepKey(originalSteps[next]), `removed — ${removal.what}`); + next += 1; + } + }; + for (const [i, raw] of rawSteps.entries()) { + skipRemoved(); + const stepUsed = { keys: new Set(), unknown: new Set() }; + const step = resolveExpressions(raw, scope, stepUsed); + for (const unknown of stepUsed.unknown) problems.push(`job "${sharedJobName}" step ${i + 1}: ${unknown} resolves to nothing`); + const id = step.id; + const ifAbsent = id ? decl.addedIfAbsent?.[id] : undefined; + if (ifAbsent && !originalSteps.some(ifAbsent.present)) { + counts.added += 1; + report('+', stepKey(step), `added — ${ifAbsent.why}`); + continue; + } + const theirs = originalSteps[next++]; + if (!theirs) { + counts.divergent += 1; + problems.push(`job "${sharedJobName}" step ${i + 1}: only in the shared workflow — ${stepKey(step)}`); + report('✗', stepKey(step), 'only in the shared workflow'); + continue; + } + const mine = without(step, ['id']); // an id is a label; nothing in these jobs reads one + const plain = without(raw, ['id']); + if (canon(plain) === canon(theirs)) { + counts.identical += 1; + report('✓', stepKey(step), 'identical'); + continue; + } + const stripped = rewrite(mine, decl.stepAdded ?? [], 'strip'); + const target = rewrite(theirs, decl.stepIntended ?? [], 'apply'); + const spec = id ? behaviour[id] : undefined; + if (spec) { + // Everything but the body must match; the body must BEHAVE the same. + const structural = { ...stripped.value, env: without(stripped.value.env, spec.added), run: undefined }; + const structuralTarget = { ...target.value, run: undefined }; + if (Object.keys(structural.env).length === 0) delete structural.env; + if (canon(structural) !== canon(structuralTarget)) { + counts.divergent += 1; + problems.push(`job "${sharedJobName}" step ${i + 1} "${stepKey(mine)}" differs beyond its body\n had: ${canon(structuralTarget)}\n gets: ${canon(structural)}`); + report('✗', stepKey(mine), 'DIVERGENT (keys)'); + continue; + } + const envs = String(plan.outputs.environments ?? '').split(' ').filter(Boolean); + const runs = behaves(spec.scenarios, envs, theirs, mine, originalJobRaw.env, sharedJobRaw.env); + const bad = runs.filter((r) => !r.same); + if (bad.length) { + counts.divergent += 1; + for (const r of bad) { + problems.push(`job "${sharedJobName}" step ${i + 1} "${stepKey(mine)}" behaves differently — ${r.scenario}\n had: exit ${r.before.status} ${JSON.stringify(r.before.stdout + r.before.stderr)}\n gets: exit ${r.after.status} ${JSON.stringify(r.after.stdout + r.after.stderr)}`); + } + report('✗', stepKey(mine), `DIVERGENT in ${bad.length}/${runs.length} scenarios`); + } else { + counts.behaviour += 1; + const why = [`run side by side: same output and exit in ${runs.length} scenarios (${spec.scenarios})`, ...target.applied, ...stripped.applied.map((w) => `+ ${w}`)]; + report('≡', stepKey(mine), why.join('; ')); + } + continue; + } + if (canon(stripped.value) === canon(target.value)) { + const why = [...target.applied.map((w) => `intended: ${w}`), ...stripped.applied.map((w) => `+ ${w}`)]; + if (target.applied.length || stripped.applied.length) counts.intended += 1; else counts.derived += 1; + report(target.applied.length || stripped.applied.length ? '~' : '✓', stepKey(mine), + [why.join('; '), stepUsed.keys.size ? `derived: ${[...stepUsed.keys].join(', ')}` : ''].filter(Boolean).join('; ')); + continue; + } + counts.divergent += 1; + problems.push(`job "${sharedJobName}" step ${i + 1} "${stepKey(mine)}" differs\n had: ${canon(target.value)}\n gets: ${canon(stripped.value)}`); + report('✗', stepKey(mine), 'DIVERGENT'); + } + skipRemoved(); + for (; next < originalSteps.length; next += 1) { + counts.divergent += 1; + problems.push(`job "${sharedJobName}": the original's step "${stepKey(originalSteps[next])}" is missing from the shared workflow`); + report('✗', stepKey(originalSteps[next]), 'MISSING from the shared workflow'); + } + } + + for (const problem of problems) console.log(` ✗ ${problem}`); + const steps = Object.values(counts).reduce((a, b) => a + b, 0); + console.log(` ${steps} steps: ${counts.identical} identical, ${counts.derived} identical once derived, ${counts.intended} intended, ` + + `${counts.behaviour} rewritten and run side by side, ${counts.added} added, ${counts.removed} removed, ${counts.divergent} divergent` + + `${problems.length ? ` — ${problems.length} problem(s)` : ''}`); + problemsTotal += problems.length; + for (const k of Object.keys(totals)) totals[k] += counts[k]; + } + } + + console.log(); + console.log(` ${entries.length} estate(s): ${totals.identical} identical, ${totals.derived} identical once derived, ${totals.intended} intended, ` + + `${totals.behaviour} run side by side, ${totals.added} added, ${totals.removed} removed, ${totals.divergent} divergent step(s)`); + console.log(problemsTotal === 0 + ? ' every caller + db-migrations-pr.yml / db-drift-env.yml reproduces the workflow it replaces' + : ` ${problemsTotal} problem(s): a caller would not do what its workflow did before`); + process.exit(problemsTotal === 0 ? 0 : 1); +}; + // ───────────────────────────────────────────────────────────────────────────── const [first, ...rest] = process.argv.slice(2); if (!first) usage(); if (first === 'infra-diff') infraDiff(rest); +else if (first === 'db') databaseWorkflows(rest); else if (existsSync(first)) libraries(first); else usage(); diff --git a/test/cli/npm-check/package.json b/test/cli/npm-check/package.json new file mode 100644 index 0000000..e1a1d02 --- /dev/null +++ b/test/cli/npm-check/package.json @@ -0,0 +1,10 @@ +{ + "name": "npm-check-fixture", + "private": true, + "dependencies": { + "@beplus/configuration": "0.127.0" + }, + "devDependencies": { + "@beplus/fixture-local": "workspace:*" + } +}