From d881ab0690a666c9e8b202daba0658683091a9c4 Mon Sep 17 00:00:00 2001 From: bea-claude Date: Thu, 8 Oct 2026 10:59:46 +0200 Subject: [PATCH] Release through dev, stage, prod and main, like every @beplus library, and publish to each CodeArtifact (v0.11.0) npm.beplus.cloud (dev's CodeArtifact) had served @beplus/be 0.7.0 as latest since July: the release published only to public npm and GitHub Packages. An estate release that installed be after `beplus npm auth` got 0.7.0's `be auto`, which picks a Node version, and failed (bepluscloud's dev Deploy, 2026-10-08). be now follows library-publish's model, one version per commit, built once, promoted: - dev: pack once, tag vX.Y.Z with a GitHub pre-release, publish the tarball to GitHub Packages (dist-tag dev) and to dev's CodeArtifact; - stage, prod: publish the bytes dev put in GitHub Packages to their own CodeArtifact (GitHub Packages tags them stage / latest); prod makes the GitHub Release a full release; - main: the public channel, fast-forwarded from prod, publishes the same bytes to public npm. Each job assumes github-actions-beplus-be--npm-publishing in its GitHub environment's BE_AWS_ACCOUNT_ID, and `beplus aws codeartifact auth --env` picks the repository. stage, prod and main refuse a version dev never released, and main refuses to be ahead of prod. PRs go into dev; scripts/install.sh clones main by name, now that dev is the default branch. Co-Authored-By: Igor Lamos --- .github/PULL_REQUEST_TEMPLATE.md | 2 +- .github/workflows/ci.yml | 9 +- .github/workflows/release.yml | 271 ++++++++++++++++------- CHANGELOG.md | 15 +- bin/be | 2 +- bin/bump | 9 +- docs/releasing.md | 81 ++++--- package-lock.json | 4 +- package.json | 2 +- scripts/install.sh | 4 +- scripts/release/check-environment.sh | 24 ++ scripts/release/fetch-github-packages.sh | 30 +++ scripts/release/publish-codeartifact.sh | 33 +++ scripts/release/verify-version.sh | 109 +++++---- 14 files changed, 441 insertions(+), 154 deletions(-) create mode 100755 scripts/release/check-environment.sh create mode 100755 scripts/release/fetch-github-packages.sh create mode 100755 scripts/release/publish-codeartifact.sh diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index e5843a3..2677937 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -21,7 +21,7 @@ Show the fixed output if appropriate. ## Release - [ ] Releases vX.Y.Z: ran `bin/bump …` and described it in CHANGELOG.md diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 775d9c3..4853491 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,7 +1,7 @@ name: CI -# Pull requests run it directly; the Release workflow calls it on every push to main, so the -# merge result is tested again before anything is published. +# Pull requests run it directly; the Release workflow calls it on every push to dev, stage, prod +# and main, so the merge result is tested again before anything is published or promoted. on: pull_request: @@ -40,8 +40,9 @@ jobs: version: name: Version - # Only a PR into main cuts a release, so only there do the release rules apply. - if: github.event_name == 'pull_request' && github.base_ref == 'main' + # Only a PR into dev cuts a release (stage and prod are fast-forwarded from it), so only there + # do the release rules apply. + if: github.event_name == 'pull_request' && github.base_ref == 'dev' runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index c6208c2..05dbc7c 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -1,24 +1,39 @@ name: Release -# A merge into main that bumped the version publishes it to public npm, tags it with a GitHub -# Release, and mirrors it to GitHub Packages, as every @beplus/* repository does; any other merge -# finds nothing to do. Every step checks what is already done, so re-running a failed run is safe. -# See docs/releasing.md. +# One version per commit, built once, promoted — the model every @beplus library follows +# (beplus/setup-beplus's library-publish.yml), for a package that is one bash script and no Rush: # -# npm trusts this file by name (Trusted Publisher: beplus/be, release.yml, environment npm), so -# there is no npm token anywhere: the publish job authenticates with its OIDC token. GitHub Packages -# takes the workflow's own GITHUB_TOKEN. +# dev A push that carries a new version packs it ONCE, tags it vX.Y.Z with a GitHub +# pre-release, and publishes that tarball to GitHub Packages (dist-tag `dev`), then to +# dev's CodeArtifact — npm.beplus.cloud, where the estates' builds install `be` from. +# stage Builds nothing: fetches the tarball dev published from GitHub Packages and publishes it +# byte for byte to stage's CodeArtifact. GitHub Packages tags it `stage`. +# prod The same into prod's CodeArtifact (GitHub Packages tags it `latest`), and the GitHub +# Release becomes a full release, marked latest. +# main The public channel, and the one thing `be` has that the libraries do not: the same +# bytes to the public npm registry (trusted publishing, with provenance). +# +# stage, prod and main are fast-forwarded (dev → stage → prod → main), never committed to; a +# version GitHub Packages does not have is one dev never released, and is refused there, and main +# publishes only what prod already has. Every step checks what is already done, +# so re-running a failed run is safe. See docs/releasing.md. +# +# Each environment's job assumes github-actions-beplus-be--npm-publishing in the account its +# GitHub environment names (vars.BE_AWS_ACCOUNT_ID) — the role name library-publish derives — and +# `beplus aws codeartifact auth --env ` picks that environment's CodeArtifact repository. +# npm trusts this file by name (Trusted Publisher: beplus/be, release.yml, environment npm), so no +# npm token exists anywhere. GitHub Packages takes the workflow's own GITHUB_TOKEN. on: push: - branches: [main] + branches: [dev, stage, prod, main] workflow_dispatch: permissions: contents: read concurrency: - group: release + group: release-${{ github.ref_name }} cancel-in-progress: false jobs: @@ -29,7 +44,7 @@ jobs: plan: name: Plan needs: ci - if: github.ref == 'refs/heads/main' + if: contains(fromJSON('["dev", "stage", "prod", "main"]'), github.ref_name) runs-on: ubuntu-latest permissions: contents: read @@ -39,119 +54,225 @@ jobs: tag: ${{ steps.verify.outputs.tag }} dist_tag: ${{ steps.verify.outputs.dist_tag }} source_sha: ${{ steps.verify.outputs.source_sha }} - need_publish: ${{ steps.verify.outputs.need_publish }} need_release: ${{ steps.verify.outputs.need_release }} need_github_packages: ${{ steps.verify.outputs.need_github_packages }} + need_npm: ${{ steps.verify.outputs.need_npm }} steps: - uses: actions/checkout@v7 + with: + fetch-depth: 0 # main publishes only what prod has: HEAD must be on origin/prod - id: verify - run: scripts/release/verify-version.sh release + run: scripts/release/verify-version.sh release "${GITHUB_REF_NAME}" env: GITHUB_PACKAGES_TOKEN: ${{ github.token }} - publish: - name: Publish ${{ needs.plan.outputs.tag }} + dev: + name: Release ${{ needs.plan.outputs.tag }} on dev needs: plan - if: needs.plan.outputs.need_publish == 'true' || needs.plan.outputs.need_release == 'true' + if: github.ref_name == 'dev' runs-on: ubuntu-latest - environment: npm + environment: dev permissions: - contents: write # tag + GitHub Release - id-token: write # npm trusted publishing and provenance + contents: write # the tag and the GitHub pre-release + packages: write # GitHub Packages + id-token: write # the CodeArtifact role + env: + BE_ENVIRONMENT: ${{ vars.BE_ENVIRONMENT }} + BE_AWS_ACCOUNT_ID: ${{ vars.BE_AWS_ACCOUNT_ID }} + VERSION: ${{ needs.plan.outputs.version }} + TAG: ${{ needs.plan.outputs.tag }} + DIST_TAG: ${{ needs.plan.outputs.dist_tag }} + SOURCE_SHA: ${{ needs.plan.outputs.source_sha }} steps: + # The commit the version was cut from, which is its tag once it has one: a later push that + # bumps nothing still publishes exactly what was released. - uses: actions/checkout@v7 + with: + ref: ${{ needs.plan.outputs.source_sha }} - uses: actions/setup-node@v7 with: node-version: 24 - registry-url: https://registry.npmjs.org - - name: Make sure npm can publish with OIDC (>= 11.5.1) - run: | - if ! node -e ' - const [a, b, c] = process.argv[1].split(".").map(Number); - process.exit(a > 11 || (a === 11 && (b > 5 || (b === 5 && c >= 1))) ? 0 : 1); - ' "$(npm --version)"; then - npm install --global npm@11 - fi - npm --version + - name: Check the prerequisites + run: scripts/release/check-environment.sh dev - - name: Publish to npm - if: needs.plan.outputs.need_publish == 'true' - run: npm publish --provenance --tag "${DIST_TAG}" - env: - DIST_TAG: ${{ needs.plan.outputs.dist_tag }} + - name: Pack it once + run: | + mkdir -p npm + npm pack --pack-destination npm + ls -l npm - - name: Tag and create the GitHub Release + - name: Tag it and create the GitHub pre-release if: needs.plan.outputs.need_release == 'true' run: | scripts/release/changelog-notes.sh "${TAG}" > "${RUNNER_TEMP}/notes.md" - if [[ "${DIST_TAG}" == "latest" ]]; then - channel=(--latest) - else - channel=(--prerelease --latest=false) - fi - gh release create "${TAG}" --target "${GITHUB_SHA}" --title "${TAG}" \ - --notes-file "${RUNNER_TEMP}/notes.md" "${channel[@]}" + gh release create "${TAG}" --target "${SOURCE_SHA}" --title "${TAG}" \ + --notes-file "${RUNNER_TEMP}/notes.md" --prerelease --latest=false env: GH_TOKEN: ${{ github.token }} - TAG: ${{ needs.plan.outputs.tag }} - DIST_TAG: ${{ needs.plan.outputs.dist_tag }} + + # publishConfig pins the @beplus scope to npmjs, and only command-line flags outrank it, so + # every publish names its registry on the command line; the token stays in the environment. + - name: Publish to GitHub Packages + if: needs.plan.outputs.need_github_packages == 'true' + run: | + echo '//npm.pkg.github.com/:_authToken=${NODE_AUTH_TOKEN}' > "${RUNNER_TEMP}/.npmrc-github" + # `./`: to npm, a bare `npm/.tgz` is a GitHub `owner/repo` shorthand. + npm publish ./npm/*.tgz --userconfig "${RUNNER_TEMP}/.npmrc-github" \ + --registry=https://npm.pkg.github.com/ --@beplus:registry=https://npm.pkg.github.com/ \ + --tag dev + env: + NODE_AUTH_TOKEN: ${{ github.token }} + + - uses: aws-actions/configure-aws-credentials@v4 + with: + role-to-assume: arn:aws:iam::${{ vars.BE_AWS_ACCOUNT_ID }}:role/github-actions-beplus-${{ github.event.repository.name }}-${{ github.ref_name }}-npm-publishing + aws-region: us-east-1 + + - uses: beplus/setup-beplus/cli@v2 + with: + BE_CLI_VERSION: ${{ vars.BE_CLI_VERSION }} + + - name: Publish to CodeArtifact + run: scripts/release/publish-codeartifact.sh ./npm/*.tgz - name: Summary run: | { - echo "### Released ${TAG}" + echo "### Released ${TAG} on dev" echo - echo "- npm: https://www.npmjs.com/package/@beplus/be/v/${VERSION}" - echo "- GitHub: ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/releases/tag/${TAG}" + echo "- GitHub (pre-release): ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/releases/tag/${TAG}" + echo "- GitHub Packages: ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/pkgs/npm/be (dist-tag \`dev\`)" + echo "- dev's CodeArtifact (npm.beplus.cloud): dist-tag \`${DIST_TAG}\`" } >> "${GITHUB_STEP_SUMMARY}" - env: - TAG: ${{ needs.plan.outputs.tag }} - VERSION: ${{ needs.plan.outputs.version }} - - github-packages: - name: GitHub Packages ${{ needs.plan.outputs.tag }} - needs: [plan, publish] - # After npm, so the mirror never has a version npm refused; also when there was nothing left to - # do on npm (publish skipped), which is how a version missing only here gets published. - if: >- - always() && needs.plan.result == 'success' - && needs.plan.outputs.need_github_packages == 'true' - && (needs.publish.result == 'success' || needs.publish.result == 'skipped') + + promote: + name: Promote ${{ needs.plan.outputs.tag }} to ${{ github.ref_name }} + needs: plan + if: github.ref_name == 'stage' || github.ref_name == 'prod' runs-on: ubuntu-latest + environment: ${{ github.ref_name }} permissions: - contents: read - packages: write + contents: write # prod: the GitHub Release becomes a full release + packages: write # the stage/latest dist-tag in GitHub Packages + id-token: write # the CodeArtifact role + env: + BE_ENVIRONMENT: ${{ vars.BE_ENVIRONMENT }} + BE_AWS_ACCOUNT_ID: ${{ vars.BE_AWS_ACCOUNT_ID }} + VERSION: ${{ needs.plan.outputs.version }} + TAG: ${{ needs.plan.outputs.tag }} + DIST_TAG: ${{ needs.plan.outputs.dist_tag }} steps: - # The commit the version was cut from, which is its tag once it has one: a later merge that - # bumps nothing still publishes exactly what npm has. - uses: actions/checkout@v7 - with: - ref: ${{ needs.plan.outputs.source_sha }} - uses: actions/setup-node@v7 with: node-version: 24 - # publishConfig pins the @beplus scope to npmjs, and only command-line flags outrank it, so - # the .npmrc carries just the token, read from the environment rather than written to disk. - - name: Publish to GitHub Packages + - name: Check the prerequisites + run: scripts/release/check-environment.sh "${GITHUB_REF_NAME}" + + - name: Fetch it from GitHub Packages + run: scripts/release/fetch-github-packages.sh "${VERSION}" npm + env: + NODE_AUTH_TOKEN: ${{ github.token }} + + - uses: aws-actions/configure-aws-credentials@v4 + with: + role-to-assume: arn:aws:iam::${{ vars.BE_AWS_ACCOUNT_ID }}:role/github-actions-beplus-${{ github.event.repository.name }}-${{ github.ref_name }}-npm-publishing + aws-region: us-east-1 + + - uses: beplus/setup-beplus/cli@v2 + with: + BE_CLI_VERSION: ${{ vars.BE_CLI_VERSION }} + + - name: Publish to CodeArtifact + run: scripts/release/publish-codeartifact.sh ./npm/*.tgz + + # How far a version got, in the one registry every environment shares: `stage`, and prod's + # `latest`. A registry that refuses a tag warns rather than stopping a promotion. + - name: Tag it in GitHub Packages + continue-on-error: true run: | echo '//npm.pkg.github.com/:_authToken=${NODE_AUTH_TOKEN}' > "${RUNNER_TEMP}/.npmrc-github" - npm publish --userconfig "${RUNNER_TEMP}/.npmrc-github" \ - --registry=https://npm.pkg.github.com/ --@beplus:registry=https://npm.pkg.github.com/ \ - --tag "${DIST_TAG}" + npm dist-tag add "@beplus/be@${VERSION}" "${PROMOTED_TAG}" \ + --userconfig "${RUNNER_TEMP}/.npmrc-github" --@beplus:registry=https://npm.pkg.github.com/ \ + || echo "::warning::GitHub Packages did not tag @beplus/be@${VERSION} ${PROMOTED_TAG}." env: NODE_AUTH_TOKEN: ${{ github.token }} - DIST_TAG: ${{ needs.plan.outputs.dist_tag }} + PROMOTED_TAG: ${{ github.ref_name == 'prod' && 'latest' || github.ref_name }} + + - name: Make the GitHub release a full release + if: github.ref_name == 'prod' + run: | + if [[ "$(gh release view "${TAG}" --repo "${GITHUB_REPOSITORY}" --json isPrerelease --jq .isPrerelease)" == "true" ]]; then + if [[ "${DIST_TAG}" == "latest" ]]; then latest=(--latest); else latest=(--latest=false); fi + gh release edit "${TAG}" --repo "${GITHUB_REPOSITORY}" --prerelease=false "${latest[@]}" + fi + env: + GH_TOKEN: ${{ github.token }} - name: Summary run: | { - echo "### Mirrored ${TAG} to GitHub Packages" + echo "### Promoted ${TAG} to ${GITHUB_REF_NAME}" echo - echo "- ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/pkgs/npm/be" + echo "- ${GITHUB_REF_NAME}'s CodeArtifact: dist-tag \`${DIST_TAG}\`, the bytes dev published" } >> "${GITHUB_STEP_SUMMARY}" + + # main only: the public registry. Its own job because npm's trusted publisher is bound to the + # `npm` environment, whose deployment branch is main. + public: + name: Publish ${{ needs.plan.outputs.tag }} to npm + needs: plan + if: github.ref_name == 'main' + runs-on: ubuntu-latest + environment: npm + permissions: + contents: read + packages: read # the tarball, from GitHub Packages + id-token: write # npm trusted publishing and provenance + env: + VERSION: ${{ needs.plan.outputs.version }} + TAG: ${{ needs.plan.outputs.tag }} + DIST_TAG: ${{ needs.plan.outputs.dist_tag }} + steps: + - uses: actions/checkout@v7 + + - uses: actions/setup-node@v7 + with: + node-version: 24 + registry-url: https://registry.npmjs.org + + - name: Make sure npm can publish with OIDC (>= 11.5.1) + if: needs.plan.outputs.need_npm == 'true' + run: | + if ! node -e ' + const [a, b, c] = process.argv[1].split(".").map(Number); + process.exit(a > 11 || (a === 11 && (b > 5 || (b === 5 && c >= 1))) ? 0 : 1); + ' "$(npm --version)"; then + npm install --global npm@11 + fi + npm --version + + - name: Fetch it from GitHub Packages + if: needs.plan.outputs.need_npm == 'true' + run: scripts/release/fetch-github-packages.sh "${VERSION}" npm env: - TAG: ${{ needs.plan.outputs.tag }} + NODE_AUTH_TOKEN: ${{ github.token }} + + - name: Publish to npm + if: needs.plan.outputs.need_npm == 'true' + run: | + npm publish ./npm/*.tgz --provenance --tag "${DIST_TAG}" \ + --registry=https://registry.npmjs.org/ --@beplus:registry=https://registry.npmjs.org/ + + - name: Summary + run: | + { + echo "### Released ${TAG}" + echo + echo "- npm: https://www.npmjs.com/package/@beplus/be/v/${VERSION}" + echo "- GitHub: ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/releases/tag/${TAG}" + } >> "${GITHUB_STEP_SUMMARY}" diff --git a/CHANGELOG.md b/CHANGELOG.md index 036a009..719b4ed 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,7 +9,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -https://github.com/beplus/be/compare/v0.10.0...main +https://github.com/beplus/be/compare/v0.11.0...dev + +## [v0.11.0] (10/08/2026) + +`@beplus/be` now releases the way every `@beplus/*` library does: one version per commit, built +once, promoted. A merge into `dev` that bumps the version tags it with a GitHub pre-release and +publishes the tarball to GitHub Packages and then to dev's CodeArtifact, `npm.beplus.cloud`, where +the estates' builds install `be` from. That registry had served 0.7.0 as `latest` since July, +because nothing published `be` there; a release that installed `be` after `beplus npm auth` got +0.7.0's `be auto`, which picks a Node version, and failed. Fast-forwarding `stage` publishes the +same bytes to stage's CodeArtifact. Fast-forwarding `prod` publishes them to prod's CodeArtifact +and turns the GitHub Release into a full release. Fast-forwarding `main`, the public channel, +publishes them to the public npm registry. `dev` is the default branch, and `scripts/install.sh` +installs from `main`. `bin/be` itself is unchanged. See docs/releasing.md. ## [v0.10.0] (10/07/2026) diff --git a/bin/be b/bin/be index 14e284a..b8e8059 100755 --- a/bin/be +++ b/bin/be @@ -64,7 +64,7 @@ function n_grep() { # Setup and state # -VERSION="v0.10.0" +VERSION="v0.11.0" BE_PREFIX="${BE_PREFIX-/usr/local}" BE_PREFIX=${BE_PREFIX%/} diff --git a/bin/bump b/bin/bump index f76022e..b94a9a2 100755 --- a/bin/bump +++ b/bin/bump @@ -6,7 +6,8 @@ # # Writes the new version into package.json, package-lock.json and bin/be, and opens its section in # CHANGELOG.md, moving any notes written under [Unreleased] into it. Commits, tags and publishes -# nothing: put the result in a PR into main, and merging that PR releases it (docs/releasing.md). +# nothing: put the result in a PR into dev, and merging that PR releases it on dev; fast-forwarding +# stage and then prod promotes it (docs/releasing.md). # # On a _versions/vX.Y.Z branch the argument defaults to X.Y.Z, and any other version is refused. # Re-running it for the version already set only re-syncs the files. @@ -46,7 +47,7 @@ sed -i.bak "s/^VERSION=\".*\"\$/VERSION=\"v${version}\"/" bin/be rm -f bin/be.bak # -# CHANGELOG.md: point [Unreleased] at what main has on top of this version, and open the version's +# CHANGELOG.md: point [Unreleased] at what dev has on top of this version, and open the version's # section with whatever [Unreleased] collected (kept as is when the section already exists). # @@ -58,7 +59,7 @@ fi awk \ -v tag="v${version}" \ -v date="$(date +%m/%d/%Y)" \ - -v link="$(node -p 'require("./package.json").homepage')/compare/v${version}...main" \ + -v link="$(node -p 'require("./package.json").homepage')/compare/v${version}...dev" \ -v has_section="${has_section}" ' function flush( i, first, last) { print "" @@ -91,4 +92,4 @@ echo "Bumped to v${version}: package.json, package-lock.json, bin/be, CHANGELOG. if ! scripts/release/changelog-notes.sh "v${version}" > /dev/null 2>&1; then echo "Next: describe the release under \"## [v${version}]\" in CHANGELOG.md." fi -echo "Then commit and open a PR into main; merging it publishes v${version}." +echo "Then commit and open a PR into dev; merging it releases v${version} on dev." diff --git a/docs/releasing.md b/docs/releasing.md index 14064c1..f34c09d 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -1,15 +1,26 @@ # Releasing -A release is cut by merging a pull request into `main` that bumps the version. Nobody publishes -by hand. The [Release workflow](../.github/workflows/release.yml) publishes `@beplus/be` to the -**public npm registry**, creates the `vX.Y.Z` tag and GitHub Release, and then mirrors the version -to **GitHub Packages**, as every `@beplus/*` repository does. +`@beplus/be` releases the way every `@beplus/*` library does (beplus/setup-beplus's +`library-publish.yml`): **one version per commit, built once, promoted.** A release is cut by +merging a pull request into `dev` that bumps the version; `stage`, `prod` and `main` are +fast-forwarded from it and build nothing. Nobody publishes by hand. The +[Release workflow](../.github/workflows/release.yml) does it all: + +| branch | what a push does | +| --- | --- | +| `dev` | packs the version **once**, tags it `vX.Y.Z` with a GitHub **pre-release**, publishes the tarball to **GitHub Packages** (dist-tag `dev`), then to **dev's CodeArtifact** — `npm.beplus.cloud`, where the estates' builds install `be` from | +| `stage` | fetches that tarball from GitHub Packages and publishes it, byte for byte, to **stage's CodeArtifact**; GitHub Packages tags it `stage` | +| `prod` | the same into **prod's CodeArtifact** (GitHub Packages tags it `latest`), and the GitHub Release becomes a **full release**, marked latest | +| `main` | the public channel, and the one thing `be` has that the libraries don't: the same bytes to the **public npm registry**, with provenance. `main` publishes only what `prod` already has | ## The flow ``` -feature/… ──▶ (optional) _versions/vX.Y.Z ──▶ PR into main ──▶ npm + tag + GitHub Release - ──▶ GitHub Packages +feature/… ──▶ (optional) _versions/vX.Y.Z ──▶ PR into dev ──▶ tag + GitHub pre-release + ──▶ GitHub Packages ──▶ dev's CodeArtifact +dev ──fast-forward──▶ stage ──▶ stage's CodeArtifact +stage ──fast-forward──▶ prod ──▶ prod's CodeArtifact ──▶ full GitHub Release +prod ──fast-forward──▶ main ──▶ public npm ``` 1. On your branch, run `bin/bump patch|minor|major` (or `bin/bump X.Y.Z`). It writes the version @@ -18,18 +29,22 @@ feature/… ──▶ (optional) _versions/vX.Y.Z ──▶ PR into main nothing. 2. Describe the release under its `## [vX.Y.Z]` heading in `CHANGELOG.md`. That text becomes the GitHub Release notes. -3. Open a PR into `main`. CI runs the tests and lint, and checks the version. -4. Once the PR is approved and merged, the Release workflow publishes the version. +3. Open a PR into `dev`. CI runs the tests and lint, and checks the version. +4. Once the PR is approved and merged, the Release workflow releases the version on `dev`. +5. Promote it: fast-forward `stage` to `dev` (`git push origin origin/dev:stage`); once it has + proven itself there, `prod` to `stage` (`git push origin origin/stage:prod`); and to make it + public, `main` to `prod` (`git push origin origin/prod:main`). Never commit to `stage`, `prod` + or `main` directly. A PR that doesn't touch `bin/be` (docs, tests, CI) can merge without a bump, and then releases -nothing. A PR that changes `bin/be` must bump. `scripts/install.sh` installs straight from `main`, -so whatever lands there must be a released version. +nothing. A PR that changes `bin/be` must bump. `scripts/install.sh` installs from `main`, so +whatever reaches `main` is a released, public version. To batch several PRs into one release, collect them on a `_versions/vX.Y.Z` branch first. Bump -on that branch, where `bin/bump` defaults to `X.Y.Z`, then open one PR from it into `main`. CI +on that branch, where `bin/bump` defaults to `X.Y.Z`, then open one PR from it into `dev`. CI checks that the branch name and the version agree. -`dev` is retired. Nothing targets it any more. +The release branches are `dev` (the default), `stage`, `prod` and `main`. ## What CI checks @@ -37,39 +52,55 @@ checks that the branch name and the version agree. the PR check and the release run use it. - `bin/be`, `package.json` and `package-lock.json` carry the same version. -- A bump is newer than `main`'s version, isn't tagged or on npm yet, and has a non-empty +- A bump is newer than `dev`'s version, isn't tagged or on npm yet, and has a non-empty `CHANGELOG.md` section. - A change to `bin/be` comes with a bump. - A `_versions/vX.Y.Z` branch carries exactly `vX.Y.Z`. +- `stage`, `prod` and `main` promote only a version `dev` released: tagged `vX.Y.Z` and in GitHub + Packages. Anything else is refused before a registry is touched. +- `main` is on `origin/prod`, so the public registry only ever gets what `prod` already has. ## When a release run fails -Re-run it from the Actions tab. Each step skips what's already done (on npm, tagged, in GitHub -Packages), so a re-run picks up where the failed one stopped. If a run was cancelled while it waited behind another -release, re-run that run too, because it holds its own commit. +Re-run it from the Actions tab. Each step skips what's already done (tagged, in GitHub Packages, +in that environment's CodeArtifact, on npm), so a re-run picks up where the failed one stopped. If +a run was cancelled while it waited behind another release on the same branch, re-run that run +too, because it holds its own commit. ## One-time setup +- **Branches.** `dev` (the default branch), `stage`, `prod` and `main`. Protect `stage`, `prod` and + `main` so they only move by fast-forward. +- **GitHub environments** `dev`, `stage` and `prod`, each limited to its own branch, each with two + variables: + - `BE_ENVIRONMENT`: `dev`, `stage` or `prod`. It picks the CodeArtifact repository + (`beplus aws codeartifact auth --env`), and the job refuses to run without it, because the CLI + would otherwise fall back to `dev`. + - `BE_AWS_ACCOUNT_ID`: the account holding that environment's CodeArtifact. + - Optionally `BE_CLI_VERSION`, as in library-publish. +- **AWS.** In each of those accounts, the role `github-actions-beplus-be--npm-publishing`, + trusting `repo:beplus/be:environment:` and `repo:beplus/be:ref:refs/heads/` — the role + every library's publish assumes, named the same way. - **npm.** On npmjs.com, go to `@beplus/be` → Settings → Trusted Publisher → GitHub Actions and enter organization `beplus`, repository `be`, workflow `release.yml`, environment `npm`. No npm token exists anywhere: the workflow authenticates with OIDC, and npm attaches provenance to every version. Under Publishing access, choose "Require two-factor authentication and disallow tokens". -- **GitHub.** The `npm` environment is created on the first run. Limit its deployment branches to - `main`. + Limit the `npm` GitHub environment's deployment branches to `main`. - **GitHub Packages.** Nothing to set up: the workflow publishes with its own `GITHUB_TOKEN`, and the first publish creates the `be` package linked to this repository. A `be` package that already exists but isn't linked to `beplus/be` makes that publish fail with a 403. Either delete it, or give this repository the Write role under the package's settings → Manage Actions access. -GitHub Packages is a mirror. Publishing there waits for npm, and a version that is on npm but missing -from GitHub Packages is published by the next release run, from the commit its tag points at. +GitHub Packages is where stage and prod take the bytes from, so it is published first, from the +commit the version's tag points at. A version that is tagged but missing from GitHub Packages is +published by the next `dev` run; stage, prod and main refuse it until then. `package.json` pins both `publishConfig.registry` and `publishConfig["@beplus:registry"]` to `https://registry.npmjs.org/`. A beplus machine maps the `@beplus` scope to `npm.beplus.cloud`, and -that mapping beats a plain `registry` pin, so the scoped key is what keeps `@beplus/be` from being -published there. The same mapping also overrides `npm view --registry`. Check public npm with -`curl -s https://registry.npmjs.org/@beplus%2fbe`. +that mapping beats a plain `registry` pin, so the scoped key is what keeps a hand-run +`npm publish` of `@beplus/be` from landing there. The same mapping also overrides +`npm view --registry`. Check public npm with `curl -s https://registry.npmjs.org/@beplus%2fbe`. The pin beats a `--userconfig` too, which is how the other `@beplus/*` repositories point npm at -GitHub Packages. So the workflow passes `--registry` and `--@beplus:registry` on the command line, -because only command-line flags outrank `publishConfig`. +GitHub Packages. So every publish in the workflow passes `--registry` and `--@beplus:registry` on +the command line, because only command-line flags outrank `publishConfig`. diff --git a/package-lock.json b/package-lock.json index 8bd2ab7..519ad53 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@beplus/be", - "version": "0.10.0", + "version": "0.11.0", "lockfileVersion": 2, "requires": true, "packages": { "": { "name": "@beplus/be", - "version": "0.10.0", + "version": "0.11.0", "license": "MIT", "os": [ "!win32" diff --git a/package.json b/package.json index e5c1354..691d67b 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "@beplus/be", "description": "Interactively Manage beplus CLI Versions", - "version": "0.10.0", + "version": "0.11.0", "author": "Igor Lamos ", "homepage": "https://github.com/beplus/be", "bugs": "https://github.com/beplus/be", diff --git a/scripts/install.sh b/scripts/install.sh index f34003d..7761e96 100644 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -29,10 +29,12 @@ if [[ "$CI_MODE" == "0" ]]; then } >> "$ZSHRC" fi +# main is the public channel, what the public npm registry has: dev runs ahead of every release, +# and main is fast-forwarded from prod. The clone names it, because dev is the default branch. if [ -d "$BE_TOOLS_SRC_DIR/be/.git" ]; then git -C "$BE_TOOLS_SRC_DIR/be" pull origin main else - git clone https://github.com/beplus/be "$BE_TOOLS_SRC_DIR/be" + git clone --branch main https://github.com/beplus/be "$BE_TOOLS_SRC_DIR/be" fi echo "" diff --git a/scripts/release/check-environment.sh b/scripts/release/check-environment.sh new file mode 100755 index 0000000..43ffd3b --- /dev/null +++ b/scripts/release/check-environment.sh @@ -0,0 +1,24 @@ +#!/usr/bin/env bash +# +# check-environment.sh — the release job's GitHub environment names the account +# and the CodeArtifact environment it publishes to, and both must be set: `beplus aws codeartifact +# auth` silently falls back to "dev" without an environment, which would publish a prod promotion +# into dev. + +set -euo pipefail + +readonly expected="${1:?usage: check-environment.sh }" +errors=0 + +if [[ -z "${BE_ENVIRONMENT:-}" ]]; then + echo "::error::BE_ENVIRONMENT is empty - set it to ${expected} on the '${expected}' GitHub environment." + errors=1 +elif [[ "${BE_ENVIRONMENT}" != "${expected}" ]]; then + echo "::error::BE_ENVIRONMENT is '${BE_ENVIRONMENT}' on the '${expected}' GitHub environment; it must be ${expected}." + errors=1 +fi +if [[ ! "${BE_AWS_ACCOUNT_ID:-}" =~ ^[0-9]{12}$ ]]; then + echo "::error::BE_AWS_ACCOUNT_ID must be the 12-digit account of ${expected}'s CodeArtifact, set on the '${expected}' GitHub environment." + errors=1 +fi +exit "${errors}" diff --git a/scripts/release/fetch-github-packages.sh b/scripts/release/fetch-github-packages.sh new file mode 100755 index 0000000..ebeb72c --- /dev/null +++ b/scripts/release/fetch-github-packages.sh @@ -0,0 +1,30 @@ +#!/usr/bin/env bash +# +# fetch-github-packages.sh — download @beplus/be@ from GitHub Packages, the +# tarball dev published, into . stage and prod promote exactly those bytes; a version GitHub +# Packages does not have is one dev never released, and is refused. Needs $NODE_AUTH_TOKEN. + +set -euo pipefail + +readonly version="${1:?usage: fetch-github-packages.sh }" +readonly dir="${2:?usage: fetch-github-packages.sh }" +readonly spec="@beplus/be@${version}" +readonly registry="https://npm.pkg.github.com/" + +# The token stays in the environment: npm expands ${NODE_AUTH_TOKEN} itself. +npmrc="${RUNNER_TEMP:-$(mktemp -d)}/.npmrc-github" +# shellcheck disable=SC2016 # npm expands it, not the shell +echo '//npm.pkg.github.com/:_authToken=${NODE_AUTH_TOKEN}' > "${npmrc}" + +if ! out="$(npm view "${spec}" version --userconfig "${npmrc}" --@beplus:registry="${registry}" 2>&1)"; then + if grep -q E404 <<< "${out}"; then + echo "::error::${spec} is not in GitHub Packages, so dev never released it. Promote a commit dev released." + else + echo "::error::Cannot tell whether GitHub Packages has ${spec}: ${out}" + fi + exit 1 +fi + +mkdir -p "${dir}" +npm pack "${spec}" --pack-destination "${dir}" --userconfig "${npmrc}" --@beplus:registry="${registry}" > /dev/null +ls -l "${dir}" diff --git a/scripts/release/publish-codeartifact.sh b/scripts/release/publish-codeartifact.sh new file mode 100755 index 0000000..3c2254a --- /dev/null +++ b/scripts/release/publish-codeartifact.sh @@ -0,0 +1,33 @@ +#!/usr/bin/env bash +# +# publish-codeartifact.sh — publish the tarball to this environment's CodeArtifact +# ($BE_ENVIRONMENT's; dev's is npm.beplus.cloud) under $DIST_TAG, unless it already has that version. +# Runs after the job assumed the environment's npm-publishing role and installed the beplus CLI. +# +# publishConfig pins the @beplus scope to npmjs, and only command-line flags outrank it, so the +# registry `beplus aws codeartifact auth` writes into ~/.npmrc (with its token) is also named on +# every npm command line here. + +set -euo pipefail + +readonly tarball="${1:?usage: publish-codeartifact.sh }" +readonly dist_tag="${DIST_TAG:?DIST_TAG is not set}" + +auth="$(beplus aws codeartifact auth --env "${BE_ENVIRONMENT:?BE_ENVIRONMENT is not set}" --json)" +registry="https://$(jq -r '.data.registry // empty' <<< "${auth}")" +if [[ "${registry}" == "https://" ]]; then + echo "::error::beplus aws codeartifact auth --env ${BE_ENVIRONMENT} named no registry: ${auth}" + exit 1 +fi + +spec="$(tar -xOzf "${tarball}" package/package.json | jq -r '"\(.name)@\(.version)"')" +if out="$(npm view "${spec}" version --registry="${registry}" --@beplus:registry="${registry}" 2>&1)" && [[ -n "${out}" ]]; then + echo "Skipping ${spec}: already in ${BE_ENVIRONMENT}'s CodeArtifact" + exit 0 +elif ! grep -q E404 <<< "${out}"; then + echo "::error::Cannot tell whether ${BE_ENVIRONMENT}'s CodeArtifact has ${spec}: ${out}" + exit 1 +fi + +echo "Publishing ${spec} to ${BE_ENVIRONMENT}'s CodeArtifact (dist-tag ${dist_tag})" +npm publish "${tarball}" --tag "${dist_tag}" --registry="${registry}" --@beplus:registry="${registry}" diff --git a/scripts/release/verify-version.sh b/scripts/release/verify-version.sh index c59b6e3..955ef62 100755 --- a/scripts/release/verify-version.sh +++ b/scripts/release/verify-version.sh @@ -3,21 +3,25 @@ # The release rules, in one place: the pull-request check and the release workflow both run this. # # scripts/release/verify-version.sh pr [] -# scripts/release/verify-version.sh release +# scripts/release/verify-version.sh release # # Both modes first check that bin/be, package.json and package-lock.json carry the same version. # -# pr A PR into main either bumps the version or leaves bin/be alone, because whatever lands -# on main is what scripts/install.sh installs. A bump must be newer than the base, not yet -# tagged or on npm, and described in CHANGELOG.md. A _versions/vX.Y.Z branch must arrive -# with exactly vX.Y.Z. +# pr A PR into dev either bumps the version or leaves bin/be alone, because whatever lands +# on dev is released, and main (fast-forwarded from dev through stage and prod) is what +# scripts/install.sh installs. A bump must be newer than the base, not yet tagged or on +# npm, and described in CHANGELOG.md. A _versions/vX.Y.Z branch must arrive with exactly +# vX.Y.Z. # -# release Decides what a push to main still has to do, and writes it as key=value lines to -# $GITHUB_OUTPUT (stdout when unset): version, tag, dist_tag, need_publish (not on npm -# yet), need_release (not tagged yet), need_github_packages (not in GitHub Packages yet, -# checked only when $GITHUB_PACKAGES_TOKEN is set) and source_sha (the commit the version -# was cut from: its tag when it has one, else HEAD). Re-checks CHANGELOG.md when any of -# them has to happen. +# release Decides what a push to dev, stage, prod or main still has to do, and writes it as +# key=value lines to $GITHUB_OUTPUT (stdout when unset): version, tag, dist_tag, +# source_sha (the commit the version was cut from: its tag when it has one, else HEAD), +# need_release (dev: not tagged yet), need_github_packages (dev: not in GitHub Packages +# yet) and need_npm (main: not on public npm yet). stage, prod and main build nothing, so +# there the version must already be tagged and in GitHub Packages: anything else is a +# commit dev never released, and is refused. main must also be on origin/prod, so the +# public registry only ever gets what prod already has. GitHub Packages is read with $GITHUB_PACKAGES_TOKEN. Re-checks +# CHANGELOG.md when dev still has something to release. set -euo pipefail @@ -213,60 +217,87 @@ function verify_pr() { } function verify_release() { + local branch="${1:?usage: verify-version.sh release }" finish - local need_publish=false need_release=false need_github_packages=false tag_commit source_sha + local need_release=false need_github_packages=false need_npm=false tag_commit source_sha local dist_tag=latest found=0 - npm_has_version "${NAME}" "${VERSION}" || need_publish=true tag_commit="$(remote_tag_commit "${TAG}")" - [[ -n "${tag_commit}" ]] || need_release=true source_sha="${tag_commit:-$(git rev-parse HEAD)}" [[ "${VERSION}" != *-* ]] || dist_tag=next - # GitHub Packages is a mirror: a registry that cannot be read must not hold up the npm release, - # so an unclear answer means "try to publish", and that job reports the actual error. if [[ -n "${GITHUB_PACKAGES_TOKEN:-}" ]]; then registry_has_version "${GITHUB_PACKAGES_REGISTRY}" "${NAME}" "${VERSION}" "${GITHUB_PACKAGES_TOKEN}" || found=$? - case "${found}" in - 0) ;; - 1) need_github_packages=true ;; - *) need_github_packages=true - warning "Cannot tell whether GitHub Packages has ${NAME}@${VERSION}, so publishing it is attempted." ;; - esac else + found=2 notice "GITHUB_PACKAGES_TOKEN is not set, so GitHub Packages is not checked." fi - # Publishing must not put a commit's code under a tag that names a different commit. - if [[ "${need_publish}" == "true" && -n "${tag_commit}" && "${tag_commit}" != "$(git rev-parse HEAD)" ]]; then - error "${NAME}@${VERSION} is not on npm, but tag ${TAG} points at ${tag_commit}, not HEAD. Publish from that commit or bump the version." - fi - if [[ "${need_publish}" == "true" || "${need_release}" == "true" || "${need_github_packages}" == "true" ]]; then - check_changelog "${TAG}" - fi + case "${branch}" in + dev) + [[ -n "${tag_commit}" ]] || need_release=true + # A registry that gives no clear answer means "try to publish"; that step reports the error. + case "${found}" in + 0) ;; + 1) need_github_packages=true ;; + *) need_github_packages=true + warning "Cannot tell whether GitHub Packages has ${NAME}@${VERSION}, so publishing it is attempted." ;; + esac + if [[ "${need_release}" == "true" || "${need_github_packages}" == "true" ]]; then + check_changelog "${TAG}" + fi + ;; + stage | prod | main) + [[ -n "${tag_commit}" ]] \ + || error "${TAG} is not tagged, so dev never released ${NAME}@${VERSION}. Fast-forward ${branch} to a commit dev released." + case "${found}" in + 0) ;; + 1) error "${NAME}@${VERSION} is not in GitHub Packages, so dev never released it. Fast-forward ${branch} to a commit dev released." ;; + *) error "Cannot tell whether GitHub Packages has ${NAME}@${VERSION}, and ${branch} promotes only what dev published there." ;; + esac + if [[ "${branch}" == "main" ]]; then + if ! git merge-base --is-ancestor HEAD origin/prod 2> /dev/null; then + error "main is ahead of prod: the public registry gets only what prod has. Fast-forward main to prod (git push origin origin/prod:main)." + fi + npm_has_version "${NAME}" "${VERSION}" || need_npm=true + fi + ;; + *) + error "A release runs on dev, stage, prod or main, not on ${branch}." + ;; + esac finish - if [[ "${need_publish}" == "true" || "${need_release}" == "true" || "${need_github_packages}" == "true" ]]; then - notice "Releasing ${NAME}@${VERSION} from ${source_sha} (npm: ${need_publish}, tag and GitHub Release: ${need_release}, GitHub Packages: ${need_github_packages})." - elif [[ -n "${GITHUB_PACKAGES_TOKEN:-}" ]]; then - notice "Nothing to release: ${NAME}@${VERSION} is already on npm and in GitHub Packages, and tagged ${TAG}." - else - notice "Nothing to release: ${NAME}@${VERSION} is already on npm and tagged ${TAG}." - fi + case "${branch}" in + dev) + if [[ "${need_release}" == "true" || "${need_github_packages}" == "true" ]]; then + notice "Releasing ${NAME}@${VERSION} on dev from ${source_sha} (tag and GitHub pre-release: ${need_release}, GitHub Packages: ${need_github_packages}; dev's CodeArtifact when it lacks it)." + else + notice "${NAME}@${VERSION} is tagged ${TAG} and in GitHub Packages; dev's CodeArtifact gets it if it lacks it." + fi ;; + main) + if [[ "${need_npm}" == "true" ]]; then + notice "Publishing ${NAME}@${VERSION} (${TAG}) to the public npm registry." + else + notice "Nothing to publish: ${NAME}@${VERSION} is already on the public npm registry." + fi ;; + *) + notice "Promoting ${NAME}@${VERSION} (${TAG}) to ${branch}'s CodeArtifact$([[ "${branch}" == "prod" ]] && echo ", and making its GitHub Release a full release")." ;; + esac { echo "version=${VERSION}" echo "tag=${TAG}" echo "dist_tag=${dist_tag}" echo "source_sha=${source_sha}" - echo "need_publish=${need_publish}" echo "need_release=${need_release}" echo "need_github_packages=${need_github_packages}" + echo "need_npm=${need_npm}" } >> "${GITHUB_OUTPUT:-/dev/stdout}" } case "${1:-}" in pr) shift; verify_pr "$@" ;; - release) verify_release ;; - *) echo "usage: verify-version.sh pr [] | release" >&2; exit 2 ;; + release) shift; verify_release "$@" ;; + *) echo "usage: verify-version.sh pr [] | release " >&2; exit 2 ;; esac