diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7c415e9..3fe57dd 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -24,7 +24,7 @@ jobs: - name: Verify committed Action bundle run: git diff --exit-code -- dist - name: Check shell scripts - run: shellcheck scripts/*.sh + run: shellcheck scripts/*.sh test/scripts/*.sh - name: Verify reproducible image artifact run: | scripts/package-runner-image.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 003ecbc..47ab91a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,3 +10,5 @@ - Lifecycle supervisor with fresh Docker startup and self-termination. - Direct AWS CLI bootstrap, image build tooling, CI, release SBOMs, and examples. +- One-command Classic PAT and static IAM-user Quickstart, with OIDC and GitHub + App credentials retained as the advanced path. diff --git a/README.md b/README.md index c47e649..7cb83f9 100644 --- a/README.md +++ b/README.md @@ -16,37 +16,37 @@ The Action implements and tests: - typed GitHub and AWS adapters with mocked-boundary integration tests. The production AL2023 runner image is implemented and validated locally and -through the AWS image build hooks with production `overlay2`. Private-repository -end-to-end validation remains a release gate. +through the AWS image build hooks with production `overlay2`. The complete +private-repository workflow is validated for success, job failure, cancellation, +startup timeout, service containers, and the maximum-duration backstop. -## Minimal setup +## Quickstart -The setup is two direct scripts. It does not require an infrastructure -framework: +Install the AWS CLI, GitHub CLI, `jq`, Docker, and Node.js 24. Authenticate both +CLIs, create a classic GitHub PAT with the `repo` scope, then run: ```bash export AWS_REGION=us-east-1 export GITHUB_REPOSITORY=OWNER/PRIVATE_REPOSITORY -scripts/bootstrap-aws.sh -scripts/build-microvm-image.sh +scripts/setup-quickstart.sh ``` -The first command idempotently creates the private S3 artifact bucket, -CloudWatch log groups, GitHub OIDC provider, and three least-privilege IAM -roles. It saves the discovered resource values to `build/aws-setup.json`. The -second command consumes that file automatically and saves the active image -details to `build/microvm-image.json`. +The script creates the AWS resources and runner image, configures the +repository, creates a dedicated IAM user, rotates its static access key directly +into GitHub Actions secrets, and prompts for the classic PAT. It does not use an +infrastructure framework or write the AWS secret access key to disk. -## Usage +The Quickstart IAM user deliberately includes bootstrap, image build, and runner +lifecycle permissions. Use it only with private repositories and trusted +workflow changes. See [advanced credentials](docs/advanced-credentials.md) to +replace both long-lived credentials with GitHub OIDC and a GitHub App. -The workflow needs an existing active MicroVM image, a least-privilege MicroVM -execution role, AWS credentials obtained through GitHub OIDC, and a short-lived -GitHub App installation token with repository Administration write access. +## Usage -Copy [examples/basic.yml](examples/basic.yml) into a private repository's -`.github/workflows/` directory, configure the referenced variables and secret, -then pin this Action and its dependencies to reviewed immutable commits. +Copy [examples/basic.yml](examples/basic.yml) into the private repository's +`.github/workflows/` directory. The Quickstart script configures every variable +and secret referenced by this workflow. The start job emits a unique label for one target job. The runner is JIT-only and single-use. Its supervisor self-terminates after that job; the explicit stop @@ -74,6 +74,7 @@ installation. Detailed guides: - [installation](docs/installation.md) +- [advanced credentials](docs/advanced-credentials.md) - [security model](docs/security.md) - [operations and quotas](docs/operations.md) - [testing and release gates](docs/testing.md) diff --git a/action.yml b/action.yml index b3be2d7..22f9678 100644 --- a/action.yml +++ b/action.yml @@ -9,8 +9,8 @@ inputs: required: true github-token: description: - GitHub App installation token or compatible fine-grained PAT used to - create a repository JIT runner + Classic PAT with repo scope, compatible fine-grained PAT, or GitHub App + installation token used to create a repository JIT runner required: false image-id: description: Lambda MicroVM image ARN or identifier diff --git a/docs/advanced-credentials.md b/docs/advanced-credentials.md new file mode 100644 index 0000000..0f5a9f9 --- /dev/null +++ b/docs/advanced-credentials.md @@ -0,0 +1,54 @@ +# Advanced credentials + +The advanced path replaces the Quickstart's static AWS access key and classic +GitHub PAT with short-lived credentials. Runner behavior and AWS resources are +otherwise identical. + +## 1. Create the AWS resources and image + +Use local AWS credentials that can create IAM roles, an IAM OIDC provider, an S3 +bucket, and CloudWatch log groups: + +```bash +export AWS_REGION=us-east-1 +export GITHUB_REPOSITORY=OWNER/PRIVATE_REPOSITORY + +scripts/bootstrap-aws.sh +scripts/build-microvm-image.sh +scripts/configure-github.sh +``` + +The bootstrap creates a GitHub OIDC launch role trusted only for the +repository's `main` branch. Set `GITHUB_DEFAULT_BRANCH` for another branch, or +set `GITHUB_OIDC_SUBJECT` to an exact GitHub Environment or ref subject. Do not +use a wildcard subject for untrusted pull-request refs. + +No IAM user or stored AWS access key is required by GitHub in this mode. + +## 2. Create a GitHub App + +Create and install a GitHub App only on the runner repository. Grant repository +Administration read/write permission so it can create, inspect, and delete JIT +runners. + +Record its App ID and download its private key, then configure them: + +```bash +gh variable set RUNNER_APP_ID --body APP_ID +gh secret set RUNNER_APP_PRIVATE_KEY < path/to/app.private-key.pem +``` + +The helper can configure these values with the other repository settings: + +```bash +RUNNER_APP_ID=APP_ID \ +RUNNER_APP_PRIVATE_KEY_FILE=path/to/app.private-key.pem \ +scripts/configure-github.sh +``` + +## 3. Configure the workflow + +Copy [the advanced workflow](../examples/advanced.yml) into +`.github/workflows/microvm-runner.yml`. It requests `id-token: write`, assumes +the repository-scoped AWS launch role, and mints a short-lived GitHub App +installation token for each start job. diff --git a/docs/installation.md b/docs/installation.md index 37d6650..9d1759f 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -1,103 +1,73 @@ # Installation Version 1 is for private repositories with trusted workflow changes. It requires -an ARM64-capable Lambda MicroVM Region and an AWS account with enough MicroVM -memory quota for at least one 4 GiB runner. +an ARM64-capable Lambda MicroVM Region and enough regional MicroVM memory quota +for at least one 4 GiB runner. -## 1. Create the AWS resources +## Quickstart -Use local AWS credentials that can create IAM roles, an IAM OIDC provider, an S3 -bucket, and CloudWatch log groups: +Install these local prerequisites: -```bash -export AWS_REGION=us-east-1 -export GITHUB_REPOSITORY=OWNER/PRIVATE_REPOSITORY - -scripts/bootstrap-aws.sh -``` - -This direct, idempotent script creates: - -- one private, encrypted, versioned S3 artifact bucket; -- build and runtime CloudWatch log groups with 30-day retention; -- the account-level GitHub Actions OIDC provider if it is absent; -- an image build role; -- a restricted MicroVM runtime role; -- a GitHub OIDC launch role trusted only for the repository's `main` branch. +- AWS CLI with credentials allowed to create IAM, S3, CloudWatch Logs, and + Lambda MicroVM resources; +- GitHub CLI authenticated to the target repository; +- `jq`, Docker, and Node.js 24. -It writes the resulting values to `build/aws-setup.json`. Run it again to -reconcile the same resources. - -For a different default branch, set `GITHUB_DEFAULT_BRANCH`. For a GitHub -Environment or another exact OIDC subject, set `GITHUB_OIDC_SUBJECT` explicitly. -Do not use a wildcard subject for untrusted pull-request refs. - -No IAM user or stored AWS access key is needed by GitHub. - -## 2. Build the MicroVM image - -The build command reads `build/aws-setup.json` automatically: +Create a classic GitHub personal access token with the `repo` scope. Then clone +this repository and run: ```bash -scripts/build-microvm-image.sh -``` - -It packages and uploads a content-addressed artifact, creates or updates the -image, waits for validation, activates the successful version, and keeps a -bounded rollback set. The active ARN and version are written to -`build/microvm-image.json`. - -## 3. Create a GitHub App - -Create and install a GitHub App only on the runner repository. Grant repository -Administration read/write permission so it can create, inspect, and delete JIT -runners. - -Record its App ID and download its private key. A compatible fine-grained PAT -can be passed directly, but short-lived installation tokens are preferred. - -## 4. Configure the GitHub repository - -With `gh auth status` working, set the generated AWS and image values: +export AWS_REGION=us-east-1 +export GITHUB_REPOSITORY=OWNER/PRIVATE_REPOSITORY -```bash -scripts/configure-github.sh +scripts/setup-quickstart.sh ``` -Then set the GitHub App credentials: +Paste the classic PAT when prompted. Alternatively, provide it for unattended +setup: ```bash -gh variable set RUNNER_APP_ID --body APP_ID -gh secret set RUNNER_APP_PRIVATE_KEY < path/to/app.private-key.pem +GH_PERSONAL_ACCESS_TOKEN=TOKEN scripts/setup-quickstart.sh ``` -Alternatively, configure everything in the helper invocation: +The script: -```bash -RUNNER_APP_ID=APP_ID \ -RUNNER_APP_PRIVATE_KEY_FILE=path/to/app.private-key.pem \ -scripts/configure-github.sh -``` +1. creates or reconciles the S3 bucket, CloudWatch log groups, and image build + and runtime IAM roles; +2. packages, uploads, validates, and activates the runner image; +3. configures the repository variables; +4. creates a dedicated `lambda-microvm-github-runner-quickstart` IAM user; +5. grants that user bootstrap, image build, and runner lifecycle permissions; +6. rotates its access key directly into the `AWS_ACCESS_KEY_ID` and + `AWS_SECRET_ACCESS_KEY` GitHub Actions secrets; +7. sets the PAT as `GH_PERSONAL_ACCESS_TOKEN`. -The helper creates these repository variables: +The secret access key is never written to the setup output or printed. +Re-running the script reconciles resources, builds a new image version, and +rotates the dedicated access key. -- `MICROVM_AWS_REGION`; -- `MICROVM_LAUNCH_ROLE_ARN`; -- `MICROVM_EXECUTION_ROLE_ARN`; -- `MICROVM_RUNTIME_LOG_GROUP`; -- `MICROVM_RUNNER_IMAGE_ARN`; -- `MICROVM_RUNNER_IMAGE_VERSION`. +> **Quickstart security boundary:** These are long-lived credentials with broad +> product permissions. Use them only in private repositories where workflow +> changes are trusted. Never expose them to untrusted `pull_request_target` +> workflows. Copy [the basic workflow](../examples/basic.yml) into -`.github/workflows/microvm-runner.yml`. Pin every third-party Action and this -Action to reviewed immutable commits before production use. +`.github/workflows/microvm-runner.yml`. Pin Actions to reviewed immutable +versions before production use. -## 5. Verify +## Verify Run the workflow manually and confirm: 1. start emits a unique label and MicroVM ID; -2. the target runs on ARM64 and `docker info`, Buildx, and Compose succeed; +2. the target runs on ARM64 and Docker, Buildx, and Compose succeed; 3. the JIT runner processes only that job; 4. the MicroVM reaches `TERMINATED`; -5. no GitHub token or JIT payload appears in Actions or CloudWatch logs. +5. no GitHub token, AWS credential, or JIT payload appears in logs. + +## Advanced credentials + +For short-lived credentials, use GitHub OIDC for AWS and a GitHub App +installation token instead. The standalone bootstrap enables the OIDC provider +and launch role by default. See [advanced credentials](advanced-credentials.md) +and [the advanced workflow](../examples/advanced.yml). diff --git a/docs/operations.md b/docs/operations.md index d97257f..31e0c86 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -1,5 +1,24 @@ # Operations +## Quickstart credential rotation + +Re-run the credential helper to rotate the dedicated IAM user's access key and +replace both GitHub Actions secrets: + +```bash +export GITHUB_REPOSITORY=OWNER/REPOSITORY +scripts/configure-quickstart-credentials.sh +``` + +The helper installs the new secret pair before deleting the previous key. Rotate +the classic PAT separately with: + +```bash +gh secret set GH_PERSONAL_ACCESS_TOKEN --repo "${GITHUB_REPOSITORY}" +``` + +Delete the dedicated IAM user and PAT when the integration is no longer used. + ## Quotas MicroVM API and memory quotas are shared per AWS account and Region. The diff --git a/docs/security.md b/docs/security.md index b24abe2..b7755e0 100644 --- a/docs/security.md +++ b/docs/security.md @@ -8,20 +8,26 @@ and trusted workflow changes only. Public fork pull requests are unsupported. ## Credentials -- Start uses a short-lived GitHub App installation token. The token is masked - before validation or external work. +- Quickstart stores a classic PAT with `repo` scope and a dedicated IAM user's + access key as GitHub Actions secrets. The IAM user can reconcile this + product's bootstrap resources, build images, and manage runner MicroVMs. +- Quickstart is limited to private repositories with trusted workflow changes. + Rotate or delete both credentials when they are no longer needed. +- Advanced setup uses a short-lived GitHub App installation token and obtains + AWS credentials through GitHub OIDC. +- GitHub tokens are masked before validation or external work. - The encoded JIT configuration and compressed payload are masked and never included in errors, outputs, or supervisor logs. -- GitHub-hosted start and stop jobs obtain AWS credentials through OIDC. - The default MicroVM execution role can write its logs and terminate runner MicroVMs. It has no application deployment permissions. - The runtime role's only unscoped resource permission is `lambda:TerminateMicrovm`, because that API does not expose a per-instance IAM resource ARN. No other Lambda or application action is granted by it. -- Deployment jobs should assume a separate role through GitHub OIDC. +- Deployment jobs should use a separate identity and must not inherit the + Quickstart IAM user's bootstrap permissions. -The GitHub App private key never enters the MicroVM. No long-lived AWS key is -required or documented. +The classic PAT, AWS secret access key, and GitHub App private key never enter +the MicroVM. ## Network diff --git a/docs/testing.md b/docs/testing.md index 9d8c462..94d9d03 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -5,15 +5,16 @@ ```bash npm ci npm run check -shellcheck scripts/*.sh +shellcheck scripts/*.sh test/scripts/*.sh scripts/package-runner-image.sh npm run test:image npm audit --audit-level=high ``` -`npm run check` covers strict TypeScript, 58 Action tests, 17 supervisor tests, -and the bundled Action. Supervisor tests also run successfully under the image's -Python 3.9 runtime. +`npm run check` covers strict TypeScript, 58 Action tests, Quickstart IAM +credential creation and rotation tests, 17 supervisor tests, and the bundled +Action. Supervisor tests also run successfully under the image's Python 3.9 +runtime. `npm run test:image` requires an ARM64 Docker host. It verifies: diff --git a/examples/advanced.yml b/examples/advanced.yml new file mode 100644 index 0000000..3d99136 --- /dev/null +++ b/examples/advanced.yml @@ -0,0 +1,63 @@ +name: Lambda MicroVM runner + +on: + workflow_dispatch: + +permissions: + contents: read + id-token: write + +jobs: + start-runner: + runs-on: ubuntu-latest + outputs: + label: ${{ steps.start.outputs.label }} + microvm-id: ${{ steps.start.outputs.microvm-id }} + region: ${{ steps.start.outputs.region }} + steps: + - uses: aws-actions/configure-aws-credentials@v6 + with: + role-to-assume: ${{ vars.MICROVM_LAUNCH_ROLE_ARN }} + aws-region: ${{ vars.MICROVM_AWS_REGION }} + + - uses: actions/create-github-app-token@v3 + id: app-token + with: + app-id: ${{ vars.RUNNER_APP_ID }} + private-key: ${{ secrets.RUNNER_APP_PRIVATE_KEY }} + + - uses: neebs12/lambda-microvm-github-runner@v1 + id: start + with: + mode: start + github-token: ${{ steps.app-token.outputs.token }} + image-id: ${{ vars.MICROVM_RUNNER_IMAGE_ARN }} + image-version: ${{ vars.MICROVM_RUNNER_IMAGE_VERSION }} + execution-role-arn: ${{ vars.MICROVM_EXECUTION_ROLE_ARN }} + cloudwatch-log-group: ${{ vars.MICROVM_RUNTIME_LOG_GROUP }} + maximum-duration-seconds: "3600" + + job: + needs: start-runner + runs-on: ${{ needs.start-runner.outputs.label }} + steps: + - uses: actions/checkout@v6 + - run: uname -a + - run: docker info + - run: docker buildx version + - run: docker compose version + + stop-runner: + if: ${{ always() }} + needs: [start-runner, job] + runs-on: ubuntu-latest + steps: + - uses: aws-actions/configure-aws-credentials@v6 + with: + role-to-assume: ${{ vars.MICROVM_LAUNCH_ROLE_ARN }} + aws-region: ${{ needs.start-runner.outputs.region }} + + - uses: neebs12/lambda-microvm-github-runner@v1 + with: + mode: stop + microvm-id: ${{ needs.start-runner.outputs.microvm-id }} diff --git a/examples/basic.yml b/examples/basic.yml index 01740b9..2c8ab47 100644 --- a/examples/basic.yml +++ b/examples/basic.yml @@ -5,7 +5,6 @@ on: permissions: contents: read - id-token: write jobs: start-runner: @@ -15,23 +14,17 @@ jobs: microvm-id: ${{ steps.start.outputs.microvm-id }} region: ${{ steps.start.outputs.region }} steps: - # Pin third-party actions to reviewed commit SHAs in production. - uses: aws-actions/configure-aws-credentials@v6 with: - role-to-assume: ${{ vars.MICROVM_LAUNCH_ROLE_ARN }} + aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: ${{ vars.MICROVM_AWS_REGION }} - - uses: actions/create-github-app-token@v3 - id: app-token - with: - app-id: ${{ vars.RUNNER_APP_ID }} - private-key: ${{ secrets.RUNNER_APP_PRIVATE_KEY }} - - uses: neebs12/lambda-microvm-github-runner@v1 id: start with: mode: start - github-token: ${{ steps.app-token.outputs.token }} + github-token: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }} image-id: ${{ vars.MICROVM_RUNNER_IMAGE_ARN }} image-version: ${{ vars.MICROVM_RUNNER_IMAGE_VERSION }} execution-role-arn: ${{ vars.MICROVM_EXECUTION_ROLE_ARN }} @@ -55,7 +48,8 @@ jobs: steps: - uses: aws-actions/configure-aws-credentials@v6 with: - role-to-assume: ${{ vars.MICROVM_LAUNCH_ROLE_ARN }} + aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: ${{ needs.start-runner.outputs.region }} - uses: neebs12/lambda-microvm-github-runner@v1 diff --git a/package.json b/package.json index 71241c4..339aa3c 100644 --- a/package.json +++ b/package.json @@ -16,9 +16,10 @@ "lint": "eslint src test", "test": "vitest run", "test:image": "scripts/test-runner-image.sh", + "test:scripts": "bash test/scripts/quickstart-credentials.test.sh", "test:supervisor": "python3 -m unittest discover -s runner-image/test -p 'test_*.py'", "typecheck": "tsc --noEmit", - "check": "npm run format:check && npm run lint && npm run typecheck && npm test && npm run test:supervisor && npm run build" + "check": "npm run format:check && npm run lint && npm run typecheck && npm test && npm run test:scripts && npm run test:supervisor && npm run build" }, "dependencies": { "@actions/core": "3.0.1", diff --git a/scripts/bootstrap-aws.sh b/scripts/bootstrap-aws.sh index dee2ef3..76ed045 100755 --- a/scripts/bootstrap-aws.sh +++ b/scripts/bootstrap-aws.sh @@ -11,6 +11,7 @@ readonly GITHUB_REPOSITORY="${GITHUB_REPOSITORY:-}" readonly PROJECT_NAME="${PROJECT_NAME:-lambda-microvm-github-runner}" readonly GITHUB_DEFAULT_BRANCH="${GITHUB_DEFAULT_BRANCH:-main}" readonly GITHUB_OIDC_SUBJECT="${GITHUB_OIDC_SUBJECT:-repo:${GITHUB_REPOSITORY}:ref:refs/heads/${GITHUB_DEFAULT_BRANCH}}" +readonly ENABLE_GITHUB_OIDC="${ENABLE_GITHUB_OIDC:-true}" readonly LOG_RETENTION_DAYS="${LOG_RETENTION_DAYS:-30}" readonly OUTPUT_FILE="${OUTPUT_FILE:-${REPOSITORY_ROOT}/build/aws-setup.json}" @@ -34,6 +35,9 @@ fail() { fail "LOG_RETENTION_DAYS must be a positive integer" ((LOG_RETENTION_DAYS > 0)) || fail "LOG_RETENTION_DAYS must be a positive integer" +[[ "${ENABLE_GITHUB_OIDC}" == "true" || + "${ENABLE_GITHUB_OIDC}" == "false" ]] || + fail "ENABLE_GITHUB_OIDC must be true or false" for command in aws jq mktemp sed tr; do command -v "${command}" >/dev/null 2>&1 || @@ -183,27 +187,29 @@ aws s3api put-bucket-tagging \ create_log_group "${BUILD_LOG_GROUP}" create_log_group "${RUNTIME_LOG_GROUP}" -if oidc_json="$( - aws iam get-open-id-connect-provider \ - --open-id-connect-provider-arn "${OIDC_PROVIDER_ARN}" \ - --output json 2>/dev/null -)"; then - log "Using GitHub Actions OIDC provider ${OIDC_PROVIDER_ARN}" - if ! jq -e '.ClientIDList | index("sts.amazonaws.com")' \ - <<<"${oidc_json}" >/dev/null; then - aws iam add-client-id-to-open-id-connect-provider \ +if [[ "${ENABLE_GITHUB_OIDC}" == "true" ]]; then + if oidc_json="$( + aws iam get-open-id-connect-provider \ --open-id-connect-provider-arn "${OIDC_PROVIDER_ARN}" \ - --client-id sts.amazonaws.com + --output json 2>/dev/null + )"; then + log "Using GitHub Actions OIDC provider ${OIDC_PROVIDER_ARN}" + if ! jq -e '.ClientIDList | index("sts.amazonaws.com")' \ + <<<"${oidc_json}" >/dev/null; then + aws iam add-client-id-to-open-id-connect-provider \ + --open-id-connect-provider-arn "${OIDC_PROVIDER_ARN}" \ + --client-id sts.amazonaws.com + fi + else + log "Creating GitHub Actions OIDC provider" + aws iam create-open-id-connect-provider \ + --url "https://token.actions.githubusercontent.com" \ + --client-id-list sts.amazonaws.com \ + --tags \ + "Key=Project,Value=${PROJECT_NAME}" \ + "Key=ManagedBy,Value=lambda-microvm-github-runner" \ + >/dev/null fi -else - log "Creating GitHub Actions OIDC provider" - aws iam create-open-id-connect-provider \ - --url "https://token.actions.githubusercontent.com" \ - --client-id-list sts.amazonaws.com \ - --tags \ - "Key=Project,Value=${PROJECT_NAME}" \ - "Key=ManagedBy,Value=lambda-microvm-github-runner" \ - >/dev/null fi jq -n '{ @@ -319,11 +325,13 @@ upsert_role \ "${temporary_directory}/lambda-trust.json" \ "${temporary_directory}/runtime-permissions.json" \ "Runtime permissions for single-use Lambda MicroVM GitHub runners" -upsert_role \ - "${GITHUB_ROLE_NAME}" \ - "${temporary_directory}/github-trust.json" \ - "${temporary_directory}/github-permissions.json" \ - "GitHub OIDC role for launching and terminating runner MicroVMs" +if [[ "${ENABLE_GITHUB_OIDC}" == "true" ]]; then + upsert_role \ + "${GITHUB_ROLE_NAME}" \ + "${temporary_directory}/github-trust.json" \ + "${temporary_directory}/github-permissions.json" \ + "GitHub OIDC role for launching and terminating runner MicroVMs" +fi mkdir -p "$(dirname -- "${OUTPUT_FILE}")" jq -n \ @@ -335,6 +343,7 @@ jq -n \ --arg buildLogGroup "${BUILD_LOG_GROUP}" \ --arg runtimeLogGroup "${RUNTIME_LOG_GROUP}" \ --arg githubOidcSubject "${GITHUB_OIDC_SUBJECT}" \ + --argjson githubOidcEnabled "${ENABLE_GITHUB_OIDC}" \ '{ region: $region, artifactBucket: $artifactBucket, @@ -343,7 +352,8 @@ jq -n \ githubLaunchRoleArn: $githubLaunchRoleArn, buildLogGroup: $buildLogGroup, runtimeLogGroup: $runtimeLogGroup, - githubOidcSubject: $githubOidcSubject + githubOidcSubject: $githubOidcSubject, + githubOidcEnabled: $githubOidcEnabled }' | tee "${OUTPUT_FILE}" log "AWS setup saved to ${OUTPUT_FILE}" diff --git a/scripts/configure-github.sh b/scripts/configure-github.sh index 2d5aa6e..fbd8af5 100755 --- a/scripts/configure-github.sh +++ b/scripts/configure-github.sh @@ -42,9 +42,11 @@ set_variable() { set_variable \ MICROVM_AWS_REGION \ "$(jq -r '.region // empty' "${SETUP_FILE}")" -set_variable \ - MICROVM_LAUNCH_ROLE_ARN \ - "$(jq -r '.githubLaunchRoleArn // empty' "${SETUP_FILE}")" +if [[ "$(jq -r '.githubOidcEnabled // true' "${SETUP_FILE}")" == "true" ]]; then + set_variable \ + MICROVM_LAUNCH_ROLE_ARN \ + "$(jq -r '.githubLaunchRoleArn // empty' "${SETUP_FILE}")" +fi set_variable \ MICROVM_EXECUTION_ROLE_ARN \ "$(jq -r '.executionRoleArn // empty' "${SETUP_FILE}")" diff --git a/scripts/configure-quickstart-credentials.sh b/scripts/configure-quickstart-credentials.sh new file mode 100755 index 0000000..dbde4ef --- /dev/null +++ b/scripts/configure-quickstart-credentials.sh @@ -0,0 +1,304 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +readonly SCRIPT_DIR +REPOSITORY_ROOT="$(cd -- "${SCRIPT_DIR}/.." && pwd)" +readonly REPOSITORY_ROOT +readonly SETUP_FILE="${AWS_SETUP_FILE:-${REPOSITORY_ROOT}/build/aws-setup.json}" +readonly PROJECT_NAME="${PROJECT_NAME:-lambda-microvm-github-runner}" +readonly USER_NAME="${QUICKSTART_IAM_USER_NAME:-${PROJECT_NAME}-quickstart}" +readonly POLICY_NAME="${QUICKSTART_IAM_POLICY_NAME:-${PROJECT_NAME}-quickstart}" + +log() { + echo "$*" >&2 +} + +fail() { + log "ERROR: $*" + exit 1 +} + +for command in aws gh jq mktemp tr; do + command -v "${command}" >/dev/null 2>&1 || + fail "Required command is unavailable: ${command}" +done + +[[ -f "${SETUP_FILE}" ]] || + fail "Run scripts/bootstrap-aws.sh first; ${SETUP_FILE} does not exist" +[[ "${USER_NAME}" =~ ^[A-Za-z0-9+=,.@_-]{1,64}$ ]] || + fail "QUICKSTART_IAM_USER_NAME is invalid" +[[ "${POLICY_NAME}" =~ ^[A-Za-z0-9+=,.@_-]{1,128}$ ]] || + fail "QUICKSTART_IAM_POLICY_NAME is invalid" + +repository="${GITHUB_REPOSITORY:-}" +if [[ -z "${repository}" ]]; then + repository="$(gh repo view --json nameWithOwner --jq '.nameWithOwner')" +fi +[[ "${repository}" =~ ^[^/[:space:]]+/[^/[:space:]]+$ ]] || + fail "GITHUB_REPOSITORY must use owner/repository format" + +REGION="$(jq -er '.region' "${SETUP_FILE}")" +readonly REGION +ARTIFACT_BUCKET="$(jq -er '.artifactBucket' "${SETUP_FILE}")" +readonly ARTIFACT_BUCKET +BUILD_ROLE_ARN="$(jq -er '.buildRoleArn' "${SETUP_FILE}")" +readonly BUILD_ROLE_ARN +EXECUTION_ROLE_ARN="$(jq -er '.executionRoleArn' "${SETUP_FILE}")" +readonly EXECUTION_ROLE_ARN +GITHUB_ROLE_ARN="$(jq -er '.githubLaunchRoleArn' "${SETUP_FILE}")" +readonly GITHUB_ROLE_ARN +BUILD_LOG_GROUP="$(jq -er '.buildLogGroup' "${SETUP_FILE}")" +readonly BUILD_LOG_GROUP +RUNTIME_LOG_GROUP="$(jq -er '.runtimeLogGroup' "${SETUP_FILE}")" +readonly RUNTIME_LOG_GROUP + +export AWS_MAX_ATTEMPTS=6 +export AWS_RETRY_MODE=standard +export AWS_PAGER="" + +identity_json="$(aws sts get-caller-identity --region "${REGION}" --output json)" +readonly identity_json +account_id="$(jq -er '.Account' <<<"${identity_json}")" +readonly account_id +caller_arn="$(jq -er '.Arn' <<<"${identity_json}")" +readonly caller_arn +partition="${caller_arn#arn:}" +partition="${partition%%:*}" +readonly partition + +readonly USER_ARN="arn:${partition}:iam::${account_id}:user/${USER_NAME}" +readonly OIDC_PROVIDER_ARN="arn:${partition}:iam::${account_id}:oidc-provider/token.actions.githubusercontent.com" +readonly BUCKET_ARN="arn:${partition}:s3:::${ARTIFACT_BUCKET}" +readonly BUILD_LOG_ARN="arn:${partition}:logs:${REGION}:${account_id}:log-group:${BUILD_LOG_GROUP}:*" +readonly RUNTIME_LOG_ARN="arn:${partition}:logs:${REGION}:${account_id}:log-group:${RUNTIME_LOG_GROUP}:*" +readonly BUILD_LOG_GROUP_ARN="arn:${partition}:logs:${REGION}:${account_id}:log-group:${BUILD_LOG_GROUP}" +readonly RUNTIME_LOG_GROUP_ARN="arn:${partition}:logs:${REGION}:${account_id}:log-group:${RUNTIME_LOG_GROUP}" +readonly INTERNET_EGRESS_ARN="arn:${partition}:lambda:${REGION}:aws:network-connector:aws-network-connector:INTERNET_EGRESS" +readonly NO_INGRESS_ARN="arn:${partition}:lambda:${REGION}:aws:network-connector:aws-network-connector:NO_INGRESS" + +temporary_directory="$(mktemp -d)" +cleanup() { + rm -rf "${temporary_directory}" +} +trap cleanup EXIT + +jq -n \ + --arg userArn "${USER_ARN}" \ + --arg bucketArn "${BUCKET_ARN}" \ + --arg buildRoleArn "${BUILD_ROLE_ARN}" \ + --arg executionRoleArn "${EXECUTION_ROLE_ARN}" \ + --arg githubRoleArn "${GITHUB_ROLE_ARN}" \ + --arg oidcProviderArn "${OIDC_PROVIDER_ARN}" \ + --arg buildLogArn "${BUILD_LOG_ARN}" \ + --arg runtimeLogArn "${RUNTIME_LOG_ARN}" \ + --arg buildLogGroupArn "${BUILD_LOG_GROUP_ARN}" \ + --arg runtimeLogGroupArn "${RUNTIME_LOG_GROUP_ARN}" \ + --arg internetEgressArn "${INTERNET_EGRESS_ARN}" \ + --arg noIngressArn "${NO_INGRESS_ARN}" \ + '{ + Version: "2012-10-17", + Statement: [ + { + Sid: "IdentifyAccount", + Effect: "Allow", + Action: "sts:GetCallerIdentity", + Resource: "*" + }, + { + Sid: "ManageArtifactBucket", + Effect: "Allow", + Action: [ + "s3:CreateBucket", + "s3:GetBucketLocation", + "s3:ListBucket", + "s3:PutBucketEncryption", + "s3:PutBucketPublicAccessBlock", + "s3:PutBucketTagging", + "s3:PutBucketVersioning" + ], + Resource: $bucketArn + }, + { + Sid: "ManageImageArtifacts", + Effect: "Allow", + Action: [ + "s3:DeleteObject", + "s3:GetObject", + "s3:GetObjectVersion", + "s3:PutObject" + ], + Resource: ($bucketArn + "/*") + }, + { + Sid: "DiscoverLogGroups", + Effect: "Allow", + Action: "logs:DescribeLogGroups", + Resource: "*" + }, + { + Sid: "ManageProjectLogs", + Effect: "Allow", + Action: [ + "logs:CreateLogGroup", + "logs:CreateLogStream", + "logs:PutLogEvents", + "logs:PutRetentionPolicy", + "logs:TagResource" + ], + Resource: [ + $buildLogGroupArn, + $runtimeLogGroupArn, + $buildLogArn, + $runtimeLogArn + ] + }, + { + Sid: "ManageProjectRoles", + Effect: "Allow", + Action: [ + "iam:CreateRole", + "iam:GetRole", + "iam:PutRolePolicy", + "iam:TagRole", + "iam:UpdateAssumeRolePolicy" + ], + Resource: [$buildRoleArn, $executionRoleArn, $githubRoleArn] + }, + { + Sid: "PassProjectRoles", + Effect: "Allow", + Action: "iam:PassRole", + Resource: [$buildRoleArn, $executionRoleArn] + }, + { + Sid: "ManageGitHubOidcProvider", + Effect: "Allow", + Action: [ + "iam:AddClientIDToOpenIDConnectProvider", + "iam:CreateOpenIDConnectProvider", + "iam:GetOpenIDConnectProvider", + "iam:TagOpenIDConnectProvider" + ], + Resource: $oidcProviderArn + }, + { + Sid: "ManageOwnQuickstartCredentials", + Effect: "Allow", + Action: [ + "iam:CreateAccessKey", + "iam:CreateUser", + "iam:DeleteAccessKey", + "iam:GetUser", + "iam:ListAccessKeys", + "iam:PutUserPolicy", + "iam:TagUser" + ], + Resource: $userArn + }, + { + Sid: "ManageLambdaMicrovms", + Effect: "Allow", + Action: [ + "lambda:*Microvm*", + "lambda:ListTags", + "lambda:TagResource", + "lambda:UntagResource" + ], + Resource: "*" + }, + { + Sid: "PassManagedNetworkConnectors", + Effect: "Allow", + Action: "lambda:PassNetworkConnector", + Resource: [$internetEgressArn, $noIngressArn] + } + ] + }' >"${temporary_directory}/quickstart-policy.json" + +if aws iam get-user --user-name "${USER_NAME}" >/dev/null 2>&1; then + log "Updating IAM user ${USER_NAME}" + aws iam tag-user \ + --user-name "${USER_NAME}" \ + --tags \ + "Key=Project,Value=${PROJECT_NAME}" \ + "Key=ManagedBy,Value=lambda-microvm-github-runner" +else + log "Creating IAM user ${USER_NAME}" + aws iam create-user \ + --user-name "${USER_NAME}" \ + --tags \ + "Key=Project,Value=${PROJECT_NAME}" \ + "Key=ManagedBy,Value=lambda-microvm-github-runner" \ + >/dev/null +fi + +aws iam put-user-policy \ + --user-name "${USER_NAME}" \ + --policy-name "${POLICY_NAME}" \ + --policy-document "file://${temporary_directory}/quickstart-policy.json" + +old_access_keys=() +old_access_key_count=0 +while IFS= read -r access_key_id; do + if [[ -n "${access_key_id}" ]]; then + old_access_keys[old_access_key_count]="${access_key_id}" + old_access_key_count=$((old_access_key_count + 1)) + fi +done < <( + aws iam list-access-keys \ + --user-name "${USER_NAME}" \ + --query 'AccessKeyMetadata[].AccessKeyId' \ + --output text | + tr '\t' '\n' +) + +old_access_key_start=0 +if ((old_access_key_count >= 2)); then + log "Removing the oldest access key before rotation" + aws iam delete-access-key \ + --user-name "${USER_NAME}" \ + --access-key-id "${old_access_keys[0]}" + old_access_key_start=1 +fi + +new_access_key_json="$(aws iam create-access-key --user-name "${USER_NAME}")" +new_access_key_id="$(jq -er '.AccessKey.AccessKeyId' <<<"${new_access_key_json}")" +new_secret_access_key="$( + jq -er '.AccessKey.SecretAccessKey' <<<"${new_access_key_json}" +)" +unset new_access_key_json + +if ! printf 'AWS_ACCESS_KEY_ID=%s\nAWS_SECRET_ACCESS_KEY=%s\n' \ + "${new_access_key_id}" \ + "${new_secret_access_key}" | + gh secret set --repo "${repository}" --app actions --env-file -; then + aws iam delete-access-key \ + --user-name "${USER_NAME}" \ + --access-key-id "${new_access_key_id}" || + true + fail "GitHub secret update failed; the new AWS access key was revoked" +fi + +for (( + index = old_access_key_start; + index < old_access_key_count; + index++ +)); do + aws iam delete-access-key \ + --user-name "${USER_NAME}" \ + --access-key-id "${old_access_keys[index]}" +done + +unset new_secret_access_key +gh variable set MICROVM_AWS_REGION \ + --repo "${repository}" \ + --body "${REGION}" + +jq --arg quickstartUserArn "${USER_ARN}" \ + '. + {quickstartUserArn: $quickstartUserArn}' \ + "${SETUP_FILE}" >"${temporary_directory}/aws-setup.json" +mv "${temporary_directory}/aws-setup.json" "${SETUP_FILE}" + +log "Set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY for ${repository}" +log "Quickstart IAM user: ${USER_ARN}" diff --git a/scripts/setup-quickstart.sh b/scripts/setup-quickstart.sh new file mode 100755 index 0000000..efe5d37 --- /dev/null +++ b/scripts/setup-quickstart.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +readonly SCRIPT_DIR + +log() { + echo "$*" >&2 +} + +: "${AWS_REGION:=${AWS_DEFAULT_REGION:-}}" +: "${AWS_REGION:?Set AWS_REGION or AWS_DEFAULT_REGION}" +: "${GITHUB_REPOSITORY:?Set GITHUB_REPOSITORY to owner/repository}" +readonly ENABLE_GITHUB_OIDC=false +export AWS_REGION GITHUB_REPOSITORY ENABLE_GITHUB_OIDC + +"${SCRIPT_DIR}/bootstrap-aws.sh" +"${SCRIPT_DIR}/build-microvm-image.sh" +"${SCRIPT_DIR}/configure-github.sh" +"${SCRIPT_DIR}/configure-quickstart-credentials.sh" + +if [[ -n "${GH_PERSONAL_ACCESS_TOKEN:-}" ]]; then + printf '%s' "${GH_PERSONAL_ACCESS_TOKEN}" | + gh secret set GH_PERSONAL_ACCESS_TOKEN \ + --repo "${GITHUB_REPOSITORY}" \ + --app actions + unset GH_PERSONAL_ACCESS_TOKEN +else + log "Paste a classic GitHub PAT with the repo scope when prompted." + gh secret set GH_PERSONAL_ACCESS_TOKEN \ + --repo "${GITHUB_REPOSITORY}" \ + --app actions +fi + +log "Quickstart setup complete for ${GITHUB_REPOSITORY}" diff --git a/test/scripts/quickstart-credentials.test.sh b/test/scripts/quickstart-credentials.test.sh new file mode 100755 index 0000000..3b0a32c --- /dev/null +++ b/test/scripts/quickstart-credentials.test.sh @@ -0,0 +1,145 @@ +#!/usr/bin/env bash +set -euo pipefail + +REPOSITORY_ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)" +readonly REPOSITORY_ROOT +temporary_directory="$(mktemp -d)" +cleanup() { + rm -rf "${temporary_directory}" +} +trap cleanup EXIT + +mkdir -p "${temporary_directory}/bin" + +cat >"${temporary_directory}/bin/aws" <<'EOF' +#!/usr/bin/env bash +set -euo pipefail +echo "$*" >>"${MOCK_AWS_LOG}" + +case "$1 $2" in + "sts get-caller-identity") + jq -n '{ + Account: "123456789012", + Arn: "arn:aws:iam::123456789012:user/bootstrap" + }' + ;; + "iam get-user") + [[ "${MOCK_USER_EXISTS}" == "true" ]] + ;; + "iam create-user" | "iam tag-user") + ;; + "iam put-user-policy") + for argument in "$@"; do + if [[ "${argument}" == file://* ]]; then + cp "${argument#file://}" "${MOCK_POLICY_CAPTURE}" + fi + done + ;; + "iam list-access-keys") + printf '%s\n' "${MOCK_OLD_KEYS}" + ;; + "iam create-access-key") + jq -n \ + --arg id "${MOCK_NEW_ACCESS_KEY_ID}" \ + --arg secret "${MOCK_NEW_SECRET_ACCESS_KEY}" \ + '{AccessKey: {AccessKeyId: $id, SecretAccessKey: $secret}}' + ;; + "iam delete-access-key") + ;; + *) + echo "Unexpected aws invocation: $*" >&2 + exit 1 + ;; +esac +EOF + +cat >"${temporary_directory}/bin/gh" <<'EOF' +#!/usr/bin/env bash +set -euo pipefail +echo "$*" >>"${MOCK_GH_LOG}" + +case "$1 $2" in + "secret set") + secret_input="$(cat)" + expected="$( + printf 'AWS_ACCESS_KEY_ID=%s\nAWS_SECRET_ACCESS_KEY=%s' \ + "${MOCK_NEW_ACCESS_KEY_ID}" \ + "${MOCK_NEW_SECRET_ACCESS_KEY}" + )" + [[ "${secret_input}" == "${expected}" ]] + ;; + "variable set") + ;; + *) + echo "Unexpected gh invocation: $*" >&2 + exit 1 + ;; +esac +EOF + +chmod 0755 "${temporary_directory}/bin/aws" "${temporary_directory}/bin/gh" + +run_case() { + local case_name="$1" + local user_exists="$2" + local old_keys="$3" + local case_directory="${temporary_directory}/${case_name}" + mkdir -p "${case_directory}" + + jq -n '{ + region: "us-east-1", + artifactBucket: "runner-artifacts", + buildRoleArn: "arn:aws:iam::123456789012:role/runner-build", + executionRoleArn: "arn:aws:iam::123456789012:role/runner-runtime", + githubLaunchRoleArn: "arn:aws:iam::123456789012:role/runner-launch", + buildLogGroup: "/lambda-microvms/runner/build", + runtimeLogGroup: "/lambda-microvms/runner/runtime" + }' >"${case_directory}/setup.json" + + export MOCK_AWS_LOG="${case_directory}/aws.log" + export MOCK_GH_LOG="${case_directory}/gh.log" + export MOCK_POLICY_CAPTURE="${case_directory}/policy.json" + export MOCK_USER_EXISTS="${user_exists}" + export MOCK_OLD_KEYS="${old_keys}" + export MOCK_NEW_ACCESS_KEY_ID="AKIAQUICKSTARTTEST" + export MOCK_NEW_SECRET_ACCESS_KEY="quickstart-secret-must-not-leak" + + output="$( + PATH="${temporary_directory}/bin:${PATH}" \ + AWS_SETUP_FILE="${case_directory}/setup.json" \ + GITHUB_REPOSITORY="owner/repository" \ + PROJECT_NAME="runner" \ + "${REPOSITORY_ROOT}/scripts/configure-quickstart-credentials.sh" 2>&1 + )" + + [[ "${output}" != *"${MOCK_NEW_SECRET_ACCESS_KEY}"* ]] + jq -e ' + .quickstartUserArn == + "arn:aws:iam::123456789012:user/runner-quickstart" + ' "${case_directory}/setup.json" >/dev/null + jq -e ' + any(.Statement[]; + (.Action | type) == "array" and + (.Action | index("lambda:*Microvm*")) != null + ) and + any(.Statement[]; + (.Action | type) == "array" and + (.Action | index("iam:CreateRole")) != null + ) and + any(.Statement[]; + (.Action | type) == "array" and + (.Action | index("s3:PutObject")) != null + ) + ' "${case_directory}/policy.json" >/dev/null + grep -q "secret set.*--env-file -" \ + "${case_directory}/gh.log" +} + +run_case "create" "false" "" +grep -q "iam create-user" "${temporary_directory}/create/aws.log" + +run_case "rotate" "true" "AKIAOLDKEY" +grep -q "iam delete-access-key.*AKIAOLDKEY" \ + "${temporary_directory}/rotate/aws.log" + +echo "Quickstart credential tests passed"