Skip to content

feat: add Packer build composite actions and reusable workflow - #31

Open
posquit0 wants to merge 3 commits into
mainfrom
feat/packer-build
Open

posquit0 wants to merge 3 commits into
mainfrom
feat/packer-build

Conversation

@posquit0

Copy link
Copy Markdown
Member

👋 Background

Building the images, as the counterpart to the checks in #1. One caller has to serve three events — a push that builds only what changed, a schedule that builds everything, and a manual dispatch — and one build directory has to be able to produce several images from several variable files.

🔗 Related Issues

⚙️ Description

New composite actions

Action Purpose
packer.build packer init then packer build for one template directory, with a summary of what it produced
packer.build-targets Resolve the build directories and their variable files into a job matrix

packer.build streams the build with tee instead of going through shell.run. shell.run redirects both streams to files and prints them once the command returns, which is right for a check that takes seconds and wrong for an image build that takes minutes.

It reads the artifacts and the errors from the two sections Packer closes with, so a summary can name the AMIs a build created without the template carrying a manifest post-processor. Worth noting against the discussion on #1: packer build does expose these as structured records under -machine-readable, unlike packer validate. It is still not used, for a different reason — it replaces the human-readable log that the same job needs.

packer.build-targets resolves targets in two modes. changed walks up from each changed directory to the nearest one holding a Packer template, so editing a variable file under <build>/vars/ still builds <build>. all enumerates the build directories under builds_dir.

Variants

A build directory with a vars/ subdirectory yields one target per variable file in it:

builds/ubuntu-2604/
  source.pkr.hcl
  datarize.auto.pkrvars.hcl   <- shared, Packer loads it for every variant
  vars/
    prod.pkrvars.hcl          <- one build
    staging.pkrvars.hcl       <- another build

A directory without vars/ is built once with no -var-file, so nothing has to change for a build that has only one variant today. vars_dir and vars_pattern make the convention configurable, and builds / variants filter it for a manual dispatch.

New reusable workflow

Workflow Jobs Purpose
packer.templates.build.yaml targets, build, report Resolve the targets, fan out one build per target, publish one report

scope defaults to auto: changed directories on a push or a pull_request, everything on any other event. That is what lets a single caller cover all three events:

on:
  push:
    branches: [main]
    paths: ['builds/**']
  schedule:
  - cron: "0 18 * * 0"
  workflow_dispatch:
    inputs:
      builds:
        description: "Comma-separated build names. Empty builds everything."
        required: false
      variants:
        description: "Comma-separated variant names. Empty builds every variant."
        required: false

permissions:
  contents: read
  id-token: write

jobs:
  build:
    uses: tedilabs/github-actions/.github/workflows/packer.templates.build.yaml@main
    with:
      builds: ${{ inputs.builds }}
      variants: ${{ inputs.variants }}
      aws_region: ap-northeast-2
      aws_github_oidc_iam_role: arn:aws:iam::000000000000:role/github-actions-packer

Each target gets its own concurrency group, so a scheduled full build and the build of a freshly merged change never overlap on the same image. max_parallel defaults to 4: images are built on real instances, so a full build should stay clear of cloud quotas rather than take whatever the runner limits allow.

✅ Verification

yamllint, actionlint, and shellcheck pass.

The target resolver was run standalone against a fixture tree (two build directories, one with two variable files, one directory without a template, one unrelated directory):

Case Result
all Three targets; the directory without a template is skipped
changed from builds/ubuntu-2604/vars Walks up to builds/ubuntu-2604, both variants
changed from unrelated directories No targets
builds + variants filters One target
all over an empty builds/ No targets

The build action's parsing was run against real packer build output (packer 1.16.0, null builder) and two fixtures for the AMI shapes:

Case Result
Successful build 1 artifact(s), artifact line in the summary
Failed provisioner Failed, > [!CAUTION] callout with the error line
amazon-ebs with two regions Both ami- lines kept under the --> line
Partly failed build (both sections) Errors first, then the artifact that still succeeded

🚨 To Reviewers

Three things worth a second opinion.

  • name in the matrix entry. The Terraform workflows repeat ${{ matrix.project }}${{ matrix.workspace && format(' / {0}', matrix.workspace) || '' }} at each use. Here the resolver emits name instead, because the build workflow needs it in four places. It is a small deviation from the existing style; say the word and I will repeat the expression instead.
  • No automatic pull request lookup. On a push after a merge there is no pull request context, so the report goes to the job summary unless the caller passes pr_number. github.pr.head-run from feat: add Terraform plan and apply workflows for workspaces #5 could resolve the merged pull request the way terraform.workspaces.apply does, but it requires a workflow to look a run up in, which a build workflow has no use for. Easy to add once feat: add Terraform plan and apply workflows for workspaces #5 lands if you want the comment.
  • The AMI output shapes are fixtures. The success and failure parsing was verified against real packer build runs, but the amazon-ebs cases could not be: they need AWS credentials. Those two rows are hand-written fixtures in the documented format, so they carry some hallucination risk until a real build confirms them.

Run `packer init` and `packer build` for one template directory and
render what it produced.

The build is streamed with `tee` rather than captured through
`shell.run`, which only prints the output once the command has returned:
an image build runs for minutes, so a silent job log is not an option.

The artifacts and the errors are read from the two sections Packer
closes with, so the summary names the AMIs a build created without the
template needing a manifest post-processor. `-machine-readable` does
expose them as structured records, unlike `packer validate`, but it
replaces the readable log the same job needs.
Resolve the build directories to build and the variable files they build
with, as one matrix entry each.

`changed` mode walks up from each changed directory to the nearest one
holding a Packer template, so editing a variable file under
`<build>/vars/` still builds `<build>`. `all` mode enumerates the build
directories instead, for a scheduled or manually dispatched full build.

A build directory with a `vars/` subdirectory yields one target per
variable file in it, which is how one directory turns into several
builds. Any `*.auto.pkrvars.hcl` beside the template stays shared, since
Packer loads it for every one of them.
Wire the target resolution, the build matrix, and the report together,
so one caller serves a push, a schedule, and a manual dispatch: `scope`
defaults to `auto`, which builds the changed directories on a `push` or
a `pull_request` and everything on any other event.

Each target takes its own concurrency group, so a scheduled full build
and the build of a freshly merged change never overlap on the same
image, and `max_parallel` keeps a full build clear of cloud quotas.

The report follows the Terraform plan and apply workflows: every target
collects a `headline` and a `details` section, which `github.matrix-report`
publishes as one table plus a collapsed section per build.
@github-actions

Copy link
Copy Markdown

👋 Welcome! Looks like this is your first pull request.

Hey, thanks for your contribution! Please give us a bit of time to review it. 😄

Please check out our contributing guidelines.

@github-actions github-actions Bot added the size/L Large size issue or PR. label Sep 21, 2026
@posquit0

Copy link
Copy Markdown
Member Author

Heads-up that is not about this pull request: the repository's actionlint / check-jsonschema lint is currently broken on main, so it did not run here.

793fe44 (chore: update .github/workflows/github-actions.integration.yaml via Terraform) replaced the reusable github-actions.integration.yaml with a copy of the caller, so that file now ends in:

jobs:
  integration:
    uses: tedilabs/github-actions/.github/workflows/github-actions.integration.yaml@main

which is itself. It no longer has on: workflow_call, so both it and github-actions.yaml (the real caller) fail at startup. main fails the same way at 793fe44, and succeeded at 2899f46, so this is not from this branch. The Auto Label / Add labels by Labeler failure is from the same batch: .github/labeler.yaml does not exist in the repository.

Ran the missing checks locally instead, all clean: yamllint, actionlint, shellcheck, and check-jsonschema (vendor.github-workflows on the workflow, vendor.github-actions on both actions).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L Large size issue or PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant