From a7bc16bc46e5bead9e28d1c53904c02431113574 Mon Sep 17 00:00:00 2001 From: Jason Aricheta Date: Sun, 26 Jul 2026 15:44:45 +1200 Subject: [PATCH 1/4] Highlight warm build server workflow --- README.md | 138 ++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 119 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index b8cbad9..15089a0 100644 --- a/README.md +++ b/README.md @@ -123,25 +123,125 @@ The start job emits a unique label for one target job. The runner is JIT-only and single-use. Its supervisor self-terminates after that job; the explicit stop job and platform maximum duration are independent cleanup backstops. -### Experimental warm cache - -Warm mode reuses the MicroVM and its local Docker cache while creating a fresh -JIT runner for every job. Set a human-readable `server` name on `start`, pass -the opaque `server` output to `stop`, and configure the Quickstart-created -`MICROVM_WARM_STATE_TABLE` for reuse across workflow runs. The informational -`warm-hit` output reports whether an existing member was reused. - -> [!WARNING] A reused machine is a cache, not an isolation boundary. Enable warm -> mode only for trusted workflows in the same private repository. Fork pull -> requests are rejected. The Action uses authenticated MicroVM control traffic, -> conditional DynamoDB leases, and the platform lifetime as its natural cleanup -> backstop; it does not run a scheduled garbage collector. - -See [the warm-cache example](examples/warm-cache.yml) and the -[implementation and testing design](docs/warm-cache.md). `server-capacity` is an -optional request-local creation bound: an available member always wins; if all -members are busy, omission permits another member, while a supplied bound fails -once the current active count reaches it. +## Warm build servers (experimental) + +Ordinary `start` and `stop` remain fully ephemeral: they create and terminate a +MicroVM for one job. Warm mode is an explicit opt-in that lets trusted jobs +reuse a suspended MicroVM as a short-lived build server. Its memory, Docker +layers, package caches, and downloaded toolchains remain available, but every +job still receives a fresh, single-use GitHub JIT runner registration. + +Give `start` a stable, human-readable `server` pool name and the +Quickstart-created `MICROVM_WARM_STATE_TABLE`. Pass the opaque `server` output +to `stop`; warm `stop` releases the lease and suspends the MicroVM instead of +terminating it. + +One warm pool member follows this lifecycle: + +```text + server: docker-builds + + first start target job #1 stop + | | | + v v v + create MicroVM -> fresh JIT runner -> cache on disk -> suspend + | + next start | + | | + +--------------- resume same MicroVM <--------------+ + | + v + fresh JIT runner + | + v + target job #2 -> suspend + | + cached state is still present +``` + +That makes warm mode useful for repeated, compatible workloads: + +- Docker can reuse its native image and build-layer cache without exporting a + cache archive between jobs. +- Package managers and build tools can reuse downloads, intermediate outputs, + and installed toolchains left on the machine. +- A named pool can serve multiple workflow runs. Conditional DynamoDB leases + ensure that only one job owns a particular MicroVM at a time. +- Suspended members retain memory and disk state without remaining active + between jobs. Lambda's maximum lifetime still provides a final cleanup + boundary. + +The complete pattern uses the `server` output—not the human-readable pool +name—when releasing the exact lease: + +```yaml +jobs: + start-runner: + runs-on: ubuntu-latest + outputs: + label: ${{ steps.start.outputs.label }} + server: ${{ steps.start.outputs.server }} + warm-hit: ${{ steps.start.outputs.warm-hit }} + region: ${{ steps.start.outputs.region }} + steps: + - uses: aws-actions/configure-aws-credentials@v6 + with: + aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} + aws-region: ${{ vars.MICROVM_AWS_REGION }} + + - uses: neebs12/lambda-microvm-github-runner@v1 + id: start + with: + mode: start + github-token: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }} + image-id: ${{ vars.MICROVM_RUNNER_IMAGE_ARN }} + image-version: ${{ vars.MICROVM_RUNNER_IMAGE_VERSION }} + execution-role-arn: ${{ vars.MICROVM_EXECUTION_ROLE_ARN }} + server: docker-builds + server-capacity: "2" + state-table: ${{ vars.MICROVM_WARM_STATE_TABLE }} + max-lifetime-seconds: "7200" + + build: + needs: start-runner + runs-on: ${{ needs.start-runner.outputs.label }} + steps: + - uses: actions/checkout@v6 + - run: docker build --tag app:ci . + + stop-runner: + if: ${{ always() && needs.start-runner.outputs.server != '' }} + needs: [start-runner, build] + runs-on: ubuntu-latest + steps: + - uses: aws-actions/configure-aws-credentials@v6 + with: + aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} + aws-region: ${{ needs.start-runner.outputs.region }} + + - uses: neebs12/lambda-microvm-github-runner@v1 + with: + mode: stop + server: ${{ needs.start-runner.outputs.server }} +``` + +`warm-hit` reports whether `start` resumed an existing member. An available +member always wins. When every member is busy, `server-capacity` optionally +limits whether that request may create another member; omitting it leaves pool +growth unbounded by the Action. + +> [!WARNING] A reused machine is a cache, not an isolation boundary. A trusted +> job has root-equivalent access through Docker and can alter state consumed by +> later jobs. Use a warm pool only for equally trusted workflows in the same +> private repository. Fork pull requests are rejected. Warm caches are +> temporary, may expire, and do not guarantee that every workload becomes +> faster. + +See the copy-ready [warm-cache workflow](examples/warm-cache.yml) and the +[warm-cache design and testing guide](docs/warm-cache.md) for lifecycle, +security, failure-recovery, and capacity details. ## Status From 59f79a144c9cfa740345165e97ad6700874e3f98 Mon Sep 17 00:00:00 2001 From: Jason Aricheta Date: Sun, 26 Jul 2026 15:47:46 +1200 Subject: [PATCH 2/4] Refresh audited development dependencies --- package-lock.json | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/package-lock.json b/package-lock.json index 74325de..0448d57 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1660,16 +1660,16 @@ "license": "MIT" }, "node_modules/brace-expansion": { - "version": "5.0.7", - "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.7.tgz", - "integrity": "sha512-7oFy703dxfY3/NLxC1fh2SUCQ0H9rmAY+5EpDVfXjUTTs+HEwR2nYaqLv+GWcTsumwxPfiz6CzCNkwXwBUwqCA==", + "version": "5.0.8", + "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.8.tgz", + "integrity": "sha512-JZyDyq3D4AUifKTPOB7DELf6XsB3WdPuNxCtob1vFXPsSXhdAiHBWJ/tJ8HAc9aH84BK+5JFZLNkJKx3G9kzQg==", "dev": true, "license": "MIT", "dependencies": { "balanced-match": "^4.0.2" }, "engines": { - "node": "18 || 20 || >=22" + "node": "20 || >=22" } }, "node_modules/chai": { @@ -2487,9 +2487,9 @@ "license": "MIT" }, "node_modules/nanoid": { - "version": "3.3.15", - "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.15.tgz", - "integrity": "sha512-y7Wygv/7mEOvxTuEQDB8StXdMRBWf1kR/tlhAzBRUFkB2jfcLOAxO/SHmOO2zgz1pVgK29/kyupn059/bCHdjA==", + "version": "3.3.16", + "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.16.tgz", + "integrity": "sha512-bzlKTyNJ7+LdGIIwy8ijFpIqEQIvafahV7eYykJ8Cvh42EdJeODoJ6gUJXpQJvej1BddH8OqTXZNE/KfbWAu8Q==", "dev": true, "funding": [ { @@ -2631,9 +2631,9 @@ } }, "node_modules/postcss": { - "version": "8.5.16", - "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.16.tgz", - "integrity": "sha512-vuwillviilfKZsg0VGj5R/YwwcHx4SLsIOI/7K6mQkWx+l5cUHTjj5g0AasTBcyXsbfTgrwsUNmVUb5xVwyPwg==", + "version": "8.5.23", + "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.23.tgz", + "integrity": "sha512-g50586zr4bZmwFiTlflMu8E0bDTb5I5gertgwAKmsdUlTQIhZtunzUlD1WSzwcVWPoAVpsrA6vlfCD7oXvRwgg==", "dev": true, "funding": [ { @@ -2651,7 +2651,7 @@ ], "license": "MIT", "dependencies": { - "nanoid": "^3.3.12", + "nanoid": "^3.3.16", "picocolors": "^1.1.1", "source-map-js": "^1.2.1" }, From 14e26d56c1b368a83adbb915a190b3f67763d794 Mon Sep 17 00:00:00 2001 From: Jason Aricheta Date: Sun, 26 Jul 2026 15:58:17 +1200 Subject: [PATCH 3/4] Simplify warm cache explanation --- README.md | 66 +++++++++++++++++++++---------------------------------- 1 file changed, 25 insertions(+), 41 deletions(-) diff --git a/README.md b/README.md index 15089a0..19e9147 100644 --- a/README.md +++ b/README.md @@ -125,54 +125,40 @@ job and platform maximum duration are independent cleanup backstops. ## Warm build servers (experimental) -Ordinary `start` and `stop` remain fully ephemeral: they create and terminate a -MicroVM for one job. Warm mode is an explicit opt-in that lets trusted jobs -reuse a suspended MicroVM as a short-lived build server. Its memory, Docker -layers, package caches, and downloaded toolchains remain available, but every -job still receives a fresh, single-use GitHub JIT runner registration. +By default, every job gets a new MicroVM. To opt into warm reuse, give `start` a +human-readable `server` pool name and the Quickstart-created +`MICROVM_WARM_STATE_TABLE`. `start` creates or resumes a pool member, and `stop` +suspends it for another job instead of terminating it. -Give `start` a stable, human-readable `server` pool name and the -Quickstart-created `MICROVM_WARM_STATE_TABLE`. Pass the opaque `server` output -to `stop`; warm `stop` releases the lease and suspends the MicroVM instead of -terminating it. +The MicroVM is reused; the GitHub runner registration is not. Every target job +still gets a fresh, single-use JIT runner. -One warm pool member follows this lifecycle: +One member of the `docker-builds` pool looks like this: ```text - server: docker-builds - - first start target job #1 stop - | | | - v v v - create MicroVM -> fresh JIT runner -> cache on disk -> suspend - | - next start | - | | - +--------------- resume same MicroVM <--------------+ - | - v - fresh JIT runner - | - v - target job #2 -> suspend - | - cached state is still present +RUN 1: start -> [ MicroVM A | fresh runner #1 | build populates cache ] -> stop -> SUSPEND + || + memory + disk preserved + \/ +RUN 2: start -> RESUME -> [ MicroVM A | fresh runner #2 | build can reuse cache ] + -> stop -> SUSPEND ``` -That makes warm mode useful for repeated, compatible workloads: +This is useful when repeated, compatible workloads can reuse local state: - Docker can reuse its native image and build-layer cache without exporting a cache archive between jobs. - Package managers and build tools can reuse downloads, intermediate outputs, and installed toolchains left on the machine. -- A named pool can serve multiple workflow runs. Conditional DynamoDB leases - ensure that only one job owns a particular MicroVM at a time. +- A named pool can serve multiple workflow runs. Each MicroVM is leased to only + one job at a time. - Suspended members retain memory and disk state without remaining active between jobs. Lambda's maximum lifetime still provides a final cleanup boundary. -The complete pattern uses the `server` output—not the human-readable pool -name—when releasing the exact lease: +There are only two warm-specific additions to the normal workflow: set the pool +name and state table on `start`, then pass the opaque `server` output to `stop`. +The output is a lease handle, not the human-readable pool name. ```yaml jobs: @@ -198,10 +184,9 @@ jobs: image-id: ${{ vars.MICROVM_RUNNER_IMAGE_ARN }} image-version: ${{ vars.MICROVM_RUNNER_IMAGE_VERSION }} execution-role-arn: ${{ vars.MICROVM_EXECUTION_ROLE_ARN }} + # Human-readable pool name server: docker-builds - server-capacity: "2" state-table: ${{ vars.MICROVM_WARM_STATE_TABLE }} - max-lifetime-seconds: "7200" build: needs: start-runner @@ -224,6 +209,7 @@ jobs: - uses: neebs12/lambda-microvm-github-runner@v1 with: mode: stop + # Opaque lease handle returned by start server: ${{ needs.start-runner.outputs.server }} ``` @@ -232,12 +218,10 @@ member always wins. When every member is busy, `server-capacity` optionally limits whether that request may create another member; omitting it leaves pool growth unbounded by the Action. -> [!WARNING] A reused machine is a cache, not an isolation boundary. A trusted -> job has root-equivalent access through Docker and can alter state consumed by -> later jobs. Use a warm pool only for equally trusted workflows in the same -> private repository. Fork pull requests are rejected. Warm caches are -> temporary, may expire, and do not guarantee that every workload becomes -> faster. +> [!WARNING] A warm cache is not an isolation boundary. Jobs can read or alter +> state left by other jobs, so share a pool only between equally trusted +> workflows in the same private repository. Warm caches are temporary, and cache +> reuse does not guarantee that every workload becomes faster. See the copy-ready [warm-cache workflow](examples/warm-cache.yml) and the [warm-cache design and testing guide](docs/warm-cache.md) for lifecycle, From 7680de80543607047c3221d93711ec8cf0079b33 Mon Sep 17 00:00:00 2001 From: Jason Aricheta Date: Sun, 26 Jul 2026 16:01:38 +1200 Subject: [PATCH 4/4] Make warm lifecycle diagram vertical --- README.md | 39 +++++++++++++++++++++++++++++++++------ 1 file changed, 33 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 19e9147..6a5180a 100644 --- a/README.md +++ b/README.md @@ -136,12 +136,39 @@ still gets a fresh, single-use JIT runner. One member of the `docker-builds` pool looks like this: ```text -RUN 1: start -> [ MicroVM A | fresh runner #1 | build populates cache ] -> stop -> SUSPEND - || - memory + disk preserved - \/ -RUN 2: start -> RESUME -> [ MicroVM A | fresh runner #2 | build can reuse cache ] - -> stop -> SUSPEND +RUN 1 + | +start + | + v ++-----------------------+ +| MicroVM A | +| fresh runner #1 | +| build populates cache | ++-----------------------+ + | +stop + | + v +SUSPEND +memory + disk preserved + | + v +RUN 2 + | +start + resume + | + v ++-----------------------+ +| same MicroVM A | +| fresh runner #2 | +| build can reuse cache | ++-----------------------+ + | +stop + | + v +SUSPEND ``` This is useful when repeated, compatible workloads can reuse local state: