perf: split the Daimon runtime image into cacheable layers - #8
Open
apresmoi wants to merge 2 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebuilding the Daimon runtime image after a one-line Daimon change cost a full
rebuild — apt, a 166 MB grok download, a 56 MB AGY download and the whole npm
install — because the image was built in a single
RUNwithCOPY --from=daimon_package /daimon.tgzabove it. Docker cached nothing.Measured: 105 s → 48 s for a Daimon-only rebuild. Cold build is unchanged
(103 s vs 105 s).
The layering
Registry vs offline-bundle mode is selected by interpolating a global
DAIMON_DEPENDENCY_MODEbuild arg intoFROMstage names, so BuildKit neverbuilds the stages of the mode that was not selected. The offline chain contains
no
apt-getand nocurl.What each cached step was costing, from the cold build: apt 27.9 s, grok 6.9 s,
AGY 6.3 s, codex npm 15.3 s. A shared BuildKit
/root/.npmcache mount took theremaining Daimon npm step from 40.3 s to 27.1 s.
This is a layering change, not a content change
Both images were exported and compared file-for-file: 41,108 entries, 37,814
file digests, 17 symlinks, zero differences in path, mode, size, content or
symlink target.
package.jsonandpackage-lock.jsoncome out byte-identicaleven though the single
npm installwas split into two.Verification
RUNstill runs. Checks moved into thestage that owns the artifact they guard; none were dropped, weakened or made
conditional.
examples/daimon-public-host(a realruntime: daimonorg) compiles againstSPAWNFILE_DAIMON_LOCAL_RUNTIME_IDENTITYand the generated org image builds,so the capability-receipt and contract-manifest assertions in
src/runtime/container.tsran for real against the new image.npm run test:source-provenance-dockerpasses (7 m 17 s) — the mandatorynetwork-disabled offline gate, including the
--target offline_dependency_probebuild and its tamper-rejection assertions.
npm run typecheck,npm run test:boundaries, and the Vitest suite pass.The layering is now a tested guarantee
The second commit adds a test that parses the Dockerfile into a
{stage -> parent}graph and asserts:daimon.tgzorsource-inputs.json— the property the whole restructure exists for;apt-getappears only inbase_registry,curlonly in the two registrysource stages.
Mutation-verified four ways — reparent the Daimon stage, move the tarball
COPYup into the codex layer, add
apt-getto the offline base, swap the offline grokCOPYfor acurl— all four go red, restoring goes green.It also wires
scripts/build-local-daimon-runtime.test.mjsinto.github/workflows/test.yml. That file had never run in CI: Vitest includessrc/**/*.test.tsonly, and the workflow'snode --teststep named only thenative-helper tests. Every Dockerfile guarantee in it was vacuous on a PR.
Known, not fixed
is Daimon's own dependency closure reinstalling. Moving it into the cached
layer would get to roughly 25 s, but every clean way to do that changes
${RUNTIME_ROOT}/package.json, a file that ships inside the attested image.Deferred deliberately.
.github/workflows/runtime-images.ymlstill omits Daimon from its buildmatrix, so ordinary CI never builds this Dockerfile. Out of scope here.
Stacked on
feat/operational-ecosystem; this PR is the two commits above it.