Skip to content

feat(earthlyoutputs): serve original content of digest-pinned images - #43

Draft
kmannislands wants to merge 1 commit into
mainfrom
digest-passthrough-registry
Draft

kmannislands wants to merge 1 commit into
mainfrom
digest-passthrough-registry

Conversation

@kmannislands

@kmannislands kmannislands commented Oct 7, 2026 •

Copy link
Copy Markdown

Summary

Lets the embedded registry serve the original content of a digest-pinned image, so a docker daemon using it as a mirror can pull name@sha256:… with no network access.

A local-registry export of a pulled image is a rebuilt image: BuildKit writes a new config, so its manifest digest differs from upstream. A client asking for the pinned reference (e.g. compose with image: alpine@sha256:… inside earth's WITH DOCKER) could never be served locally. This is the BuildKit half of the follow-up to EarthBuild/earthbuild#513.

Changes

  • New ref metadata key export-image-local-registry-source (earthlyoutputs exporter). When set to the digest-pinned reference an image was pulled from, the exporter also registers the original index, the manifest for the exported platform, and its config with the embedded registry. Unmodified layers keep their digests and are already served, so the pinned reference can be pulled in full.
    • The original content is read from the content store: the puller leases an image's non-layer blobs to its cache record, so the common case needs no network. Anything missing is fetched with the session's credentials.
    • Best effort: a failure is logged and the export carries on; clients then fall back to upstream.
  • eodriver: unknown manifests and blobs are a 404 (PathNotFoundError), not a 500. Most requests to a mirror are for content it doesn't have.
  • Capability earthly.localregistry.source so the earth CLI only relies on this when BuildKit supports it.

Testing

  • Unit tests:
    • sourceContent against a real local content store with a resolver that fails any network call. It returns the index, the matching platform manifest and the config. A missing root goes to the resolver.
    • imagePlatform, parseSourceRef, and the eodriver 404 mapping.
  • docker buildx bake lint-default (amd64 + arm64) is clean.
  • End to end with earth: feat: resolve digest-pinned Docker Hub images in WITH DOCKER locally earthbuild#1008. The dind daemon pulls alpine@sha256:… and alpine:3.24.2@sha256:… from the embedded registry with all upstream access cut off (dockerd proxied to a dead port). An unpinned control pull fails, proving it was offline. The same test fails on the current release.

Notes

  • Merge first; the earth PR then needs go.mod's replace and buildkitd/Earthfile's BUILDKIT_BASE_IMAGE repointed to the merge commit.
  • Only the exported platform's manifest is served, which is what a daemon of that platform requests.

🤖 Generated with Claude Code

A local registry export of a pulled image is a rebuilt image: its config,
and so its manifest digest, differ from upstream. A client asking for the
upstream reference by digest (e.g. compose with `image: alpine@sha256:...`)
can therefore not be served locally and has to reach the upstream registry.

Add the `export-image-local-registry-source` ref metadata key. When set to
the digest-pinned reference an image was pulled from, the exporter also
registers the original index, platform manifest and config with the
embedded registry. Unmodified layers keep their digests and are already
served, so the pinned reference can be pulled from the embedded registry
in full, e.g. by a docker daemon using it as a registry mirror.

The original content is read from the content store, where BuildKit keeps
it leased to the pulled image's cache record, so no network is needed in
the common case; anything missing is fetched with the session's
credentials. Failure is logged and not fatal: clients fall back to the
upstream registry.

Advertise support via the earthly.localregistry.source capability, and
report unknown manifests and blobs as 404 rather than 500, as most
requests to a mirror are for content it does not have.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 7, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown

⚠️ Are we earthbuild yet?

Warning: "earthly" occurrences have increased by 6 (2.76%)

📈 Overall Progress

Branch Total Count
main 217
This PR 223
Difference +6 (2.76%)

📁 Changes by file type:

File Type Change
Go files (.go) ❌ +6
Documentation (.md) ➖ No change
Earthfiles ➖ No change

Keep up the great work migrating from Earthly to Earthbuild! 🚀

💡 Tips for finding more occurrences

Run locally to see detailed breakdown:

./.github/scripts/count-earthly.sh

Note that the goal is not to reach 0.
There is anticipated to be at least some occurences of earthly in the source code due to backwards compatibility with config files and language constructs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-assisted Authored with AI assistance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant