Skip to content

Serve the workspace with gitfs behind BLACKSMITH_USE_GITFS - #62

Draft
max-blacksmith wants to merge 3 commits into
mainfrom
devin/1792207525-gitfs-checkout
Draft

max-blacksmith wants to merge 3 commits into
mainfrom
devin/1792207525-gitfs-checkout

Conversation

@max-blacksmith

@max-blacksmith max-blacksmith commented Oct 17, 2026 •

Copy link
Copy Markdown

Summary

Per-job opt-in to serve $GITHUB_WORKSPACE from the git mirror sticky disk with gitfs instead of fetch + checkout. The agent sets BLACKSMITH_USE_GITFS=true from the installation's use_gitfs_checkout flag (FastActions/fa#6520, useblacksmith/web#18247); gitfs mounting a --mirror bare repo in place is useblacksmith/gitfs#2, the cold (streaming) mount is useblacksmith/gitfs#3.

Flow in getSourceInner:

# mirror setup (unchanged) except:
if BLACKSMITH_USE_GITFS=true and eligible and no mirror on the disk:
  initEmptyMirror(mirror, url)                                   # git init --bare + remote add --mirror=fetch; no blocking clone --mirror
  performedHydration = true; mirrorFresh = false
# gitfs branch, between the mirror setup and git init:
if BLACKSMITH_USE_GITFS=true:
  reason = gitfsIneligibilityReason(settings, {mirrorPath})      # linux, mirror mounted, not in a container,
                                                                 # no submodules/lfs/sparse/filter/dissociate, empty workspace
  revision = resolveInMirror(mirror, mirrorRevisionCandidates(ref, commit))
  if revision:  gitfs mount <mirror> --ref <sha> <workspace> --daemon          # warm: mounted in ms
  else:         sha = gitfs mount <mirror> --ref <coldRevision(ref, commit)> <workspace> --daemon --url <repo>   # GITFS_TOKEN in env
                                                                 # gitfs fetches --depth 1 into the mirror, serves the pack as it arrives,
                                                                 # prints the commit once it is in, completes the mirror in the background
                setBlacksmithCacheMirrorChanged(true)
  setGitfsMountPoint(workspace)
# unchanged: git init / alternates / remote add / auth — all through the mount
if mounted:  fetchRefsFromMirror(...); pointWorkspaceAt(...)     # refs/remotes/origin/*, local branch or detached HEAD, read-tree <sha>
else:        fetchAndCheckout(...)                               # the previous fetch/checkout-info/checkout block, moved verbatim

The cold branch covers the empty sticky disk (first job on a repo: no clone --mirror before the first step — babel from GitHub: checkout 3.0 s vs 13.0 s, whole job 69.9 s vs 77.1 s) and a ref the mirror has not synced (pull-request merge refs), replacing the blocking targeted fetch. The mirror ends the job as a complete ordinary mirror either way; the post step commits it.

pointWorkspaceAt produces what git checkout -B <b> origin/<b> / detached checkout would (branch config, refs/remotes/pull/N/merge for PR refs) without touching the tree; read-tree without update-index --refresh so nothing stats every file through FUSE — the first git status verifies contents instead.

Post step (main.ts cleanup, before blacksmithCache.cleanup): gitfs.teardown runs gitfs unmount (ends a fetch still in flight; a mirror left shallow is completed by the next job's mount), then waits for / SIGTERMs any gitfs mount <mirror> daemon found in /proc, because the daemon maps the mirror's packs and the sticky-disk umount would otherwise be busy.

Not in the cohort → [gitfs] not used: <reason> and the regular path; any gitfs failure → warning and the regular path (with an empty mirror that is a network fetch into the workspace, and the post step's deferred sync fills the mirror). serving_mode telemetry is unchanged (closed enum on the agent side). Caveat documented in BLACKSMITH.md: the workspace has the full history via alternates regardless of fetch-depth.

Tests: __test__/gitfs.test.ts — eligibility, candidate mapping, coldRevision, and two end-to-end tests that run when gitfs is on PATH (skipped otherwise): a warm mount of a real --mirror clone (.git through the mount, status clean and HEAD stable before and after the swap, mirror untouched) and a cold mount from an empty mirror over file:// (sha reported, file readable, mirror ends complete and unshallowed).

Testing this on the fleet

  1. Merge useblacksmith/gitfs#2 and Bump actions/upload-artifact from 4 to 5 #3, tag v0.1.0 there (the release workflow publishes static gitfs-<tag>.linux-{amd64,arm64} + .sha256).
  2. fa: the prepared rootfs PR (download by version + sha256 into /usr/local/bin/gitfs, both patchroot and stage_bootstrap paths) with the hashes from step 1; merge with FastActions/fa#6520 and roll the agent.
  3. Merge useblacksmith/web#18247 and this PR; release the action.
  4. Flip use_gitfs_checkout on one internal installation. In the checkout step look for [gitfs] … serves <sha> from the mirror (mounted in N ms) (warm) or … streamed from the remote (cold), and in the post step [gitfs] Unmounting …. [gitfs] not used: <reason> means the regular path ran. Kill switch: turn the flag off (or bypass_blacksmith_checkout); nothing persists on the disk but an ordinary mirror.

Link to Devin session: https://app.devin.ai/sessions/5af1665023e24d5f97ffbb031bcb4920
Open in Devin Desktop: https://app.devin.ai/desktop/session/5af1665023e24d5f97ffbb031bcb4920?variant=devin
Requested by: @max-blacksmith


View with [code]smith Prove this works
Need help on this PR? Tag @codesmith-bot with what you need.

When the job's installation has use_gitfs_checkout on, mount the requested
revision from the git mirror sticky disk with gitfs instead of fetching and
checking it out, create .git through the mount so git works from the first
step, and take gitfs down in the post step before the mirror is unmounted.
Anything outside the plain Linux cohort, or any gitfs failure, falls through
to the regular checkout.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

max-blacksmith and others added 2 commits October 17, 2026 04:03
…n mount)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ing first

An empty sticky disk no longer costs a blocking git clone --mirror when
the job uses gitfs: the action creates an empty bare mirror (same config
as --mirror) and runs gitfs mount --ref <rev> --url <repo>, which fetches
the revision --depth 1 into the mirror, serves the pack as it arrives and
completes the mirror in the background.  The same path replaces the
blocking targeted fetch for a ref the mirror has not synced (pull-request
merge refs).  gitfs prints the mounted commit, which is how the action
learns the sha before git has written the ref.

The e2e test (self-skips without the binary) mounts from an empty mirror
over file:// and checks the mirror ends up complete.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant