Repository navigation
fix(server): checkpoints keep big files staged under batch fsync - #39927
juliusmarminge wants to merge 2 commits into
Conversation
…er small new files Reproduces #39913: with core.fsyncMethod=batch (darwin, win32), git add fails with exit 128 when a loose object is written before a file over core.bigFileThreshold is streamed into a pack. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On macOS and Windows checkpoint staging uses core.fsyncMethod=batch, and Git 2.43 through 2.56.1 loses a file over core.bigFileThreshold when a smaller loose object was written earlier in the same add (exit 128). Retry a failed batch add once with per-object fsync, and stage the embedded-repository recovery with per-object fsync too. Fixes #39913 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Thread transfer impact✅ Thread transfer remains within every enforced ceiling.
Other scenariosAll 64 scenario measurements match All 64 scenario measurements
Baseline: Scenario and decoded snapshot size10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.
Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed. |
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
|
The automated pipeline stopped on this pull request and will not merge it: the fix is written and reviewed, but proving it needs something no worker has:
It stays open for a person to finish or close. |
|
Note Written by Claude Opus 5.5 on behalf of Navid The pipeline stopped this for lack of native macOS, so here is a run on one: macOS 27.0.1 (arm64), Apple Git 2.54.0 (Apple Git-157), Node 26.13.1, no This PR's rows (
The Raw Git on the same Mac, outside T3 (repo with one empty commit,
Whole file ( To reproduce from a checkout of main gh pr diff 39927 | git apply --include=apps/server/src/vcs/GitVcsDriver.test.ts
(cd apps/server && vp test run src/vcs/GitVcsDriver.test.ts -t "captures a big file") # 2 failed, exit 128
gh pr diff 39927 | git apply --include=apps/server/src/vcs/GitVcsDriver.ts
(cd apps/server && vp test run src/vcs/GitVcsDriver.test.ts -t "captures a big file") # 2 passed
(cd apps/server && vp test run src/vcs/GitVcsDriver.test.ts) # 114 passed, 3 failed
gh pr diff 37404 | git apply --include=apps/server/src/vcs/GitVcsDriver.test.ts
(cd apps/server && vp test run src/vcs/GitVcsDriver.test.ts) # 117 passedMade with Claude Opus 5.5 via Claude Code. Reviewed by GPT-6.1 Sol via Codex. |
On macOS and Windows, a checkpoint could be lost when a turn added a small file plus a file larger than Git's big-file threshold (512 MiB by default, such as a video). Under the batch fsync mode checkpoints use there, Git 2.43–2.56.1 drops the big file's pack, and
git addexits 128. Checkpoint staging now retries a failed batch-fsync add once with per-object fsync. The existing embedded-repository recovery also stages with per-object fsync, so it can no longer hit the same Git bug. Linux is unchanged.Fixes #39913
Reproduction on main
Commit origin/main a1b9e1c, on Linux. (1) Raw Git, with the system Git 2.43.0 and with Git 2.56.1 built from source. The reporter's exact steps passed in every variant, because only the big file is new. Adding a small new file that sorts first reproduces it:
git init; git commit --allow-empty -m init; echo small > a.txt; head -c 2097152 /dev/urandom > big.bin; mkdir .git/objects/tmp_objdir-priv; GIT_OBJECT_DIRECTORY=$PWD/.git/objects/tmp_objdir-priv GIT_INDEX_FILE=../idx git -c core.bigFileThreshold=1m -c core.fsync=objects -c core.fsyncMethod=batch add -Aexits 128 withfatal: unable to rename temporary file to '.../tmp_objdir-priv/pack/pack-<sha>.pack': No such file or directory. The same happens without the private object folder. With core.fsyncMethod=fsync it succeeds in every case. (2) Through T3: in GitVcsDriver.test.ts, captureCheckpoint was run with HostProcess.Platform set to darwin or win32 on a repo with a small new file, a 2 MiB file and bigFileThreshold=1m. The logged Git commands show the firstadd -A(with core.fsyncMethod=batch) exiting 128. The recovery retryadd --ignore-errors -Athen succeeded, which hides the bug in small cases. That retry gives up after 5 s or when more than 64 untracked embedded repositories are listed. With 65 committed nested repositories the checkpoint fails with VcsProcessExitError exit 128 on darwin and win32, and passes on linux as a control. (3) Realistic size: with a 600 MiB file plus a small new file, the first add exited 128 after about 13 s and the second took about 14 s. That is well over the 5 s recovery timeout, so a real checkpoint with a file over the default 512 MiB threshold is lost. Not run on real macOS or Windows; batch mode is a Git setting that behaves the same on Linux.Evidence
Being captured: before and after in the real app. This section is filled in before the pull request can merge.
Tests
vp test run src/vcs/GitVcsDriver.test.ts -t "captures a big file"before the fix: 2 failed (darwin, win32), exit 128, which reproduces the bugvp test run src/vcs/GitVcsDriver.test.tsafter the fix: 114 passed, 1 failed. The failure is "lists the fetch and push URLs of a partial clone remote", caused by this VM's globalurl.insteadOfrewrite and unrelated to this changeGIT_CONFIG_GLOBAL=/dev/null vp test run src/vcs/GitVcsDriver.test.ts: 115 passedvp run check:changed: format, knip and typecheck passed; lint failed onprefer-catch-tags(now fixed and clean undervp lint); tests failed only on the insteadOf test abovevp fmt,vp lint,knip:checkpassedNot in this change
url.insteadOfrewrite.git addon macOS and Windows was left as is.Who did the work
Run by the t3code factory in T3 Code.