Skip to content

Each untracked file's patch in the Diff panel copies and rewrites the whole Git index on Windows: 4 extra git and ~510 ms per file (1.26 s vs 0.75 s for a tracked file) #40247

Closed

Description

@juliusmarminge

What is slow

The Diff panel loads file patches lazily, 4 at a time, as you scroll. Each file is its own review.getDiffPreview request with file: {path, previousPath, sourceKind}. When the file is untracked, the server builds a temporary review index for that single path. It copies the repository's whole index, expands it, and then adds the one file with add --intent-to-add. It does this again for every untracked file. On a large repository, every untracked file's patch waits about half a second longer than a tracked file's, and costs 4 more git processes.

Measured

  • Windows Server 2025, 4 vCPU, 16 GB, Defender real-time scanning on. Commit 511447a9f2, server from source serving the production web build, headless Chromium through Playwright.
  • Seed: 6 projects, each a clone of this repository (32,168 tracked files; .git/index is 4.5 MB); 18 devReplay threads. In one project (no remote, branch feature-4): 400 untracked files (8 new folders of 50 small .ts files) and 40 modified tracked .ts files.
  • Open a thread in that project, press Ctrl+D, scroll the diff with the mouse wheel for 25 s. 3 runs:
untracked file tracked file
single-file requests (server trace) 188 219
git processes per request 10 6
request duration, median (range) 1,262 ms (1,149-2,221) 754 ms (666-2,436)
prepareReviewIndex span 511 ms median -
  • Untracked files loaded per 25-s run: 64 (3/3 runs), each requested once. That is 256 extra git processes and 64 copies of a 4.5 MB index per run.
  • Per-step medians inside an untracked request: readUnifiedWorkingTreeReviewDiff.stagedDeletions 177 ms, .indexPath 110 ms, .expandSplitIndex 121 ms, .addUntracked 183 ms, plus the index copy and utimes.

How to measure it again

  1. Check out the commit, vp i, cd apps/web && vp build.
  2. git clone --local <a t3code checkout> p4, git remote remove origin, git checkout -b feature-4, commit one file. Then, in PowerShell:
$r='<path>\p4'
foreach($d in 0..7){ $dir="$r\newmod$d"; New-Item -ItemType Directory -Force $dir | Out-Null; foreach($k in 0..49){ Set-Content -LiteralPath "$dir\file$k.ts" -Value ("export const v$k = $k;`n" * 20) } }
$f = git -C $r ls-files -- '*.ts' | Select -First 40; foreach($x in $f){ Add-Content -LiteralPath (Join-Path $r $x) -Value '// edit' }
  1. From apps/server: T3CODE_DEV_SEED=0 node src/bin.ts serve --base-dir <home> --port 4800 --host 127.0.0.1 --dev-url http://localhost:5733; node src/bin.ts project add <p4> --base-dir <home>; node src/bin.ts thread start --base-dir <home> --provider devReplay --model todo-list --wait --json <projectId> "task 4-0". Stop it and start it again without --dev-url, so it serves the production build. Leave Defender on.
  2. node src/bin.ts pair --ttl 1h --base-dir <home> and set the URL's host to 127.0.0.1:4800. Then run SCROLL=25000 node diff.cjs <node_modules/.pnpm/playwright@*/node_modules/playwright> <pairUrl> 3 4-0 out.json (the script is in issue Scrolling a large diff on Windows runs 6 git commands per file, and 3 of them re-resolve the branch, base and merge base the preview just resolved (about 1,650 git processes to view 272 files) #40182, "How to measure it again"). Copy <home>/userdata/logs/server.trace.ndjson* right away, because the trace rotates at ~10 MB.
  3. N=10 node steps.cjs <trace files> (untracked file requests) and N=6 node steps.cjs <trace files> (tracked):
// steps.cjs <trace.ndjson>...: review.getDiffPreview requests with exactly N git processes: duration and per-step medians
const fs=require("fs");const sp=new Map();
for(const f of process.argv.slice(2))for(const l of fs.readFileSync(f,"utf8").split("\n")){try{const r=JSON.parse(l);if(r.type==="effect-span")sp.set(r.spanId,r)}catch{}}
const kids=new Map();for(const s of sp.values()){const k=kids.get(s.parentSpanId)||[];k.push(s);kids.set(s.parentSpanId,k)}
const desc=s=>{const o=[],q=[s];while(q.length){const x=q.pop();for(const c of kids.get(x.spanId)||[]){o.push(c);q.push(c)}}return o};
const agg={};let n=0;const tot=[];
for(const s of [...sp.values()].filter(s=>s.name==="ws.rpc.review.getDiffPreview")){const d=desc(s);const g=d.filter(x=>x.name==="runGitCommand");if(g.length!==+(process.env.N||10))continue;n++;tot.push(s.durationMs);
 for(const x of d.filter(x=>/prepareReviewIndex|prepareUntrackedReviewIndex/.test(x.name)))(agg["SPAN "+x.name]=agg["SPAN "+x.name]||[]).push(x.durationMs);
 for(const x of g){const k=sp.get(x.parentSpanId)?.name;(agg[k]=agg[k]||[]).push(x.durationMs)}}
const med=v=>{v=[...v].sort((a,b)=>a-b);return Math.round(v[v.length>>1])};
console.log("n",n,"median",med(tot),"range",Math.round(Math.min(...tot)),Math.round(Math.max(...tot)));
for(const [k,v] of Object.entries(agg))console.log(med(v)+"ms",(v.length/n).toFixed(1)+"x",k);

Where the time goes

In apps/server/src/vcs/GitVcsDriverCore.ts, getReviewDiffPreview calls prepareUntrackedReviewIndex(cwd, pathArgs, input.file?.path), which leads to prepareUntrackedReviewIndexOrReason. That runs ls-files --others -- <path>. When the path is untracked, prepareReviewIndex takes over. It runs diff --cached --diff-filter=D (stagedDeletions) and rev-parse --git-path index, then copyFiles the full index to a temp file and runs update-index --no-split-index on it, which rewrites all 32k entries. Only then does it run add --intent-to-add --pathspec-from-file=- for the one path. The stat and patch come after that. For a single path, almost all of that index work concerns entries the pathspec excludes.

Why it is avoidable

The request diffs one new file against an empty side. Copying and rewriting a 32k-entry index, and the two extra reads before it, happen again for every untracked file even though nothing about the rest of the index matters to the answer. The same is true of the preview's own result for the file.

What a fix has to show

The same 25-s scroll, 3 runs before and after, on the same seed:


Found by the automated Windows performance sweep on 511447a.


Found by the automated Windows performance sweep on 511447a.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions