Skip to content

Scrolling a large diff on Windows runs 6 git commands per file, and 3 of them re-resolve the branch, base and merge base the preview just resolved (about 1,650 git processes to view 272 files) #40182

Closed

Description

@juliusmarminge

What is slow

The Diff panel loads file patches lazily, 4 at a time, as you scroll. Each file is its own review.getDiffPreview request (with file: {path, previousPath, sourceKind}), and each runs 6 git processes on the server. Three of them answer the same question for every file of the same comparison: what the current branch is, whether the base ref exists, and what the merge base is. Scrolling through a big change on Windows therefore starts thousands of git processes. Every patch waits for those three before its own diff starts.

Measured

  • Windows Server 2025, 4 vCPU, 16 GB, Defender real-time scanning on. Commit 80871270f9, server from source, production web build served by the server, headless Chromium through Playwright.
  • Seed: 6 projects, each a clone of this repository (~32k files); 18 devReplay threads. In one project (no remote, branch feature-3), 799 tracked .ts files modified (one line appended to each).
  • Open a thread in that project, press Ctrl+D, then scroll the diff with the mouse wheel for 60 s. n=3:
per 60-s scroll run
unique files loaded 272 (3/3 runs)
single-file review.getDiffPreview requests 276
git processes per single-file request 6 (872 of 872 requests over the 3 runs)
git processes for those requests, 3 runs 5,232
  • Per single-file request (server trace, n=32 in the open-only runs): 1,220 ms median (770–1,702). The three shared reads take 557 ms median (295–737) of it: GitVcsDriver.review.currentBranch 126 ms, GitVcsDriver.branchExists 208 ms, GitVcsDriver.resolveReviewMergeBase 215 ms. The file's own work: review.listUntracked 207 ms, getReviewDiffPreview.stat 512 ms, getReviewDiffPreview.patch 188 ms.
  • Run alone on the same repository, each of these commands takes 42–72 ms (rev-parse 41–60 ms, a single-path diff --raw --numstat 58–72 ms). The in-server numbers are higher because the per-file requests and their repeats run concurrently.

How to measure it again

Setup (Windows, fresh machine)

  1. Check out the commit, vp i, then cd apps/web && vp build.
  2. Seed 6 repositories: git clone --local <a t3code checkout> p<i> for i = 0..5. For p0–p2 set origin to https://github.com/pingdotgg/t3code.git, git fetch origin main, git branch -u origin/main main. For p3–p5, git remote remove origin. In each, git checkout -b feature-<i> and commit one new file.
  3. From apps/server, start a seeding server: T3CODE_DEV_SEED=0 node src/bin.ts serve --base-dir <home> --port 4800 --host 127.0.0.1 --dev-url http://localhost:5733. Then from apps/server run node seed.cjs <home> <reposDir> 0 6 (below). Stop the server and restart it the same way without --dev-url, so it serves the production build. Leave Defender on with no exclusions.
  4. In p3, modify 800 tracked TypeScript files (PowerShell): $f = git -C p3 ls-files -- '*.ts' | Select -First 800; foreach($x in $f){ Add-Content -LiteralPath (Join-Path p3 $x) -Value '// edit' }. git status shows 799 modified files.
  5. Pair: node src/bin.ts pair --ttl 1h --base-dir <home>. Take the http…pair#token=… URL and set its host to 127.0.0.1:4800. Use a fresh token for each driver run.
  6. SCROLL=60000 node diff.cjs <node_modules/.pnpm/playwright@*/node_modules/playwright> <pairUrl> 3 3-0 out.json (3-0 matches a thread title in p3). Then run node reviewspans.cjs <server.trace.ndjson files…> on <home>/userdata/logs/server.trace.ndjson* (copy them right away; they rotate at ~10 MB).
// seed.cjs <home> <reposDir> <from> <to>: run from apps/server; project add + 3 devReplay threads per repo p<i>
const { execFile } = require("node:child_process"); const path = require("node:path");
const [home, reposDir, from, to] = process.argv.slice(2);
const run = (a) => new Promise((res, rej) => execFile(process.execPath, ["src/bin.ts", ...a], { env: { ...process.env, T3CODE_DEV_SEED: "0" }, maxBuffer: 1 << 26 }, (e, o, er) => e ? rej(new Error(String(er || e.message))) : res(o)));
(async () => { for (let i = +from; i < +to; i++) { const out = await run(["project", "add", path.join(reposDir, `p${i}`), "--base-dir", home]);
  const pid = /Added project (\S+)/.exec(out)[1];
  for (let t = 0; t < 3; t++) await run(["thread", "start", "--base-dir", home, "--provider", "devReplay", "--model", "todo-list", "--wait", "--json", pid, `task ${i}-${t}`]); }
  console.log("DONE"); })();
// diff.cjs <playwrightDir> <pairUrl> <loads> <threadText> <out.json>: per load, open the thread, make sure the Diff panel is closed, press Ctrl+D,
// record every review.* request (offset ms, id, payload, answer offset), interrupts, first file time and long tasks. SCROLL=<ms> scrolls the diff after it settles.
const [pw, pairUrl, loads, threadText, out] = process.argv.slice(2); const { chromium } = require(pw); const fs = require("node:fs");
(async () => { const b = await chromium.launch(); const ctx = await b.newContext({ viewport: { width: 1600, height: 1000 } }); const res = [];
  let p = await ctx.newPage(); await p.goto(pairUrl); await p.waitForTimeout(8000); const origin = new URL(pairUrl).origin; await p.close();
  for (let i = 0; i < +loads; i++) { p = await ctx.newPage(); let cur = null;
    await p.addInitScript(() => { window.__lt = []; new PerformanceObserver((l) => { for (const e of l.getEntries()) window.__lt.push([e.startTime, e.duration]); }).observe({ type: "longtask", buffered: true }); });
    p.on("websocket", (ws) => { ws.on("framesent", (f) => { if (!cur) return; let m; try { m = JSON.parse(String(f.payload)); } catch { return; }
        if (m.tag && /review\./.test(m.tag)) cur.reqs.push([Date.now() - cur.w0, m.id, m.tag, JSON.stringify(m.payload).slice(0, 400)]);
        if (m._tag === "Interrupt") cur.reqs.push([Date.now() - cur.w0, "INT", m.requestId]); });
      ws.on("framereceived", (f) => { if (!cur) return; const s = String(f.payload); if (!s.includes('"Exit"')) return; try { for (const x of [].concat(JSON.parse(s))) if (x._tag === "Exit") { const r = cur.reqs.find((q) => String(q[1]) === String(x.requestId)); if (r) r.push(Date.now() - cur.w0, s.length, x.exit?._tag); } } catch {} }); });
    await p.goto(origin + "/"); await p.waitForTimeout(6000);
    const c0 = Date.now(); cur = { load: i, reqs: [], w0: c0 };
    await p.locator("[data-thread-item]", { hasText: threadText }).first().click(); await p.waitForTimeout(5000);
    if (await p.locator("[data-diff-file-key]").count() > 0) { await p.keyboard.press("Control+d"); await p.waitForTimeout(3000); }
    cur.reqs.push([Date.now() - c0, "OPEN"]); const w0 = Date.now(); const t0 = await p.evaluate(() => performance.now());
    await p.keyboard.press("Control+d"); let first = null, lastN = -1, stable = Date.now();
    while (Date.now() - w0 < 40000) { const n = await p.locator("[data-diff-file-key]").count(); if (n > 0 && first === null) first = Date.now() - w0; if (n !== lastN) { lastN = n; stable = Date.now(); } if (n > 0 && Date.now() - stable > 4000) break; await p.waitForTimeout(100); }
    if (process.env.SCROLL) { const box = await p.locator("[data-diff-file-key]").first().boundingBox(); await p.mouse.move(box.x + 200, box.y + 200);
      const s0 = Date.now(); while (Date.now() - s0 < +process.env.SCROLL) { await p.mouse.wheel(0, 2500); await p.waitForTimeout(250); } await p.waitForTimeout(5000); }
    const lt = await p.evaluate((t0) => window.__lt.filter(([s]) => s >= t0), t0);
    Object.assign(cur, { firstFileMs: first, files: lastN, longTasks: lt.length, longTaskMs: Math.round(lt.reduce((a, [, d]) => a + d, 0)) });
    console.log(JSON.stringify(cur)); res.push(cur); cur = null; await p.close(); }
  fs.writeFileSync(out, JSON.stringify(res)); await b.close(); })();
// reviewspans.cjs <trace.ndjson>...: every ws.rpc.review.getDiffPreview request: start, duration, git processes, and the operation of each git process
const fs = require("fs"); const sp = new Map();
for (const f of process.argv.slice(2)) for (const l of fs.readFileSync(f, "utf8").split("\n")) { try { const r = JSON.parse(l); if (r.type === "effect-span") sp.set(r.spanId, r); } catch {} }
const kids = new Map(); for (const s of sp.values()) { const k = kids.get(s.parentSpanId) || []; k.push(s); kids.set(s.parentSpanId, k); }
const desc = (s) => { const o = [], q = [s]; while (q.length) { const x = q.pop(); for (const c of kids.get(x.spanId) || []) { o.push(c); q.push(c); } } return o; };
const st = (s) => Number(BigInt(s.startTimeUnixNano) / 1000000n); const hist = {};
for (const s of [...sp.values()].filter((s) => s.name === "ws.rpc.review.getDiffPreview").sort((a, b) => st(a) - st(b))) {
  const git = desc(s).filter((x) => x.name === "runGitCommand"); hist[git.length] = (hist[git.length] || 0) + 1;
  console.log(new Date(st(s)).toISOString().slice(11, 23), Math.round(s.durationMs) + "ms", git.length, s.exit._tag[0], git.map((g) => sp.get(g.parentSpanId)?.name.replace("GitVcsDriver.", "")).join(",")); }
console.log("git per request:", hist);

A single-file request prints 6 with getReviewDiffPreview.patch,getReviewDiffPreview.stat,review.listUntracked,resolveReviewMergeBase,branchExists,review.currentBranch (in some order).

Where the time goes

  • Client: apps/web/src/components/diffs/useReviewFilePatches.ts (reviewFileQueries) creates one reviewEnvironment.diffFilePatch query per file, and loadNextFiles adds 4 at a time. packages/client-runtime/src/state/review.ts (diffFilePatch) sends each one as its own review.getDiffPreview with file set and the resolved baseRef.
  • Server: apps/server/src/vcs/GitVcsDriverCore.ts, getReviewDiffPreview. For every request, resolveReviewRepository reads the branch fresh (GitVcsDriver.review.currentBranch, symbolic-ref --quiet HEAD), and resolveReviewMergeBase(cwd, branch, explicitBaseRef) runs branchExists (show-ref --verify refs/heads/<base>) and merge-base --end-of-options <comparisonRef> HEAD. Only after those three come the file's own ls-files --others -- <path>, diff --raw --numstat <mergeBase> -- <path> and diff --patch.

Why it is avoidable

Every file loaded under one preview compares the same HEAD with the same base. Re-deriving the branch, base ref and merge base for each file repeats an answer the preview request computed moments earlier, and repeats it 4 times in parallel for each batch. Reading the branch fresh is deliberate (resolveReviewRepository: "The cached branch outlives a checkout made outside T3 Code, so it is read fresh"). Doing it once per batch keeps that guarantee.

What a fix has to show

The same 60-s scroll, 3 runs before and after, on the same seed:

  • Git processes per loaded file drop from 6 to 3 (the file's untracked check, stat and patch), plus at most a constant number per batch or per preview for the branch, base and merge-base reads. For 272 files that is about 820–900 git processes per run, from ~1,650.
  • Median single-file patch latency under the same load drops by the shared part, about 0.5 s (from 1.22 s).
  • What stays: one fresh read of HEAD/branch per batch or preview revision, so a checkout made outside T3 Code is still noticed; the per-file untracked check, stat and patch; results identical to today's for renamed, untracked and binary files.
  • Not counted here: base-branch lookups without an explicit baseRef (perf(server): resolve a branch's base once while its refs and config are unchanged #40125 caches resolveBaseBranchForNoUpstream), and the duplicated first batch when the panel opens (filed separately).

Found by the automated Windows performance sweep on 8087127.


Found by the automated Windows performance sweep on 8087127.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions