Skip to content

Use Git's fsmonitor daemon for status, via a gitoxide fork - #16858

Open
schacon wants to merge 3 commits into
masterfrom
gix-fsmonitor
Open

schacon wants to merge 3 commits into
masterfrom
gix-fsmonitor

Conversation

@schacon

@schacon schacon commented Nov 2, 2026

Copy link
Copy Markdown
Member

🧢 Changes

  • Use a gitoxide fork whose status uses Git's fsmonitor. gix and gix-testtools point at schacon/gitoxide gitbutler-fsmonitor. That branch is the previously pinned upstream rev 4dc73c31a plus two commits:
    • gix status asks git fsmonitor--daemon which paths changed and uses the index's untracked cache, when core.fsmonitor is set.
    • It skips the daemon for indexes under 2,000 entries, where one round trip costs more than a full scan.
  • Keep fsmonitor data across libgit2 checkouts. libgit2's checkout_index drops the FSMN and UNTR index extensions. That forced a full worktree scan after branch new, apply and unapply. They are now carried into the rewritten index.
  • Write the workspace index with gix. update_workspace_commit rewrote the index with libgit2, which dropped the same extensions and reset stat data. gix now writes it, keeping stat data and sparse-checkout flags for unchanged entries.

☕️ Reasoning

Report with charts and per-commit explanations: https://claude.ai/artifact/RuqYj5PkxTJkGRntXwWPTA

With the daemon running, compared with #16856 alone:

Deep dirs (20k files) before after
status 342 ms 64 ms
show 344 ms 43 ms
branch new 1178 ms 200 ms
commit one file 792 ms 177 ms
reword 759 ms 108 ms
  • On a repo with 100k files: 2–3.5× faster.
  • On the GitButler repo: 1.1–1.7× faster.
  • Small repos are unchanged.

Trade-offs:

  • The gix-written workspace index no longer has Git's TREE (cache-tree) extension. Git rebuilds it on its next commit.
  • Windows fsmonitor (named pipe) isn't supported yet and falls back to a full scan.
  • This should go back to GitoxideLabs/gitoxide once the fork's changes land upstream.

@github-actions github-actions Bot added the rust Pull requests that update Rust code label Nov 2, 2026
Point gix and gix-testtools at schacon/gitoxide's gitbutler-fsmonitor
branch: upstream 4dc73c31a, the rev we pinned before, plus two commits.

The first commit makes gix status use Git's fsmonitor daemon and the
untracked cache when core.fsmonitor is set:
- It reads and writes the index's FSMN and UNTR extensions.
- It asks `git fsmonitor--daemon` which paths changed since the token.
- It skips stat checks for every other tracked entry.
- It reuses cached directory listings for directories whose mtime has
  not changed.

The second commit skips the daemon for indexes with fewer than 2000
entries, where its ~10 ms round trip costs more than a full scan.

With the daemon running, on repositories with deep trees or about 100k
files: status takes 64 ms instead of 342 ms, `show` 43 ms instead of
344 ms, `branch new` 200 ms instead of 1178 ms, and committing one file
177 ms instead of 792 ms. Small repositories measure the same. Output
matched the upstream build for status after `branch new`, commit,
uncommit, adding an untracked file, and deleting a branch.

Go back to GitoxideLabs once this lands upstream.
libgit2's checkout_index rewrites the index without Git's FSMN and UNTR
extensions. The next status then has no fsmonitor token and no
untracked cache, so it scans the whole worktree again. On a deep
repository that put a full scan after every `branch new`, `apply` and
`unapply`.

Before the checkout, keep the gix index if it has fsmonitor data. After
the checkout, copy that data into the rewritten index with
carry_over_fsmonitor_data_from() and write it. That function marks the
paths the checkout changed as dirty, so they are still checked. It does
nothing when fsmonitor is off.
update_workspace_commit used libgit2 to read the workspace tree into the
index and write it. That write dropped the fsmonitor and untracked-cache
extensions, and it reset the stat data of every entry. Every command
that updates the workspace commit then forced a full worktree scan on
the next status.

gix now builds the index from the tree. Entries whose object id did not
change keep their stat data and SKIP_WORKTREE flags, unless that stat
data is racy. The new index carries over the fsmonitor data and is
written. The libgit2 Repository then re-reads the index from disk, so
callers that still hold it see the new index.

Trade-off: Git's TREE (cache-tree) extension is no longer written. Only
plain git's index-vs-HEAD comparison uses it, and git rebuilds it on its
next commit.
@Byron

Byron commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

Especially those with bigger repos or those on Windows will have core.fsmonitor enabled, so supporting it seems like a must.
Let's see what it takes so this can land in main.

@Byron

Byron commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

Please give me some days to review this: GitoxideLabs/gitoxide#3153

@Byron Byron mentioned this pull request Nov 3, 2026
7 tasks

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rust Pull requests that update Rust code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants