Repository navigation
notify and more - #3191
Draft
Sebastian Thiel (Byron) wants to merge 8 commits into
Draft
notify and more#3191Sebastian Thiel (Byron) wants to merge 8 commits into
Sebastian Thiel (Byron) wants to merge 8 commits into
Conversation
Sebastian Thiel (Byron)
force-pushed
the
notify-and-more
branch
5 times, most recently
from
November 11, 2026 06:29
99b01d0 to
4f52c81
Compare
Knowing how the venerable `notify` crate sometimes doesn't work as one would expect, particularly on Linux, it's probably worth it to give it another shot with assistance, thinking that todays models can probably do a better job, ultimately. Walked through the watcher tests (high-level API) and actually tried to understand and follow. Many changes were done based on my feedback. <!-- agent --> Long-running applications such as Tix need bounded event intake, explicit coverage loss, replaceable subscriptions, and predictable shutdown without owning platform notification machinery themselves. Provide these facilities in the Git-independent `gix-notify` crate so repository monitoring and other event loops can share the same contract. Bound queued events and diagnostic samples, wake consumers without holding the queue lock, and invalidate coverage on overflow, disconnection, or root replacement. Canonicalize registrations, preserve native path representations, and keep file, nonrecursive-directory, and recursive-directory interests separate from the native registrations used to implement them. Own notification delivery on macOS through FSEvents, on Linux through inotify, and on Windows through `ReadDirectoryChangesW` and a completion-port worker. macOS and Linux support administrative-marker delivery fences. Windows uses bounded asynchronous reads, validates native records, and drains cancelled operations before releasing their buffers; delivery synchronization remains explicitly unsupported there. Retain an upstream `notify` compatibility backend for other platforms. Include the workspace registration, dependencies, documentation, example, queue tests, and native backend tests with the complete crate. Repository-specific watch inventories and application integration can build on this independently. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>
Added application-like test so there is more cross-platform coverage of what apps would be expected to do. Also made hardcoded 'check-root-interval' value configurable. Finally, make use of error classes more consistent. <!-- agent --> Long-running `tix` monitors exposed directory descriptors retained by FSEvents `WatchRoot` for every root and ancestor. Sibling roots duplicate their shared ancestors even in one stream. Remove `WatchRoot` and validate reduced native root identities once per second, including while idle, and around delivery fences. Drain and release a failed stream before reporting one backend error. Consumers can re-register identical paths without repeated errors or wakeups. Root checks do not scan directories and cannot detect every transient rename and restoration between checks. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>
This minimal addition provides callers with to-be-included paths right before it attempts to include them. This reveals even those paths that don't exist yet, or otherwise can't yet be included. Is it slightly paranoid to not just rely on the files that are actually included? Certainly... but it's also the most correct way of doing it. Alternative: create empty sections, but there is no concept of an 'invisible' section with metadata. So no other, simpler, way to do this. <!-- agent --> Tix's repository monitor needs to track configuration dependencies even when an active include is missing or empty. Parsed sections alone cannot identify those paths, so observing include resolution lets the monitor notice when a missing file appears or an empty file gains configuration. Add `File::resolve_includes_with_observer()` to report each resolved active include before filesystem checks and reads. Preserve existing include resolution behavior through the observer-free entry points. Root configuration paths remain the caller's responsibility. Keep the integration tests with this API. They cover missing, empty, nested, and conditional includes, disabled resolution, and unavailable interpolation. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>
Quick review, minimal change to get FreeBSD integrated into the QA tool. <!-- agent --> The Linux, macOS and Windows CI jobs exercise owned notification backends, leaving the `notify` compatibility backend without native platform coverage. Run its existing tests against kqueue in a FreeBSD VM. Allow packages to opt into additional baseline test platforms with `package.metadata.qa.extra-platforms`. Keep the FreeBSD image selection in the QA catalogue and use the existing matrix, shared runner and blocking pass gate. Both fast and exhaustive plans restrict FreeBSD to opted-in packages, and fast plans retain affected-package selection. The VM receives the planned job and copies sources onto its own filesystem. Android support is deferred. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>
Tix needs to keep history and worktree views current without retaining a repository while idle. Add `gix::notify::RepositoryMonitor` on `gix-notify` and replace Tix's direct watcher management with it, making repository-aware cache invalidation and coverage recovery reusable by other long-running Git clients. Expose `Repository::monitor(Options)` through the opt-in `notify` feature. The monitor owns paths and native watchers, reopening repositories only when needed to update coverage. Event loops call `service()` for bounded processing and `next_timeout()` for scheduling. Outcomes distinguish reference, index, configuration, ignore, attribute, operation, layout, and worktree invalidations, retaining errors alongside simultaneous changes. Notifications are conservative cache invalidations rather than atomic Git transactions, and this API consumes native events independently of the Git fsmonitor protocol. Select metadata and worktree monitoring independently. Default `IgnoreAware` coverage follows the index and ignore rules, stopping at nested repositories. `WorktreeCoverage::All` observes the entire worktree, including ignored paths and nested repositories. With metadata disabled, complete coverage needs no index parsing or repository maintenance, so malformed or inaccessible indexes do not prevent monitoring. Consume an unstarted, worktree-only monitor with complete coverage through `into_worktree_stream()` to obtain a detached `WorktreeStream`. It exposes raw native events, explicit registration, bounded draining, delivery fences, and wakeup callbacks on one stream, including additional caller-owned watches. Registration must precede the baseline scan; loss and errors require recovery. Conversion rejects bare repositories and started monitors to avoid discarding queued events. The raw stream leaves scheduling and verification to its caller. Confine repository-monitor subscriptions and missing-directory fallback to the worktree and its private and shared Git directories. Observe local configuration roots, active includes, ignore and attribute dependencies, and missing permitted files, including symlink replacement and local targets. External dependencies add no watches. Track linked-worktree membership and administrative roots non-recursively, including incomplete worktrees before `gitdir` exists; retain recursive coverage for shared refs and the current worktree's refs and operation directories. Avoid extra parent subscriptions for replacement of an entire root. Reconcile subscriptions after changes or coverage loss. Publish worktree invalidations on the next service call, replacing Tix's 75 ms debounce. Metadata defaults to 100 ms of quiet with a 250 ms maximum latency; discovery and registration failures retry after five seconds. A configurable 60-second safety refresh rediscovers coverage and invalidates caches after silent notification gaps. Bound worktree scopes by count and bytes, escalating to a full refresh on overflow. Bound monitor diagnostics and Tix response logs to 16 sample paths and 4 KiB of path bytes, reporting omitted occurrences without retaining an unbounded set. Establish Tix's metadata coverage before its initial history snapshot and enable worktree subscriptions only while its worktree block is shown. Update the Tix specification and regression tests alongside the integration. Tests cover nested ignores, complete coverage with malformed indexes, raw-stream registration and recovery, cache invalidation, linked worktrees, local configuration dependencies, excluded external paths, and missing roots. Recorded validation on macOS, using the locked dependency graph: - `cargo test -p gix --no-default-features --features sha1,notify,status --lib`: all 36 tests passed. - `cargo test -p gix-tix --features sha1 --lib watch`: both watcher tests passed. Full workspace QA and native Linux/Windows runtime tests were not run. CI follow-up: `lint@linux-x86_64` rejected debug formatting of a path in the raw-stream assertion, and `housekeeping@linux-x86_64` rejected the fixture script's missing executable bit. Use display formatting and make the shebang-bearing fixture executable. Windows native events carry verbatim paths, while the repository layout could retain ordinary paths, hiding loose-reference updates from Tix. Canonicalize the nearest existing path prefix on Windows, preserving missing dependency suffixes. Add public coverage for the native metadata layout. Linked-worktree removal may report an expected missing watched root or coverage conflict; verify successful re-registration after that diagnostic instead of requiring the removal to be silent. Follow-up validation: 27 monitor unit tests and six public monitor integration tests pass on macOS. `cargo clippy -p gix --all-targets --locked -- -D warnings -A unknown-lints --no-deps` passes. The Windows ARM64 all-target check passes with `sha1,notify,status`; native Windows execution remains with CI. Co-authored-by: Byron <sebastian.thiel@icloud.com>
… monitor server <!-- agent --> Applications and monitored status need reusable Git filesystem monitor clients and servers without depending on porcelain or a command-line parser. Introduce `gix-fsmonitor` with a client API and optional `server` feature. Keep repository discovery, configuration, and process launching in `gix-fsmonitor-daemon`, which provides the standalone `gix-fsmonitor` executable. Support hook protocols 1 and 2 and native Simple IPC over Unix sockets and Windows named pipes. Native clients can start Git's daemon on macOS and Windows; the standalone daemon also runs on Linux. Callers supply paths, opaque tokens, command context, and connection and response limits. Trace protocol outcomes without logging tokens or path contents, and leave trust decisions, index checkpoints, and full-verification policy to callers. Long Unix socket paths use private aliases without changing the process working directory. The server watches all worktree paths independently of ignore rules and client indexes. A bounded journal serves multiple token baselines, requiring full scans after restarts, expired or malformed tokens, overflow, lost coverage, or uncertifiable delivery. macOS and Linux fence queries through administrative markers. Windows uses Git-style cookie observation to return incremental results, including for linked worktrees with external administrative directories. For Windows cookies, publish complete native batches together with immediately ready worktree reads obtained by rearming notification handles. This prevents an external cookie from being exposed before older buffered worktree changes. Bound the drain and report loss rather than publishing a partial burst. Injected sources must opt into the required ordering; cookie timeouts, diagnostics, and coverage changes remain conservative. Remove query cookies on every exit path, and keep the native watcher's public cross-directory synchronization unsupported. `server::spawn(Context)` returns after binding, cookie preparation, and initial watch registration. Its handle supports shutdown requests, waiting, and shutdown with joining; dropping it also stops the owned instance. Startup failures release resources, and an old handle cannot stop a replacement endpoint owner. Normalize roots to native filesystem spelling and preserve Git's endpoint and startup-lock protocol. Each linked worktree has its own endpoint and watch lifetime. `spawn_with_source()` accepts an `EventSource` factory while keeping the server independent of porcelain. Native and injected sources share the journal, recovery, lifetime, and protocol implementation; worktree, cookie, and sentinel watches stay on one stream. The daemon adapts an unfiltered, detached `WorktreeStream` from `gix::Repository::monitor()` and drops repository state after constructing the source on the worker thread. The CLI supports foreground operation, detached startup, hook autostart, stop, status, flush, and raw queries. Optional idle shutdown is disabled by default; completed client exchanges and status probes restart its timer, while filesystem activity does not. Active exchanges finish before shutdown releases the watches and endpoint. Native delivery checks cannot detect every silent notification gap, so applications needing that protection must arrange independent verification. Tests cover client protocols, injected sources, lifecycle and idle shutdown, endpoint ownership, journal and cookie loss, immediate writes, malformed indexes, ignored paths, alternate indexes, linked worktrees, and Unicode or case aliases. Recorded validation includes the macOS server, client, and daemon suites and Clippy for both fsmonitor packages. Windows validation includes: - `cargo test -p gix-notify --offline` - `cargo test -p gix-fsmonitor --all-features` - `cargo test -p gix-fsmonitor-daemon --test interoperability` - `cargo clippy -p gix-fsmonitor -p gix-fsmonitor-daemon -p gix-notify --all-targets --all-features` - `cargo fmt --all -- --check` Native Git interoperability also passed outside the Windows sandbox, whose ancestor-directory restrictions otherwise prevent Git's `GetLongPathNameW` call. Assisted-by: GPT 6.0 Astra Co-authored-by: Codex <codex@openai.com>
Applications that already have a `Repository` should be able to serve Git's filesystem monitor protocol without reconstructing repository layout or retaining that repository for the lifetime of the server. Add the optional `fsmonitor-server` feature and `gix::fsmonitor` re-export. `Repository::fsmonitor_server(options)` derives worktree and administrative paths from the repository, uses its filesystem settings, and forwards the server options, including idle shutdown. The resulting handle owns the watches and private Git directory's endpoint independently of the repository. Document the API and its lifetime controls alongside the daemon. This serves external Git clients; it does not enable filesystem-monitor acceleration in `gix` status operations or alter repository configuration. Validation: both public server integration tests pass on macOS with `GIX_TEST_IGNORE_ARCHIVES=1`, covering linked-worktree layout, idle timeout forwarding, operation after dropping the repository, default startup, and bare-repository rejection. The combined tree is unchanged by the commit split.
<!-- agent --> Tix should use native notifications for routine updates and leave an explicit full refresh as the last resort when users notice stale information. Resetting all caches every sixty seconds does unnecessary work without predicting when that recovery is actually needed. Remove `gix::notify::Options::safety_interval` and its deadline, validation, and service-loop branch. Update callers to omit the removed field. Native events and reported coverage loss continue to drive normal reconciliation. Use the existing `RepositoryMonitor::rescan()` API for manual full recovery. Queue every enabled invalidation immediately, as well as rebuilding watches and rediscovering subscriptions. A queued worktree error must not postpone metadata invalidation while native recovery waits for its retry deadline. CI follow-up: `tests@macos-aarch64` failed because the public manual-refresh test assumed that two startup service calls settle every asynchronous native notification. A later event legitimately reopened the repository, triggering the test's panic. Keep the assertions about explicit full invalidation and failed rediscovery, but check stable subscriptions during verification instead of requiring silence from the native stream. Validation: the focused `manual_monitor_refresh_invalidates_every_enabled_cache` integration test passes on macOS with the `notify` feature enabled. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>
Sebastian Thiel (Byron)
force-pushed
the
notify-and-more
branch
from
November 11, 2026 07:11
4f52c81 to
8b4961c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A basis for #3153 , so it can merge earlier.
Tasks
notifybackends