GitIgnoreSpec: `X/**` reports the bare directory as ignored (5 cases), and the excluded-ancestor rule outlives its own re-inclusion (1 case)

[KaizenShogun]

Following up on your request in #132.

Everything below is measured on master at 5f14318, against the same corpus I used there — 9,852 (.gitignore, path) cases from 43 real repositories, git check-ignore as the oracle, KaizenShogun/gitignore-conformance — with GitIgnoreSpec on all three backends. The six remaining divergences are two bugs, not six.

A. X/** reports the directory X itself as ignored — 5 of the 6, pre-existing

from pathspec.gitignore import GitIgnoreSpec

GitIgnoreSpec.from_lines(['d/**']).match_file('d/')   # True
GitIgnoreSpec.from_lines(['d/*']).match_file('d/')    # False

git ignores d in neither case. d/** excludes what is under d, not d itself, and that difference is exactly what makes a re-inclusion work under d/** and fail under d/:

.gitignore	git stages d/keep.txt	match_file('d/')	match_file('d/keep.txt')
d/** + !d/keep.txt	yes	True	False
d/ + !d/keep.txt	no	True	True

On the first row pathspec calls the directory ignored and the file inside it not ignored. A caller that prunes a walk with match_file(append_dir_sep(p)) — the idiom append_dir_sep() was added for in #65 — never reaches the file git would stage.

The mechanism is in the compiled regex. A trailing ** emits a bare '/' (spec.py:324), so d/** becomes ^d/, which the string d/ matches with nothing left over — and with no ps_d group it lands in the file accumulator, not the directory one:

'd/**'  ->  ^d/
'd/*'   ->  ^d/[^/]+(?:(?P<ps_d>/)|$)
'd/'    ->  ^(?:.+/)?d(?P<ps_d>/)

d/* is already right on the same question, which is what makes this look like an oversight rather than a decision. GitIgnoreBasicPattern has the same shape (basic.py:280).

Appending one character there — out_parts.append('/.') — takes the bench from 6 to 1 on all three backends. Of the 212 tests in tests/, three then fail, and all three are assertEqual on the literal regex text: test_03_child_double_asterisk, test_03_duplicate_leading_double_asterisk_edge_case, test_14_issue_81_a. The match_file assertions inside those same tests still hold, and **/api/** still compiles equal to **/**/api/**/**. I'm not putting that forward as the fix — it's a one-character probe that says where the five live. Happy to send it as a PR with those three assertions updated if you want it in that shape.

The five corpus rows, all the same shape: .settings/ under .settings/** (kubernetes), .superset/ under **/.superset/** (supabase), and tensorflow/lite/gen/, tensorflow/lite/tools/make/downloads/, tensorflow/lite/tools/make/gen/ under their /…/** (tensorflow).

B. The excluded-ancestor rule outlives the negation that re-includes the ancestor — 1 of the 6, new in #132

GitIgnoreSpec.from_lines(['.*', '!**/node_modules/**']).match_file(
    'vendor/deps/npm/node_modules/.bin/x.txt')      # True; git: not ignored

93e0179 (master before the merge) answered False here on all three backends; from f404ca47 on, True. Reduced from nodejs/node's real .gitignore with git check-ignore arbitrating each step. Swap .bin for bin and the answer is right again, so it is the hidden ancestor doing it.

.* is the only pattern that produces a _DIR_MARK match, so it sets dir_include = True for the ancestor .bin. !**/node_modules/** compiles to ^(?:.+/)?node_modules/ — no dir mark — so it only ever reaches file_include, and if dir_include: wins. But that same negation also re-includes .bin itself: git names it as the deciding pattern for the bare ancestor path, and a canary dropped inside .bin stages. pathspec agrees, when you ask it directly:

s = GitIgnoreSpec.from_lines(['.*', '!**/node_modules/**'])
s.match_file('vendor/deps/npm/node_modules/.bin')          # False — ancestor not excluded
s.match_file('vendor/deps/npm/node_modules/.bin/x.txt')    # True  — because that ancestor is excluded

So the ancestor rule fires on an ancestor the same spec says is not excluded. The directory accumulator is resolved over _DIR_MARK matches only, while whether an ancestor ends up excluded is decided by every pattern that matches it. I don't have a measured fix for this one — it looks like an ordering question inside the accumulator rather than a design problem, but I haven't got a patch I'd stand behind yet.

How git was asked, since half of this is about directories

The obvious ways to ask git about a directory are both traps, and they cost me a probe script each. check-ignore on d/ with the trailing slash matches d/*, because * happily matches the empty string — that is the question answering itself, not git ignoring anything. And !! d/ in status --ignored fires just as readily when a directory merely has nothing left to stage.

So directory verdicts here come from a real tree: check-ignore -v on the bare path, plus a canary probe — append !d/canary, create it, git add -A -n, and staged means the parent was never excluded, per the "not possible to re-include a file if a parent directory is excluded" rule in gitignore(5). That is the only thing I found that separates d/* from d/. Corpus cases were kept only where the independent ways agreed; disagreements were dropped and counted, never guessed at. git 2.55.0 throughout.

— Midas

[KaizenShogun]

Two measurements that were sitting in my duplicate of this issue (#136, now closed) and belong here instead.

Before/after, 93e0179 — the first parent of the merge commit, so the comparison isolates #132 and nothing else — against 5f14318:

backend	pre	post
simple	69	6
re2	65	6
hyperscan	65	6

Compared as sets rather than counts: 64 fixed, 1 new, 5 pre-existing on simple (60 / 1 / 5 on re2 and hyperscan, whose pre-merge sets were already smaller). The same six paths survive on all three backends.

The two halves land on different APIs, which may change what the fix should be:

• §A, the five. match_tree_files gets all five right. They only bite callers that run their own os.walk and prune on match_file("some_dir/") — which is what black's gen_python_files does, and it's common enough that I built a second bench around that pattern. So that half may be a documentation matter (directory queries belong to the tree API) rather than a code one; your call.
• §B, the regression. This one reaches match_tree_files too, so it isn't the same kind of thing. With the file actually on disk in a real repo: git status --untracked-files=all lists it, match_tree_files(repo, negate=True) returns it on 93e0179, and returns nothing on 5f14318.

Bench, corpus and the reduction scripts: https://github.com/KaizenShogun/gitignore-conformance — happy to run any candidate fix through it, same as with #132.

— Midas