Skip to content

fix(grep): match GNU grep 3.11 output, options and POSIX regex dialects - #2609

Merged
chaliy merged 1 commit into
mainfrom
claude/project-thread-1mku65
Oct 7, 2026
Merged

chaliy merged 1 commit into
mainfrom
claude/project-thread-1mku65

Conversation

@chaliy

@chaliy chaliy commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Requested by Михайло · project thread

What changed

  • Regex dialects follow GNU. Basic and extended patterns are translated with POSIX bracket rules, \{m,n\}, \+ \? \|, \< \> \b \B \w \W \s \S and repeated operators.
  • Back-references work in plain patterns, e.g. \(abc\)\1. They run on the backtracking engine, which already has a step cap (TM-DOS-025 updated).
  • Leftmost-longest -o. Matches are leftmost-longest, reusing awk's POSIX matcher.
  • Pattern errors use glibc's exact messages (Unmatched ( or \(, Invalid range end) and exit 2.
  • Output order follows GNU:
    • Prefixes print as FILE:LINE:BYTE, with - on context lines.
    • A -- separator goes between groups and between files whenever any context option is given.
    • --group-separator and --no-group-separator are supported.
    • After -m, trailing context is printed in full.
  • Options:
    • Added -NUM, -T, --label, -d, -D, -R (follows symlinks; -r does not), -y and --exclude-from.
    • - reads stdin.
    • Long options accept unique prefixes; an ambiguous or unknown one prints GNU's message.
  • Inputs:
    • An empty -f file matches nothing.
    • A directory operand without -r reports "Is a directory".
    • A line that isn't valid UTF-8 makes the file binary.
    • \r is part of the line, so x$ no longer matches x\r.
  • One line engine now serves both buffered and streaming runs. The old ad-hoc basic-to-extended converter is gone.

Why

grep scored 64% on the Debian-oracle bench from pseudo-linus. Agents write grep -o, back-references and context flags the GNU way, and they hit wrong output or "backreferences are not supported" errors.

Before / After

Each case below was also run through GNU grep 3.11 on the host, and the After output matches it in every case.

$ echo abcabc | grep -o '\(abc\)\1'
before: grep: invalid pattern: ... error: backreferences are not supported
after:  abcabc

$ grep -E '(' f; echo "exit=$?"
before: grep: invalid pattern: regex parse error: ( ... unclosed group / exit=2
after:  grep: Unmatched ( or \( / exit=2

$ printf 'one\ntwo\n' | grep --label=in -H o -
before: grep: unrecognized option '--label=in'
after:  in:one / in:two

$ printf 'ab\n' | grep -o 'a\|ab'
before: a
after:  ab

$ printf 'x\r\n' | grep -c 'x$'
before: 1
after:  0

Debian-oracle bench:

  • grep: 60.5% to 96.8% strict (185 cases).
  • grep cases in the regex set: 663 to 877 of 879.
  • The whole regex set: 64.1% to 81.7% lenient. The sed and awk cases in that set are unchanged.

The 6 grep cases still failing differ only in recursive directory order (L-GREP-003).

Risk

  • Medium. grep is heavily used, and this rewrites its option parsing and output path.
  • Four old unit tests encoded behavior GNU grep doesn't have and were rewritten. One example: -om1 now prints every match on the first line, because -m counts lines.
  • Known gaps are recorded as L-GREP-002: no "stray \" warnings, -o binary detection works per line, and NUL is not a line separator in binary files.

Checklist

  • Tests added or updated (new grep_gnu_output_tests, grep_pattern unit tests, limitation evidence tests). The full lib suite (2866) and integration suite (1368) pass. The bash comparison run shows no grep mismatches.
  • Backward compatibility considered (behavior now matches GNU grep 3.11)

Generated by Claude Code

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bashkit 347fcef Commit Preview URL

Branch Preview URL
Oct 07 2026, 11:30 PM

- Output follows GNU prtext/prline: FILE:LINE:BYTE: prefix order, `-`
  for context lines, group separators between groups and files whenever a
  context option is given (-NUM, --group-separator, --no-group-separator),
  full trailing context after -m, -T, -o skips empty matches and gives
  each match's offset with -b.
- New grep_pattern module translates GNU BRE/ERE (POSIX brackets,
  intervals, \< \> \b \w, literal leading * in BRE, dropped leading ERE
  operator with GNU's warning, glibc error messages) onto awk's
  leftmost-longest AwkRegex; back-references work via fancy-regex under
  the existing backtrack cap.
- getopt_long-style option parsing (unique prefixes, --label, -d, -D,
  -R, -y, --exclude-from), `-` as stdin, empty -f file matches nothing,
  directory operands report "Is a directory", invalid UTF-8 output turns
  the file binary, \r is line data. One FileScan engine serves buffered
  and streaming runs.

Debian-oracle grep score 60.5% -> 96.8% (185 cases); grep probes in the
regex set 663 -> 877 of 879. Gaps recorded as L-GREP-002/003.
@chaliy
chaliy force-pushed the claude/project-thread-1mku65 branch from 6b6e6c1 to 347fcef Compare October 7, 2026 23:29
@chaliy
chaliy merged commit adebd09 into main Oct 7, 2026
47 checks passed
@chaliy
chaliy deleted the claude/project-thread-1mku65 branch October 7, 2026 23:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant