Skip to content

feat(builtins): stream grep and tr through pipelines - #2587

Merged
chaliy merged 1 commit into
mainfrom
claude/stream-grep-tr
Oct 7, 2026
Merged

chaliy merged 1 commit into
mainfrom
claude/stream-grep-tr

Conversation

@chaliy

@chaliy chaliy commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Requested by Михайло · project thread

What changed

grep and tr now stream when they are a pipeline stage, following plain cat (#2584). They read input in ~4 KiB pieces and write results as they go. grep cuts its pieces on record ends and tr on UTF-8 character ends. grep -m stops reading once its budget is spent. Options that need the whole input stay buffered and drain the pipe first:

  • grep -c/-l/-L/-q, context lines, -b, -r
  • tr -s

The shared piece reader (InputChunks) lives in builtins/mod.rs so more filters can adopt it.

Why

This is roadmap step 2 (concurrent/streaming execution). producer | grep pat | head is one of the most common agent pipelines. Before this change, a grep or tr stage downstream of an endless or large producer read everything, so it ran until a limit fired instead of stopping at SIGPIPE.

Before / After

Script Before After (= bash)
while :; do echo y; done | grep y | head -2 runs until a limit y y, PIPESTATUS 141 141 0
yes | tr y n | head -1 runs until a limit n, 141 141 0
yes | tr -d '\n' | head -c 5 runs until a limit yyyyy, 141 141 0
grep . big | head -1; echo ${PIPESTATUS[*]} 0 0 141 0
yes | grep -o -m3 y | cat runs until a limit 3 lines, 141 0 0

Proof: l_pipe_001_grep_tr_stream and l_pipe_001_buffered_filter_options_drain_pipe (bash-verified). I also ran a differential script over grep -c/-v/-n/-i/-H/-z/-a, multiple files, a missing file, binary input and the tr variants. Its only differences are existing ones outside streaming: stderr ordering, byte-wise tr 'é' e, and \r stripping.

Risk

Medium-low. The buffered paths are unchanged apart from draining a streaming pipe. tr's logic moved into TrSpec::apply unchanged. Binary input in a stream is reported once with binary file matches from the point the NUL is seen.

Checklist

  • Failing test first
  • cargo test -p bashkit --lib / --test integration / bash_comparison
  • clippy -D warnings, fmt, check-okf, check-doc-links
  • knowledge: parallel-execution, limitations L-PIPE-001, log

https://claude.ai/code/session_019aFikmptPc91Fj4N2iDXQA


Generated by Claude Code

grep and tr read their input pipe in record/UTF-8 aligned pieces and
write results as they go, so `loop | grep y | head -2` and
`yes | tr y n | head -1` end with SIGPIPE like bash. grep -m stops
reading at its budget. Summarizing options stay buffered and drain
the pipe (Context::stdin_to_end).

Claude-Session: https://claude.ai/code/session_019aFikmptPc91Fj4N2iDXQA
@chaliy chaliy self-assigned this Oct 7, 2026
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bashkit 9388c4a Commit Preview URL

Branch Preview URL
Oct 07 2026, 07:02 PM

@chaliy
chaliy merged commit 88e8e2e into main Oct 7, 2026
53 of 55 checks passed
@chaliy
chaliy deleted the claude/stream-grep-tr branch October 7, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant