Skip to content

fix(cli): centralize JSONL stdin scanning - #13

Merged
GuoMonth merged 2 commits into
mainfrom
fix/cli-jsonl-scanner
Apr 17, 2026
Merged

GuoMonth merged 2 commits into
mainfrom
fix/cli-jsonl-scanner

Conversation

@GuoMonth

@GuoMonth GuoMonth commented Apr 17, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • add shared CLI JSONL stdin helpers with an explicit 10 MiB scanner buffer and clear oversized-line errors
  • refactor gm rm to reuse the helper while keeping its malformed-record and max-ID semantics
  • reject the next JSONL ID before append once the max batch size is reached
  • document that future CLI stdin / JSON / JSONL handling must stay centralized in internal/cli/stdin.go
  • add focused CLI tests for valid JSONL id extraction, oversized lines, and max-ID enforcement

Validation

  • go test ./internal/cli
  • make check
  • make validate

Closes #9

GuoMonth and others added 2 commits April 17, 2026 15:56
Move rm's JSONL scanner behavior into shared stdin helpers so large lines use an explicit 10 MiB scanner buffer and return a clear invalid-input error instead of the default bufio token-too-long path. Keep rm's semantics for ignoring malformed records and extracting id values.

Add focused CLI tests for valid JSONL id extraction, oversized JSONL lines, and max-ID enforcement.

Closes #9

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Document the rule that future CLI stdin / JSON / JSONL parsing must extend internal/cli/stdin.go, and tighten the JSONL ID limit check so oversized batches are rejected before appending the next ID.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@GuoMonth
GuoMonth merged commit 7646da7 into main Apr 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cli(rm): bufio.Scanner uses default 64 KiB buffer, large JSONL lines fail opaquely

1 participant