Skip to content

fix(cli): reject unsupported go test flags - #67

Open
not-matthias wants to merge 1 commit into
mainfrom
cod-4005-handle-unsupported-go-runner-flags-without-leaving-orphaned
Open

not-matthias wants to merge 1 commit into
mainfrom
cod-4005-handle-unsupported-go-runner-flags-without-leaving-orphaned

Conversation

@not-matthias

@not-matthias not-matthias commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Reject unsupported go test flags instead of skipping only the flag.

The parser warned and continued for unknown flags, but their values were left behind and parsed as packages. With go test -p 1 ./... -bench ., the leftover 1 reached Go as a package, so the run failed with package 1 is not in std only after every benchmark had passed.

This is a user-visible behavior change: every flag the runner does not support now aborts with exit 1 before Go runs, instead of being warned about and ignored. This includes common flags such as -count, -benchmem, and -v, not just -p. The help text also no longer advertises --dry-run, which the runner never implemented.

The arity of an arbitrary unsupported flag is unknown, so the runner cannot safely remove a separate value. It now fails before running Go with an error naming the flag. Flag parsing no longer stops at the first package, so unsupported flags are caught wherever they appear; supported flags after package arguments remain valid.

Regression tests cover separate and =-attached values, flags before and after packages, and supported flags after packages. cargo test --lib -- --skip integration_tests, cargo check --all-targets, and cargo clippy --all-targets -- -D warnings pass. The integration cases need the testdata submodules initialized; locally they fail because those submodules are absent.

Closes COD-4005

@codspeed

codspeed Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ Unable to measure the execution of 1 benchmark

This benchmark measured no execution time, so its result is not comparable and counts as unchanged. This usually happens when the compiler removes the code under test.

Prevent compiler optimizations

See affected benchmarks

✅ 49 untouched benchmarks
⏩ 1 skipped benchmark1


Comparing cod-4005-handle-unsupported-go-runner-flags-without-leaving-orphaned (0458739) with main (a5f6b7c)

Open in CodSpeed

Footnotes

  1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports. ↩

The parser warned about unsupported flags but dropped only the flag, so a
separate value was collected as a package. In `go test -p 1 ./... -bench .`,
`1` became a package and the run failed with `package 1 is not in std` after
the benchmarks completed.

Go flag arity is not known to the runner, so after an unsupported flag the next
token is dropped too unless it looks like a package pattern (relative or
absolute path, `...` wildcard, reserved word, or domain-style import path).
Inline `-flag=value` forms still drop a single token. The run keeps going, as
before, and never fails before Go is invoked.

Closes COD-4005
@not-matthias
not-matthias force-pushed the cod-4005-handle-unsupported-go-runner-flags-without-leaving-orphaned branch from 52a0d05 to 0458739 Compare October 30, 2026 18:20
@not-matthias
not-matthias marked this pull request as ready for review October 30, 2026 18:20
@not-matthias
not-matthias requested a review from lvaroqui October 30, 2026 18:21
@greptile-apps

greptile-apps Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The change affects command-line parsing across Go runner benchmark runs.

Fix unsupported flag rejection and use tracing for the added warning before merging.

Fix All in Claude CodeFindings

  1. P1 Flags change package selection ▶
  2. P2 Warning bypasses tracing ▶
Fix with agent prompt
### Issue 1
go-runner/src/cli.rs:133-139
With `test -v fmt`, the new code consumes `fmt` as a flag value and runs the default `./...` instead of the requested package. Conversely, `test -cpuprofile ./cpu.out ./...` keeps `./cpu.out` as a package, leaving the original failure unfixed.

Return `CliExit::UnknownFlag` before Go runs, as the PR description requires, rather than guessing which arguments belong to unsupported flags. Update the tests to expect rejection.

### Issue 2
go-runner/src/cli.rs:140
The new warning uses `eprintln!`, but the repository rule prohibits `println!` and `eprintln!` in Rust and requires tracing macros instead. Use `tracing::warn!` here to satisfy that requirement before merging.

```suggestion
                        tracing::warn!("ignoring value '{value}' of unsupported flag '{s}'");
```

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR adds a helper that guesses whether an argument is a package and skips presumed values of unsupported flags.

  • The parser keeps many unsupported flag values out of the package list.

Reviews (1) · Last reviewed commit: "fix(cli): drop the value of unsupported ..." · Reviewed by Greptile

Comment thread go-runner/src/cli.rs
Comment on lines +133 to +139
if s.contains('=') {
continue;
}
// The flag's arity is unknown. A bare token that does not look like a
// package is the flag's value, and must not be collected as a package.
let is_flag_value = |a: &String| !a.starts_with('-') && !looks_like_package(a);
if let Some(value) = args.next_if(is_flag_value) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Flags change package selection

With test -v fmt, the new code consumes fmt as a flag value and runs the default ./... instead of the requested package. Conversely, test -cpuprofile ./cpu.out ./... keeps ./cpu.out as a package, leaving the original failure unfixed.

Return CliExit::UnknownFlag before Go runs, as the PR description requires, rather than guessing which arguments belong to unsupported flags. Update the tests to expect rejection.

Prompt To Fix With AI
This is a comment left during a code review.
Path: go-runner/src/cli.rs
Line: 133-139

Comment:
**Flags change package selection**

With `test -v fmt`, the new code consumes `fmt` as a flag value and runs the default `./...` instead of the requested package. Conversely, `test -cpuprofile ./cpu.out ./...` keeps `./cpu.out` as a package, leaving the original failure unfixed.

Return `CliExit::UnknownFlag` before Go runs, as the PR description requires, rather than guessing which arguments belong to unsupported flags. Update the tests to expect rejection.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code Fix in Codex

Comment thread go-runner/src/cli.rs
// package is the flag's value, and must not be collected as a package.
let is_flag_value = |a: &String| !a.starts_with('-') && !looks_like_package(a);
if let Some(value) = args.next_if(is_flag_value) {
eprintln!("warning: ignoring value '{value}' of unsupported flag '{s}'");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Warning bypasses tracing

The new warning uses eprintln!, but the repository rule prohibits println! and eprintln! in Rust and requires tracing macros instead. Use tracing::warn! here to satisfy that requirement before merging.

Suggested change
eprintln!("warning: ignoring value '{value}' of unsupported flag '{s}'");
tracing::warn!("ignoring value '{value}' of unsupported flag '{s}'");

Rule Used: Never use println!/eprintln! in Rust; use tracing macros (info!/debug!/warn!/error!). (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: go-runner/src/cli.rs
Line: 140

Comment:
**Warning bypasses tracing**

The new warning uses `eprintln!`, but the repository rule prohibits `println!` and `eprintln!` in Rust and requires tracing macros instead. Use `tracing::warn!` here to satisfy that requirement before merging.

```suggestion
                        tracing::warn!("ignoring value '{value}' of unsupported flag '{s}'");
```

**Rule Used:** Never use `println!`/`eprintln!` in Rust; use tracing macros (`info!`/`debug!`/`warn!`/`error!`). ([source](https://app.greptile.com/codspeed/-/custom-context?memory=f04f734b-e556-482b-98b9-64c7ce663663))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Claude Code Fix in Codex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant