Allow reading args from `@path` by matthewhughes934 · Pull Request #7220 · rust-lang/rustfmt

matthewhughes934 · 2026-10-29T06:49:34Z

That is, something like

$ printf "--config path/to/config.toml\n--check\n" > @args-file
$ rustfmt @args-file path/to/format.rs
# same as: rustfmt --config path/to/config.toml --check path/to/format.rs

The motivation for this is that cargo fmt collects the list of files to be formatted and passes them to rustfmt as arguments. But this list can get so long that it reaches OS limits, see #6934.

The process for making this change was:

Go and look for inspiration from other tools
Find out the rustdoc supports this pattern[1] so just re-use its approach[2]

Conveniently, that approach uses helpers from internal compiler crates that are also available to us. Note, those helpers expect UTF-8 everywhere, which is what the current code also does, but this will be an additional place to consider in the future if we want to move away from that (see #7210).

Link: https://doc.rust-lang.org/rustdoc/command-line-arguments.html#path-load-command-line-flags-from-a-path [1]
Link: https://github.com/rust-lang/rust/blob/ab019e71a1a1f4688bcc4d5bf06d2c5a424a31a5/src/librustdoc/lib.rs#L827 [2]

I did not use an LLM to create a change in this PR.
I used an LLM to create a change in this PR, and I have explained below how it was used.

ytmimi · 2026-10-29T17:06:48Z

+    let cmdline_args = env::args().skip(1).collect::<Vec<String>>();
+    // if we saw a `@path` arg, read arguments from that file and merge them with the other
+    // arguments.
+    let args = rustc_driver::args::arg_expand_all(&early_dcx, &cmdline_args);


It's nice that we can leverage existing code from rustc_driver. Will this enforce that options are newline separated? For example, can we pass more than one argument on a single line? I think we should add some tests for this.

I see that your tests below dynamically create files with build_args_file. We can probably create some sample @path files and store them in the tests/ directory.

// Is this okay? --config=wrap_comments=true --config=comment_width=80

How is whitespace handled?

// is this okay? --config=wrap_comments=true --config=comment_width=80 // there's trailing whitespace here ^^^^^^^^^^^

View changes since the review

Will this enforce that options are newline separated?

Yep, it just reads lines from the file and adds them as args, so

// Is this okay? --config=wrap_comments=true --config=comment_width=80

This will add a single arg like: "--config=wrap_comments=true --config=comment_width=80"

How is whitespace handled?

// is this okay? --config=wrap_comments=true --config=comment_width=80 // there's trailing whitespace here ^^^^^^^^^^^

It's left untouched, so that would result in two args:

[" --config=wrap_comments=true"., "--config=comment_width=80"]

Additionally it means it can't accepts any arguments with newlines in it e.g.

path/to/my/file_that_has_a_newline in_its_name.rs

Would present as two arguments

Cool. I wouldn't expect us to handle newlines in arguments so that seems fine to me.

ytmimi · 2026-10-30T20:18:17Z

Is this necessary?

View changes since the review

Is this necessary?

From experience, yes: otherwise git on Windows will try and convert these to plain \n (IIRC, that happens under default config, but I can't remember all the details, see the eol attribute if you want to go down a documentation rabbit hole

Which files are we worried about having their line endings changed? I guess I'm just confused because we don't have this setup anywhere else.

Which files are we worried about having their line endings changed?

args_full_carriage_return uses \r\n for its line endings, so I want to preserve those

I guess I'm just confused because we don't have this setup anywhere else.

We have an equivalent setup for some files in the top level .gitattributes, e.g. tests/source/issue-3494/crlf.rs -text

Which disables the text attribute, and since that attribute means:

This attribute marks the path as a text file, which enables end-of-line conversion

Disabling it means we don't do any eol conversion. I prefer setting the eol attribute as it's more explicit plus it means if e.g. your editor decides to write the file with just \n instead, then git will convert them back.

Poking around, I'm wondering if these tests are silently doing nothing:

$ git grep --files-with-matches $'\r' | git check-attr --stdin text | awk '$NF!="unset"' tests/source/preserves_carriage_return_for_unix.rs: text: auto tests/source/preserves_carriage_return_for_windows.rs: text: auto tests/target/preserves_carriage_return_for_unix.rs: text: auto tests/target/preserves_carriage_return_for_windows.rs: text: auto tests/writemode/target/stdin.xml: text: auto

I.e. with text=auto I suspect git will convert all those \r\n to \n (in both source and target files) so the tests pass but we don't actually test anything

Thanks for elaborating. Let's leave a comment somewhere to make it clear that this is for args_full_carriage_return

We can look into those other tests later.

Added comment to the attributes: f169a87

After more experimentation: -text is indeed the right choice:

Unsetting the text attribute on a path tells Git not to attempt any end-of-line conversion upon checkin or checkout.

I.e. tell git to just leave the file alone cb331d2

We can look into those other tests later.

Turns out those tests just ignored line endings: #7233

ytmimi · 2026-10-30T20:18:43Z

Do we need a whole args_file cargo project? Is it enough to just have the src dir with .rs files?

View changes since the review

Do we need a whole args_file cargo project? Is it enough to just have the src dir with .rs files?

Nope, plain .rs should be fine: c70847a

matthewhughes934 · 2026-11-01T09:01:09Z

@rustbot ready

ytmimi

@matthewhughes934 things are looking good. I also wanted to double double check if this supports multiple @path declarations? Maybe arg_expand_all already handles that?

If that's the case maybe we need to update the help text to make it clear that more than one @path can be passed.

View changes since this review

matthewhughes934 · 2026-11-05T18:09:17Z

@matthewhughes934 things are looking good. I also wanted to double double check if this supports multiple @path declarations? Maybe arg_expand_all already handles that?

Yep, the added test test_args_file_multiple_files_merged covers this behaviour

If that's the case maybe we need to update the help text to make it clear that more than one @path can be passed.

Done a7b1781, I also added a full example in the README since I don't think the short help alone is fully clear (at least for me, since I wasn't familiar with args-file via the @ syntax before this)

matthewhughes934 · 2026-11-05T18:22:09Z

@rustbot ready

ytmimi · 2026-11-05T18:23:22Z

Yep, the added test test_args_file_multiple_files_merged covers this behaviour

Sorry, a lot going on and I missed that.

Done a7b1781, I also added a full example in the README since I don't think the short help alone is fully clear (at least for me, since I wasn't familiar with args-file via the @ syntax before this)

Agreed. This is also something I didn't know what possible. Thank you for updating the README.

That is, something like $ printf "--config path/to/config.toml\n--check\n" > @args-file $ rustfmt @args-file path/to/format.rs # same as: rustfmt --config path/to/config.toml --check path/to/format.rs The motivation for this is that `cargo fmt` collects the list of files to be formatted and passes them to `rustfmt` as arguments. But this list can get so long that it reaches OS limits, see rust-lang#6934. The process for making this change was: 1. Go and look for inspiration from other tools 2. Find out the `rustdoc` supports this pattern[1] so just re-use its approach[2] Conveniently, that approach uses helpers from internal compiler crates that are also available to us. Note, those helpers expect UTF-8 everywhere, which is what the current code also does, but this will be an additional place to consider in the future if we want to move away from that (see rustfmt/issues#7210). Link: https://doc.rust-lang.org/rustdoc/command-line-arguments.html#path-load-command-line-flags-from-a-path [1] Link: https://github.com/rust-lang/rust/blob/ab019e71a1a1f4688bcc4d5bf06d2c5a424a31a5/src/librustdoc/lib.rs#L827 [2]

rustbot · 2026-11-05T18:25:44Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

ytmimi · 2026-11-05T18:26:35Z

@matthewhughes934 thanks for rebasing! was just about to say that I saw some fixup! commits

matthewhughes934 · 2026-11-05T20:48:07Z

Just noting, I commented on the original issue about this being a breaking change for anyone running rustfmt over a file starting with @, e.g. rustfmt @some_file.rs would break. But since rustdoc and rustc both support this I'm leaning towards this being acceptable. rustfmt ./@some_file.rs will still work fine too.

matthewhughes934 · 2026-11-05T20:51:11Z

ah, thanks @ytmimi I removed it from queue because I wasn't sure if there was subtree syncing going at the moment (so I should avoid merges to keep things simple)

While working on rust-lang#7220 I got curious about how the integration tests handle line endings, specifically CRLF, and it turns out they don't. Given we have features that explicitly add/remove these line endings we should ensure the tests correctly cover this case to avoid regressions. We use `diff::lines` to generate diffs, and this calls `str::lines` under the hood, which strip any trailing `\r\n` or `\n`. So the only way we can tell that line-endings differ is by comparing the two contents, previously we performed this comparison in a way that would ignore if one file had `\r\n` endings and another had `\n` endings. This meant we never actually asserted that we saw the expected line endings. Note: we can only tell the files have differing line endings, not the lines where this occurs. It _could_ be a future improvement to make this more specific, but I think the approach is reasonable since there aren't any tests (or really any use case) for mixed line-endings in a single file. This reflects the behaviour of how we report such errors with e.g. `emitter::diff::DiffEmitter::emit_formatted_file` This required updating some test files that used `newline_style=Windows` but never actually added those line endings in the corresponding `target`. In these cases carriage returns are added to the `target` files, but not the `source` ones so these tests also assert that we do indeed convert the line endings. The files `tests/{target,source}/configs/enum_discrim_align_threshold/40.rs` had CRLF line endings, but not corresponding `newline_style` option and the feature does not depend on line endings, so these were converted back to used Unix line endings

While working on rust-lang#7220 I got curious about how the integration tests handle line endings, specifically CRLF, and it turns out they don't. Given we have features that explicitly add/remove these line endings we should ensure the tests correctly cover this case to avoid regressions. We use `diff::lines` to generate diffs, and this calls `str::lines` under the hood, which strip any trailing `\r\n` or `\n`. So the only way we can tell that line-endings differ is by comparing the two contents, previously we performed this comparison in a way that would ignore if one file had `\r\n` endings and another had `\n` endings. This meant we never actually asserted that we saw the expected line endings. Note: we can only tell the files have differing line endings, not the lines where this occurs. It _could_ be a future improvement to make this more specific, but I think the approach is reasonable since there aren't any tests (or really any use case) for mixed line-endings in a single file. This reflects the behaviour of how we report such errors with e.g. `emitter::diff::DiffEmitter::emit_formatted_file`

rustbot added the S-waiting-on-author Status: awaiting some action (such as code changes or more information) from the author. label Oct 29, 2026

matthewhughes934 force-pushed the args-from-path branch from 53d0976 to 765b765 Compare October 29, 2026 07:05

ytmimi reviewed Oct 29, 2026

View reviewed changes

Comment thread tests/rustfmt/main.rs Resolved

ytmimi reviewed Oct 29, 2026

View reviewed changes

Comment thread tests/rustfmt/main.rs Resolved

ytmimi reviewed Oct 29, 2026

View reviewed changes

matthewhughes934 force-pushed the args-from-path branch from b768351 to 2a70ce7 Compare October 30, 2026 19:19

matthewhughes934 marked this pull request as ready for review October 30, 2026 19:21

rustbot added S-waiting-on-review Status: awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: awaiting some action (such as code changes or more information) from the author. labels Oct 30, 2026

ytmimi reviewed Oct 30, 2026

View reviewed changes

ytmimi reviewed Nov 4, 2026

View reviewed changes

ytmimi added S-waiting-on-author Status: awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: awaiting review from the assignee but also interested parties. labels Nov 4, 2026

jieyouxu added A-CLI Area: command-line interface A-io Area: related to I/O by rustfmt (fs, stdin/stdout, etc.) labels Nov 5, 2026

matthewhughes934 force-pushed the args-from-path branch from 0a4169e to a7b1781 Compare November 5, 2026 18:15

rustbot added S-waiting-on-review Status: awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: awaiting some action (such as code changes or more information) from the author. labels Nov 5, 2026

ytmimi approved these changes Nov 5, 2026

View reviewed changes

matthewhughes934 force-pushed the args-from-path branch from a7b1781 to c2856f3 Compare November 5, 2026 18:25

matthewhughes934 added this pull request to the merge queue Nov 5, 2026

matthewhughes934 removed this pull request from the merge queue due to a manual request Nov 5, 2026

ytmimi added this pull request to the merge queue Nov 5, 2026

Merged via the queue into rust-lang:main with commit 059b793 Nov 5, 2026
33 checks passed

rustbot added release-notes Needs an associated changelog entry and removed S-waiting-on-review Status: awaiting review from the assignee but also interested parties. labels Nov 5, 2026

matthewhughes934 deleted the args-from-path branch November 5, 2026 21:02

matthewhughes934 mentioned this pull request Nov 5, 2026

system/idempotence tests: also assert on line endings #7233

Draft

1 of 2 tasks

Conversation

matthewhughes934 commented Oct 29, 2026 • edited by ytmimi Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ytmimi Oct 29, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

matthewhughes934 Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ytmimi Oct 30, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ytmimi Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ytmimi Oct 30, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

matthewhughes934 commented Nov 1, 2026

Uh oh!

ytmimi left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

matthewhughes934 commented Nov 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

matthewhughes934 commented Nov 5, 2026

Uh oh!

ytmimi commented Nov 5, 2026

Uh oh!

rustbot commented Nov 5, 2026

Uh oh!

ytmimi commented Nov 5, 2026

Uh oh!

matthewhughes934 commented Nov 5, 2026

Uh oh!

Uh oh!

matthewhughes934 commented Nov 5, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

matthewhughes934 commented Oct 29, 2026 •

edited by ytmimi

Loading

ytmimi Oct 29, 2026 •

edited by rustbot

Loading

matthewhughes934 Oct 29, 2026 •

edited

Loading

ytmimi Oct 30, 2026 •

edited by rustbot

Loading

ytmimi Oct 31, 2026 •

edited

Loading

ytmimi Oct 30, 2026 •

edited by rustbot

Loading

ytmimi left a comment •

edited

Loading

matthewhughes934 commented Nov 5, 2026 •

edited

Loading