Build a new incr comp session dir from scratch every time by bjorn3 · Pull Request #159287 · rust-lang/rust

bjorn3 · 2026-07-14T12:21:59Z

Rather than copying the old incr comp dir and then modifying it. This saves a copy/hardlink for files that are modified. And it removes the need for accurate work product tracking to avoid accumulating cruft, which is non-trivial. We don't accurately track the pre-LTO bitcode files for ThinLTO for example.

Part of rust-lang/compiler-team#908

rustbot · 2026-08-05T18:45:45Z

r? @nnethercote

rustbot has assigned @nnethercote.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

Owners of files modified in this PR: compiler, incremental
compiler, incremental expanded to 75 candidates
Random selection from 18 candidates

lewisl9029 · 2026-09-03T05:26:43Z

Fwiw, I think this could also fix a nasty cache corruption bug I ran into when a rustc compile gets interrupted with incremental and opt-level >= 1.

I had my agent whip up a quick repro here for anyone interested in digging deeper: https://github.com/lewisl9029/rustc-incremental-lto-repro

TL;DR: it can either show cryptic errors that don't match the source, or (worse) successfully build a new binary that doesn't match the source, depending on what gets mixed up in the cache from unintentionally overwriting hardlinks pointing to previous finalized session state without a new finalized session taking its place (due to getting interrupted, even from just a ctrl+c, no SIGKILL needed). Rebuilding doesn't fix the issue either, only a cargo clean will. Originally encountered on 1.94.1, but still repros on 1.98.0.

bjorn3 · 2026-09-21T11:41:41Z

@rustbot ready

Depend on lockfiles to prevent GC of the current session Instead of manually skipping it right before the remove_dir. This will be simpler once we stop immediately copying from the old incr comp dir. Previously this wasn't possible due to the usage of process-based fcntl locking, but I added proper fd-based locking recently in rust-lang/rust#162602. Will probably simplify rust-lang/rust#159287 a bit.

Rather than copying the old incr comp dir and then modifying it. This saves a copy/hardlink for files that are modified. And it removes the need for accurate work product tracking to avoid accumulating cruft, which is non-trivial. We don't accurately track the pre-LTO bitcode files for ThinLTO for example.

nnethercote · 2026-09-24T10:59:57Z

@bjorn3 and I had some discussions about this via DM. I think it's now ready for merging.

@bors r+

rust-bors · 2026-09-24T11:00:00Z

📌 Commit 5ed0180 has been approved by nnethercote

It is now in the queue for this repository.

…tch, r=nnethercote Build a new incr comp session dir from scratch every time Rather than copying the old incr comp dir and then modifying it. This saves a copy/hardlink for files that are modified. And it removes the need for accurate work product tracking to avoid accumulating cruft, which is non-trivial. We don't accurately track the pre-LTO bitcode files for ThinLTO for example. Part of rust-lang/compiler-team#908

…uwer Rollup of 7 pull requests Successful merges: - #163218 (stdarch subtree update) - #159287 (Build a new incr comp session dir from scratch every time) - #163222 (Enable EII tests for cg_gcc) - #163251 (cg_gcc subtree sync 2026-09-24) - #162595 (avoid accessing uninferred closure upvars in diagnostics) - #162937 (do generic_const_args Const::Ty hack only on projection consts) - #163238 (Fix incorrect use of await in parser suggestion)

Rollup merge of #159287 - bjorn3:incr_comp_session_from_scratch, r=nnethercote Build a new incr comp session dir from scratch every time Rather than copying the old incr comp dir and then modifying it. This saves a copy/hardlink for files that are modified. And it removes the need for accurate work product tracking to avoid accumulating cruft, which is non-trivial. We don't accurately track the pre-LTO bitcode files for ThinLTO for example. Part of rust-lang/compiler-team#908

rust-timer · 2026-09-24T19:31:15Z

Note

This PR was benchmarked as part of triage of its containing rollup: triage URL.

Finished benchmarking commit (255e1eb): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Our benchmarks found a performance regression caused by this PR.
This might be an actual regression, but it can also be just noise.

Next Steps:

If the regression was expected or you think it can be justified,
please write a comment with sufficient written justification, and add
@rustbot label: +perf-regression-triaged to it, to mark the regression as triaged.
If you think that you know of a way to resolve the regression, try to create
a new PR with a fix for the regression.
If you do not understand the regression or you think that it is just noise,
you can ask the @rust-lang/wg-compiler-performance working group for help (members of this group
were already notified of this PR).

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.2%	[0.1%, 0.3%]	5
Improvements ✅ (primary)	-0.8%	[-0.8%, -0.8%]	1
Improvements ✅ (secondary)	-0.5%	[-0.9%, -0.2%]	14
All ❌✅ (primary)	-0.8%	[-0.8%, -0.8%]	1

Max RSS (memory usage)

Results (primary -5.0%, secondary -1.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-5.0%	[-5.0%, -5.0%]	1
Improvements ✅ (secondary)	-1.4%	[-1.4%, -1.4%]	1
All ❌✅ (primary)	-5.0%	[-5.0%, -5.0%]	1

Cycles

Results (primary 6.0%, secondary -4.8%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	6.0%	[6.0%, 6.0%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-4.8%	[-6.1%, -2.8%]	3
All ❌✅ (primary)	6.0%	[6.0%, 6.0%]	1

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: missing data
Artifact size: 406.16 MiB -> 406.20 MiB (0.01%)

bjorn3 · 2026-09-24T19:58:34Z

Wall time seems to improve a bit on average and max rss decreases a bit on average.

…uwer Rollup of 7 pull requests Successful merges: - rust-lang/rust#163218 (stdarch subtree update) - rust-lang/rust#159287 (Build a new incr comp session dir from scratch every time) - rust-lang/rust#163222 (Enable EII tests for cg_gcc) - rust-lang/rust#163251 (cg_gcc subtree sync 2026-09-24) - rust-lang/rust#162595 (avoid accessing uninferred closure upvars in diagnostics) - rust-lang/rust#162937 (do generic_const_args Const::Ty hack only on projection consts) - rust-lang/rust#163238 (Fix incorrect use of await in parser suggestion)

panstromek · 2026-09-29T06:55:51Z

perf triage:

Improvements outweigh regressions, but most of the changes look like noise to me - ctfe just became bimodal again, deep-vector goes back to the previous state in next PR.

@rustbot label: +perf-regression-triaged

…uwer Rollup of 7 pull requests Successful merges: - rust-lang/rust#163218 (stdarch subtree update) - rust-lang/rust#159287 (Build a new incr comp session dir from scratch every time) - rust-lang/rust#163222 (Enable EII tests for cg_gcc) - rust-lang/rust#163251 (cg_gcc subtree sync 2026-09-24) - rust-lang/rust#162595 (avoid accessing uninferred closure upvars in diagnostics) - rust-lang/rust#162937 (do generic_const_args Const::Ty hack only on projection consts) - rust-lang/rust#163238 (Fix incorrect use of await in parser suggestion)

…nathanBrouwer Rollup of 7 pull requests Successful merges: - rust-lang#163218 (stdarch subtree update) - rust-lang#159287 (Build a new incr comp session dir from scratch every time) - rust-lang#163222 (Enable EII tests for cg_gcc) - rust-lang#163251 (cg_gcc subtree sync 2026-09-24) - rust-lang#162595 (avoid accessing uninferred closure upvars in diagnostics) - rust-lang#162937 (do generic_const_args Const::Ty hack only on projection consts) - rust-lang#163238 (Fix incorrect use of await in parser suggestion)

bjorn3 mentioned this pull request Jul 14, 2026

Move LTO from the codegen coordinator thread to link_binary rust-lang/compiler-team#908

Closed

15 of 23 tasks

This comment has been minimized.

Sign in to view

bjorn3 added the A-incr-comp Area: Incremental compilation label Jul 14, 2026

This comment has been minimized.

Sign in to view

bjorn3 force-pushed the incr_comp_session_from_scratch branch 2 times, most recently from bf1a4e3 to 50ba28a Compare July 21, 2026 09:35

This comment has been minimized.

Sign in to view

bjorn3 force-pushed the incr_comp_session_from_scratch branch from 50ba28a to 68660f2 Compare July 21, 2026 14:00

This comment has been minimized.

Sign in to view

bjorn3 force-pushed the incr_comp_session_from_scratch branch from 68660f2 to 1f2a9e3 Compare July 21, 2026 14:43

This comment has been minimized.

Sign in to view

bjorn3 force-pushed the incr_comp_session_from_scratch branch from 1f2a9e3 to e1094a1 Compare July 23, 2026 10:32

bjorn3 mentioned this pull request Jul 24, 2026

Incremental ThinLTO doesn't reuse unchanged CGUs originating form upstream crates #158955

Open

bjorn3 force-pushed the incr_comp_session_from_scratch branch from e1094a1 to d9dd28d Compare July 28, 2026 13:39

This comment has been minimized.

Sign in to view

bjorn3 force-pushed the incr_comp_session_from_scratch branch from d9dd28d to decf8a0 Compare August 5, 2026 18:45

bjorn3 marked this pull request as ready for review August 5, 2026 18:45

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 5, 2026

rustbot assigned nnethercote Aug 5, 2026

rustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Aug 5, 2026

bjorn3 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 13, 2026

bjorn3 mentioned this pull request Aug 19, 2026

[PATCH] bootstrap: use fcntl locking on Solaris #161357

Open

This comment has been minimized.

Sign in to view

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-blocked Status: Blocked on something else such as an RFC or other implementation work. labels Sep 21, 2026

bjorn3 force-pushed the incr_comp_session_from_scratch branch from 9c951e2 to 5ed0180 Compare September 24, 2026 10:34

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 24, 2026

This was referenced Sep 24, 2026

Rollup of 6 pull requests #163260

Closed

Rollup of 7 pull requests #163265

Merged

rust-bors Bot merged commit f0f6653 into rust-lang:main Sep 24, 2026
13 checks passed

rustbot added this to the 1.100.0 milestone Sep 24, 2026

bjorn3 deleted the incr_comp_session_from_scratch branch September 24, 2026 17:43

rustbot added the perf-regression Performance regression. label Sep 24, 2026

Mark-Simulacrum mentioned this pull request Sep 24, 2026

Rollup of 16 pull requests #162767

Merged

rustbot added the perf-regression-triaged The performance regression has been triaged. label Sep 29, 2026

github-actions Bot mentioned this pull request Oct 2, 2026

Toolchain upgrade to nightly-2026-09-25 flux-rs/flux#1803

Merged

vikng-dev mentioned this pull request Nov 6, 2026

Bump the Rust toolchain to 1.100 when it ships, ending link failures after a killed build vikng-dev/overmatch#303

Open

Navigation Menu

Repository navigation

Build a new incr comp session dir from scratch every time - #159287

Build a new incr comp session dir from scratch every time#159287
rust-bors[bot] merged 1 commit into
rust-lang:mainfrom
bjorn3:incr_comp_session_from_scratch

bjorn3 commented Jul 14, 2026 •

edited by rustbot

Loading

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rustbot commented Aug 5, 2026

Uh oh!

lewisl9029 commented Sep 3, 2026

Uh oh!

This comment has been minimized.

bjorn3 commented Sep 21, 2026

Uh oh!

nnethercote commented Sep 24, 2026

Uh oh!

rust-bors Bot commented Sep 24, 2026

Uh oh!

Uh oh!

rust-timer commented Sep 24, 2026

Uh oh!

bjorn3 commented Sep 24, 2026

Uh oh!

panstromek commented Sep 29, 2026 •

edited

Loading

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Conversation

bjorn3 commented Jul 14, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rustbot commented Aug 5, 2026

Uh oh!

lewisl9029 commented Sep 3, 2026

Uh oh!

This comment has been minimized.

bjorn3 commented Sep 21, 2026

Uh oh!

nnethercote commented Sep 24, 2026

Uh oh!

rust-bors Bot commented Sep 24, 2026

Uh oh!

Uh oh!

rust-timer commented Sep 24, 2026

Overall result: ❌✅ regressions and improvements - please read:

Instruction count

Max RSS (memory usage)

Cycles

Binary size

Uh oh!

bjorn3 commented Sep 24, 2026

Uh oh!

panstromek commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

bjorn3 commented Jul 14, 2026 •

edited by rustbot

Loading

panstromek commented Sep 29, 2026 •

edited

Loading