Make `Proxy` weak reference as it seems that it is not dropped at all by aerooneqq · Pull Request #165506 · rust-lang/rust

aerooneqq · 2026-11-05T16:08:41Z

Trying to fix tests that fail when we enable two threads in frontend, this is a fix for a test failure when we did not return all tokens for job server. I tested it in top of #162848 with 2 and 222 threads and did not reproduce after 200 launches, while it was certainly reproducing after 20-30 launches without this fix.

There is a second failure which is an infinite hang that I managed to get one or two times locally (with 222 threads). I don't know the reason for it, but maybe this fix will also help as it did not reproduce with it.

cc @Kobzol @petrochenkov

…nnel

aerooneqq · 2026-11-06T10:06:08Z

@bors try jobs=test-x86_64-gnu-aux

Make `Proxy` weak reference as it seems that it is not dropped at all try-job: test-x86_64-gnu-aux

rust-bors · 2026-11-06T11:09:44Z

💔 Test for 23ce799 failed: CI. Failed job:

try - test-x86_64-gnu-aux (web logs, enhanced plaintext logs)

rust-log-analyzer · 2026-11-06T11:09:47Z

The job test-x86_64-gnu-aux failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

aerooneqq · 2026-11-06T11:24:44Z

To sum up the current state:

The first idea (the error in tokens calculation and the rewrite of it) didn't work locally as there is no error,
The second idea that Proxy is not dropped and something bad happens because of it fixed the issue with tokens locally, but it encounetered hang after ~300 launches of ./x t src/tools/cargo --stage 1 -- jobserver
Then I changed the command to ./x t src/tools/cargo --stage 1 -- jobserver --test-threads=1 and it survived 609 launches until I stopped it

In CI the failure now differs: locally I had only diff between tokens in test jobserver_and_j, here we have a failure of the code that is in https://github.com/rust-lang/cargo/blob/3c747ea3ead23951c1b43a5edfcbdb449135ddeb/tests/testsuite/jobserver.rs#L42, so it happened when this test code was executed, locally I haven't ever encountered this error:

failures:
2026-11-06T11:09:35.9053866Z 
2026-11-06T11:09:35.9054359Z ---- jobserver::runner_inherits_jobserver stdout ----
2026-11-06T11:09:35.9055125Z running `/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/cargo install --path .`
2026-11-06T11:09:35.9055739Z running `make run -j2`
2026-11-06T11:09:35.9055911Z 
2026-11-06T11:09:35.9056303Z thread 'jobserver::runner_inherits_jobserver' (129959) panicked at src/tools/cargo/tests/testsuite/jobserver.rs:198:10:
2026-11-06T11:09:35.9056783Z 
2026-11-06T11:09:35.9056902Z test failed running `make run -j2`
2026-11-06T11:09:35.9057405Z error: process exited with code 2 (expected 0)
2026-11-06T11:09:35.9058050Z --- stdout
2026-11-06T11:09:35.9058846Z make[1]: Entering directory '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/tmp/cit/src/tools/cargo/testsuite/jobserver/runner_inherits_jobserver/foo'
2026-11-06T11:09:35.9059895Z /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/cargo run
2026-11-06T11:09:35.9060953Z make[1]: Leaving directory '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/tmp/cit/src/tools/cargo/testsuite/jobserver/runner_inherits_jobserver/foo'
2026-11-06T11:09:35.9061905Z 
2026-11-06T11:09:35.9062075Z --- stderr
2026-11-06T11:09:35.9063481Z    Compiling cargo-jobserver-check v0.0.1 (/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/tmp/cit/src/tools/cargo/testsuite/jobserver/runner_inherits_jobserver/foo)
2026-11-06T11:09:35.9065215Z     Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.38s
2026-11-06T11:09:35.9065953Z      Running `target/debug/cargo-jobserver-check`
2026-11-06T11:09:35.9066359Z 
2026-11-06T11:09:35.9066612Z thread 'main' (130227) panicked at src/main.rs:26:44:
2026-11-06T11:09:35.9067826Z called `Result::unwrap()` on an `Err` value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }
2026-11-06T11:09:35.9068962Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2026-11-06T11:09:35.9069642Z make[1]: *** [Makefile:2: run] Error 101
2026-11-06T11:09:35.9070281Z make[1]: INTERNAL: Exiting with 1 jobserver tokens available; should be 2!

If I understand correctly (I haven't debugged it yet) the cause of the missing token is not straight in the compiler but because we failed to write it back in main.

petrochenkov · 2026-11-06T12:58:02Z

            helper: OnceLock::new(),
        });
-        let proxy_ = Arc::clone(&proxy);
+        let weak_proxy = Arc::downgrade(&proxy);


I agree that it's better to drop the Proxy rather than to leak it through an Arc cycle, but I'm still not sure what bad may happen if we don't do it (the only nontrivial piece of logic there is Drop for HelperThread).

View changes since the review

I think the helper thread could still have a token acquired but not yet dropped if a token was requested, but then rustc exits because of an error. By avoiding the arc cycle, the helper thread gets a chance to release the token before the process exits.

petrochenkov · 2026-11-06T13:00:52Z

-                if let Ok(token) = token {
-                    let mut data = proxy_.data.lock();
+                if let Ok(token) = token
+                    && let Some(proxy) = weak_proxy.upgrade()


Suggested change

&& let Some(proxy) = weak_proxy.upgrade()

&& let proxy = weak_proxy.upgrade().unwrap()

?
If we are here, then the helper thread is still running, and it means the Arc<Proxy> is not yet destroyed.

View changes since the review

petrochenkov · 2026-11-06T13:23:46Z

            assert_eq!(read.read(&mut buf).unwrap(), 1);
            assert_eq!(write.write(&buf).unwrap(), 1);

WouldBlock during the write means that the pipe is full, but I'm not sure how it's possible if we just read from it.
Perhaps it makes sense to check first whether read and write actually refer to the same pipe.

petrochenkov · 2026-11-06T13:26:41Z

(I think we can merge the Drop fix without blocking on investigation of the next jobserver::runner_inherits_jobserver failure.)

rust-bors · 2026-11-07T00:02:26Z

☔ The latest upstream changes (presumably #165563) made this pull request unmergeable. Please resolve the merge conflicts by rebasing.

Don't rely on used when deciding whether to release a jobserver token

9c9e65e

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Nov 5, 2026

bjorn3 reviewed Nov 5, 2026

View reviewed changes

Comment thread compiler/rustc_data_structures/src/jobserver.rs Resolved

aerooneqq changed the title ~~Don't rely on used when deciding whether to release a jobserver token~~ Make Proxy weak reference as it seems that it is not dropped at all Nov 6, 2026

aerooneqq and others added 6 commits November 6, 2026 13:04

Make Proxy weak reference as it seems that it is not dropped at all

ad84f1b

Use 2 parallel frontend threads by default on the nightly and dev cha…

8731613

…nnel

Run UI tests with -Zthreads=1, unless overridden

786ae74

Make obtain-borrowck compatible with multiple threads

2750a72

Force Clippy tests to use one frontend thread

85cd2b3

Use a single frontend job in run-make tests

958a88e

aerooneqq force-pushed the count-external-tokens branch from 387ff40 to 958a88e Compare November 6, 2026 10:05

This comment has been minimized.

Sign in to view

rust-bors Bot pushed a commit that referenced this pull request Nov 6, 2026

Auto merge of #165506 - aerooneqq:count-external-tokens, r=<try>

23ce799

Make `Proxy` weak reference as it seems that it is not dropped at all try-job: test-x86_64-gnu-aux

petrochenkov self-assigned this Nov 6, 2026

petrochenkov added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 6, 2026

petrochenkov reviewed Nov 6, 2026

View reviewed changes

petrochenkov removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 6, 2026

petrochenkov added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Nov 6, 2026

Navigation Menu

Repository navigation

Make `Proxy` weak reference as it seems that it is not dropped at all - #165506

Make `Proxy` weak reference as it seems that it is not dropped at all#165506
aerooneqq wants to merge 7 commits into
rust-lang:mainfrom
aerooneqq:count-external-tokens

aerooneqq commented Nov 5, 2026 •

edited

Loading

Uh oh!

Uh oh!

aerooneqq commented Nov 6, 2026

Uh oh!

This comment has been minimized.

rust-bors Bot commented Nov 6, 2026

Uh oh!

rust-log-analyzer commented Nov 6, 2026

Uh oh!

aerooneqq commented Nov 6, 2026 •

edited

Loading

Uh oh!

petrochenkov Nov 6, 2026 •

edited by rustbot

Loading

Uh oh!

bjorn3 Nov 6, 2026

Uh oh!

petrochenkov Nov 6, 2026 •

edited by rustbot

Loading

Uh oh!

petrochenkov commented Nov 6, 2026

Uh oh!

petrochenkov commented Nov 6, 2026

Uh oh!

rust-bors Bot commented Nov 7, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

	&& let Some(proxy) = weak_proxy.upgrade()
	&& let proxy = weak_proxy.upgrade().unwrap()

Conversation

aerooneqq commented Nov 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

aerooneqq commented Nov 6, 2026

Uh oh!

This comment has been minimized.

rust-bors Bot commented Nov 6, 2026

Uh oh!

rust-log-analyzer commented Nov 6, 2026

Uh oh!

aerooneqq commented Nov 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

petrochenkov Nov 6, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bjorn3 Nov 6, 2026

Choose a reason for hiding this comment

Uh oh!

petrochenkov Nov 6, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

petrochenkov commented Nov 6, 2026

Uh oh!

petrochenkov commented Nov 6, 2026

Uh oh!

rust-bors Bot commented Nov 7, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

aerooneqq commented Nov 5, 2026 •

edited

Loading

aerooneqq commented Nov 6, 2026 •

edited

Loading

petrochenkov Nov 6, 2026 •

edited by rustbot

Loading

petrochenkov Nov 6, 2026 •

edited by rustbot

Loading