Skip to content

fix(happens-before): a set's addAll withdraws what did not go in (#1425) - #1446

Merged
PIsberg merged 3 commits into
mainfrom
fix/p4-addall-retract
Nov 21, 2026
Merged

PIsberg merged 3 commits into
mainfrom
fix/p4-addall-retract

Conversation

@PIsberg

@PIsberg PIsberg commented Nov 20, 2026

Copy link
Copy Markdown
Owner

Closes #1425. Stacked on #1444, review the last commit.

Why

The woven addAll released every element before the call. For queues, #806 already withdraws what did not go in. For every other collection that publishes its elements (concurrent sets, Vector, synchronized wrappers), nothing was withdrawn. An element a set refused because it held an equal one, or an element a call throwing part way never added, stayed published. A thread that reached it some other way was then ordered after the producer by a hand-off that never happened, which can hide a race. That is a missed finding, never a false one.

HappensBefore.retract could not do this. It keeps only the thread's last release, and forgets it at the thread's next stamped access, which the element's own equals/hashCode makes inside addAll.

Change

  • HappensBefore.releaseWithdrawable(sync) releases as release does and returns the release: the clock, what it held before, and what was installed. HappensBefore.withdraw puts the clock back by compare-and-set only while it still holds this release, so a release another thread made since stays. That is the same exact-or-nothing rule retract follows.
  • AgentCollectionHooks.collectionAddAll, non-queue path: on a false return, withdraws every element's release, since the collection did not change. On a throw, withdraws the releases of elements the collection does not hold by identity. The queue paths are unchanged.
  • Cost: one toArray and one array of releases per addAll into a publishing collection. The identity walk runs only on the throw path.

Verified

  • AgentHappensBeforeFeedTest.aSetAddAllThatAddsNothingPublishesNothing (a newKeySet already holding equal tags) and aSetAddAllThatThrowsPartWayPublishesOnlyWhatWentIn (addAll stopping at a null) failed before the change and pass after. The twin, where the tag that went in stays published, passes throughout.
  • Library -P fast: 4338 tests green. Full corpus (6 lanes) green after a fresh install. Static gates and the javadoc jar are clean.

Docs

🤖 Generated with Claude Code

PIsberg and others added 3 commits November 20, 2026 18:59
…#1317, part)

#1315 made set, getAndSet, compareAndSet and the arithmetic forms of
AtomicBoolean/AtomicInteger/AtomicLong release and get acquire. A field
published through lazySet, setRelease, updateAndGet/getAndUpdate/
accumulateAndGet/getAndAccumulate, a won compareAndExchange(Release) or
a won weakCompareAndSetVolatile/Release, and read after a get that saw
the value, was still reported by AtomicityValidator as an unordered
publication on correct code. And a reader whose read was getAndSet's
previous value, a compareAndExchange witness, or what an add or update
returned acquired nothing.

Those hooks now call primitiveSlotStoring (before the store; after a
won swap for the compare forms) and HappensBefore.acquireVolatile on
the value read. The update and accumulate forms run as a CAS loop like
addPublishing, so the value to be stored is known before the store.
setPlain, setOpaque and weakCompareAndSetPlain stay unordered, as the
JMM says; AtomicFlagPublicationWeavingTest pins setPlain.

Verified failing first: 23 new AtomicFlagPublicationWeavingTest rows
(one per form) failed before the hook change and pass after; the
setPlain negative passed throughout. Library atomic, spinlock,
happens-before and atomicity tests plus MinTrustGateTest under -P e2e,
the full agent suite under -P e2e and the corpus eval are green;
static gates and the javadoc jar are clean.

Still open on #1317: getAcquire, intValue and longValue reads, the
AtomicLong forms the weaver does not substitute, and the updater,
VarHandle and array forms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The previous commit made the value an increment, add, update or
accumulate returns acquire. The corpus idioms lane then filled a 2 GB
heap within seconds on CI (Java 26 OOM; 21 and 25 hung past 25 min),
reproduced locally under -Xmx2g. A heap histogram of the dead fork
showed 1.38 GB of long[] and 690 MB of int[] averaging 32 KB and 14 KB:
HappensBefore stamps near MAX_ENTRIES (4096). The add releases before
its store, so a counter every thread bumps (CorpusRecorder's static
BODY_EXECUTIONS, bumped once per body execution) collects every
bumper's clock; acquiring on the read handed that whole clock to each
bumper, and every access it recorded afterwards pinned a copy. A user
suite with a global id counter would hit the same OOM.

The add and update forms now release only, as on main before #1317.
getAndSet and the compareAndExchange witness still acquire, and a get
of the counter still acquires what it published.

Verified failing first: AtomicCounterClockTest (11 forms, 200 earlier
bumpers) failed with a clock of 201 threads on every form and passes
after. AtomicFlagPublicationWeavingTest's two add/update-result rows
are inverted to assertRaced, pinning the decision. Idioms lane under
-Xmx2g: OOM before, 220/220 in 37 s after; full corpus (6 lanes)
green; library -P fast 4336 tests green; static gates and javadoc jar
clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The woven addAll released every element before the call and, outside
queues, withdrew nothing. An element a concurrent set refused (it held
an equal one) or one a call throwing part way never added stayed
published, so a thread that reached it another way was ordered after
the producer by a hand-off that never happened, hiding a race.
HappensBefore.retract could not be reused: it keeps only the thread's
last release and forgets it at the next stamped access, which the
element's own equals/hashCode makes inside addAll.

HappensBefore.releaseWithdrawable hands each release back and withdraw
takes it back exactly (CAS of the element's clock from what the release
installed to what it held before; a later release by another thread
keeps it). collectionAddAll withdraws every element's release when the
call returns false, and those of elements the collection does not hold
by identity when it throws.

Verified failing first: AgentHappensBeforeFeedTest's two new tests
(newKeySet already holding equal tags; newKeySet.addAll stopping at a
null) failed before and pass after; the twin (the element that went in
stays published) passes throughout. Library -P fast 4338 green, full
corpus green after a fresh install, static gates and javadoc jar clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 20, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P4] A failed addAll withdraws only the last element's release: HappensBefore.retract is single-slot

1 participant