Repository navigation
Conversation
A task records where it read a flow from so that the completion of its push can finish that flow: mark the copy as transferred and let go of the source it read. Only the push that acquires a flow records anything, and the tasks come from a mempool, so a flow this task did not bring in reads whatever the previous occupant of that slot left behind. Nobody noticed because the completion skipped every flow whose input was already on the device, which is exactly the case where nothing is recorded. Letting an emptied copy through that skip, so that the push refilling it can complete it, exposed the stale read: a flow being filled by another task now reaches a body that dereferences the candidate, and a never-written slot is as likely to be zero as to be a freed copy. Leave the field empty when a task is built, in both interfaces and in the two tasks the engine builds for itself, so that "nothing recorded" is something the completion can recognise, and skip such a flow: another task is filling the copy and will complete it.
A task announces that it is reading a copy by counting itself among the readers of that copy, and that count is what stops the device owning it from reclaiming the memory. The communication engine reads copies too, from its own thread, for as long as a message is outstanding, and it never made that announcement. It only ever retained the copy, which keeps the object alive and says nothing about its memory. That was survivable for as long as a copy anybody else pointed at was refused for eviction outright, because the retain was then protection by accident. It stopped being survivable once such a copy could be emptied on the grounds that its value is recoverable from the host: the network is not asked, cannot be, and has no way to notice. A tile on its way out was emptied under an outstanding put and the transport was handed a null address to register, which it rightly refused. Count the engine among the readers for as long as it holds a copy, at the three points where it takes one and the one where it gives it up, and across the local reshape, which reads two copies in place. The readers are consulted before anything else in the victim search, so this is all the protection needed. Eviction and the network can still race for a copy, and the engine loses that race gracefully: it is told that the copy cannot be read rather than being handed memory that is about to go away. The host mirror is no substitute for a device copy here. For a matrix in LAPACK storage the mirror is a strided view of a larger matrix while the device holds a packed tile, and the message describes the layout of the copy the reshape produced, so sending the mirror instead would send the wrong bytes. While here, stop the eager release of a host mirror from handing back memory that something other than a device still points at. It only looked for other device copies, and a host task holding the mirror as an input, or a message not yet sent, reads that buffer just the same.
There is one host copy per data, so a write-back coming down from an accelerator lands on the same bytes a task or the communication engine may be reading at that moment. Nothing stopped it: readers was a notion the accelerators kept among themselves, and the pin taken on behalf of a reader deliberately did nothing once the copy turned out to live on the host. A write-back could therefore change a value under its reader, and the only reason it rarely showed is that write-backs were rare. Count readers on host copies too, and let both producers of a write-back consult them: the background drain passes over a copy whose mirror is being read and comes back to it later, and a task pushout settles the question for all of its flows before staging any of them, since an enqueued transfer cannot be taken back. Counting is only worth something if the count is true, and it was not. Transferring ownership took a reader for every read access and nothing ever gave it back, so the number grew without meaning for every caller that is done reading by the time it transfers ownership. Ownership says who may hold the value, not who is reading it; only the caller knows how long it intends to read. The accelerator stage in, which is the caller that keeps reading until its task pops, now takes its own, and the DTD release that used to hand back the free reader on a host task input goes away with the reader it was compensating for: what a body reads is already counted for as long as the body runs. A count that can go below zero is worse than no count at all, since a gate asking whether anybody is reading would then answer yes forever, so say so where the decrement happens.
A host copy now carries a reader count that a write-back consults, so what that count leaves out is what a write-back is allowed to overwrite. Three readers were left out. A task body reading its input on the host was not one of them. It reads through a pointer into the host mirror for as long as the hook runs, and the runtime brackets that call, so it can say so on the body's behalf without every body having to remember to. It says it around the call rather than inside the generated code because a body may return from anywhere, and it remembers what it pinned instead of looking again afterwards, since a task that hands its completion to somebody else is no longer ours to read. Neither was the transfer that carries a value from the host to an accelerator. It reads the mirror for as long as it is in flight, which is precisely the window a write-back must not land in. The push already acquired a source on a peer and recorded it; now it acquires the host mirror the same way, and hands it back where the peer is handed back, when the value has arrived. The protection follows the value: the mirror while the bytes are on their way, the device copy from then until the task pops. The third was of our own making. DTD cleared the count when it bound a tile, which was harmless while nothing on a host copy could be counted and is now a way to forget a reader the network or a body is holding.
A reservation hands a device copy to a flow and binds it as its data_out long before any transfer into it starts. From that moment the copy is reachable from its data like any other one, and nothing said that a task was about to use it: its reader count was zero and the only reference to it was the data's own. A release of the data as a whole would therefore find the copy unused and free it, and the push that followed would fill, and then read back, memory that had been given away. Under memory pressure that showed up either as a segmentation fault in the push completion or, worse, as a wrong result. Take the reader where the copy is handed over, which is the moment the task starts depending on it, and give it back where the task is done with it. The stage in used to take that reader, for read flows only, which left both the window above and every write-only flow uncovered. A push can be retried between the two, so remember per flow whether the reader is held, rather than deduce it from the access mode: a reservation that gives up hands back everything it took, and a pop gives back exactly what the reservation took. The one task that pops a copy it never reserved is the D2D completion command, which exists to hand a reader over, so it says so when it is built.
Deciding that a data is used by nothing but its own copies, and then freeing those copies, were two steps with nothing holding them together: a reader taken in between was a reader of memory that was about to be given back. The eviction has an answer for this already, which is to claim a copy by driving its readers below zero, so that whoever comes along to read it sees the claim and steps back out. Take the same claim here, on every copy, and let go of the whole data if any copy resists. A copy still being filled resists too. Whoever started the transfer comes back to it on completion, to mark it complete and to hand its version over to the data, and neither the readers nor the references cover that: a task counts itself on the copy it reads from, which for a copy being filled is the source rather than this one.
bosilca
force-pushed
the
topic/data-copy-readers
branch
from
September 27, 2026 17:46
b0558a1 to
91f562a
Compare
bosilca
force-pushed
the
topic/device-copy-placeholder
branch
from
September 27, 2026 17:46
f2ac004 to
a1680a4
Compare
Owner
Author
|
Moved upstream as ICLDisco#814 so it is visible there. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A device may only reclaim memory nobody is using, and
readersis how a user says so. Several users were not counting themselves: the communication engine reading a copy for the duration of a message, a task body reading a host copy, and a reservation holding a device copy from the moment it binds it asdata_outuntil the transfer starts. That last one let a copy still bound to a live task be freed with its data, which showed up as a crash in the push completion or as a wrong result.Also claims the copies of a data before releasing it, so that finding them unused and freeing them is a single step.
Stacked on #6.