cuda-core: safe element casts require AnyBitPattern; bool is DeviceCopy

[elibol]

Closes #1506. First change under #1460; the bool decision recorded there and in #1461 and #1503 is implemented here.

The hole

DeviceBuffer::cast_elem, DeviceBuffer::cast_chunks, and Tensor::reinterpret are safe and checked only size and alignment, so a u8 buffer holding 2 could become a DeviceBuffer<bool> or a Tensor<bool>. Reading that value as a bool is undefined behavior, on the host through to_host_vec and on the device inside a kernel.

The fix

The three casts now require their target to implement bytemuck::AnyBitPattern: every bit pattern is a valid value, so no byte written as T can become an invalid A. bool is not AnyBitPattern, and no downstream crate can make it one, so casting into bool is a compile error. The trybuild cases under cuda-core/tests/device_buffer_casts/ and cutile/tests/ui/reinterpret_bool.rs pin the diagnostics.

bool implements DeviceCopy. The trait documentation now states the rule: DeviceCopy promises a byte-copyable type with a valid zero pattern, not that every bit pattern is valid. A typed buffer holds valid values by construction (from_host, zeroed, kernels writing through &mut T, unsafe raw constructors), and the casts are the one place arbitrary bytes could become typed values, which the bound closes. bool buffers can now be allocated, filled, launched, and read back like any other element type.

cuda-core takes bytemuck as a dependency and re-exports it (cuda_core::bytemuck) so downstream crates implement the marker traits against the same crate. The DType newtypes (tf32, the f8* types, f4e2m1fn, f4e2m1fnx2, i4) implement Pod, and half's bytemuck feature covers f16 and bf16, so every numeric Tensor::reinterpret target still compiles.

Atomics: wrap_elem and peel_elem

bytemuck::AnyBitPattern requires Copy, and the cuda-device atomic wrappers hold an UnsafeCell, so they can never satisfy the new cast_elem bound. The documented atomic-slice mapping from #151 moves to two new methods: DeviceBuffer::wrap_elem<W: TransparentWrapper<T>> views a scalar buffer as its wrapper, and peel_elem goes back so the result can be read on the host. TransparentWrapper promises the same layout and the same valid values as the inner type without requiring Copy, which is exactly the relation the atomics have to their integers.

cuda-device change, for Nihal's review: cuda-device takes bytemuck = { version = "1", default-features = false } and the two atomic macros gain unsafe impl bytemuck::TransparentWrapper<$ty> for $Name {}. The orphan rule leaves no other place for that impl: cuda-core cannot name these types, and cuda-device cannot depend on cuda-core. The vector types (F32x4 and friends) implement Pod, so floats.cast_chunks::<F32x4>() works as the module docs describe. The examples that cast into atomics (hashmap, hashmap_v2, hashmap_v3, legacy_atomic_fadd, legacy_atomic_rmw_cas) call wrap_elem and peel_elem; cast_chunks's Quad implements Pod. Every example lock and the device-only fixture lock gain cuda-device's new dependency at the same bytemuck 1.25.0 the cuda-oxide and cutile-rs locks already resolve to; no other pin moved.

The earlier draft of #1506 also wanted T: DeviceCopy on KernelSliceArg. That is dropped, and the issue says why: the atomic wrappers are legitimate kernel element types but not DeviceCopy, and the constructor invariant above already covers the launch boundary.

Compatibility

Adding a bound to cast_elem, cast_chunks, can_cast_chunks, and Tensor::reinterpret is a breaking change for callers casting into types that are not AnyBitPattern. In this tree those were only the atomic casts, now wrap_elem/peel_elem. Tuples are no longer valid cast targets either, since their layout is unspecified; tuple element types still work everywhere else. The cutile-rs changelog records the reinterpret change under Unreleased.

Checks

• Root: cargo fmt --check, cargo clippy -p cuda-core --all-targets -D warnings, cargo test -p cuda-core (all suites, GPU included), cargo doc with warnings denied.
• cutile-rs: cargo fmt --check, cargo clippy -p cutile --all-targets -D warnings, cargo test -p cutile --test ui, cargo test -p cutile --test tensor_reinterpret (GPU).
• cuda-oxide: cargo clippy -p cuda-device -p cuda-host --all-targets -D warnings, cargo test -p cuda-device, cargo oxide fmt --check, and cargo oxide run for the six touched examples on an RTX 5090, which also shows bytemuck compiles through the device backend.
• Guards: sync-example-locks.sh --check, check-dependency-licenses.sh (new bytemuck row), check-device-only-build.sh, check-spdx-headers.sh, check-source-references.sh, and cargo deny check for the root, cuda-oxide, backend, device-only, and cutile-rs workspaces.
• cargo semver-checks -p cuda-core against the crates.io 0.4.0 release reports no semver update required.

One pre-existing thing I did not touch: cutile/tests/ui/graph_scope_rejects_allocating_input.stderr has not matched since the monorepo move (it records a $WORKSPACE-relative path into cuda-async, which trybuild can no longer normalize from cutile-rs/), and run_cpu_tests.sh does not run the ui target, so CI does not notice. Every other ui case, including the new one, passes locally. Worth a follow-up.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[nihalpasham]

So, as a first step I think the DeviceCopy / AnyBitPattern split makes sense. Under the revised contract, bool can implement DeviceCopy while remaining excluded from AnyBitPattern. Requiring AnyBitPattern on reinterpretation targets rejects cast_elem::<bool>(), preventing byte 2 from becoming a bool through that safe cast.

But we need to address two safety issues that popped up from trying to break the above setup via adversarial examples.

1. The first one is the atomic TransparentWrapper implementations expose invalid shared-reference conversions.

The intended operation is converting an owned integer buffer into an atomic buffer. But implementing TransparentWrapper also exposes safe methods on the atomic type:

use bytemuck::TransparentWrapper;

// Device code:
fn overwrite(value: &u32) {
    let atomic = DeviceAtomicU32::wrap_ref(value);
    atomic.store(7, AtomicOrdering::Relaxed);
}

The conversion aliases the same memory:

&u32 → &DeviceAtomicU32 → store(7)

The original &u32 does not permit that mutation. Matching size and alignment cannot grant that permission. The reverse conversion, peel_ref, also allows an ordinary shared reference to coexist with mutation through the atomic.

Proposed fix: remove the atomic TransparentWrapper implementations and retain wrap_elem / peel_elem using a narrower unsafe marker, provisionally named DeviceRepr<Inner>:

pub unsafe trait DeviceRepr<Inner>: Sized {}

Its documented contract would require matching layout, valid initialized representations in both directions, and no additional ownership or destruction obligations. It would provide no reference-conversion methods.

The buffer’s wrap_elem bound becomes W: DeviceRepr<T>, with the corresponding change to peel_elem. Existing atomic-buffer usage remains available:

let atomics = integers.wrap_elem::<DeviceAtomicU32>();
// Run the kernel and complete its accesses.
let integers = atomics.peel_elem::<u32>();

DeviceAtomicU32::wrap_ref(&value) becomes unavailable because the atomic no longer implements TransparentWrapper. Allocation ownership, lifetimes, and ordering must remain preserved.

[nihalpasham]

2. zeroed() can return before the buffer actually contains zeros.

Suppose an allocation still contains old byte 2. We request a buffer of bool, initialized to false—byte 0:

// writer and reader are independent GPU streams.
let flags = DeviceBuffer::<bool>::zeroed(&writer, 1)?;
let values = flags.to_host_vec(&reader)?;

Both calls are safe Rust. However, zeroed only queues the zero-fill on writer. This execution is possible:

Time ───────────────────────────────────────►

writer: [earlier work................][write 0]
reader:      [copy existing byte 2 to CPU]

to_host_vec waits for the copy on reader; it does not establish that writer finished initializing the allocation first. It can therefore copy byte 2 into a Rust bool, causing undefined behavior.

AnyBitPattern cannot prevent this: there is no cast to bool. The buffer was created as DeviceBuffer<bool> directly.

Proposed fix: the simplest repair is to finish initialization before safe zeroed() returns:

// After queuing the zero-fill:
stream.synchronize()?;
Ok(buf)

To preserve asynchronous initialization, retain its completion dependency and make every safe consumer honor it, including readback and kernel launches. Asking callers to synchronize in documentation does not repair the safe API.

Our GPU probe observed byte 2 throughout a 1 MiB buffer before synchronizing writer, then zeros afterward. The probe inspected u8 bytes, avoiding invalid Rust booleans.

I think the initialization-ordering code predates this PR and it only surfaced after admitting bool which exposed its invalid-value consequence.