Generic GPU sparse arrays: storage-parametric formats owned by GPUArrays by maleadt · Pull Request #745 · JuliaGPU/GPUArrays.jl

maleadt · 2026-06-23T20:26:11Z

Breaking v13 redesign: GPUArrays owns immutable sparse vectors, CSR, CSC and row-major
COO, portable operations, high-level vendor integration and algorithm selection.
Backends supply storage/runtime and native executors, so future portable operations
and algorithm choices stay central.

Compressed pointers and coordinates have separate storage/index widths. Raw
constructors borrow; structural mutation is preflighted. Duplicate combination
preserves order; custom combiners must be associative but need not commute.
See the interface contract for the API and unsupported cases.

Coordinated CUDA, AMDGPU, oneAPI, Metal and OpenCL branches preserve vendor solves,
factorizations, BSR and batching where supported. oneAPI has explicit handle,
queue, buffer and prepared-operation lifetimes.
Metal integration is in JuliaGPU/Metal.jl#982.

The handoff records exact tested revisions and limits;
the performance report records timings and temporary memory. Metal was spot-tested
only. NVIDIA dense-allocation churn also reproduces on the v12 baseline and remains
unresolved.

Not release-ready: final coordinated CI, oneAPI artifact audit/publication, compatible
releases and registered resolution remain gates. The single TEMP CI commit pins the
unreleased stack and explicitly develops dependencies on Julia 1.10.

kshyatt · 2026-06-23T20:54:11Z

+
+A sparse array can't share the `AbstractGPUArray` supertype — that is a `DenseArray`,
+whereas a sparse array must be an `AbstractSparseArray` — so GPUArrays keeps a parallel
+sparse hierarchy with its own generic functionality. Integrating a back-end has three


Weird phrasing here, let me think a bit about how to make this more natural

kshyatt · 2026-06-23T20:54:45Z

+| `AbstractGPUSparseMatrixCSR{Tv,Ti}` | `rowPtr`, `colVal`, `nzVal`, `dims`, `nnz` |
+| `AbstractGPUSparseMatrixCOO{Tv,Ti}` | `rowInd`, `colInd`, `nzVal`, `dims`, `nnz` |
+
+The pointer/index/value arrays are the back-end's own dense vector type. Provide only the


Should we give an example (e.g. CuVector for CUDA)

kshyatt · 2026-06-23T20:55:21Z

+
+### Interface to implement
+
+  * **Constructors** — from component arrays (`MyCSR(rowPtr, colVal, nzVal, dims)`),


What about to/from dense arrays?

kshyatt · 2026-06-23T20:56:59Z

-JLArray(x::JLSparseVector)    = JLArray(collect(SparseVector(x)))
-JLArray(x::JLSparseMatrixCSC) = JLArray(collect(SparseMatrixCSC(x)))
-JLArray(x::JLSparseMatrixCSR) = JLArray(collect(SparseMatrixCSC(x)))
+JLArray(x::JLSparseVector)    = GPUArrays.to_dense(x)


should we make this an eval block?

kshyatt · 2026-06-23T20:58:50Z

 SparseArrays.SparseMatrixCSC(x::AbstractGPUSparseMatrixCSC) = SparseMatrixCSC(size(x)..., Array(SparseArrays.getcolptr(x)), Array(SparseArrays.rowvals(x)), Array(SparseArrays.nonzeros(x)))

+function check_sparse_target(::Type{ST}, ::Type{Tv}, ::Type{Ti}) where {ST,Tv,Ti}
+    Tv isa Type && Ti isa Type ||


This looks really funky to me. Shouldn't Ti also check it's a <: Integer?

kshyatt · 2026-06-23T20:59:08Z


+function check_sparse_target(::Type{ST}, ::Type{Tv}, ::Type{Ti}) where {ST,Tv,Ti}
+    Tv isa Type && Ti isa Type ||
+        throw(ArgumentError("sparse target type $ST must specify value and index eltypes"))


Should we include Tv and Ti in the error message?

kshyatt · 2026-06-23T21:01:44Z

+@kernel function densify_vector_kernel!(out, iPtr, nzVal)
+    k = @index(Global, Linear)
+    if k <= length(nzVal)
+        @inbounds out[iPtr[k]] = nzVal[k]


it's funny how Claude made this a one liner here but not above

maleadt · 2026-06-23T21:06:16Z

Yeah this hasn't passed my personal sniff test yet, hence draft, but thanks for the review. FWIW, the initial motivation was Metal support + SpMV/SpMM (which happens to be easier to express in COO), with the rest following out of it. The API simplification, replacing *_array_type / dense_array_type / sparse_array_type with something like similar feels like a genuine simplification though. (tried to get rid of the remaining csc_type/csr_type/... too, but that proved too hard)

kshyatt · 2026-06-23T21:12:10Z

Ah I can eff off until it's ready if you prefer, sorry!

gdalle · 2026-06-24T08:27:34Z

I'd love to review this one too once it's ready!

maleadt · 2026-07-07T13:36:44Z

Note to self: add a findnz regression test here to catch issues like JuliaGPU/CUDA.jl#3189

kshyatt · 2026-08-14T15:53:52Z

Can we resurrect this?

gdalle · 2026-09-29T16:22:40Z

This is exciting!!!

GPUArrays now implements the sparse formats itself: GPUSparseVector and GPUSparseMatrixCSR/CSC/COO are immutable structs parametrized on the dense vectors that store their indices and values. Generic code allocates results with similar() on the buffers it already has, so the trait hooks (dense_array_type, sparse_array_type, csc_type, csr_type, coo_type, _sptranspose, _spadjoint) are gone, and a back-end needs no sparse-specific code: it can alias the types for its storage and add a transfer policy for its adaptor. The same struct is used on the device, replacing the GPUSparseDevice* types and the per-back-end adapt rules. The formats are canonical, as in SparseArrays (COO in row-major order). Constructors check host metadata only; check_structure verifies the invariants on the device. similar, copy, copyto!, zero and adapt follow SparseArrays and Adapt semantics: copies and conversions never share buffers, and adapting to a storage type keeps the format and types. Conversions between all formats run on the device (generic_regroup for CSR <-> CSC, generic_expand/generic_compress for CSR <-> COO). Broadcasting and reductions are ported to the new types. Broadcasting no longer reuses the input's pointer buffer for its output, and handles matrices without rows or columns. AbstractGPUSparseMatrixCSR/CSC/COO remain as deprecated aliases of the concrete types for downstream dispatch. JLArrays keeps only aliases, constructors on them, and host versions of sortperm, accumulate! and findall. The testsuite builds its inputs from dense buffers and covers every format for every back-end.

Dense GPU arrays convert to every sparse format on the device, through findall in column-major order (or on the dense transpose for row-major formats), and sparse(A; fmt) selects the format like CUDA.jl and AMDGPU.jl do. copy(transpose(A)), copy(adjoint(A)) and permutedims produce the same format by regrouping, and the format constructors accept transposed operands. findnz returns the coordinates in column-major order like SparseArrays, for every format.

sparse(I, J, V, m, n, combine; fmt) and sparsevec(I, V, n, combine) on GPU vectors sort packed coordinate keys, combine the values of repeated coordinates with a per-run fold (reduce_by_key), and compress. As in SparseArrays, repeated entries are folded in input order, with + by default (| for Bool) and explicit zeros kept, so that a call gives the same result on every back-end. Because the device sort need not be stable, a second sort on keys that are unique per entry restores the input order, unless the order cannot change the result (+ and * on integers, max, min, |, &). spdiagm builds on the same path.

The sparse redesign removes the back-end hooks and changes the sparse types, which is breaking for back-ends and for JLArrays' sparse types.

…tion

…y tests]

…t CUDA baseline

…ence

…nly tests]

…ream migration

…lidation [only tests]

…ates

kshyatt reviewed Jun 23, 2026

View reviewed changes

michel2323 mentioned this pull request Sep 24, 2026

Make oneMKL sparse matrices AbstractGPUSparseArrays JuliaGPU/oneAPI.jl#642

Merged

This was referenced Sep 25, 2026

Compact sparse-vector broadcast output on the device #795

Merged

Fix sparse mapreduce for operators other than + #796

Merged

maleadt force-pushed the tb/sparse branch from 5115577 to 2a50d28 Compare September 25, 2026 18:23

maleadt changed the base branch from main to tb/permutedims-empty September 25, 2026 18:23

maleadt added this pull request to stack #797 September 25, 2026 18:23

This was referenced Sep 25, 2026

Don't launch a kernel to permute empty arrays #798

Merged

Sparse arrays on top of GPUArrays' generic formats JuliaGPU/Metal.jl#982

Draft

maleadt changed the title ~~Generic sparse arrays: CSC/CSR/COO formats, on-device conversions, SpMM/SpMV~~ Generic GPU sparse arrays: storage-parametric formats owned by GPUArrays Sep 25, 2026

christiangnrd mentioned this pull request Oct 1, 2026

Add GPUArrays sparse array support JuliaGPU/Metal.jl#909

Open

maleadt added the arrays The array abstraction: broadcasting, mapreduce, indexing, views, generic linear algebra. label Oct 19, 2026

This was referenced Oct 19, 2026

fixes the kron implementation for sparse + diagonal matrix JuliaGPU/CUDA.jl#2804

Closed

Fix sparse mul! with integer inputs and in-place CSC products JuliaGPU/CUDA.jl#3416

Merged

maleadt added 4 commits October 21, 2026 10:55

Bump GPUArrays to 12.0.0 and JLArrays to 0.5.0

676993d

The sparse redesign removes the back-end hooks and changes the sparse types, which is breaking for back-ends and for JLArrays' sparse types.

maleadt removed this pull request from stack #797 October 21, 2026 13:00

maleadt changed the base branch from tb/permutedims-empty to main October 21, 2026 13:00

maleadt added 13 commits October 21, 2026 17:17

Materialize structured sparse operands for mixed addition and subtrac…

5313858

…tion

docs: record coordinated CI, hardware parity and remaining gates [onl…

54522bd

…y tests]

Record PR retarget, validated entrypoint and SpGEMM publication contract

0b1460f

Benchmark migrated sparse execution and observed device memory agains…

688778b

…t CUDA baseline

Own dense sparse entrypoint policy while preserving backend defaults

262d8bb

Add reproducible backend sparse validation and portable-only driver

820f806

Record final-stack checks, policy migration and OpenCL isolation evid…

256a629

…ence

Update validation-driver checkpoint

ddba8ce

Keep rectangular identity regressions independent of stdlib restrictions

974c082

Record exact shared-suite results and backend CI wiring checkpoint [o…

91b31f1

…nly tests]

Retain deprecated canonical-format dispatch names and document downst…

3ceeb6e

…ream migration

Record downstream sparse API migration audit [only tests]

c0134e1

Record OpenCL compilation routes and unsupported-type preflight in va…

d9350ef

…lidation [only tests]

maleadt force-pushed the tb/sparse branch 4 times, most recently from 4e20de2 to 099665d Compare October 21, 2026 17:27

maleadt added 8 commits October 21, 2026 19:44

TEMP CI: validate the pinned unreleased sparse-v13 stack [only tests]

8609e8d

Keep sparse broadcast combinations on the checked GPU path

f1b0abd

Include implicit zeros in negative-order sparse norms

d26c2dd

Test sparse entry rendering independently of CPU MIME presentation

6b04c67

Use scalar magnitudes for nonfinite complex norm references

f10a01f

Document final sparse contracts, validation and coordinated release g…

8b6fb9f

…ates

Keep zero-dimensional GPU fusion inside sparse broadcast preflight

08f0630

Reject unsupported broadcast dimensions before generic fallback

5e6c98d

maleadt force-pushed the tb/sparse branch from 099665d to 9a9126f Compare October 21, 2026 17:44

Record artifact validation and consolidate the sparse release handoff

76600f3

maleadt force-pushed the tb/sparse branch from 9a9126f to 76600f3 Compare October 21, 2026 18:27


		### Interface to implement

		* Constructors — from component arrays (`MyCSR(rowPtr, colVal, nzVal, dims)`),

Conversation

maleadt commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kshyatt Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

kshyatt Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

kshyatt Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

kshyatt Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

kshyatt Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

kshyatt Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

kshyatt Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

maleadt commented Jun 23, 2026

Uh oh!

kshyatt commented Jun 23, 2026

Uh oh!

gdalle commented Jun 24, 2026

Uh oh!

maleadt commented Jul 7, 2026

Uh oh!

kshyatt commented Aug 14, 2026

Uh oh!

gdalle commented Sep 29, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

maleadt commented Jun 23, 2026 •

edited

Loading