Skip to content

reduce doesn't query src backend if backend specified - #23

Closed
dstahlke wants to merge 2 commits into
JuliaGPU:mainfrom
dstahlke:mappedarray
Closed

reduce doesn't query src backend if backend specified#23
dstahlke wants to merge 2 commits into
JuliaGPU:mainfrom
dstahlke:mappedarray

Conversation

@dstahlke

@dstahlke dstahlke commented Feb 16, 2025

Copy link
Copy Markdown
Contributor

This allows calling reduce on MappedArrays types. That's nice because you can avoid allocating a large array to store the input. For example,

julia> using AcceleratedKernels, CUDA, MappedArrays
julia> AcceleratedKernels.sum(mappedarray(sin, 1:10_000_000_000), CUDABackend(false,false))

I'm creating a bare-bones pull request to get feedback on whether this is desired. If so, I can add a unit test.

@anicusan

anicusan commented Feb 16, 2025

Copy link
Copy Markdown
Member

This is great, one of the primary reasons I added an explicit backend argument to all functions was to allow unmaterialised types such as ranges - e.g. having any work over eachindex(v) instead of v directly for more complex conditions. I haven't tested that on reduce, for which the temp check was failing.

A test for such unmaterialised types - MappedArrays or simple ranges - would be very useful; and perhaps including such tests for the other functions, thanks!

@dstahlke

dstahlke commented Feb 17, 2025

Copy link
Copy Markdown
Contributor Author

Unit tests added for range and mappedvector.

I tried to also get this working for mapreduce_nd. There are a variety of InvalidIRError, for example "unsupported dynamic function invocation (call to convert)" and "unsupported dynamic function invocation (call to print_to_string". Test case:

AK.mapreduce(x->x[1], +, CartesianIndices((12:345, 67:89)), CUDABackend(), init=10)

@maleadt maleadt mentioned this pull request Oct 1, 2026
Merged
@maleadt

maleadt commented Oct 1, 2026

Copy link
Copy Markdown
Member

#137

@maleadt maleadt closed this Oct 1, 2026
maleadt added a commit that referenced this pull request Oct 22, 2026
@maleadt @dstahlke
Support lazy arrays such as MappedArrays as read-only sources
79ae594 
Backend resolution asked every array that is not a range, CartesianIndices or LinearIndices
(or a Base view, reshape or permutation of one) for its backend with KernelAbstractions'
`get_backend`, which recurses through `parent` and throws when it reaches a range or a tuple
of parents. So `AK.sum(mappedarray(abs2, 1:n))`, and every other operation on a
MappedArrays.jl array over ranges or Cartesian indices, threw "Implement get_backend"
without an explicit `backend`, and `mappedarray(+, A, B)` threw a MethodError.

With `backend` given, reductions, predicates, findall and searchsorted already worked on such
arrays, since an `isbits` mapped array is passed to kernels as it is. But the operations that
allocate a result (map, accumulate, reverse, sort, sortperm) used `similar(v)`, which for a
wrapper without its own `similar` is Base's host `Array`, so their kernels got a host array.
Copies of such wrappers (`sort`, the keys of the merge sort permutation, the host finish of
small reductions) went through `copy`, `copyto!` or `Array`, whose fallbacks scalar-index
device arrays.

Now a wrapper votes like the arrays it wraps: `get_backend` is still asked where it can answer
or where the wrapper defines its own method, and a wrapper of only backend-free arrays does not
vote, as Base's wrappers of ranges already did not. A wrapper that gets Base's `similar`
fallback, or wraps no memory, has its results allocated on `backend`. On kernel backends,
wrappers that neither GPUArrays (its wrappers of device arrays) nor Base (wrappers of host
memory) can copy are copied with a kernel; lazy arrays over ranges are not evaluated on the
host either, since their function may read device arrays it captures. Everything else is
allocated and copied as before.

`reverse!` without `dims` indexed `dst` with the source's `eachindex`, which is Cartesian for a
strided view or a lazy array over CartesianIndices, and threw a MethodError; it now pairs
elements in linear order.

A lazy array of device arrays also needs an Adapt.jl rule for its type to reach kernels.
MappedArrays.jl has none, so `mappedarray(f, ::CuArray)` still only works on the host; that
rule belongs in MappedArrays.jl. The tests use a lazy wrapper with such a rule for the device
path.

Supersedes #23, which allocated the 1D reduction's scratch on the given backend instead of
with `similar(src)`, and whose range and mapped-array sums are among the tests.

Co-authored-by: Dan Stahlke <dan@stahlke.org>
maleadt added a commit that referenced this pull request Oct 23, 2026
@maleadt @dstahlke
Support lazy arrays such as MappedArrays as read-only sources
9dbaa7f 
Backend resolution asked every array that is not a range, CartesianIndices or LinearIndices
(or a Base view, reshape or permutation of one) for its backend with KernelAbstractions'
`get_backend`, which recurses through `parent` and throws when it reaches a range or a tuple
of parents. So `AK.sum(mappedarray(abs2, 1:n))`, and every other operation on a
MappedArrays.jl array over ranges or Cartesian indices, threw "Implement get_backend"
without an explicit `backend`, and `mappedarray(+, A, B)` threw a MethodError.

With `backend` given, reductions, predicates, findall and searchsorted already worked on such
arrays, since an `isbits` mapped array is passed to kernels as it is. But the operations that
allocate a result (map, accumulate, reverse, sort, sortperm) used `similar(v)`, which for a
wrapper without its own `similar` is Base's host `Array`, so their kernels got a host array.
Copies of such wrappers (`sort`, the keys of the merge sort permutation, the host finish of
small reductions) went through `copy`, `copyto!` or `Array`, whose fallbacks scalar-index
device arrays.

Now a wrapper votes like the arrays it wraps: `get_backend` is still asked where it can answer
or where the wrapper defines its own method, and a wrapper of only backend-free arrays does not
vote, as Base's wrappers of ranges already did not. A wrapper that gets Base's `similar`
fallback, or wraps no memory, has its results allocated on `backend`. On kernel backends,
wrappers that neither GPUArrays (its wrappers of device arrays) nor Base (wrappers of host
memory) can copy are copied with a kernel; lazy arrays over ranges are not evaluated on the
host either, since their function may read device arrays it captures. Everything else is
allocated and copied as before.

`reverse!` without `dims` indexed `dst` with the source's `eachindex`, which is Cartesian for a
strided view or a lazy array over CartesianIndices, and threw a MethodError; it now pairs
elements in linear order.

A lazy array of device arrays also needs an Adapt.jl rule for its type to reach kernels.
MappedArrays.jl has none, so `mappedarray(f, ::CuArray)` still only works on the host; that
rule belongs in MappedArrays.jl. The tests use a lazy wrapper with such a rule for the device
path.

Supersedes #23, which allocated the 1D reduction's scratch on the given backend instead of
with `similar(src)`, and whose range and mapped-array sums are among the tests.

Co-authored-by: Dan Stahlke <dan@stahlke.org>
maleadt added a commit that referenced this pull request Oct 26, 2026
@maleadt @dstahlke
Support lazy arrays such as MappedArrays as read-only sources
274b5e1 
Backend resolution asked every array that is not a range, CartesianIndices or LinearIndices
(or a Base view, reshape or permutation of one) for its backend with KernelAbstractions'
`get_backend`, which recurses through `parent` and throws when it reaches a range or a tuple
of parents. So `AK.sum(mappedarray(abs2, 1:n))`, and every other operation on a
MappedArrays.jl array over ranges or Cartesian indices, threw "Implement get_backend"
without an explicit `backend`, and `mappedarray(+, A, B)` threw a MethodError.

With `backend` given, reductions, predicates, findall and searchsorted already worked on such
arrays, since an `isbits` mapped array is passed to kernels as it is. But the operations that
allocate a result (map, accumulate, reverse, sort, sortperm) used `similar(v)`, which for a
wrapper without its own `similar` is Base's host `Array`, so their kernels got a host array.
Copies of such wrappers (`sort`, the keys of the merge sort permutation, the host finish of
small reductions) went through `copy`, `copyto!` or `Array`, whose fallbacks scalar-index
device arrays.

Now a wrapper whose parents are not a single array with memory votes like the arrays it
wraps, so a wrapper of only ranges does not vote, as Base's wrappers of ranges already did
not; other arrays are still asked with `get_backend`. A wrapper that gets Base's `similar`
fallback, or wraps no memory, has its results allocated on `backend`, and is copied with a
kernel there; other wrappers keep their own `copy`, so an OffsetArray's copy keeps its axes.
Copies into existing arrays and to the host also use a kernel for wrappers that GPUArrays (its
wrappers of device arrays) and Base (wrappers of host memory) do not copy, since Base's
fallbacks index device memory element by element. Lazy arrays over ranges are not evaluated
on the host either, since their function may read device arrays it captures.

A lazy array of device arrays also needs an Adapt.jl rule for its type to reach kernels.
MappedArrays.jl has none, so `mappedarray(f, ::CuArray)` still only works on the host; that
rule belongs in MappedArrays.jl. The tests use a lazy wrapper with such a rule for the device
path.

Supersedes #23, which allocated the 1D reduction's scratch on the given backend instead of
with `similar(src)`, and whose range and mapped-array sums are among the tests.

Co-authored-by: Dan Stahlke <dan@stahlke.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@dstahlke @anicusan @maleadt