Repository navigation
Fold smaller workgroups of N-d launches on hardware that runs them fast - #918
Merged
Merged
Conversation
Contributor
|
Documentation preview: https://juliagpu.github.io/KernelAbstractions.jl/previews/PR918/ |
Contributor
Benchmark ResultsShow table
Benchmark PlotsA plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR. |
A tuned launch of an iteration space with more than three dimensions is
only folded onto the N-d grid if the first three dimensions hold 64
work-items. Below that, NVIDIA GPUs run the folded launch slower than a
linear one, but other hardware keeps gaining from it. Folded over
linear, for a kernel scaling an array, by work-items in the first three
dimensions:
64 32 16 8
CUDA, RTX 5080 1.35 0.72 - 0.45
CUDA, RTX 6000 Ada 0.85 0.51 - 0.50
AMDGPU, gfx1036 2.85 2.85 1.43 0.73
Metal, M1 1.93 1.92 1.02 0.63
oneAPI, Arc A770 1.41 1.42 1.17 0.97
PoCL, 8 threads 4.3 6.1 - 2.3
Make the threshold a per-backend `fold_threshold`: 1 on PoCL, 16 on
AMDGPU, Metal and oneAPI, and 64 otherwise, which also covers OpenCL
devices from NVIDIA. Like the tunables of AcceleratedKernels, the values
live in KernelAbstractions (in extensions on the back ends), so they can
be changed without releasing those.
maleadt
force-pushed
the
tb/fold-threshold
branch
from
October 25, 2026 14:14
0bf46d3 to
5bc49c9
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## tb/ndlaunch-fold #918 +/- ##
====================================================
+ Coverage 82.87% 82.91% +0.03%
====================================================
Files 28 31 +3
Lines 2494 2499 +5
====================================================
+ Hits 2067 2072 +5
Misses 427 427 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#919 folds tuned launches of iteration spaces with more than three dimensions onto the N-d grid only when the first three dimensions hold at least 64 work-items. Below that, NVIDIA GPUs run the folded launch slower than a linear one, but other hardware keeps gaining from folding well below 64. Folded over linear, by work-items in the first three dimensions:
This makes the threshold a per-backend
fold_threshold: 1 on PoCL, whose work-items are iterations of a loop, 16 on AMDGPU, Metal and oneAPI, and 64 otherwise. The default also covers the OpenCL back end, which serves NVIDIA devices too (0.90 at 64 on the RTX 6000 Ada).Like the tunables of AcceleratedKernels, the values live in KernelAbstractions, as extensions on AMDGPU, Metal and oneAPI, rather than in the back ends, so they can be tuned without releasing those. The extensions load on Julia 1.10 and 1.13, even though their triggers depend on KernelAbstractions.
The AMDGPU numbers come from an integrated RDNA2 GPU; discrete RDNA and wave64 CDNA GPUs weren't measured.
Tested with the KA test suite on PoCL, and the KernelAbstractions tests of AMDGPU.jl (gfx1036), Metal.jl (M1) and oneAPI.jl (Arc A770), each with the extension loaded.