Repository navigation
Cache per-device element-type support - #705
Merged
Merged
Conversation
check_eltype runs for every oneArray construction and queried the driver each time: module_properties twice, plus the BFloat16 check. On an Iris Xe with NEO 26.35 that took 5.2 µs and 32 allocations, out of 6.6 µs for a small allocate+free. Cache the result per device, which brings the check down to ~90 ns without allocating. Also test the capability bits in module_properties().flags. The checks were masking fp16flags/fp64flags, which are ze_device_fp_flags_t (denorm, inf/nan, ...), with ZE_DEVICE_MODULE_FLAG_FP16/FP64. That gave the right answer only because devices report either no FP64 flags or all of them. The compiler configuration now uses the same cached flags. Fixes #690
Contributor
|
Your PR requires formatting changes to meet the project's style guidelines. Click here to view the suggested changes.diff --git a/src/array.jl b/src/array.jl
index f2a8b3b..fea0081 100644
--- a/src/array.jl
+++ b/src/array.jl
@@ -52,33 +52,35 @@ const EltypeSupport = @NamedTuple{float16::Bool, float64::Bool, bfloat16::Bool}
const eltype_support_cache = Dict{ZeDevice, EltypeSupport}()
const eltype_support_lock = ReentrantLock()
-function eltype_support(dev::ZeDevice=device())
- Base.@lock eltype_support_lock begin
- get!(eltype_support_cache, dev) do
- flags = oneL0.module_properties(dev).flags
- (; float16 = flags & oneL0.ZE_DEVICE_MODULE_FLAG_FP16 != 0,
- float64 = flags & oneL0.ZE_DEVICE_MODULE_FLAG_FP64 != 0,
- bfloat16 = _device_supports_bfloat16(dev))
+function eltype_support(dev::ZeDevice = device())
+ return Base.@lock eltype_support_lock begin
+ get!(eltype_support_cache, dev) do
+ flags = oneL0.module_properties(dev).flags
+ (;
+ float16 = flags & oneL0.ZE_DEVICE_MODULE_FLAG_FP16 != 0,
+ float64 = flags & oneL0.ZE_DEVICE_MODULE_FLAG_FP64 != 0,
+ bfloat16 = _device_supports_bfloat16(dev),
+ )
+ end
end
- end
end
function check_eltype(T)
Base.allocatedinline(T) || error("oneArray only supports element types that are stored inline")
Base.isbitsunion(T) && error("oneArray does not yet support isbits-union arrays")
- support = eltype_support()
- if !support.float16
+ support = eltype_support()
+ if !support.float16
contains_eltype(T, Float16) && error("Float16 is not supported on this device")
end
- if !support.float64
+ if !support.float64
contains_eltype(T, Float64) && error("Float64 is not supported on this device")
end
- @static if isdefined(Core, :BFloat16)
- if !support.bfloat16
- contains_eltype(T, Core.BFloat16) && error("BFloat16 is not supported on this device")
+ @static if isdefined(Core, :BFloat16)
+ if !support.bfloat16
+ contains_eltype(T, Core.BFloat16) && error("BFloat16 is not supported on this device")
end
- end
- return
+ end
+ return
end
"""
diff --git a/test/array.jl b/test/array.jl
index 33d2686..e393740 100644
--- a/test/array.jl
+++ b/test/array.jl
@@ -19,17 +19,17 @@ import Adapt
end
@testset "element type checks" begin
- if !float64_supported
- @test_throws "Float64 is not supported" oneArray{Float64}(undef, 1)
- @test_throws "Float64 is not supported" oneArray{ComplexF64}(undef, 1)
- end
- if !float16_supported
- @test_throws "Float16 is not supported" oneArray{Float16}(undef, 1)
- end
+ if !float64_supported
+ @test_throws "Float64 is not supported" oneArray{Float64}(undef, 1)
+ @test_throws "Float64 is not supported" oneArray{ComplexF64}(undef, 1)
+ end
+ if !float16_supported
+ @test_throws "Float16 is not supported" oneArray{Float16}(undef, 1)
+ end
- # every array construction performs this check, so it shouldn't query the device
- oneAPI.check_eltype(Float32)
- @test @allocated(oneAPI.check_eltype(Float32)) == 0
+ # every array construction performs this check, so it shouldn't query the device
+ oneAPI.check_eltype(Float32)
+ @test @allocated(oneAPI.check_eltype(Float32)) == 0
end
@testset "adapt" begin |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #705 +/- ##
==========================================
- Coverage 83.70% 83.10% -0.61%
==========================================
Files 52 51 -1
Lines 4376 4262 -114
==========================================
- Hits 3663 3542 -121
- Misses 713 720 +7 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
check_eltyperuns on everyoneArrayconstruction and queried the driver each time (module_propertiestwice, plus the BFloat16 check). This caches the result per device, likedevice_limitsdoes for KernelAbstractions.On an Iris Xe (TGL, NEO 26.35):
oneAPI.check_eltype(Float32)unsafe_free!(oneArray{Float32}(undef, 16))The issue measured 29 µs on NEO 26.18, where the BFloat16 check went through the full
propertiesquery. Since #701, NEO advertisesZE_extension_bfloat16_conversions, so that path is cheaper now, but the remaining queries still made up most of a small allocation.This also fixes the capability test noted in the issue:
fp16flags/fp64flagsareze_device_fp_flags_t(denorm, inf/nan, ...), so masking them withZE_DEVICE_MODULE_FLAG_FP16/FP64checked the wrong bits. The checks now usemodule_properties().flags, incheck_eltype, in the compiler configuration (which now reads the same cache), and intest/runtests.jl.The new test checks that unsupported element types are rejected, and that
check_eltypedoesn't allocate.Fixes #690