Skip to content

Cache per-device element-type support - #705

Merged
maleadt merged 1 commit into
mainfrom
tb/cache-device-caps
Oct 18, 2026
Merged

maleadt merged 1 commit into
mainfrom
tb/cache-device-caps

Conversation

@maleadt

@maleadt maleadt commented Oct 16, 2026

Copy link
Copy Markdown
Member

check_eltype runs on every oneArray construction and queried the driver each time (module_properties twice, plus the BFloat16 check). This caches the result per device, like device_limits does for KernelAbstractions.

On an Iris Xe (TGL, NEO 26.35):

before after
oneAPI.check_eltype(Float32) 5.2 µs, 32 allocs 89 ns, 0 allocs
unsafe_free!(oneArray{Float32}(undef, 16)) 6.6 µs, 46 allocs 0.65 µs, 14 allocs

The issue measured 29 µs on NEO 26.18, where the BFloat16 check went through the full properties query. Since #701, NEO advertises ZE_extension_bfloat16_conversions, so that path is cheaper now, but the remaining queries still made up most of a small allocation.

This also fixes the capability test noted in the issue: fp16flags/fp64flags are ze_device_fp_flags_t (denorm, inf/nan, ...), so masking them with ZE_DEVICE_MODULE_FLAG_FP16/FP64 checked the wrong bits. The checks now use module_properties().flags, in check_eltype, in the compiler configuration (which now reads the same cache), and in test/runtests.jl.

The new test checks that unsupported element types are rejected, and that check_eltype doesn't allocate.

Fixes #690

check_eltype runs for every oneArray construction and queried the driver
each time: module_properties twice, plus the BFloat16 check. On an Iris Xe
with NEO 26.35 that took 5.2 µs and 32 allocations, out of 6.6 µs for a
small allocate+free. Cache the result per device, which brings the check
down to ~90 ns without allocating.

Also test the capability bits in module_properties().flags. The checks
were masking fp16flags/fp64flags, which are ze_device_fp_flags_t
(denorm, inf/nan, ...), with ZE_DEVICE_MODULE_FLAG_FP16/FP64. That gave
the right answer only because devices report either no FP64 flags or all
of them. The compiler configuration now uses the same cached flags.

Fixes #690
@github-actions

Copy link
Copy Markdown
Contributor

Your PR requires formatting changes to meet the project's style guidelines.
Please consider running Runic (git runic main) to apply these changes.

Click here to view the suggested changes.
diff --git a/src/array.jl b/src/array.jl
index f2a8b3b..fea0081 100644
--- a/src/array.jl
+++ b/src/array.jl
@@ -52,33 +52,35 @@ const EltypeSupport = @NamedTuple{float16::Bool, float64::Bool, bfloat16::Bool}
 const eltype_support_cache = Dict{ZeDevice, EltypeSupport}()
 const eltype_support_lock = ReentrantLock()
 
-function eltype_support(dev::ZeDevice=device())
-  Base.@lock eltype_support_lock begin
-    get!(eltype_support_cache, dev) do
-      flags = oneL0.module_properties(dev).flags
-      (; float16 = flags & oneL0.ZE_DEVICE_MODULE_FLAG_FP16 != 0,
-         float64 = flags & oneL0.ZE_DEVICE_MODULE_FLAG_FP64 != 0,
-         bfloat16 = _device_supports_bfloat16(dev))
+function eltype_support(dev::ZeDevice = device())
+    return Base.@lock eltype_support_lock begin
+        get!(eltype_support_cache, dev) do
+            flags = oneL0.module_properties(dev).flags
+            (;
+                float16 = flags & oneL0.ZE_DEVICE_MODULE_FLAG_FP16 != 0,
+                float64 = flags & oneL0.ZE_DEVICE_MODULE_FLAG_FP64 != 0,
+                bfloat16 = _device_supports_bfloat16(dev),
+            )
+        end
     end
-  end
 end
 
 function check_eltype(T)
   Base.allocatedinline(T) || error("oneArray only supports element types that are stored inline")
   Base.isbitsunion(T) && error("oneArray does not yet support isbits-union arrays")
-  support = eltype_support()
-  if !support.float16
+    support = eltype_support()
+    if !support.float16
     contains_eltype(T, Float16) && error("Float16 is not supported on this device")
   end
-  if !support.float64
+    if !support.float64
     contains_eltype(T, Float64) && error("Float64 is not supported on this device")
   end
-  @static if isdefined(Core, :BFloat16)
-    if !support.bfloat16
-      contains_eltype(T, Core.BFloat16) && error("BFloat16 is not supported on this device")
+    @static if isdefined(Core, :BFloat16)
+        if !support.bfloat16
+            contains_eltype(T, Core.BFloat16) && error("BFloat16 is not supported on this device")
     end
-  end
-  return
+    end
+    return
 end
 
 """
diff --git a/test/array.jl b/test/array.jl
index 33d2686..e393740 100644
--- a/test/array.jl
+++ b/test/array.jl
@@ -19,17 +19,17 @@ import Adapt
 end
 
 @testset "element type checks" begin
-  if !float64_supported
-    @test_throws "Float64 is not supported" oneArray{Float64}(undef, 1)
-    @test_throws "Float64 is not supported" oneArray{ComplexF64}(undef, 1)
-  end
-  if !float16_supported
-    @test_throws "Float16 is not supported" oneArray{Float16}(undef, 1)
-  end
+    if !float64_supported
+        @test_throws "Float64 is not supported" oneArray{Float64}(undef, 1)
+        @test_throws "Float64 is not supported" oneArray{ComplexF64}(undef, 1)
+    end
+    if !float16_supported
+        @test_throws "Float16 is not supported" oneArray{Float16}(undef, 1)
+    end
 
-  # every array construction performs this check, so it shouldn't query the device
-  oneAPI.check_eltype(Float32)
-  @test @allocated(oneAPI.check_eltype(Float32)) == 0
+    # every array construction performs this check, so it shouldn't query the device
+    oneAPI.check_eltype(Float32)
+    @test @allocated(oneAPI.check_eltype(Float32)) == 0
 end
 
 @testset "adapt" begin

@codecov

codecov Bot commented Oct 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.50000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.10%. Comparing base (8460d3f) to head (b7e5190).

Files with missing lines Patch % Lines
src/array.jl 83.33% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #705      +/-   ##
==========================================
- Coverage   83.70%   83.10%   -0.61%     
==========================================
  Files          52       51       -1     
  Lines        4376     4262     -114     
==========================================
- Hits         3663     3542     -121     
- Misses        713      720       +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@maleadt
maleadt merged commit 4ab2df8 into main Oct 18, 2026
5 checks passed
@maleadt
maleadt deleted the tb/cache-device-caps branch October 18, 2026 13:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Array construction queries device properties on every allocation

1 participant