Skip to content

Cache device limits globally for lockless lookups - #747

Merged
maleadt merged 1 commit into
mainfrom
tb/device-limits-cache
Oct 27, 2026
Merged

maleadt merged 1 commit into
mainfrom
tb/device-limits-cache

Conversation

@maleadt

@maleadt maleadt commented Oct 27, 2026

Copy link
Copy Markdown
Member

Each KernelAbstractions launch calls device_limits several times, each requiring task-local storage and Dict lookups. Store the limits in a global Base.ImmutableDict, replaced under a lock when adding a device. Cached lookups need no lock. __init__ clears the cache because its keys are driver handles. In KI.max_work_group_size(kernel), query device limits only when the kernel's limit is unknown; coalesce evaluated both arguments.

Host time per launch in batches of 64 launches without synchronization. Arc A770, Julia 1.12.7, KernelAbstractions 0.10.3; median of 3 fresh processes:

main PR
@oneapi 2294 ns 2298 ns
KA, 1D, ndrange=1000 3103 ns 2938 ns
KA, 1D, static group size 256 2928 ns 2727 ns
KA, 3D, 30×30×30 3833 ns 3548 ns

The full test suite passes on an Arc A770 (Julia 1.12.7).

Replace repeated task-local storage and `Dict` lookups with a global
immutable dictionary. Lock only when adding a device. Query device limits
in `KI.max_work_group_size` only when the kernel's limit is unknown;
`coalesce` evaluated both arguments. This saves 165-285 ns per
KernelAbstractions launch of ~3 µs on an Arc A770.
@codecov

codecov Bot commented Oct 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.33333% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 86.65%. Comparing base (446f8ac) to head (03afc1a).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
src/oneAPIKernels.jl 92.85% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #747      +/-   ##
==========================================
- Coverage   86.89%   86.65%   -0.25%     
==========================================
  Files          55       55              
  Lines        5090     5005      -85     
==========================================
- Hits         4423     4337      -86     
- Misses        667      668       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@maleadt
maleadt merged commit a2b2df0 into main Oct 27, 2026
5 checks passed
@maleadt
maleadt deleted the tb/device-limits-cache branch October 27, 2026 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant