Repository navigation
B70 attention opt - #2
Merged
Merged
Conversation
When GPU_TARGET is not given, common.mk detects the local Intel GPU (detect_gpu_target.sh: sycl-ls, then the sysfs PCI id; PVC fallback). LARGE_GRF=auto adds -ze-opt-large-register-file on bmg targets: Xe2 AOT defaults to 128 GRF and the PVC-sized tiles spill there. bmg builds also get -DKITTENS_BMG for Battlemage launch configs. decode keeps 128 GRF (LARGE_GRF ?= 0), it is bandwidth-bound. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Chen, Sheng S <sheng.s.chen@intel.com>
The bf16_2 vec constructor lowers to a software round-to-nearest-even sequence (shr/and/add3 plus a NaN branch) on Xe2; converting the two lanes with the scalar converter emits the hardware conversion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Chen, Sheng S <sheng.s.chen@intel.com>
row_reduce now reduces all rows of a base tile with one transposing butterfly (lane L ends up with row L): 15 shuffles for 16 rows instead of 64. Shuffles lower to serialized a0-indirect moves on Xe, so this was the main stall in the attention softmax. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Chen, Sheng S <sheng.s.chen@intel.com>
On device, float max/sum row reductions of 16- and 8-row tiles use inline vISA register regions (the data movement of sycl-tla's hreduce16/8, BSD-3-Clause) instead of shuffles, so the reduction needs no a0-indirect moves at all. The shuffle butterfly stays as the generic fallback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Chen, Sheng S <sheng.s.chen@intel.com>
A WG x seq_q sweep on B70 (bf16 out, B in {1,4}, N 512..8192) with the
register-region reductions shows wg256/seq_q8 is fastest, or within noise,
for every causal and non-causal shape.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Chen, Sheng S <sheng.s.chen@intel.com>
The forward KV loop now builds one block2d address payload per K/V stream (K^T, K prefetch, V, V prefetch) before the loop and only updates the row per KV tile; all other offsets are vISA immediates. This removes the per-load payload rebuild (364 -> 4 scalar movs per 64 DPAS; loop 504 -> 280 instrs). gl_desc gains a transposed mode, a raw-surface init, compile-time load and prefetch offsets, and int storage for VNNI reads (short storage made IGC re-interleave every word). B70, H32/8 D128 bf16: causal N4096 83 -> 89.5 TF, N8192 71.5 -> 86 TF, non-causal N4096 77 -> 90 TF; all 11 standalone harnesses pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Chen, Sheng S <sheng.s.chen@intel.com>
Every subgroup of a forward WG reads the same K/V tile, yet each one prefetched the whole next tile (8 32x32 blocks per KV tile). The next tile's blocks are now split across the subgroups, one block each for WG256, which cuts the per-subgroup 2D sends per KV tile from 28 to 21. B70, H32/8 D128 bf16: causal N4096 88 -> 96 TF, N8192 86 -> 91 TF, B4 N2048 causal 78 -> 82 TF / full 83 -> 87.5 TF, N1024 +3-4%; non-causal N4096 flat (90 -> 89). A 64-key iteration (half the reductions/rescales), early V loads and a lazy O rescale were tried and dropped: they were slower or neutral, the loop is load-latency bound rather than ALU bound. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Chen, Sheng S <sheng.s.chen@intel.com>
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
3 open findings
What changed in this PR
This PR optimizes attention performance on Intel Battlemage (B70/B60/B570) by adding GPU target auto-detection, introducing Battlemage-specific build defaults (notably large-GRF handling), and updating the GQA forward kernel to use block2D descriptors and faster in-register reductions.
Changes:
- Add
detect_gpu_target.shand wirecommon.mkto auto-detectGPU_TARGET, with PVC as fallback. - Introduce
LARGE_GRFauto/override logic and Battlemage-specific compile-time defines/flags. - Optimize
gqa_forwardby switching K/V loads togl_desc(block2D) and adding register-region transpose-reductions.
| File | Description |
|---|---|
| kernels/detect_gpu_target.sh | Adds script to detect the first local Intel GPU architecture / map PCI IDs to AOT targets. |
| kernels/decode/Makefile | Overrides LARGE_GRF default for decode to protect bandwidth-bound performance on B70. |
| kernels/common.mk | Auto-detects GPU_TARGET, introduces LARGE_GRF auto logic and Battlemage compile-time defines. |
| kernels/attention/gqa_forward.dp.cpp | Replaces flat K/V loads + prefetch with block2D descriptor loads and cooperative prefetching; adds BMG tuning path. |
| include/ops/warp/register/tile/reductions.dp.hpp | Adds a faster transpose-reduce path (device-only asm) and refactors row reductions to use it. |
| include/ops/warp/memory/tile/gl_desc.dp.hpp | Extends gl_desc to support transpose loads and compile-time offsets; updates init API. |
| include/common/base_types.dp.hpp | Changes bf16_2 conversion to reuse scalar bf16 conversion to avoid slow SW RNE sequence on Xe2. |
🧠 Review effort: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Copilot stopped reviewing on behalf of
ShengchenJ due to an error
October 16, 2026 09:29
The hoisted 2D block-load payloads open-coded the per-(batch, kv_head) slice arithmetic and then reused one set of numbers for both K and V, taking the offset, pitch and surface height from Kg alone. Derive each side from its own gl via the template stride<> accessor that global_to_register and the other memory ops already use, so this site cannot drift from the layout the rest of the library assumes and V no longer depends on Kg's dims. Numerically a no-op at the harness's dimensions, where Kg and Vg are constructed identically: residuals are unchanged (causal 1.56e-03, full 2.10e-03) and a 5-trial paired A/B shows causal +0.1% and full +0.5%, both inside the per-case noise floor. No new spills. Note that this does not make the path safe for padded or arbitrarily strided tensors, because gl cannot describe one: stride<axis>() is derived from the shape (gl.dp.hpp:171) rather than stored, and operator[] indexes densely the same way. Supporting real strides is a gl-level change, not a per-kernel one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.