Repository navigation
Conversation
…e_attention.cu, block_attention.cu), checked against core/attention_ref.zig
… merge combines them in order; a slice takes its own one-warp block where four-warp blocks would leave SMs idle (TreeAttention.sms), the same bits
jschmied
force-pushed
the
cuda-moe-dense-window
branch
from
October 13, 2026 17:57
2710ae2 to
bd3e55c
Compare
…lves again, only one-warp blocks leave them to the merge (a small fixed slice area), so the scratch stays at one partial a group
jschmied
force-pushed
the
cuda-moe-dense-window
branch
from
October 13, 2026 18:56
bd3e55c to
60887de
Compare
…async stages), the same arithmetic in the same order
jschmied
force-pushed
the
cuda-moe-dense-window
branch
from
October 13, 2026 19:16
60887de to
e0a37c5
Compare
…w), partials sized to the query heads, one-warp blocks with 3 tile stages where they fit one wave of the SMs, else 2
…, and dense rows for every family, checked against float64
jschmied
force-pushed
the
cuda-moe-dense-window
branch
from
October 13, 2026 20:40
e0a37c5 to
4ffe90d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Stacked on #746 (the window test checks against its
core/attention_ref.zig). Operator-only, no model calls them yet:window_attention.cu/.zig: sliding-window attention, decode rows over a ring of keys (split by key tile, mergedin tile order) and prompt rows, grouped KV heads, window 0 = full.
moe_glue.cu/.zig: top-k by logit + bias (ties: lower id), sigmoid weights, optionally normalised and scaled,a shared-expert slot; a warp form up to 512 experts, a block form beyond; the weighted combine of the slots.
dense_rows.cu/.zig: bf16 rows times bf16 weight rows into fp32 (routers, output heads), a row's bits at any rowcount.
core/moe_ref.zig,core/dense_ref.zig: the float64 references.Users: families with sliding-window layers or sigmoid-routed MoE on CUDA (Kolibri 1's shapes in the tests; GLM and
DeepSeek-style routing are the same top-k form).
Receipt
c36b536d(1.0.9) plus cuda: tree attention and DFlash block attention for every family, checked against float64 #746 and this commit; one DGX Spark (GB10, sm_121, CUDA 13). Nocheckpoint.
tf-cuda-test window: 48/4x128 window 513 over rings of 640 and 600 (a ring end inside a tile), 8/2x64 window 100with 3 streams, 16/1x128 no window; prompt rows from position 0 and across the ring's wrap: worst 0.465, bits at
any row count and chunk.
tf-cuda-test moe-glue: 384 experts top 6 + shared, 256 top 8 normalised x2.5, 512 top 32 (warp form's widest),513 top 6 (block form's narrowest), 4096 top 64; picks exact, weights and sums worst 0.996.
tf-cuda-test dense-rows: [384, 2560], [4103, 2560], [257, 7168]: worst 0.007, bits at every row count.dropped (17,529x).
window,dense-rows,moe-glueon an RTX 5090, all passing with mutations (Qwen3.8-27B on CUDA: the qwen27 family's engine for NVIDIA GPUs, every stream drafted #608).zig build testandzig build test-cuda-host(27/27 steps, 350/350 tests).zig fmt --checkclean.lean_check.py: 538 findings, as onmain.mainuses it.Until #746 lands, this branch carries its commit; the change itself is the last commit (
2710ae2d).Checklist
"draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Noserving path changes.)