Skip to content

Virtual CSR over the DR cube; aggregation as BF16 GEMM (ndarray tile + ndarray CBLAS); benchmark of every arm - #1984

Merged
AdaWorldAPI merged 3 commits into
ccr-8062bcc7-l7lkz9-isa-labfrom
ccr-8062bcc7-l7lkz9-vcsr
Nov 8, 2026
Merged

AdaWorldAPI merged 3 commits into
ccr-8062bcc7-l7lkz9-isa-labfrom
ccr-8062bcc7-l7lkz9-vcsr

Conversation

@AdaWorldAPI

@AdaWorldAPI AdaWorldAPI commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

Stacked on #1983, and based on its branch. Retarget to main once #1983 merges. Both need AdaWorldAPI/ndarray#414.

What this adds

A virtual CSR (vcsr.rs). Graph adjacency in the lanes' own 256 × 256 cube coordinates.

  • The cube is the whole 64K universe: one lane is one 256 × 256 matrix.
  • The edge relation is identity × identity, so it is 256 × 256 tile pairs, each a 256 × 256 bitmask block.
  • Each row tile lists its nonempty column tiles, and an edge is addressed by (tile_r, tile_c, slot_r, slot_c).
  • There are no row_ptr or col_idx arrays. ExplicitCsr is the materialized form of the same edges, kept only as the oracle.

Neighbour aggregation (spmm.rs). Y[p][i] = Σ_{j ∈ row(i)} lane_p[j] (plus-times) over 16 DT64-bound lanes, as BF16 GEMMs per block:

  • ndarray strided tile GEMM (AMX TDPBF16PS on this host). A is read in place from the slab; B is staged per 16-row group.
  • ndarray's own backend::cblas_gemm_bf16bf16f32. This is the CBLAS-compatible entry (AMX, then AVX-512 BF16, then scalar), with no external library. It takes dense &[u16] and no lda, so A is staged per block.
  • Staged-A tile path, for comparison.

bench_all. Every arm of #1983 and of this PR, oracle-checked, in one table. The result is committed as crates/dr-isa-stream-lab/BENCH.md.

blasgraph's CsrStorage (BitVec values, HDR semirings) is not realized by a floating-point GEMM. Its adapter is a named seam, not wired.

Benchmark highlights (AMX host, n = 65,536, 16 lanes)

Workload Best arm Time ndarray CBLAS Scalar CSR
SUM WHERE, 16 × 16 masks, whole 64K tile GEMM, A in place 172 µs 3.4 ms —
Graph, degree 8, local scalar CSR 11.6 ms 581 ms 11.6 ms
Graph, degree 64 within the tile tile GEMM 12.5–15.3 ms 51 ms 75 ms
  • The WHERE terminal is the only arm inside the µs gate.
  • ndarray's CBLAS entry repacks on every call. matmul_bf16_to_f32 copies both operands, and bf16_tile_gemm_16x16 VNNI-packs B and allocates each call, so it runs about 20× slower than the strided tile path on the same problem.
  • Block GEMM cost is set by block area, not by edges. It loses to a scalar CSR loop at about 0.15% fill and wins 5–6× at 25% fill.
  • The block count depends on locality in the G2L order. One long-range edge per node grows it about 8×.

Decision: no production crate yet

Every graph arm misses the gate. Even the dense case is bounded at about 1.4 ms by its own 268M MACs. The best arm on the sparse graph is the scalar loop.

The WHERE terminal meets the gate. If it is promoted, it belongs in the Quack / mask-risc terminal set behind DT64 binding, not in a new crate.

Revisit conditions are in .claude/board/entries/2026-11-08-dr-virtual-csr-bf16-gemm.md.

Correctness

  • Virtual equals explicit: the virtual CSR matches the explicit CSR row by row.
  • Exact against the oracle:
    • tile GEMM (direct and staged) at n = 256, 513, 4096 and 65,536;
    • ndarray CBLAS at 513 and 65,536.
  • Lane borrows: 16 per aggregation, whatever n is.
  • Refusals: a graph in another row order, a stale frame, and fewer than 16 lanes for direct mode.
  • Orientation: a one-edge test checks that row i gets lane[j], not lane[i].
  • Disable runs, each going red:
    • the graph-space check;
    • the empty-group skip;
    • B orientation;
    • the tail K;
    • the CBLAS accumulator.
  • Totals: 27 tests pass, and clippy with -D warnings is clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XAqKMMWgnUU9g7Ldyn6sbk

@coderabbitai

coderabbitai Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c9ef7e73-6412-47d9-97e0-3eac77b35aeb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-08T00:42:45.347922Z 686c443 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@AdaWorldAPI AdaWorldAPI changed the title Virtual CSR over the DR cube, with neighbor aggregation as BF16 GEMM (AMX tile, MKL) Virtual CSR over the DR cube; aggregation as BF16 GEMM (ndarray tile + ndarray CBLAS); benchmark of every arm Nov 8, 2026

Copy link
Copy Markdown
Owner Author

excluded-crate fmt, clippy and tests is red for the same reason as #1983. On a runner without AMX, ndarray master's polyfill fallback rejects a strided A longer than 16·K (A must be 16xK BF16 row-major). The fix is AdaWorldAPI/ndarray#415, and I'll re-run this job after it merges. This branch also now carries #1983's bf16 exactness fix (merge f252621).


Generated by Claude Code

The virtual CSR keeps graph adjacency in the lanes' own 256x256 cube
coordinates: each row tile lists its nonempty column tiles, each pair one
256x256 bitmask block, with no row-pointer or column-index array. Row i's
neighbors are read in place from tile(i)/slot(i).

Neighbor aggregation (plus-times) runs as BF16 GEMMs over it: ndarray's
strided tile GEMM (AMX on this host) with A read in place from the slab,
or staged, and, behind the mkl feature, MKL cblas_gemm_bf16bf16f32 with A
a strided BF16 view of the same slab. All three are exact against an
explicit-CSR oracle, including a partial last tile and the full 64K.

bench_vcsr measures three graphs. The GEMM realizations win only when
blocks are dense (community graph, 6.4x over a scalar CSR loop) and lose
on a degree-8 local graph; every row is in milliseconds and none passes
the lance-graph ms gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMMWgnUU9g7Ldyn6sbk
The comparator arm now calls ndarray's CBLAS-compatible
backend::cblas_gemm_bf16bf16f32 instead of upstream mkl_rt, and the mkl
feature is gone. That entry takes dense u16 and no lda, so A is staged per
block; at n = 65536 the slab pitch equals K, so for the whole-universe
WHERE GEMM the slab is passed in place.

bench_all replaces both benches: every arm of both labs, oracle-checked,
in one markdown table, committed as BENCH.md. The board entry records the
decision not to promote the virtual CSR to a production crate, and why.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMMWgnUU9g7Ldyn6sbk
@AdaWorldAPI
AdaWorldAPI force-pushed the ccr-8062bcc7-l7lkz9-vcsr branch from f252621 to d563af1 Compare November 8, 2026 01:25
@AdaWorldAPI
AdaWorldAPI merged commit 8119ba8 into ccr-8062bcc7-l7lkz9-isa-lab Nov 8, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants