Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .claude/board/entries/2026-11-10-dt64-g2l-lab.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# 2026-11-10 — D-DT64-G2L-1: one witnessed translation reused across GEMMs

**Status:** MEASURED (lab). Closes the G2L row of
`dr-dt64-locality-execution-contract.md` from OPEN to measured in the lab.
Stacked on #1992 (needs ndarray#418).

**What.** A selection named in global ids is translated once, under one frame
of the source's DR `MapTable`, into the source's own row order (counted
lookups). The object is bound in the source's space; B is prepared once
through the translation; every run then reads the source's lanes in place
with zero lookups. Cross-space reads without a translation stay refused, and
the translation does not grant Gather (ADMISSION B #1640).

**Result** (one AMX VM, three runs, each quantity on its own): translate
once 83–85 µs (3200 lookups); prepare B once 46–48 µs (200 admission + 200
packing map reads, 16,384 B bytes); stage A 11–14 µs per warm run (12,800
useful bytes into a 16,384-byte padded buffer); GEMM only 0.71–0.86 µs; warm
run 20–24 µs with 0 lookups; re-translating per operation 157–186 µs. Zero
lookups is not zero movement: the sparse selection is still gathered into A on
every run, at about 15× the GEMM.

**Scope:** K is reordered by the translation. That is validated only for exact
integer GEMM (order-independent sum absent overflow), not for order-sensitive
operators.

**Falsifiers:** oracle in global ids only; an identity past the inner B table refused with OperandShape before packing (#1990 contract); reorder refuses binding,
translation and operand; stale mapping refused; prepared B bound to its
translation; unknown and repeated ids refused. Five disable runs, each red.

**Governing question:** the location of each selected id is resolved once
per translation; B's per-row identity once per preparation; per operation,
nothing.

**Trap:** a disable run leaves its build behind. The bench reported 254 µs
for preparing B from the last disable run's binary; rebuilt, 37–47 µs.
3 changes: 2 additions & 1 deletion .claude/board/entries/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately
opposite directions; the stranding this convention prevents shows up in
exactly one of them, never both.

593 entries, 2026-08-06 .. 2026-11-10.
594 entries, 2026-08-06 .. 2026-11-10.

| date | entry id | finding | file |
|---|---|---|---|
Expand All @@ -35,6 +35,7 @@ exactly one of them, never both.
| 2026-11-10 | `dt64-lane-borrow-all` | | [2026-11-10-dt64-lane-borrow-all.md](2026-11-10-dt64-lane-borrow-all.md) |
| 2026-11-10 | `dt64-int8-facet-stride` | | [2026-11-10-dt64-int8-facet-stride.md](2026-11-10-dt64-int8-facet-stride.md) |
| 2026-11-10 | `D-DT64-GEMM-SHAPE-1` | | [2026-11-10-dt64-gemm-shape.md](2026-11-10-dt64-gemm-shape.md) |
| 2026-11-10 | `D-DT64-G2L-1` | | [2026-11-10-dt64-g2l-lab.md](2026-11-10-dt64-g2l-lab.md) |
| 2026-11-08 | `dr-isa-stream-lab` | | [2026-11-08-dr-isa-stream-lab.md](2026-11-08-dr-isa-stream-lab.md) |
| 2026-11-07 | `D-DT64-7-L5b` | | [2026-11-07-dt64-l5b-historical-basis.md](2026-11-07-dt64-l5b-historical-basis.md) |
| 2026-11-07 | `D-DT64-7-L5a` | | [2026-11-07-dt64-l5a-exact-basis-witness.md](2026-11-07-dt64-l5a-exact-basis-witness.md) |
Expand Down
2 changes: 1 addition & 1 deletion .claude/knowledge/dr-dt64-locality-execution-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ field mapping, operand identity and version.
| Lab resolve on `borrow_all` (follow-up) | MEASURED, not merged | Resolve at 256 fields 1.6–2.0 µs vs 13.8–13.9 µs (#1987 as first measured) and 26–28 µs (`main` after the provenance fix); the old cost was `LaneBinding::addr`, a linear scan called per field |
| INT8 vs BF16 over the same object (follow-up) | MEASURED, not merged | `u8 × i8 → i32` end to end 66–101 µs vs BF16 103–160 µs at 256 fields on one VM; INT8 still copies A (5–7 µs) and re-packs B per call (23–29 µs, 36–38% of its GEMM), because ndarray has no strided-A / prepacked-B INT8 entry |
| #1987's "B preparation ≈ GEMM" | REJECTED-BY-MEASUREMENT | That timer included the synthetic operand's hash; from a table built once, prepared B costs about a third of the GEMM |
| G2L across different source orders | OPEN | Not demonstrated: every fixture shared one coordinate space, and differing spaces were refused, not translated |
| G2L across different source orders (D-DT64-G2L-1) | MEASURED, lab | One translation under one `MapTable` frame (counted lookups), reused across runs with 0 lookups; warm run 20–24 µs (of which A staging 11–14 µs, GEMM 0.7–0.9 µs) vs 157–186 µs re-translating per operation; reorder, stale mapping, foreign translation and an identity past the inner operand refused. Exact integer GEMM only: K order changes under translation. Still OPEN for production: who owns the translation, and frames across a restore (#1993) |
| Foundry operational-ontology parity | HYPOTHESIS | Research goal; nothing here establishes it |

All timings: one VM (Xeon with AMX-BF16, 4 vCPU), single-threaded, medians of
Expand Down
80 changes: 80 additions & 0 deletions crates/dr-isa-stream-lab/BENCH-SHAPE.md
Original file line number Diff line number Diff line change
Expand Up @@ -344,3 +344,83 @@ each pay the tile configuration (~105 ns), and the per-tile calls lose the
2×2 register blocking. Hoisting the pack was not enough; the whole-matrix
kernel had to take the prepacked bands. That is why ndarray#418 adds
`int8_gemm_packed_strided`.

## D-DT64-G2L-1: one witnessed translation, reused (`src/g2l.rs`)

The consumer names its selection in **global ids**. The source holds its u8
lanes in **its own row order** (`SPACE_B`), described by a DR `MapTable` (16
DR256 tiles over 4096 identities). Reading the source's lanes in the
consumer's order is a gather and stays refused (`CoordinateSpace`).

ADMISSION B (#1640): preparation may establish coordinates; execution operates
on co-addressed sets. So:

1. `translate` locates every selected global id once, under one frame of the
map table (counted lookups), and returns the selection **in the source's
row order**. It refuses a map whose coordinate revision is not the source's
(a stale mapping).
2. The object is bound on the source, in the source's space, as usual.
3. B is prepared once through `TranslatedB` (position → global via the map's
pointer table, one read per row); its operand identity names the
translation.
4. Every run reads the source's lanes in place, in the source's order, against
the prepared B, with **zero** lookups. A sum over K does not depend on K's
order, so the result is the consumer's answer.

One AMX VM (`int8_tile_gemm_tier()` = AMX TDPBUSD), three runs
(`results/shape-2026-11-10-g2l-run*.txt`), 64 fields × 64 columns, 200 global
ids. Each quantity is reported on its own; ranges are over the three runs:

| Step | Time | Movement |
|---|---|---|
| translate once | 83–85 µs | 3200 G2L lookups (200 ids × 16 tiles) |
| prepare B once (admit + pack) | 46–48 µs | 200 admission map reads + 200 packing map reads; 16,384 B bytes packed |
| stage A, per warm run | 11–14 µs | 12,800 useful bytes copied into a 16,384-byte padded buffer |
| GEMM only | 0.71–0.86 µs | — |
| **warm run** (resolve + stage A + GEMM) | **20–24 µs** | 0 G2L lookups, 0 map reads, 12,800 A bytes staged |
| re-translate per operation | 157–186 µs | 3200 lookups + 400 map reads + B pack + A stage |

**Zero lookups is not zero data movement.** The selection is sparse in the
source's order, so a warm run still gathers 200 values per field into a dense,
K-padded A: 12,800 useful bytes into a 16,384-byte buffer, every run. That
copy costs about 15× the GEMM it feeds. What the translation removed is the
global → local resolution; what it did not remove is the gather of a scattered
selection. Reading A in place needs the selection to be a region in the
source's order (as in the INT8 direct arm), which this fixture is not.

**Order of K.** Translation reorders K: B is packed in the source's row order,
not the consumer's. That is validated here only for exact integer GEMM, where
`Σ_k a·b` does not depend on the order of summation as long as nothing
overflows (i32 accumulation of 200 products of |a| ≤ 255, |b| ≤ 8, far inside
range). It does not carry over to order-sensitive operators: floating-point
reduction, saturating arithmetic, prefix scans, argmax with tie-breaking, or
anything that reads K positionally.

**Operand extent.** `TranslatedB` reports the inner operand's own extent (its
rows are identities) and maps every selected source position to the identity
it reads (`BValues::highest_row`). An identity past the inner table is
refused with `ShapeRefusal::OperandShape` before packing reads anything.
Before this fix it reported the source's position range and packing indexed
past the table (`tests/integration/g2l.rs`,
`a_translated_identity_past_the_inner_table_is_refused_before_packing`).

The first version of `TranslatedB` resolved position → identity once per
(row, column) and took 241–255 µs to prepare B: 64× the map reads needed. One
read per row fixed it (test-pinned: exactly 200 packing reads).

Falsifiers (`tests/integration/g2l.rs`), each disable-verified red:

| Test | Disable that turns it red |
|---|---|
| exact vs an oracle computed in global ids alone; 5 runs, 0 further lookups, 0 further map reads | per-row cache removed |
| a source reorder refuses the old binding (`CoordinatesChanged`), the translation and the operand; a fresh translation is exact | staleness check in `TranslatedB::new` removed |
| a stale mapping (lanes reordered, map not) is refused | map-vs-source coordinate check removed |
| a prepared B from one translation is refused under another | translated identity ignoring the translation |
| the untranslated cross-space read stays refused | (DT64's existing check) |
| an identity past the inner B table is refused before packing (OperandShape) | `TranslatedB::highest_row` removed: packing reads past the table |
| unknown and repeated ids are refused | — |

A measurement trap worth keeping: after the disable runs, the benchmark binary
was the one built by the last disable run (cache off), and reported 254 µs
for preparing B. Rebuilding from the committed source gave 37–47 µs. A disable
run leaves its build behind; re-measure only after a rebuild.
43 changes: 43 additions & 0 deletions crates/dr-isa-stream-lab/results/shape-2026-11-10-g2l-run1.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
Host: Intel(R) Xeon(R) Processor @ 2.10GHz; 4 logical CPUs; cache (cpu0): L1 Data 48K, L1 Instruction 32K, L2 Unified 2048K, L3 Unified 266240K
Tile tier: AMX TDPBF16PS. Single-threaded. 41 warm runs per row (median).

Sparse demand (256 fields x 256 cols, 40 scattered identities): logical selection 40 · K padded 64 · lanes borrowed 256 · A bytes copied 20480 · dense dot-product steps 512 · staged AMX 390.4 µs

G2L (64 fields x 64 cols, 200 global ids, source in its own order, 16 tiles, AMX TDPBUSD):
| Step | Time | Movement |
|---|---|---|
| translate once | 84.4 µs | 3200 G2L lookups |
| prepare B once (admit + pack) | 48.1 µs | 200 admission map reads, 200 packing map reads, 16384 B bytes packed |
| stage A (per warm run) | 13.6 µs | 12800 useful bytes copied into a 16384-byte padded buffer |
| GEMM only | 855 ns | — |
| warm run (resolve + stage A + GEMM) | 23.5 µs | 0 G2L lookups, 0 map reads, 12800 A bytes staged |
| re-translate per operation | 185.9 µs | 3200 lookups + 400 map reads + B pack + A stage |

| Metric | 64×64 | 256×256 |
|---|---|---|
| Output tiles / K=32 dot-product steps | 16 / 32 | 256 / 2048 |
| DT64 bind (cold) | 3086 ns · 2 allocs | 8644 ns · 6 allocs |
| Resolve: one check + borrows (warm) | 1171 ns · 0 allocs · 0 B | 2025 ns · 0 allocs · 0 B |
| Resolve: `lane()` per field (warm) | 11.4 µs | 154.0 µs |
| B preparation (VNNI panels) | 4796 ns · 8192 B written · 8320 B alloc | 79.5 µs · 131072 B written · 131584 B alloc |
| Direct AMX (A in place) | 4110 ns · 63.8 GMAC/s · 0 allocs · 0 B tmp | 142.1 µs · 118.1 GMAC/s · 0 allocs · 0 B tmp |
| Direct AMX, pitch 65,568 (not 2^k) | 3697 ns · 70.9 GMAC/s | 146.5 µs · 114.6 GMAC/s |
| Staged AMX (dense A tiles) | 4153 ns · 1 allocs · 2048 B tmp | 153.3 µs · 1 allocs · 8192 B tmp |
| ndarray CBLAS BF16 | 36.2 µs · 23 allocs · 54272 B tmp | 1036.1 µs · 263 allocs · 803840 B tmp |
| Source bytes copied (direct / staged / CBLAS) | 0 / 8192 / 8192 | 0 / 131072 / 131072 |
| CBLAS B materialized | 8192 B | 131072 B |
| FP32 accumulator (C) | 16384 B | 262144 B |
| End-to-end warm (resolve + direct, B reused) | 6479 ns | 144.2 µs |
| End-to-end cold (bind + prepare + resolve + direct) | 16.0 µs | 264.5 µs |
| Warm repetitions (100, per run, one prepare) | 6394 ns | 150.0 µs |
| INT8 resolve (u8 lanes, one check) | 1208 ns | 1977 ns |
| INT8 stage A (region: one copy per lane) | 308 ns · 4096 B | 4069 ns · 65536 B |
| INT8 stage A, per-value walk (baseline) | 3452 ns | 27.3 µs |
| INT8 B for a region, in place from the table | 30 ns · 0 B written | 30 ns · 0 B written |
| INT8 materialize B (row-major i8) | 6545 ns · 4096 B | 104.9 µs · 65536 B |
| INT8 GEMM `backend::gemm_i8` (packs B per call) | 3556 ns · 73.7 GMAC/s · 3 allocs · 3072 B tmp | 80.8 µs · 207.7 GMAC/s · 3 allocs · 12288 B tmp |
| INT8 B band gather + VNNI pack (done inside each gemm_i8 call) | 2110 ns · 59% of the GEMM row | 29.5 µs · 36% of the GEMM row |
| INT8 end-to-end warm (resolve + stage + GEMM) | 5827 ns | 85.9 µs |
| INT8 prepare B once (PackedI8B bands, `ndarray::simd`) | 5370 ns · 4096 B | 80.2 µs · 65536 B |
| INT8 direct GEMM (A in place, prepared B) | 1007 ns · 260.3 GMAC/s · 0 allocs · 0 B of A copied | 43.6 µs · 385.2 GMAC/s · 0 allocs · 0 B of A copied |
| INT8 direct end-to-end warm (resolve + GEMM) | 3019 ns | 44.0 µs |
43 changes: 43 additions & 0 deletions crates/dr-isa-stream-lab/results/shape-2026-11-10-g2l-run2.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
Host: Intel(R) Xeon(R) Processor @ 2.10GHz; 4 logical CPUs; cache (cpu0): L1 Data 48K, L1 Instruction 32K, L2 Unified 2048K, L3 Unified 266240K
Tile tier: AMX TDPBF16PS. Single-threaded. 41 warm runs per row (median).

Sparse demand (256 fields x 256 cols, 40 scattered identities): logical selection 40 · K padded 64 · lanes borrowed 256 · A bytes copied 20480 · dense dot-product steps 512 · staged AMX 346.8 µs

G2L (64 fields x 64 cols, 200 global ids, source in its own order, 16 tiles, AMX TDPBUSD):
| Step | Time | Movement |
|---|---|---|
| translate once | 84.9 µs | 3200 G2L lookups |
| prepare B once (admit + pack) | 47.7 µs | 200 admission map reads, 200 packing map reads, 16384 B bytes packed |
| stage A (per warm run) | 13.7 µs | 12800 useful bytes copied into a 16384-byte padded buffer |
| GEMM only | 857 ns | — |
| warm run (resolve + stage A + GEMM) | 22.6 µs | 0 G2L lookups, 0 map reads, 12800 A bytes staged |
| re-translate per operation | 183.3 µs | 3200 lookups + 400 map reads + B pack + A stage |

| Metric | 64×64 | 256×256 |
|---|---|---|
| Output tiles / K=32 dot-product steps | 16 / 32 | 256 / 2048 |
| DT64 bind (cold) | 3183 ns · 2 allocs | 8677 ns · 6 allocs |
| Resolve: one check + borrows (warm) | 1227 ns · 0 allocs · 0 B | 2075 ns · 0 allocs · 0 B |
| Resolve: `lane()` per field (warm) | 12.0 µs | 152.3 µs |
| B preparation (VNNI panels) | 4869 ns · 8192 B written · 8320 B alloc | 79.5 µs · 131072 B written · 131584 B alloc |
| Direct AMX (A in place) | 4601 ns · 57.0 GMAC/s · 0 allocs · 0 B tmp | 148.0 µs · 113.4 GMAC/s · 0 allocs · 0 B tmp |
| Direct AMX, pitch 65,568 (not 2^k) | 4130 ns · 63.5 GMAC/s | 130.9 µs · 128.2 GMAC/s |
| Staged AMX (dense A tiles) | 4416 ns · 1 allocs · 2048 B tmp | 146.5 µs · 1 allocs · 8192 B tmp |
| ndarray CBLAS BF16 | 36.4 µs · 23 allocs · 54272 B tmp | 1025.9 µs · 263 allocs · 803840 B tmp |
| Source bytes copied (direct / staged / CBLAS) | 0 / 8192 / 8192 | 0 / 131072 / 131072 |
| CBLAS B materialized | 8192 B | 131072 B |
| FP32 accumulator (C) | 16384 B | 262144 B |
| End-to-end warm (resolve + direct, B reused) | 6814 ns | 150.4 µs |
| End-to-end cold (bind + prepare + resolve + direct) | 16.0 µs | 262.6 µs |
| Warm repetitions (100, per run, one prepare) | 6583 ns | 154.3 µs |
| INT8 resolve (u8 lanes, one check) | 1195 ns | 1975 ns |
| INT8 stage A (region: one copy per lane) | 309 ns · 4096 B | 4934 ns · 65536 B |
| INT8 stage A, per-value walk (baseline) | 3439 ns | 30.9 µs |
| INT8 B for a region, in place from the table | 30 ns · 0 B written | 30 ns · 0 B written |
| INT8 materialize B (row-major i8) | 6267 ns · 4096 B | 102.1 µs · 65536 B |
| INT8 GEMM `backend::gemm_i8` (packs B per call) | 3584 ns · 73.1 GMAC/s · 3 allocs · 3072 B tmp | 78.7 µs · 213.0 GMAC/s · 3 allocs · 12288 B tmp |
| INT8 B band gather + VNNI pack (done inside each gemm_i8 call) | 2105 ns · 59% of the GEMM row | 29.6 µs · 38% of the GEMM row |
| INT8 end-to-end warm (resolve + stage + GEMM) | 5906 ns | 89.7 µs |
| INT8 prepare B once (PackedI8B bands, `ndarray::simd`) | 5327 ns · 4096 B | 79.1 µs · 65536 B |
| INT8 direct GEMM (A in place, prepared B) | 1011 ns · 259.3 GMAC/s · 0 allocs · 0 B of A copied | 50.3 µs · 333.2 GMAC/s · 0 allocs · 0 B of A copied |
| INT8 direct end-to-end warm (resolve + GEMM) | 3064 ns | 54.3 µs |
Loading
Loading