perf(kernels): SIMD128 bodies for K-quants, Q5_0/Q5_1 and dense FP32/BF16/FP16; threaded dense kernels

CHANGELOG.md

@@ -2,6 +2,17 @@
 
 ## [Unreleased]
 
+### Added
+
+- **BF16 and FP16 matmul on the wasm tier**: `WasmKernelProvider` now binds the sidecar's `skainet_bf16_matmul` / `skainet_fp16_matmul` exports (`WasmBf16MatmulKernel`, `WasmFp16MatmulKernel`), so 16-bit dense weights no longer fall to the scalar kernels in a browser; a contiguous weight stays resident in the sidecar after its first call.
+- **`WasmSimd.features()` / `WasmKernels.features()`**: mode, SIMD128, threads, cross-origin isolation, logical cores and resident weight bytes — the wasm counterpart of the JVM provider banner. `WasmSimd.install()` prints one `[SKaiNET] Using SIMD128 sidecar kernels (…)` line like the JVM factory does.
+- **`KernelStrictness` is common code** (was JVM-only): `KernelStrictness.overrideEnabled` is the programmatic switch on every target; the JVM and Android still read `-Dskainet.strict.kernels`. `KernelDispatch` fails fast before the decoding reference fallback and `DefaultCpuOps.matmul` before the generic per-element loop, so strict mode means the same thing on every tier.
+- **Kernel support matrix**: a `Float16` row, BF16/FP16 in the wasm column, Q6_K corrected to `native-ffm` on the JVM (the provider has had the kernel; the declared tier had not), and a third table, *Resident serving*, naming the sidecar-resident weights that stand in for mapped serving on JS/WASM. `KernelSupportMatrixTest` now gates the declared Panama and FFM tiers against what the providers expose.
+- **Explicit SIMD128 bodies for every sidecar matmul kernel**: Q4_K, Q5_K, Q6_K, Q5_0 and Q5_1 join Q8_0 and Q4_0 with hand-written `__wasm_simd128__` paths (the K-quant int8 dots on `i32x4.dot_i16x8_s`, the Q5 fifth-bit plane via a byte swizzle); the dense FP32 GEMM gets a 4×8 register-blocked, k-blocked micro-kernel; BF16 and FP16 widen eight weights per step. BF16 and FP16 also gain NEON bodies (none existed on any tier). Results stay bit-identical in accumulation order to the portable bodies.
+- **Threaded dense kernels**: `skainet_fp32_matmul`, `skainet_bf16_matmul` and `skainet_fp16_matmul` now split the output over the shared row-thread pool like the packed kernels do (the GEMM over the longer side of `C`, the 16-bit kernels over columns) — the mt sidecar's pthreads in a browser, the native pool on the JVM.
+- **`WasmCpuOps.matmul`**: a dense 2-D `A · B` on heap FP32 operands runs the sidecar GEMM; the common `[k, n]` loop no longer serves that shape on wasm (it was the only dense path that never reached the sidecar).
+- **`wasm-bench` runs the `engine-*` scenarios** of the JVM publish harness (`engine-fp32-gemm`, `engine-kernel-matmul`, `engine-bf16-matmul`, `engine-q8-matmul`, `engine-q4-gemm`, `engine-ternary-f32-gemv`, `engine-elementwise-add`, `engine-reductions-sum/mean`) with identical shapes and metrics on every tier it measures, so the JVM and wasm numbers sit in one table.
+
 ## [0.61.0] - 2026-10-16
 
 Headline: **the browser lane (SKEEP-007): packed SIMD128 kernels, quantize-at-load, and no more

build-logic/convention/src/main/kotlin/GenerateKernelMatrixTask.kt

@@ -49,8 +49,10 @@ abstract class GenerateKernelMatrixTask : DefaultTask() {
             appendLine(
                 "Each cell is the best (highest-priority) provider that serves " +
                     "`${module.inputDtype} × format` `${module.op}` on that platform: " +
-                    "*native-ffm* (100) → *panama-vector* (50) → *scalar* (0). An empty cell " +
-                    "(`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only).",
+                    "*native-ffm* / *native-jni* / *native-cinterop* (100) → *panama-vector* / " +
+                    "*wasm-simd128* (50) → *scalar* (0). An empty cell " +
+                    "(`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only). " +
+                    "The JS/WASM column assumes `WasmSimd.install()` has run; without the sidecar it is the scalar floor.",
             )
             appendLine("")
             if (module.formats.isEmpty() || module.platforms.isEmpty()) {
@@ -100,6 +102,32 @@ abstract class GenerateKernelMatrixTask : DefaultTask() {
                 appendLine("|===")
                 appendLine("")
             }
+            if (module.resident.isNotEmpty()) {
+                appendLine("== Resident serving (sidecar memory)")
+                appendLine("")
+                appendLine(
+                    "The browser has no files to map, so the wasm tier keeps weights *resident* in the " +
+                        "SIMD128 sidecar's linear memory instead: each weight is uploaded once on its first " +
+                        "matmul and every later call reads it in place (`WasmKernels.weightHandle`). This is " +
+                        "the JS/WASM counterpart of mapped serving above — no per-call copy of the weight, " +
+                        "no prepack — bounded by the wasm32 4 GB linear memory rather than the page cache. " +
+                        "The other platforms serve from the heap, off-heap or the mapping and have no cell here.",
+                )
+                appendLine("")
+                appendLine("[cols=\"$colSpec\", options=\"header\"]")
+                appendLine("|===")
+                append("| Weight format ")
+                module.platforms.forEach { append("| $it ") }
+                appendLine("")
+                appendLine("")
+                module.resident.forEach { fmt ->
+                    append("| `${fmt.name}` ")
+                    module.platforms.forEach { p -> append("| ${fmt.byPlatform[p] ?: "—"} ") }
+                    appendLine("")
+                }
+                appendLine("|===")
+                appendLine("")
+            }
             appendLine(
                 "See also the eager backends & kernels mindmap " +
                     "(xref:explanation/eager-execution.adoc[]) for the narrative overview and gaps.",

build-logic/convention/src/main/kotlin/models/KernelSupportModels.kt

@@ -22,6 +22,7 @@ data class KernelSupportModule(
      * prepack. Absent/empty in pre-#1189 JSON, so old files stay decodable.
      */
     val mapped: List<KernelFormatSupport> = emptyList(),
+    val resident: List<KernelFormatSupport> = emptyList(),
 )
 
 @Serializable

docs/modules/ROOT/pages/contributing/dtype-model.adoc

@@ -105,11 +105,18 @@ affordance today (W4, shipped):
 java -Dskainet.strict.kernels=true …
 ----
 
+[source,kotlin]
+----
+KernelStrictness.overrideEnabled = true   // every target; the only switch on JS, Wasm and Kotlin/Native
+----
+
 When set, `DefaultCpuOpsJvm.matmul` raises
 `NoSuchKernelException` (with the failing dtype pair and the list
 of currently-registered providers) just before its silent scalar
-fallback would have run. Default off — adaptive behaviour is
-preserved.
+fallback would have run; on every tier `KernelDispatch` raises it
+before the decoding reference kernel and `DefaultCpuOps.matmul`
+before the generic per-element loop. Default off — adaptive
+behaviour is preserved.
 
 The constraint-resolution pass (W7) raises the same exception
 shape at *graph-prep* time, before forward execution can even

docs/modules/ROOT/pages/explanation/eager-execution.adoc

@@ -52,28 +52,33 @@ mindmap
 
 == Kernel × provider (matmul, FP32 activations)
 
-[cols="2,1,1,1,1,1",options="header"]
+[cols="2,1,1,1,1,1,1",options="header"]
 |===
-| Weight format | Scalar (all targets) | Panama (JVM) | Native FFM (JVM) | Native JNI (Android) | Native cinterop (K/N)
-| FP32 | ✅ | ✅ | ✅ | ❌ | ❌
-| BF16 | ✅ | ✅ | ✅ | ❌ | ❌
-| Q8_0 | ✅ | ✅ | ✅ | ✅ | ✅
-| Q4_0 | ✅ | ✅ | ✅ | ✅ | ✅
-| Q4_K | ✅ | ✅ | ✅ | ✅ | ✅
-| Q6_K | ✅ | ✅ | ❌ | ✅ | ✅
-| Q5_K | ✅ | ✅ | ✅ | ✅ | ✅
-| Q5_1 | ✅ | ✅ | ✅ | ✅ | ✅
-| Q5_0 | ✅ | ✅ | ✅ | ✅ | ✅
-| TQ2_0 / BitNet b1.58 (int8 activations) | ✅ | — | — | ✅ NEON | —
-| BitNet b1.58 (exact FP32 activations, vendored NeoGPU kernel) | ✅ | ✅ FFM | — | ✅ JNI + fused lm_head | ✅ K/N cinterop
-| BITNET_PLANES (multi-plane ternary packing) | ✅ | via kernel pack | — | via kernel pack | via kernel pack
-| Q2_K / Q3_K / Q8_K / IQ4 | ❌ (dequant to FP32 only) | ❌ | ❌ | ❌ | ❌
+| Weight format | Scalar (all targets) | Panama (JVM) | Native FFM (JVM) | Native JNI (Android) | Native cinterop (K/N) | Wasm SIMD128 (wasmJs)
+| FP32 | ✅ | ✅ | ✅ | ❌ | ❌ | ✅
+| BF16 | ✅ | ✅ | ✅ | ❌ | ❌ | ✅
+| FP16 | ✅ | ✅ | ✅ | ❌ | ❌ | ✅
+| Q8_0 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
+| Q4_0 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
+| Q4_K | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
+| Q6_K | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
+| Q5_K | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
+| Q5_1 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
+| Q5_0 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
+| TQ2_0 / BitNet b1.58 (int8 activations) | ✅ | — | — | ✅ NEON | — | —
+| BitNet b1.58 (exact FP32 activations, vendored NeoGPU kernel) | ✅ | ✅ FFM | — | ✅ JNI + fused lm_head | ✅ K/N cinterop | ❌ (reference)
+| BITNET_PLANES (multi-plane ternary packing) | ✅ | via kernel pack | — | via kernel pack | via kernel pack | ❌ (reference)
+| Q2_K / Q3_K / Q8_K / IQ4 | ❌ (dequant to FP32 only) | ❌ | ❌ | ❌ | ❌ | ❌
 |===
 
 Resolution is by priority: *Native (100 — whichever of FFM/JNI/cinterop applies to the target) →
-Panama (50, JVM only) → Scalar (0)*. The best *available* provider that carries the kernel wins;
+Panama / Wasm SIMD128 (50) → Scalar (0)*. The best *available* provider that carries the kernel wins;
 otherwise it cascades down. At most one native tier is compiled into a given target, so the native
-columns are mutually exclusive per platform rather than stacked.
+columns are mutually exclusive per platform rather than stacked. The Wasm SIMD128 column is the
+sidecar pack (`skainet-backend-wasm-simd`), present only after `WasmSimd.install()`; every matmul
+kernel in it has an explicit SIMD128 body (the K-quant int8 dots on `i32x4.dot_i16x8_s`, a 4×8
+register-blocked dense GEMM, eight-wide BF16/FP16 widening), and the dense kernels thread over the
+sidecar's pthread pool like the packed ones when the page is cross-origin isolated.
 
 Packed weights reach a kernel through the registry by declaring their
 xref:explanation/packed-weight-layout.adoc[block order]; a canonical weight is relayouted once,
@@ -96,7 +101,11 @@ Packed-quant matmul works on every target: those formats have a `commonMain` sca
 
 == Gaps
 
-* *Native FFM Q6_K* — the only packed format the FFM C kernel set does not cover.
+* *Wasm ternary / BitNet* — the ternary gemv and planes lm-head kernels are not in the sidecar
+build yet, so BitNet models in a browser run the decoding reference kernel.
+* *Wasm beyond matmul* — elementwise, activations, reductions, normalisation and attention are the
+common scalar Kotlin paths on wasmJs; Kotlin/Wasm has no SIMD types and no shared-memory threads, so
+these follow the matmuls into the sidecar (SKEEP-007 rollout step 3).
 * *Native JNI dense FP32/BF16* — the Android JNI provider has no GEMM shim, so dense ops fall
 through to scalar there whichever `.so` tier loaded (SKaiNET#920).
 * *Dense FP32/BF16 SIMD on Kotlin/Native linux* — still the scalar floor; Apple has Accelerate

docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc

@@ -3,18 +3,19 @@
 
 Generated from `kernel-support.json` (version `0.61.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh.
 
-Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* (100) → *panama-vector* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only).
+Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* / *native-jni* / *native-cinterop* (100) → *panama-vector* / *wasm-simd128* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only). The JS/WASM column assumes `WasmSimd.install()` has run; without the sidecar it is the scalar floor.
 
 [cols="1,1,1,1,1,1", options="header"]
 |===
 | Weight format | JVM | Android | Native·linux | Native·apple | JS/WASM 
 
 | `Float32` | native-ffm | scalar | scalar | scalar | wasm-simd128 
-| `BFloat16` | native-ffm | scalar | scalar | scalar | scalar 
+| `BFloat16` | native-ffm | scalar | scalar | scalar | wasm-simd128 
+| `Float16` | native-ffm | scalar | scalar | scalar | wasm-simd128 
 | `Q8_0` | native-ffm | native-jni | native-cinterop | native-cinterop | wasm-simd128 
 | `Q4_0` | native-ffm | native-jni | native-cinterop | native-cinterop | wasm-simd128 
 | `Q4_K` | native-ffm | native-jni | native-cinterop | native-cinterop | wasm-simd128 
-| `Q6_K` | panama-vector | native-jni | native-cinterop | native-cinterop | wasm-simd128 
+| `Q6_K` | native-ffm | native-jni | native-cinterop | native-cinterop | wasm-simd128 
 | `Q5_K` | native-ffm | native-jni | native-cinterop | native-cinterop | wasm-simd128 
 | `Q5_1` | native-ffm | native-jni | native-cinterop | native-cinterop | wasm-simd128 
 | `Q5_0` | native-ffm | native-jni | native-cinterop | native-cinterop | wasm-simd128 
@@ -37,4 +38,24 @@ Kernels that read the weight in canonical row-major GGUF file order straight fro
 | `Q5_0` | ffm-rowmajor | native-jni-direct | — | — | — 
 |===
 
+== Resident serving (sidecar memory)
+
+The browser has no files to map, so the wasm tier keeps weights *resident* in the SIMD128 sidecar's linear memory instead: each weight is uploaded once on its first matmul and every later call reads it in place (`WasmKernels.weightHandle`). This is the JS/WASM counterpart of mapped serving above — no per-call copy of the weight, no prepack — bounded by the wasm32 4 GB linear memory rather than the page cache. The other platforms serve from the heap, off-heap or the mapping and have no cell here.
+
+[cols="1,1,1,1,1,1", options="header"]
+|===
+| Weight format | JVM | Android | Native·linux | Native·apple | JS/WASM 
+
+| `Float32` | — | — | — | — | wasm-resident 
+| `BFloat16` | — | — | — | — | wasm-resident 
+| `Float16` | — | — | — | — | wasm-resident 
+| `Q8_0` | — | — | — | — | wasm-resident 
+| `Q4_0` | — | — | — | — | wasm-resident 
+| `Q4_K` | — | — | — | — | wasm-resident 
+| `Q6_K` | — | — | — | — | wasm-resident 
+| `Q5_K` | — | — | — | — | wasm-resident 
+| `Q5_1` | — | — | — | — | wasm-resident 
+| `Q5_0` | — | — | — | — | wasm-resident 
+|===
+
 See also the eager backends & kernels mindmap (xref:explanation/eager-execution.adoc[]) for the narrative overview and gaps.

skainet-backends/benchmarks/wasm-bench/src/wasmJsMain/kotlin/sk/ainet/bench/wasm/Main.kt

@@ -21,7 +21,9 @@ import kotlin.time.TimeSource
  *
  * `[SKAINET_PERF_WASM] <case> m=<m> n=<n> k=<k> form=<dense|q8_0> ms=<median> gflops=<rate>`
  *
- * Every later tier (SIMD128 sidecar, quantize-at-load) is measured against these lines.
+ * Every later tier (SIMD128 sidecar, quantize-at-load) is measured against these lines. Each pass
+ * also runs the `engine-*` scenarios of the JVM publish harness ([runEngineScenarios]) so the two
+ * tiers are compared on identical shapes.
  */
 private data class Case(val name: String, val m: Int, val n: Int, val k: Int)
 
@@ -107,7 +109,8 @@ suspend fun main() {
 
 private fun runPass(tier: String) {
     val ctx = DirectCpuExecutionContext()
-    println("[SKAINET_PERF_WASM] tier=$tier backend=${ctx.ops::class.simpleName} cases=${cases.size}")
+    println("[SKAINET_PERF_WASM] tier=$tier backend=${ctx.ops::class.simpleName} cases=${cases.size} features=${WasmKernels.features()}")
+    runEngineScenarios(tier)
     for (case in cases) {
         val x = ctx.fromFloatArray<FP32, Float>(Shape(case.m, case.k), FP32::class, pseudo(case.m * case.k, 3))
         val dense = ctx.fromFloatArray<FP32, Float>(Shape(case.n, case.k), FP32::class, pseudo(case.n * case.k, 7))