Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,21 @@

## [Unreleased]

Headline: **the browser lane stops falling to the reference kernel (SKEEP-007, Tier 0).**

### Added

- **SKEEP-007 "The browser lane"** (`docs/modules/skeep/pages/007-browser-lane.adoc`): the design record for making inference in a browser usable and quick — common-code fixes now, a SIMD128 sidecar kernel pack with pthreads and quantize-at-load for SafeTensors as the next pull requests, WebGPU as a later proposal.
- **`ByteArrayRandomAccessSource`** (`skainet-io-core`, `sk.ainet.io`): the engine's in-memory `RandomAccessSource`, single array or equal-sized chunks for files above the 2 GB array limit. Every web sample had written its own.
- **`NarrowFloatDenseTensorData` is a `RowDequantSource`**: FP16 and BF16 tables serve rows on demand, so a 16-bit token-embedding table loaded `KEEP_NATIVE` is read row by row by `ops.gather` instead of being widened to FP32 at load.
- **`skainet-backends/benchmarks/wasm-bench`**: a Kotlin/Wasm executable that times the matmul shapes of the shipped models and prints `[SKAINET_PERF_WASM]` lines — the baseline for the following tiers.

### Changed

- **`matmulWeightTransposed` on a canonical packed weight** (every GGUF weight) now prefers, in order: an exact kernel for the row-major view, the one-time cached relayout into kernel feed order that the scalar packed kernels on every target read, and only then the decoding reference kernel. On JS, wasm and Kotlin/Native without the cinterop pack this replaces a ~1000× slower per-element decode on every call; the JVM overrides the path and is unaffected.
- **`matmulWeightTransposed` on a dense heap FP32 `[out, in]` weight** computes the product straight from the weight's rows (4 × 4 tiles, one sequential accumulator per output, bit-identical to the transpose-then-`matmul` result) on tiers without a vectorised `[k, n]` kernel, instead of making and caching a transposed copy of every weight. `DefaultCpuOpsBase.preferDirectWeightTransposed()` is the hook; the JVM and Apple ops keep their cached transpose.
- **`JsBlobRandomAccessSource`** no longer logs every read to the console. (The byte-by-byte `ArrayBuffer` conversion stays: Kotlin/Wasm 2.4 has no typed-array adapter; SKEEP-007 Tier 1 replaces it with the sidecar's linear-memory bridge.)

## [0.60.0] - 2026-10-15

Headline: **key-padding lengths and window metadata for attention (#1343).**
Expand Down
4 changes: 2 additions & 2 deletions docs/modules/ROOT/pages/explanation/eager-execution.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ mindmap
Android — scalar, Native-JNI
Native linux — scalar, Native-cinterop
Native apple — scalar, Accelerate, Native-cinterop
JS and WASM — scalar only
JS and WASM — scalar; packed weights relaid once into kernel feed order (SKEEP-007)
----

== Kernel × provider (matmul, FP32 activations)
Expand Down Expand Up @@ -88,7 +88,7 @@ not per call.
| *Android* | Scalar + Native-JNI | Panama (`jdk.incubator.vector`) and FFM (`java.lang.foreign`) are JDK-only — ART has neither, so Android's native tier is JNI, not a degraded JVM. Includes the NEON `bitnet_gemv` pack.
| *Kotlin/Native — linux x64/arm64* | Scalar + Native-cinterop | Static archive embedded in the klib; manual `installNativeKernels()`, no ServiceLoader on K/N.
| *Kotlin/Native — macOS/iOS* | Scalar + Apple Accelerate + Native-cinterop | Accelerate covers dense FP32 and reductions; cinterop covers packed quant with runtime `FEAT_DotProd` dispatch, so one archive serves A12 through M-series.
| *JS / WASM (Js, Wasi)* | Scalar | No SIMD.
| *JS / WASM (Js, Wasi)* | Scalar | No SIMD in Kotlin code. Packed weights are relaid once per weight into kernel feed order and served by the scalar packed kernels; dense `[out, in]` weights are multiplied in place without a transpose copy (SKEEP-007 Tier 0). A SIMD128 sidecar pack for wasmJs is SKEEP-007 Tier 1.
|===

Packed-quant matmul works on every target: those formats have a `commonMain` scalar kernel and
Expand Down
5 changes: 5 additions & 0 deletions docs/modules/ROOT/pages/explanation/kernel-selection.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,11 @@ platform support is absent (a missing native library, no vector unit).
| manual — call the pack's `install()` yourself
|===

Without a row-major pack, a canonical packed weight is relaid **once per weight** into kernel feed
order and served by the scalar packed kernels (`DefaultCpuOpsBase.matmulWeightTransposed`, SKEEP-007
Tier 0); the decoding reference kernel is the last resort, reached only for an encoding with no
scalar kernel at all.

A backend module declares its service the usual way, e.g. `skainet-backend-native-cpu` ships
`META-INF/services/sk.ainet.backend.api.kernel.ViewKernelPack` naming
`FfmRowMajorKernelPackFactory`. `ServiceLoader` needs a public no-arg constructor, which a Kotlin
Expand Down
1 change: 1 addition & 0 deletions docs/modules/skeep/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@
** xref:skeep:004-virtual-tensor-layout.adoc[SKEEP-004: Virtual tensor layout]
** xref:skeep:005-schedules-structured-concurrency.adoc[SKEEP-005: Schedules — structured concurrency]
** xref:skeep:006-input-frontends-and-model-identity.adoc[SKEEP-006: Input front ends and model identity]
** xref:skeep:007-browser-lane.adoc[SKEEP-007: The browser lane]
Loading
Loading