diff --git a/CHANGELOG.md b/CHANGELOG.md index 98845fb2..afa6ced3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,8 +2,36 @@ ## [Unreleased] +## [0.62.0] - 2026-10-21 + +Headline: **JVM ↔ Kotlin/Wasm parity for compute and IO.** In 0.61 the SIMD128 sidecar served the +packed matmuls only; everything else in a browser ran as single-threaded scalar Kotlin, and the IO +layer had gaps the JVM does not have. This release gives the wasm tier every kernel the JVM has — +explicit SIMD128 bodies for every matmul format, BF16/FP16, the ternary kernels, elementwise, +activation and reduction kernels, fused softmax, normalisation and banded attention, convolutions as +im2col + GEMM — keeps activations resident in the sidecar between ops, and closes the IO gaps: +`Long` Blob sources with worker-side blocking reads, sharded SafeTensors from Blobs, +`PlannerProfile.BROWSER`, a browser `MemoryProbe`, and `skainet-data-source` on Kotlin/JS and +Kotlin/Wasm. Parity runs in both directions: the JVM gains the direct `x · Wᵀ` kernel and the fused +softmax, normalisation and attention kernels through FFM, every new C kernel is one source for every +native build, and the `engine-*` benchmarks run with identical shapes on both tiers. Every remaining +difference between the tiers is a documented N/A with its reason (no file system, no `mmap`, no +Kotlin threads, no ServiceLoader, 2 GB arrays) in `how-to/io-readers.adoc` and the kernel support +matrix. On the JVM side, four allocation and threading fixes from the BitNet decode profile (#1357) +land as well. + +Version note: every API addition is additive (`KernelProvider` accessors with `null` defaults, +`FusedNormOps`, `TensorOps.convTranspose2d` with a throwing default, new constructors beside the +old ones). `skainet-data-source` now publishes `-js` and `-wasm-js` variants; it has no native +target yet. SKaiNET-transformers and the web samples follow on this release (`WasmSimd.install()` +is still the one call that turns the browser tier on). + ### Added +- **Operator coverage matrix from real data** (#1370): `@NotImplemented` takes `reason` and `issue`, wins over `@InProgress` and over a detected override, and is read on methods of `@Backend` classes; the docs processor also runs in `skainet-backend-cpu`, so the cpu column of `ops-status-matrix.adoc` reflects the backend (66/67) instead of `—`. New *StableHLO* and *Backward* columns from jvmTest dumps of the converter registry and the tape's dispatch table, an *NN layers* table over every concrete `nn` `Module`, ❌ cells linking their tracking issue, notes rendered (they were always empty). Tracking parents: #1371 (`TransposedConv2d` / `convTranspose2d`), #1379 (`GroupNormalization` forward), #1384 (SDPA backward), #1389 (StableHLO converter coverage). +- **`TensorOps.convTranspose2d`** (throwing default) so `TransposedConv2d.forward` has an op to call; `TransposedConv2d` and `GroupNormalization` carry `@NotImplemented` with their issues instead of a bare `NotImplementedError` / `TODO()`. +- **`./gradlew checkUnmarkedStubs`** and the `stub-markers` CI leg: production code may not contain `NotImplementedError` / `TODO(` outside a marked declaration or `config/stub-allowlist.txt` (`path|count|issue`); the docs workflow regenerates the operator matrix and fails when the committed pages are stale. + - **BF16 and FP16 matmul on the wasm tier**: `WasmKernelProvider` now binds the sidecar's `skainet_bf16_matmul` / `skainet_fp16_matmul` exports (`WasmBf16MatmulKernel`, `WasmFp16MatmulKernel`), so 16-bit dense weights no longer fall to the scalar kernels in a browser; a contiguous weight stays resident in the sidecar after its first call. - **`WasmSimd.features()` / `WasmKernels.features()`**: mode, SIMD128, threads, cross-origin isolation, logical cores and resident weight bytes — the wasm counterpart of the JVM provider banner. `WasmSimd.install()` prints one `[SKaiNET] Using SIMD128 sidecar kernels (…)` line like the JVM factory does. - **`KernelStrictness` is common code** (was JVM-only): `KernelStrictness.overrideEnabled` is the programmatic switch on every target; the JVM and Android still read `-Dskainet.strict.kernels`. `KernelDispatch` fails fast before the decoding reference fallback and `DefaultCpuOps.matmul` before the generic per-element loop, so strict mode means the same thing on every tier. @@ -30,6 +58,16 @@ - **`skainet-data-source` on Kotlin/JS and Kotlin/Wasm** (was JVM-only): `WebDataSourceResolver` resolves HTTP(S) and Hugging Face URIs over the browser's `fetch` (`KtorRemoteDataSourceFetcher` on the Ktor JS engine, the same name and contract as the JVM's CIO fetcher) with the common cache policies, checksum verification and token handling; remote artifacts are cached in memory for the life of the resolver (`InMemoryDataSourceArtifactStore`, new in common code, also the store for tests), a `file:` URI resolves to bytes the page registered under that path (a file picker, a drop) and fails otherwise, and the token comes from the request or the resolver — a tab has no file system and no environment. `Sha256DataSourceChecksum` is a SHA-256 in common Kotlin, byte-identical to `MessageDigest` (tested), for targets without a synchronous platform digest. - **`wasm-bench` runs the `engine-*` scenarios** of the JVM publish harness (`engine-fp32-gemm`, `engine-kernel-matmul`, `engine-bf16-matmul`, `engine-q8-matmul`, `engine-q4-gemm`, `engine-ternary-f32-gemv`, `engine-elementwise-add`, `engine-reductions-sum/mean`) with identical shapes and metrics on every tier it measures, so the JVM and wasm numbers sit in one table. +### Changed + +- **Ternary FP32 GEMV on the shared row pool** (#1359): the `BITNET_B1_58` projection drove the vendored kernel's own create-and-join threads on every call (about 210 per BitNet 2B4T decode step); the adapter now slices the rows over `skainet_run_rows`, the vendored file stays byte-identical, results are bit-identical. BitNet 2B4T decode on four Cortex-A78C cores: 1.90 → 2.26 tok/s. +- **Elementwise Vector API paths are intrinsified** (#1360): the shared `binaryFloat` / `unaryFloat` helpers are `inline` (one monomorphic loop per op instead of a megamorphic lambda) and the species is a static final; the Vector API's Java fallback, 61 % of all allocation in the decode profile, is gone from the allocation sites. `relu` on 1 × 2560: 31 KB → 10.5 KB per call. Output unchanged. +- **`split` on dense FP32** (#1361) copies contiguous runs instead of building every element through `get(*indices)` (two `IntArray` copies and a boxed value each); `RoPE.applyRoPESplitHalfFull` no longer shows in the decode allocation profile. +- **`MmapFloatTensorData.copyToFloatArray()`** (#1362) is one bulk `FloatBuffer` read instead of a per-element walk; the mapped RMS-norm gains that `eagerFusedRmsNorm` copies every forward cost about 10 GB per 45 s of decode before. +- 35 stale `@InProgress("cpu", …)` markers on implemented `DefaultCpuOps` methods and the placeholder `@InProgress("Metal", …)` in `VoidTensorOps` are removed (#1370). +- Internal: the classification helpers of `Accuracy` (argmax, shape validation, target dispatch, sample traversal) live in `ClassificationSupport`, shared by the upcoming precision, recall and F1 metrics (#1429, #1222). +- Toolchain: Kotlin 2.4.21, kotest 6.2.6, Gradle 9.8.1. + ## [0.61.0] - 2026-10-16 Headline: **the browser lane (SKEEP-007): packed SIMD128 kernels, quantize-at-load, and no more diff --git a/CITATION.cff b/CITATION.cff index 490800c9..38d5ceb5 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -12,8 +12,8 @@ authors: orcid: https://orcid.org/0009-0006-7981-4075 - name: SKaiNET-developers contributors website: https://github.com/SKaiNET-developers -version: "0.61.0" -date-released: "2026-10-16" +version: "0.62.0" +date-released: "2026-10-21" license: MIT repository-code: https://github.com/SKaiNET-developers/SKaiNET url: https://skainet-developers.github.io/SKaiNET/ diff --git a/README.md b/README.md index 90dfcbc1..1c338cad 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,7 @@ Add the core dependencies (Gradle Kotlin DSL): ```kotlin dependencies { // Recommended: import the umbrella BOM and drop versions on the engine modules. - implementation(platform("sk.ainet:skainet-bom:0.61.0")) + implementation(platform("sk.ainet:skainet-bom:0.62.0")) implementation("sk.ainet.core:skainet-lang-core") implementation("sk.ainet.core:skainet-backend-cpu") @@ -308,26 +308,32 @@ val withoutLabel = dataPipeline() --- -## What's New in 0.61.0 - -The browser lane (SKEEP-007): packed SIMD128 kernels, quantize-at-load, no reference-kernel -fallbacks on wasm. - -- **`skainet-backend-wasm-simd`** — the engine's own C kernels compiled by Emscripten into a - SIMD128 sidecar for wasmJs, with pthreads on cross-origin-isolated pages and under Node. One call, - `WasmSimd.install()`, registers the kernel pack and the provider and makes every context built - afterwards use them. Measured on Node: Laya's Q8_0 matmuls from 0.72 GFLOPS (scalar) to 8.5 - single-threaded and 27 with four threads — a 22-layer decision from about 29 s to about 1.2 s. -- **Quantize at load** — `WeightForm(RequantizeTo(Q8_0))` on the SafeTensors loaders (and every - encoding with a quantizer on the GGUF loader) turns dense F32/F16/BF16 weight matrices into packed - Q8_0 or Q4_0 at load; `PlannerProfile.quantizeDenseTo` lets a browser or phone profile ask for it; - `skainet-quantize` converts offline. Hugging Face checkpoints can be served packed on every target. -- **The scalar lane is no longer the reference lane** — a canonical packed weight on a target without - a row-major kernel pack is relaid once into kernel feed order and served by the scalar packed - kernels; dense `[out, in]` weights are multiplied in place without a transpose copy; 16-bit - token tables are read row by row. `ByteArrayRandomAccessSource` is the engine's in-memory source. -- The SKEEP-006 compatibility aliases, errors since 0.60, are removed as scheduled; `TokenizerFactory` - stays in `sk.ainet.io.tokenizer` for Java callers. +## What's New in 0.62.0 + +JVM ↔ Kotlin/Wasm parity: the browser tier runs the same kernels as the JVM, keeps its activations +in the sidecar, and reads models the way the JVM does — with every remaining difference documented. + +- **Every kernel on the wasm tier** — explicit SIMD128 bodies for every matmul format (K-quants, + Q5_0/Q5_1, dense FP32/BF16/FP16), the ternary BitNet kernels, elementwise, activation and + reduction kernels, fused softmax, normalisation and banded attention, and convolutions as + im2col + GEMM, all threaded over the sidecar's row pool. Chains of ops stay resident in the + sidecar's linear memory (`Storage.OffHeap` in a browser) and come back to the Kotlin heap only when + something outside needs them. +- **Both directions** — the JVM binds the same C kernels through FFM: the direct `x · Wᵀ` kernel + (no transposed weight copy), fused softmax, layer and RMS normalisation, and SDPA with + key lengths, windows and grouped-query heads. One C source per kernel serves every native build. +- **Browser IO** — `Blob` sources address `Long` positions (shards over 2 GB), copy in bulk, and + serve blocking reads inside a Web Worker; sharded SafeTensors load from Blobs; + `PlannerProfile.BROWSER`; `MemoryProbe` reports the JavaScript heap, linear memory and the + sidecar's resident bytes; `skainet-data-source` resolves HTTP and Hugging Face URIs over `fetch` + on Kotlin/JS and Kotlin/Wasm. +- **One benchmark table** — the `engine-*` scenarios of the JVM publish harness run with identical + shapes in `wasm-bench`, and the kernel support matrix names what each tier serves and why the + rest is N/A. The operator coverage matrix is generated from backend data, with StableHLO, backward + and NN-layer columns and a CI leg that rejects unmarked stubs. +- **JVM decode allocation** — four fixes from the BitNet 2B4T profile: the ternary GEMV on the + shared row pool (+19 % decode), intrinsified Vector API elementwise paths, a dense `split` fast + path and bulk copies from mapped tensors. See [CHANGELOG.md](CHANGELOG.md) for full release notes, including every prior release. @@ -353,6 +359,16 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix: Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET). +### Contributors (0.62.0) + +- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — the JVM ↔ wasm parity + series: SIMD128 bodies for every sidecar kernel, the ternary and direct `x · Wᵀ` kernels, resident + activations with the elementwise, fused and convolution kernels on both tiers, the browser IO + parity, `skainet-data-source` on JS and wasm, the operator coverage matrix, the BitNet decode + allocation fixes, and the release +- **Edwin** ([@EdinEd](https://github.com/EdinEd)) — the `ClassificationSupport` helpers extracted + from `Accuracy`, preparing the precision, recall and F1 metrics (#1222) + ### Contributors (0.61.0) - **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — SKEEP-007: the Tier 0 @@ -462,7 +478,7 @@ from it). Ready-to-paste BibTeX: author = {Harakal, Michal and {SKaiNET-developers contributors}}, title = {{SKaiNET}: A {Kotlin} Multiplatform {AI} Framework}, year = {2026}, - note = {Version 0.61.0. MIT License}, + note = {Version 0.62.0. MIT License}, howpublished = {\url{https://github.com/SKaiNET-developers/SKaiNET}} } ``` diff --git a/docs/antora.yml b/docs/antora.yml index fa17c175..3cad2f3d 100644 --- a/docs/antora.yml +++ b/docs/antora.yml @@ -15,7 +15,7 @@ asciidoc: framework_name: SKaiNET # Current SKaiNET release — bump once per release; referenced as # {skainet_version} in dependency snippets (blocks need subs="attributes+"). - skainet_version: 0.61.0 + skainet_version: 0.62.0 ksp_version: 2.2.21-2.0.5 dokka_version: 2.1.0 asciidoctorj_version: 3.0.0 diff --git a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc index 82c38089..1ffa6a83 100644 --- a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc @@ -1,7 +1,7 @@ = Kernel × platform support matrix :description: Which compute-kernel provider serves each weight format on each KMP target. -Generated from `kernel-support.json` (version `0.61.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. +Generated from `kernel-support.json` (version `0.62.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* / *native-jni* / *native-cinterop* (100) → *panama-vector* / *wasm-simd128* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only). The JS/WASM column assumes `WasmSimd.install()` has run; without the sidecar it is the scalar floor. diff --git a/docs/modules/ROOT/pages/reference/operators/generated/index.adoc b/docs/modules/ROOT/pages/reference/operators/generated/index.adoc index b3f9e096..df8efd5d 100644 --- a/docs/modules/ROOT/pages/reference/operators/generated/index.adoc +++ b/docs/modules/ROOT/pages/reference/operators/generated/index.adoc @@ -1,6 +1,6 @@ = AI-NET Operators Reference -Generated from version `0.61.0` on 2026-10-19 +Generated from version `0.62.0` on 2026-10-21 == Operators by Modality diff --git a/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc b/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc index 7619bd0c..109fc20a 100644 --- a/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc @@ -1,7 +1,7 @@ = Operator Coverage Matrix :description: Cross-backend status for every operator function in SKaiNET. -Generated from `operators.json` version `0.61.0` on 2026-10-19. +Generated from `operators.json` version `0.62.0` on 2026-10-21. Rows are `Operator.function` pairs. A gap is marked in source with `@NotImplemented(reason = …, issue = …)` (or `@InProgress`) and shows up here as ❌ (or 🚧) linked to its tracking issue; the PR that closes the gap removes the annotation. See xref:contributing/issue-taxonomy.adoc[Issue taxonomy] to pick one up. diff --git a/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc b/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc index c5da3dca..7b47e107 100644 --- a/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc +++ b/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc @@ -19,12 +19,12 @@ An Android project with Kotlin. Add the SKaiNET modules: [source,kotlin] ---- dependencies { - implementation("sk.ainet.core:skainet-lang-core:0.61.0") // tensors, DSL, training - implementation("sk.ainet.core:skainet-backend-cpu:0.61.0") // CPU ops - implementation("sk.ainet.core:skainet-compile-dag:0.61.0") // autograd (training context) - implementation("sk.ainet.core:skainet-data-api:0.61.0") // Dataset / DataBatch - implementation("sk.ainet.core:skainet-data-simple:0.61.0") // embedded Iris - runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.61.0") // NEON kernels (see below) + implementation("sk.ainet.core:skainet-lang-core:0.62.0") // tensors, DSL, training + implementation("sk.ainet.core:skainet-backend-cpu:0.62.0") // CPU ops + implementation("sk.ainet.core:skainet-compile-dag:0.62.0") // autograd (training context) + implementation("sk.ainet.core:skainet-data-api:0.62.0") // Dataset / DataBatch + implementation("sk.ainet.core:skainet-data-simple:0.62.0") // embedded Iris + runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.62.0") // NEON kernels (see below) } ---- diff --git a/docs/modules/skeep/pages/007-browser-lane.adoc b/docs/modules/skeep/pages/007-browser-lane.adoc index eda85f49..20c90d91 100644 --- a/docs/modules/skeep/pages/007-browser-lane.adoc +++ b/docs/modules/skeep/pages/007-browser-lane.adoc @@ -1,7 +1,7 @@ = SKEEP-007: The browser lane — packed SIMD kernels and quantize-at-load :description: SKaiNET proposal for making inference in a browser usable and quick: stop the scalar wasm lane from falling to the decoding reference kernel, serve dense weights without transpose copies, keep token tables narrow, add a SIMD128 kernel pack with pthreads as a sidecar wasm module, and let the SafeTensors loader quantize at load so Hugging Face checkpoints are served packed on every target. -Status: Implemented (Tiers 0, 1 and 2 in engine 0.61.0; resident activations with sidecar elementwise, activation and reduction kernels, then fused softmax / normalisation / attention kernels on the wasm and JVM tiers, then the browser IO gaps — `Long` Blob sources with bulk copies and worker-side blocking reads, sharded SafeTensors from Blobs, `PlannerProfile.BROWSER`, the browser `MemoryProbe` — and convolutions as im2col + GEMM in the sidecar, in the parity series that followed); the SKaiNET-transformers consumer and the WebGPU proposal remain open + +Status: Implemented (Tiers 0, 1 and 2 in engine 0.61.0; resident activations with sidecar elementwise, activation and reduction kernels, then fused softmax / normalisation / attention kernels on the wasm and JVM tiers, then the browser IO gaps — `Long` Blob sources with bulk copies and worker-side blocking reads, sharded SafeTensors from Blobs, `PlannerProfile.BROWSER`, the browser `MemoryProbe` — and convolutions as im2col + GEMM in the sidecar, in the parity series that followed, engine 0.62.0); the SKaiNET-transformers consumer and the WebGPU proposal remain open + Implementation: Tier 0 https://github.com/SKaiNET-developers/SKaiNET/pull/1352[#1352], Tier 2 quantize-at-load https://github.com/SKaiNET-developers/SKaiNET/pull/1353[#1353], Tier 1 `skainet-backend-wasm-simd` https://github.com/SKaiNET-developers/SKaiNET/pull/1354[#1354] — engine 0.61.0; the consumer work in SKaiNET-transformers follows, see <<_rollout_plan>>. + Audience: SKaiNET maintainers and contributors; SKaiNET-transformers maintainers (Laya and BERT in the browser); SKaiNET-examples maintainers (every web sample runs through this lane) + Created: 2026-10-16 + @@ -192,7 +192,8 @@ marked breaking only for implementors of `ParametersLoader`. `PlannerProfile.quantizeDenseTo`, the `skainet-quantize` tool in `skainet-tensor-tools`. . PR 2 (Tier 1): the sidecar build, `skainet-backend-wasm-simd`, `WasmCpuOps`, parity and bench. . PR 3 (Tier 1 follow-up): resident activations, banded attention, pthread tuning — scoped by PR 2's - measurements. + measurements. Shipped in engine 0.62.0 as the JVM ↔ wasm parity series + (https://github.com/SKaiNET-developers/SKaiNET/pull/1363[#1363], https://github.com/SKaiNET-developers/SKaiNET/pull/1364[#1364], https://github.com/SKaiNET-developers/SKaiNET/pull/1368[#1368], https://github.com/SKaiNET-developers/SKaiNET/pull/1369[#1369], https://github.com/SKaiNET-developers/SKaiNET/pull/1426[#1426], https://github.com/SKaiNET-developers/SKaiNET/pull/1427[#1427], https://github.com/SKaiNET-developers/SKaiNET/pull/1428[#1428], https://github.com/SKaiNET-developers/SKaiNET/pull/1430[#1430]). . Engine release; then SKaiNET-transformers: a common-code `LayaModelLoader` over a `RandomAccessSource`, a packed-aware `LayaWeightPreparation.splitRows`, a decision-quality gate for the quantized lane; then the SemanticEmojiFilter web verifier. diff --git a/gradle.properties b/gradle.properties index a75ada9c..1abd57e4 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.core -VERSION_NAME=0.61.0 +VERSION_NAME=0.62.0 POM_DESCRIPTION=SKaiNET POM_URL=https://github.com/SKaiNET-developers/skainet/