diff --git a/CHANGELOG.md b/CHANGELOG.md index 2b117987..0ea3a60b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,8 +2,22 @@ ## [Unreleased] +## [0.62.2] - 2026-10-24 + +Headline: **the dense FP32 `x · Wᵀ` product on the JVM native lane uses every core and, on x86-64, AVX2/FMA** +(#1441, #1442, see *Changed*). The kernel behind every resident dense weight ran on the compile-time C row pool +(four threads) with an SSE2 loop that re-read the weight once per activation row. It now splits the weight's rows +across the context `Schedule` (SKEEP-005) and, on hosts with AVX2 and FMA, runs an eight-lane register-tiled lane +selected at runtime; threaded, sliced and single calls stay bit-identical to each other. Measured on a six-core +laptop: a single 63 × 1024 × 1024 call goes from 29.6 to 131.7 GFLOP/s, and a ModernBERT-large encoder request in +the LayaMoE example from 13.0 s to 6.4 s with unchanged answers. Also in this release: added-token matching in the +text front-end is indexed by first character and length (#1443: Gemma's 6,400 added tokens, 2.0 ms → 36 µs per +encode), `CoroutineSchedule.forRange` no longer hands a body an empty, inverted range, and the Gradle wrapper +(9.8.2) and `kotlinx-serialization-json` (1.12.0) are bumped. + ### Changed +- **Dense FP32 `x · Wᵀ` on the JVM native lane runs under the context schedule** (`skainet-backend-cpu`, SKEEP-005): `DefaultCpuOpsJvm` used to hand the whole product to `skainet_fp32_matmul_wt` in one call and leave threading to the C row pool, which is sized at compile time (four threads) and re-reads the weight for every activation row, so an encoder or prefill pass on a twelve-core host ran on about three cores at the shared cache's bandwidth. A product with several activation rows now splits the weight's rows across the `Schedule` — up to `parallelism` native calls over disjoint column slices, each slice narrow enough for a core's own cache — so `DirectCpuExecutionContext()` (the hardware schedule) uses every core and `withSchedule(Schedule.Sequential)` runs the old single call. Results are bit-identical either way: every output stays the same contiguous dot product in the same order. A single-row product (a decode step) is unchanged, as is a weight in a confined arena, which stays on the calling thread. - **Dense FP32 `x · Wᵀ` uses AVX2/FMA on x86-64** (`skainet-backend-native-cpu`): `skainet_fp32_matmul_wt` had SIMD bodies for wasm and NEON only; the x86-64 library is built for baseline x86-64, so the kernel behind the JVM's resident dense weights (SKEEP-007) ran a scalar/SSE2 loop and re-read the weight once per activation row. Hosts with AVX2 and FMA now select, once per call through a cached `cpuid` probe, a lane compiled per function for that target: eight-lane FMA chains over a tile of four weight rows and two activation rows, so the weight streams from memory once per call. Every tile variant reduces an output in the same order, so threaded, sliced (`Schedule`) and single calls stay bit-identical to each other; against the previous x86 numbers an output moves within rounding, as the NEON and wasm lanes already do. Machines without AVX2, and every other architecture, run the previous code. Measured single call, m=63 n=1024 k=1024 on a 6-core laptop: see the pull request. ### Fixed @@ -11,7 +25,6 @@ - **Added-token matching no longer scans every added token at every position** (`skainet-frontend-text`, `AddedTokenMatcher` behind `SpecialTokenSplitter` and the inline matchers). Gemma's GGUF declares ~6,400 added tokens; a 70-character chat turn cost ~450k region compares per `encode`. Candidates are now indexed by first character and length (one map lookup per distinct length). Gemma-3 270M fine-tune vocab, JVM: 2.0 ms → 36 µs per chat-turn encode, ids unchanged; on a 32-bit ARM Android device the same encode took 36–166 ms. - **`CoroutineSchedule.forRange` could call its body with an empty, inverted range** (`skainet-backend-cpu`): the chunk length is rounded up, so for some `n` and task counts the last chunk started past `n` (63 elements on 12 tasks: chunks of 6, the twelfth at 66..63). The engine's own bodies loop over the range and did nothing for it; a body that passes the range length to a kernel saw a negative size. The task count now follows the chunk length, and every range is non-empty and covers `[0, n)` as the `Schedule` contract states. - ## [0.62.1] - 2026-10-21 Headline: **the native Q8_0 / Q4_0 / Q5_0 / Q5_1 kernel bindings no longer zero the earlier rows of a diff --git a/CITATION.cff b/CITATION.cff index a4feb7f4..25858088 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -12,7 +12,7 @@ authors: orcid: https://orcid.org/0009-0006-7981-4075 - name: SKaiNET-developers contributors website: https://github.com/SKaiNET-developers -version: "0.62.1" +version: "0.62.2" date-released: "2026-10-21" license: MIT repository-code: https://github.com/SKaiNET-developers/SKaiNET diff --git a/README.md b/README.md index aee13349..c5236bf3 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,7 @@ Add the core dependencies (Gradle Kotlin DSL): ```kotlin dependencies { // Recommended: import the umbrella BOM and drop versions on the engine modules. - implementation(platform("sk.ainet:skainet-bom:0.62.1")) + implementation(platform("sk.ainet:skainet-bom:0.62.2")) implementation("sk.ainet.core:skainet-lang-core") implementation("sk.ainet.core:skainet-backend-cpu") @@ -308,7 +308,18 @@ val withoutLabel = dataPipeline() --- -## What's New in 0.62.1 +## What's New in 0.62.2 + +A performance release for the JVM native lane. The dense FP32 `x · Wᵀ` kernel — what every resident dense +weight resolves to once `skainet-backend-native-cpu` is on the classpath — now splits its work across the +context `Schedule` (SKEEP-005) instead of a fixed four-thread C pool, and on x86-64 hosts with AVX2 and FMA +runs a runtime-selected eight-lane register-tiled lane; threaded, sliced and single calls stay bit-identical. +A single 63 × 1024 × 1024 call goes from 29.6 to 131.7 GFLOP/s on a six-core laptop, and a ModernBERT-large +encoder request in the LayaMoE example from 13.0 s to 6.4 s with unchanged answers. The text front-end +indexes added tokens by first character and length, so a Gemma chat turn encodes in 36 µs instead of 2.0 ms. +`CoroutineSchedule.forRange` never hands a body an empty range any more. + +### Carried from 0.62.1 A fix release on top of 0.62.0: the native FFM bindings of the Q8_0, Q4_0, Q5_0 and Q5_1 kernels copied a zero-filled staging buffer back from index 0, so a product served row by row — any @@ -370,6 +381,12 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix: Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET). +### Contributors (0.62.2) + +- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — the schedule-driven and AVX2/FMA + dense FP32 lanes with their tests and measurements, the `CoroutineSchedule` fix, the added-token index, and + the release + ### Contributors (0.62.1) - **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — the native packed-kernel @@ -495,7 +512,7 @@ from it). Ready-to-paste BibTeX: author = {Harakal, Michal and {SKaiNET-developers contributors}}, title = {{SKaiNET}: A {Kotlin} Multiplatform {AI} Framework}, year = {2026}, - note = {Version 0.62.1. MIT License}, + note = {Version 0.62.2. MIT License}, howpublished = {\url{https://github.com/SKaiNET-developers/SKaiNET}} } ``` diff --git a/docs/antora.yml b/docs/antora.yml index fb8e53fc..e1e4bb7c 100644 --- a/docs/antora.yml +++ b/docs/antora.yml @@ -15,7 +15,7 @@ asciidoc: framework_name: SKaiNET # Current SKaiNET release — bump once per release; referenced as # {skainet_version} in dependency snippets (blocks need subs="attributes+"). - skainet_version: 0.62.1 + skainet_version: 0.62.2 ksp_version: 2.2.21-2.0.5 dokka_version: 2.1.0 asciidoctorj_version: 3.0.0 diff --git a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc index 69caf002..c65f1541 100644 --- a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc @@ -1,7 +1,7 @@ = Kernel × platform support matrix :description: Which compute-kernel provider serves each weight format on each KMP target. -Generated from `kernel-support.json` (version `0.62.1`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. +Generated from `kernel-support.json` (version `0.62.2`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* / *native-jni* / *native-cinterop* (100) → *panama-vector* / *wasm-simd128* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only). The JS/WASM column assumes `WasmSimd.install()` has run; without the sidecar it is the scalar floor. diff --git a/docs/modules/ROOT/pages/reference/operators/generated/index.adoc b/docs/modules/ROOT/pages/reference/operators/generated/index.adoc index 968a07fc..25298070 100644 --- a/docs/modules/ROOT/pages/reference/operators/generated/index.adoc +++ b/docs/modules/ROOT/pages/reference/operators/generated/index.adoc @@ -1,6 +1,6 @@ = AI-NET Operators Reference -Generated from version `0.62.1` on 2026-10-21 +Generated from version `0.62.2` on 2026-10-24 == Operators by Modality diff --git a/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc b/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc index 4070ea4f..a3e4642f 100644 --- a/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc @@ -1,7 +1,7 @@ = Operator Coverage Matrix :description: Cross-backend status for every operator function in SKaiNET. -Generated from `operators.json` version `0.62.1` on 2026-10-21. +Generated from `operators.json` version `0.62.2` on 2026-10-24. Rows are `Operator.function` pairs. A gap is marked in source with `@NotImplemented(reason = …, issue = …)` (or `@InProgress`) and shows up here as ❌ (or 🚧) linked to its tracking issue; the PR that closes the gap removes the annotation. See xref:contributing/issue-taxonomy.adoc[Issue taxonomy] to pick one up. diff --git a/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc b/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc index 59cc6fc7..c89c854d 100644 --- a/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc +++ b/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc @@ -19,12 +19,12 @@ An Android project with Kotlin. Add the SKaiNET modules: [source,kotlin] ---- dependencies { - implementation("sk.ainet.core:skainet-lang-core:0.62.1") // tensors, DSL, training - implementation("sk.ainet.core:skainet-backend-cpu:0.62.1") // CPU ops - implementation("sk.ainet.core:skainet-compile-dag:0.62.1") // autograd (training context) - implementation("sk.ainet.core:skainet-data-api:0.62.1") // Dataset / DataBatch - implementation("sk.ainet.core:skainet-data-simple:0.62.1") // embedded Iris - runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.62.1") // NEON kernels (see below) + implementation("sk.ainet.core:skainet-lang-core:0.62.2") // tensors, DSL, training + implementation("sk.ainet.core:skainet-backend-cpu:0.62.2") // CPU ops + implementation("sk.ainet.core:skainet-compile-dag:0.62.2") // autograd (training context) + implementation("sk.ainet.core:skainet-data-api:0.62.2") // Dataset / DataBatch + implementation("sk.ainet.core:skainet-data-simple:0.62.2") // embedded Iris + runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.62.2") // NEON kernels (see below) } ---- diff --git a/gradle.properties b/gradle.properties index eb32112b..acdf54fe 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.core -VERSION_NAME=0.62.1 +VERSION_NAME=0.62.2 POM_DESCRIPTION=SKaiNET POM_URL=https://github.com/SKaiNET-developers/skainet/