Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,29 @@

## [Unreleased]

## [0.62.2] - 2026-10-24

Headline: **the dense FP32 `x · Wᵀ` product on the JVM native lane uses every core and, on x86-64, AVX2/FMA**
(#1441, #1442, see *Changed*). The kernel behind every resident dense weight ran on the compile-time C row pool
(four threads) with an SSE2 loop that re-read the weight once per activation row. It now splits the weight's rows
across the context `Schedule` (SKEEP-005) and, on hosts with AVX2 and FMA, runs an eight-lane register-tiled lane
selected at runtime; threaded, sliced and single calls stay bit-identical to each other. Measured on a six-core
laptop: a single 63 × 1024 × 1024 call goes from 29.6 to 131.7 GFLOP/s, and a ModernBERT-large encoder request in
the LayaMoE example from 13.0 s to 6.4 s with unchanged answers. Also in this release: added-token matching in the
text front-end is indexed by first character and length (#1443: Gemma's 6,400 added tokens, 2.0 ms → 36 µs per
encode), `CoroutineSchedule.forRange` no longer hands a body an empty, inverted range, and the Gradle wrapper
(9.8.2) and `kotlinx-serialization-json` (1.12.0) are bumped.

### Changed

- **Dense FP32 `x · Wᵀ` on the JVM native lane runs under the context schedule** (`skainet-backend-cpu`, SKEEP-005): `DefaultCpuOpsJvm` used to hand the whole product to `skainet_fp32_matmul_wt` in one call and leave threading to the C row pool, which is sized at compile time (four threads) and re-reads the weight for every activation row, so an encoder or prefill pass on a twelve-core host ran on about three cores at the shared cache's bandwidth. A product with several activation rows now splits the weight's rows across the `Schedule` — up to `parallelism` native calls over disjoint column slices, each slice narrow enough for a core's own cache — so `DirectCpuExecutionContext()` (the hardware schedule) uses every core and `withSchedule(Schedule.Sequential)` runs the old single call. Results are bit-identical either way: every output stays the same contiguous dot product in the same order. A single-row product (a decode step) is unchanged, as is a weight in a confined arena, which stays on the calling thread.
- **Dense FP32 `x · Wᵀ` uses AVX2/FMA on x86-64** (`skainet-backend-native-cpu`): `skainet_fp32_matmul_wt` had SIMD bodies for wasm and NEON only; the x86-64 library is built for baseline x86-64, so the kernel behind the JVM's resident dense weights (SKEEP-007) ran a scalar/SSE2 loop and re-read the weight once per activation row. Hosts with AVX2 and FMA now select, once per call through a cached `cpuid` probe, a lane compiled per function for that target: eight-lane FMA chains over a tile of four weight rows and two activation rows, so the weight streams from memory once per call. Every tile variant reduces an output in the same order, so threaded, sliced (`Schedule`) and single calls stay bit-identical to each other; against the previous x86 numbers an output moves within rounding, as the NEON and wasm lanes already do. Machines without AVX2, and every other architecture, run the previous code. Measured single call, m=63 n=1024 k=1024 on a 6-core laptop: see the pull request.

### Fixed

- **Added-token matching no longer scans every added token at every position** (`skainet-frontend-text`, `AddedTokenMatcher` behind `SpecialTokenSplitter` and the inline matchers). Gemma's GGUF declares ~6,400 added tokens; a 70-character chat turn cost ~450k region compares per `encode`. Candidates are now indexed by first character and length (one map lookup per distinct length). Gemma-3 270M fine-tune vocab, JVM: 2.0 ms → 36 µs per chat-turn encode, ids unchanged; on a 32-bit ARM Android device the same encode took 36–166 ms.
- **`CoroutineSchedule.forRange` could call its body with an empty, inverted range** (`skainet-backend-cpu`): the chunk length is rounded up, so for some `n` and task counts the last chunk started past `n` (63 elements on 12 tasks: chunks of 6, the twelfth at 66..63). The engine's own bodies loop over the range and did nothing for it; a body that passes the range length to a kernel saw a negative size. The task count now follows the chunk length, and every range is non-empty and covers `[0, n)` as the `Schedule` contract states.


## [0.62.1] - 2026-10-21

Headline: **the native Q8_0 / Q4_0 / Q5_0 / Q5_1 kernel bindings no longer zero the earlier rows of a
Expand Down
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ authors:
orcid: https://orcid.org/0009-0006-7981-4075
- name: SKaiNET-developers contributors
website: https://github.com/SKaiNET-developers
version: "0.62.1"
version: "0.62.2"
date-released: "2026-10-21"
license: MIT
repository-code: https://github.com/SKaiNET-developers/SKaiNET
Expand Down
23 changes: 20 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Add the core dependencies (Gradle Kotlin DSL):
```kotlin
dependencies {
// Recommended: import the umbrella BOM and drop versions on the engine modules.
implementation(platform("sk.ainet:skainet-bom:0.62.1"))
implementation(platform("sk.ainet:skainet-bom:0.62.2"))

implementation("sk.ainet.core:skainet-lang-core")
implementation("sk.ainet.core:skainet-backend-cpu")
Expand Down Expand Up @@ -308,7 +308,18 @@ val withoutLabel = dataPipeline<RawDataset>()

---

## What's New in 0.62.1
## What's New in 0.62.2

A performance release for the JVM native lane. The dense FP32 `x · Wᵀ` kernel — what every resident dense
weight resolves to once `skainet-backend-native-cpu` is on the classpath — now splits its work across the
context `Schedule` (SKEEP-005) instead of a fixed four-thread C pool, and on x86-64 hosts with AVX2 and FMA
runs a runtime-selected eight-lane register-tiled lane; threaded, sliced and single calls stay bit-identical.
A single 63 × 1024 × 1024 call goes from 29.6 to 131.7 GFLOP/s on a six-core laptop, and a ModernBERT-large
encoder request in the LayaMoE example from 13.0 s to 6.4 s with unchanged answers. The text front-end
indexes added tokens by first character and length, so a Gemma chat turn encodes in 36 µs instead of 2.0 ms.
`CoroutineSchedule.forRange` never hands a body an empty range any more.

### Carried from 0.62.1

A fix release on top of 0.62.0: the native FFM bindings of the Q8_0, Q4_0, Q5_0 and Q5_1 kernels
copied a zero-filled staging buffer back from index 0, so a product served row by row — any
Expand Down Expand Up @@ -370,6 +381,12 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix:

Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET).

### Contributors (0.62.2)

- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — the schedule-driven and AVX2/FMA
dense FP32 lanes with their tests and measurements, the `CoroutineSchedule` fix, the added-token index, and
the release

### Contributors (0.62.1)

- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — the native packed-kernel
Expand Down Expand Up @@ -495,7 +512,7 @@ from it). Ready-to-paste BibTeX:
author = {Harakal, Michal and {SKaiNET-developers contributors}},
title = {{SKaiNET}: A {Kotlin} Multiplatform {AI} Framework},
year = {2026},
note = {Version 0.62.1. MIT License},
note = {Version 0.62.2. MIT License},
howpublished = {\url{https://github.com/SKaiNET-developers/SKaiNET}}
}
```
Expand Down
2 changes: 1 addition & 1 deletion docs/antora.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ asciidoc:
framework_name: SKaiNET
# Current SKaiNET release — bump once per release; referenced as
# {skainet_version} in dependency snippets (blocks need subs="attributes+").
skainet_version: 0.62.1
skainet_version: 0.62.2
ksp_version: 2.2.21-2.0.5
dokka_version: 2.1.0
asciidoctorj_version: 3.0.0
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
= Kernel × platform support matrix
:description: Which compute-kernel provider serves each weight format on each KMP target.

Generated from `kernel-support.json` (version `0.62.1`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh.
Generated from `kernel-support.json` (version `0.62.2`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh.

Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* / *native-jni* / *native-cinterop* (100) → *panama-vector* / *wasm-simd128* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only). The JS/WASM column assumes `WasmSimd.install()` has run; without the sidecar it is the scalar floor.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
= AI-NET Operators Reference

Generated from version `0.62.1` on 2026-10-21
Generated from version `0.62.2` on 2026-10-24

== Operators by Modality

Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/pages/reference/ops-status-matrix.adoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
= Operator Coverage Matrix
:description: Cross-backend status for every operator function in SKaiNET.

Generated from `operators.json` version `0.62.1` on 2026-10-21.
Generated from `operators.json` version `0.62.2` on 2026-10-24.

Rows are `Operator.function` pairs. A gap is marked in source with `@NotImplemented(reason = …, issue = …)` (or `@InProgress`) and shows up here as ❌ (or 🚧) linked to its tracking issue; the PR that closes the gap removes the annotation. See xref:contributing/issue-taxonomy.adoc[Issue taxonomy] to pick one up.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,12 @@ An Android project with Kotlin. Add the SKaiNET modules:
[source,kotlin]
----
dependencies {
implementation("sk.ainet.core:skainet-lang-core:0.62.1") // tensors, DSL, training
implementation("sk.ainet.core:skainet-backend-cpu:0.62.1") // CPU ops
implementation("sk.ainet.core:skainet-compile-dag:0.62.1") // autograd (training context)
implementation("sk.ainet.core:skainet-data-api:0.62.1") // Dataset / DataBatch
implementation("sk.ainet.core:skainet-data-simple:0.62.1") // embedded Iris
runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.62.1") // NEON kernels (see below)
implementation("sk.ainet.core:skainet-lang-core:0.62.2") // tensors, DSL, training
implementation("sk.ainet.core:skainet-backend-cpu:0.62.2") // CPU ops
implementation("sk.ainet.core:skainet-compile-dag:0.62.2") // autograd (training context)
implementation("sk.ainet.core:skainet-data-api:0.62.2") // Dataset / DataBatch
implementation("sk.ainet.core:skainet-data-simple:0.62.2") // embedded Iris
runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.62.2") // NEON kernels (see below)
}
----

Expand Down
2 changes: 1 addition & 1 deletion gradle.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
GROUP=sk.ainet.core
VERSION_NAME=0.62.1
VERSION_NAME=0.62.2
POM_DESCRIPTION=SKaiNET

POM_URL=https://github.com/SKaiNET-developers/skainet/
Expand Down
Loading