From 8d11c748893c437b719f89182677da6dc2037e5f Mon Sep 17 00:00:00 2001 From: michalharakal Date: Mon, 12 Oct 2026 22:37:33 +0200 Subject: [PATCH 1/5] docs(changelog): 0.58.0 entry --- CHANGELOG.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 10acd926..5931c9a9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,17 @@ ## [Unreleased] +## [0.58.0] - 2026-10-12 + +Headline: **input front ends and model identity (SKEEP-006).** Tokenizers leave `skainet-io-core` +for three new modules with one front-end contract across modalities, families become string +identities with descriptors in an immutable registry, and `tokenizer.json` support grows to +Metaspace BPE, Unigram and WordPiece with a golden corpus recorded against the Hugging Face +`tokenizers` library. Every 0.57 call site keeps compiling through deprecated aliases. + +Version note: SKaiNET-transformers 0.58.0 follows on this engine release and resolves every +tokenizer through the engine; the two stay on one version line. + ### Added - **Input front ends and model identity (SKEEP-006, draft).** Three new modules: @@ -27,8 +38,8 @@ the per-format `CheckpointView` implementations the resolver consumes. ### Fixed -- `tokenizer.json` added tokens keep their `lstrip` / `rstrip` / `single_word` flags through `SpecialTokenSplitter`, so the whitespace before an `lstrip` token such as the ModernBERT / mmBERT `` belongs to the special token as in the reference library (SKEEP-006, #1324 corpus). +- `tokenizer.json` added tokens keep their `lstrip` / `rstrip` / `single_word` flags through `SpecialTokenSplitter`, so the whitespace before an `lstrip` token such as the ModernBERT / mmBERT `` belongs to the special token as in the reference library (SKEEP-006, #1324 corpus). - **`tokenizer.json` parity for byte-level BPE** (#1321): the array form of `model.merges` written by every `tokenizers` release since 0.20 loads (#1323); every `added_tokens` entry is matched atomically and counted in `vocabSize`, special or not (#1324); the `normalizer` block From 5fbaf02e074eebb1566cb346e14c81c18971629c Mon Sep 17 00:00:00 2001 From: michalharakal Date: Mon, 12 Oct 2026 22:37:33 +0200 Subject: [PATCH 2/5] docs(readme): 0.58.0 highlights and BOM coordinate --- README.md | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index c8051e07..7dad4e5f 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,7 @@ Add the core dependencies (Gradle Kotlin DSL): ```kotlin dependencies { // Recommended: import the umbrella BOM and drop versions on the engine modules. - implementation(platform("sk.ainet:skainet-bom:0.57.0")) + implementation(platform("sk.ainet:skainet-bom:0.58.0")) implementation("sk.ainet.core:skainet-lang-core") implementation("sk.ainet.core:skainet-backend-cpu") @@ -308,18 +308,21 @@ val withoutLabel = dataPipeline() --- -## What's New in 0.57.0 +## What's New in 0.58.0 -Grouped-query KV-cache graphs export with an attention mask, and the toolchain moves to Kotlin 2.4.20: +Input front ends and model identity land as SKEEP-006: -- **Masked grouped-query attention exports to valid StableHLO** — a per-head mask under GQA is split - into the K/V head groups instead of broadcast onto them, and a mask onto a dynamic key length (the - chunked-prefill shape of a KV-cache graph) uses `dynamic_broadcast_in_dim` with the expansion hints - IREE 3.11 needs. Static graphs emit exactly what they did before. -- **kotlinx-io is part of `skainet-data-source`'s API** — callers of `openSource()` / `copyTo()` no - longer need to declare it themselves. -- **Kotlin 2.4.20**, AGP 9.4.1, Ktor client 3.6.0; the `webpack` security pin is retired now that - the Kotlin toolchain's own webpack is past the fix. +- **One front-end contract for every input modality** — `skainet-frontend-core` defines + `FrontEnd`, `FrontEndSpec` and the text, audio, image and multimodal contracts; a resolver turns a + checkpoint view into the front ends a model needs. The text lane ships in `skainet-frontend-text`. +- **Tokenizers move out of io-core and cover more files** — Metaspace BPE with byte fallback, + Viterbi Unigram and WordPiece join byte-level BPE, SentencePiece and Tekken. A golden corpus + recorded with the Hugging Face `tokenizers` library guards exact id parity for GPT-2, Qwen2.5, + TinyLlama, T5, BERT, Gemma 4 and the two public Laya tokenizers. +- **Families are identities** — `skainet-model-identity` carries `ModelIdentity`, `CheckpointView` + and an immutable `ModelFamilyRegistry`; the old enum-style detection is deprecated. +- Everything that compiled against 0.57 still compiles: the io-core names are deprecated aliases + that forward to the new homes. See [CHANGELOG.md](CHANGELOG.md) for full release notes, including every prior release. @@ -345,6 +348,12 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix: Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET). +### Contributors (0.58.0) + +- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — SKEEP-006: the + front-end and model-identity modules, the tokenizer move with Metaspace BPE, Unigram and + WordPiece, the golden corpus, and the release + ### Contributors (0.57.0) - **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — masked grouped-query @@ -428,7 +437,7 @@ from it). Ready-to-paste BibTeX: author = {Harakal, Michal and {SKaiNET-developers contributors}}, title = {{SKaiNET}: A {Kotlin} Multiplatform {AI} Framework}, year = {2026}, - note = {Version 0.57.0. MIT License}, + note = {Version 0.58.0. MIT License}, howpublished = {\url{https://github.com/SKaiNET-developers/SKaiNET}} } ``` From ba08950d34e3171b2e64bfcf83b7462b92164e75 Mon Sep 17 00:00:00 2001 From: michalharakal Date: Mon, 12 Oct 2026 22:37:33 +0200 Subject: [PATCH 3/5] docs: bump version references to 0.58.0 --- CITATION.cff | 4 ++-- docs/antora.yml | 2 +- .../android-classifier-getting-started.adoc | 12 ++++++------ 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/CITATION.cff b/CITATION.cff index 0243f175..85fe4fa0 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -12,8 +12,8 @@ authors: orcid: https://orcid.org/0009-0006-7981-4075 - name: SKaiNET-developers contributors website: https://github.com/SKaiNET-developers -version: "0.57.0" -date-released: "2026-09-24" +version: "0.58.0" +date-released: "2026-10-12" license: MIT repository-code: https://github.com/SKaiNET-developers/SKaiNET url: https://skainet-developers.github.io/SKaiNET/ diff --git a/docs/antora.yml b/docs/antora.yml index dcc30cc9..c051ea44 100644 --- a/docs/antora.yml +++ b/docs/antora.yml @@ -15,7 +15,7 @@ asciidoc: framework_name: SKaiNET # Current SKaiNET release — bump once per release; referenced as # {skainet_version} in dependency snippets (blocks need subs="attributes+"). - skainet_version: 0.57.0 + skainet_version: 0.58.0 ksp_version: 2.2.21-2.0.5 dokka_version: 2.1.0 asciidoctorj_version: 3.0.0 diff --git a/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc b/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc index 82f3859b..81813ea8 100644 --- a/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc +++ b/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc @@ -19,12 +19,12 @@ An Android project with Kotlin. Add the SKaiNET modules: [source,kotlin] ---- dependencies { - implementation("sk.ainet.core:skainet-lang-core:0.57.0") // tensors, DSL, training - implementation("sk.ainet.core:skainet-backend-cpu:0.57.0") // CPU ops - implementation("sk.ainet.core:skainet-compile-dag:0.57.0") // autograd (training context) - implementation("sk.ainet.core:skainet-data-api:0.57.0") // Dataset / DataBatch - implementation("sk.ainet.core:skainet-data-simple:0.57.0") // embedded Iris - runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.57.0") // NEON kernels (see below) + implementation("sk.ainet.core:skainet-lang-core:0.58.0") // tensors, DSL, training + implementation("sk.ainet.core:skainet-backend-cpu:0.58.0") // CPU ops + implementation("sk.ainet.core:skainet-compile-dag:0.58.0") // autograd (training context) + implementation("sk.ainet.core:skainet-data-api:0.58.0") // Dataset / DataBatch + implementation("sk.ainet.core:skainet-data-simple:0.58.0") // embedded Iris + runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.58.0") // NEON kernels (see below) } ---- From 2b4a20ccac171d01e01871f9965126b21e64f1fe Mon Sep 17 00:00:00 2001 From: michalharakal Date: Mon, 12 Oct 2026 22:37:33 +0200 Subject: [PATCH 4/5] release: 0.58.0 --- gradle.properties | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gradle.properties b/gradle.properties index 9908f1a6..190f6247 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.core -VERSION_NAME=0.58.0-SNAPSHOT +VERSION_NAME=0.58.0 POM_DESCRIPTION=SKaiNET POM_URL=https://github.com/SKaiNET-developers/skainet/ From e986aa07b224b1ba32f5734787690e1bd8588e0d Mon Sep 17 00:00:00 2001 From: michalharakal Date: Mon, 12 Oct 2026 22:39:07 +0200 Subject: [PATCH 5/5] docs: regenerate operator and kernel matrices at 0.58.0 --- docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc | 2 +- .../modules/ROOT/pages/reference/operators/generated/index.adoc | 2 +- docs/modules/ROOT/pages/reference/ops-status-matrix.adoc | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc index 82367da8..a19b179e 100644 --- a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc @@ -1,7 +1,7 @@ = Kernel × platform support matrix :description: Which compute-kernel provider serves each weight format on each KMP target. -Generated from `kernel-support.json` (version `0.57.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. +Generated from `kernel-support.json` (version `0.58.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* (100) → *panama-vector* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only). diff --git a/docs/modules/ROOT/pages/reference/operators/generated/index.adoc b/docs/modules/ROOT/pages/reference/operators/generated/index.adoc index 57c23c16..292f4ea6 100644 --- a/docs/modules/ROOT/pages/reference/operators/generated/index.adoc +++ b/docs/modules/ROOT/pages/reference/operators/generated/index.adoc @@ -1,6 +1,6 @@ = AI-NET Operators Reference -Generated from version `0.57.0` on 2026-09-24 +Generated from version `0.58.0` on 2026-10-12 == Operators by Modality diff --git a/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc b/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc index a013a9cb..8154df4a 100644 --- a/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc @@ -1,7 +1,7 @@ = Operator Coverage Matrix :description: Cross-backend status for every operator function in SKaiNET. -Generated from `operators.json` version `0.57.0` on 2026-09-24. +Generated from `operators.json` version `0.58.0` on 2026-10-12. Rows are `Operator.function` pairs. The `Validated` column shows whether the function's documentation has been DARC-validated by a reviewer (see xref:contributing/darc-workflow.adoc[DARC workflow]). Remaining columns are backends that appear in any function's `statusByBackend` map — a missing entry means the backend makes no claim about the function (treat it as "unknown", not "not supported").