Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,17 @@

## [Unreleased]

## [0.58.0] - 2026-10-12

Headline: **input front ends and model identity (SKEEP-006).** Tokenizers leave `skainet-io-core`
for three new modules with one front-end contract across modalities, families become string
identities with descriptors in an immutable registry, and `tokenizer.json` support grows to
Metaspace BPE, Unigram and WordPiece with a golden corpus recorded against the Hugging Face
`tokenizers` library. Every 0.57 call site keeps compiling through deprecated aliases.

Version note: SKaiNET-transformers 0.58.0 follows on this engine release and resolves every
tokenizer through the engine; the two stay on one version line.

### Added

- **Input front ends and model identity (SKEEP-006, draft).** Three new modules:
Expand All @@ -27,8 +38,8 @@
the per-format `CheckpointView` implementations the resolver consumes.

### Fixed
- `tokenizer.json` added tokens keep their `lstrip` / `rstrip` / `single_word` flags through `SpecialTokenSplitter`, so the whitespace before an `lstrip` token such as the ModernBERT / mmBERT `<mask>` belongs to the special token as in the reference library (SKEEP-006, #1324 corpus).

- `tokenizer.json` added tokens keep their `lstrip` / `rstrip` / `single_word` flags through `SpecialTokenSplitter`, so the whitespace before an `lstrip` token such as the ModernBERT / mmBERT `<mask>` belongs to the special token as in the reference library (SKEEP-006, #1324 corpus).
- **`tokenizer.json` parity for byte-level BPE** (#1321): the array form of `model.merges`
written by every `tokenizers` release since 0.20 loads (#1323); every `added_tokens` entry is
matched atomically and counted in `vocabSize`, special or not (#1324); the `normalizer` block
Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ authors:
orcid: https://orcid.org/0009-0006-7981-4075
- name: SKaiNET-developers contributors
website: https://github.com/SKaiNET-developers
version: "0.57.0"
date-released: "2026-09-24"
version: "0.58.0"
date-released: "2026-10-12"
license: MIT
repository-code: https://github.com/SKaiNET-developers/SKaiNET
url: https://skainet-developers.github.io/SKaiNET/
Expand Down
33 changes: 21 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Add the core dependencies (Gradle Kotlin DSL):
```kotlin
dependencies {
// Recommended: import the umbrella BOM and drop versions on the engine modules.
implementation(platform("sk.ainet:skainet-bom:0.57.0"))
implementation(platform("sk.ainet:skainet-bom:0.58.0"))

implementation("sk.ainet.core:skainet-lang-core")
implementation("sk.ainet.core:skainet-backend-cpu")
Expand Down Expand Up @@ -308,18 +308,21 @@ val withoutLabel = dataPipeline<RawDataset>()

---

## What's New in 0.57.0
## What's New in 0.58.0

Grouped-query KV-cache graphs export with an attention mask, and the toolchain moves to Kotlin 2.4.20:
Input front ends and model identity land as SKEEP-006:

- **Masked grouped-query attention exports to valid StableHLO** — a per-head mask under GQA is split
into the K/V head groups instead of broadcast onto them, and a mask onto a dynamic key length (the
chunked-prefill shape of a KV-cache graph) uses `dynamic_broadcast_in_dim` with the expansion hints
IREE 3.11 needs. Static graphs emit exactly what they did before.
- **kotlinx-io is part of `skainet-data-source`'s API** — callers of `openSource()` / `copyTo()` no
longer need to declare it themselves.
- **Kotlin 2.4.20**, AGP 9.4.1, Ktor client 3.6.0; the `webpack` security pin is retired now that
the Kotlin toolchain's own webpack is past the fix.
- **One front-end contract for every input modality** — `skainet-frontend-core` defines
`FrontEnd`, `FrontEndSpec` and the text, audio, image and multimodal contracts; a resolver turns a
checkpoint view into the front ends a model needs. The text lane ships in `skainet-frontend-text`.
- **Tokenizers move out of io-core and cover more files** — Metaspace BPE with byte fallback,
Viterbi Unigram and WordPiece join byte-level BPE, SentencePiece and Tekken. A golden corpus
recorded with the Hugging Face `tokenizers` library guards exact id parity for GPT-2, Qwen2.5,
TinyLlama, T5, BERT, Gemma 4 and the two public Laya tokenizers.
- **Families are identities** — `skainet-model-identity` carries `ModelIdentity`, `CheckpointView`
and an immutable `ModelFamilyRegistry`; the old enum-style detection is deprecated.
- Everything that compiled against 0.57 still compiles: the io-core names are deprecated aliases
that forward to the new homes.

See [CHANGELOG.md](CHANGELOG.md) for full release notes, including every prior release.

Expand All @@ -345,6 +348,12 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix:

Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET).

### Contributors (0.58.0)

- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — SKEEP-006: the
front-end and model-identity modules, the tokenizer move with Metaspace BPE, Unigram and
WordPiece, the golden corpus, and the release

### Contributors (0.57.0)

- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — masked grouped-query
Expand Down Expand Up @@ -428,7 +437,7 @@ from it). Ready-to-paste BibTeX:
author = {Harakal, Michal and {SKaiNET-developers contributors}},
title = {{SKaiNET}: A {Kotlin} Multiplatform {AI} Framework},
year = {2026},
note = {Version 0.57.0. MIT License},
note = {Version 0.58.0. MIT License},
howpublished = {\url{https://github.com/SKaiNET-developers/SKaiNET}}
}
```
Expand Down
2 changes: 1 addition & 1 deletion docs/antora.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ asciidoc:
framework_name: SKaiNET
# Current SKaiNET release — bump once per release; referenced as
# {skainet_version} in dependency snippets (blocks need subs="attributes+").
skainet_version: 0.57.0
skainet_version: 0.58.0
ksp_version: 2.2.21-2.0.5
dokka_version: 2.1.0
asciidoctorj_version: 3.0.0
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
= Kernel × platform support matrix
:description: Which compute-kernel provider serves each weight format on each KMP target.

Generated from `kernel-support.json` (version `0.57.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh.
Generated from `kernel-support.json` (version `0.58.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh.

Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* (100) → *panama-vector* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only).

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
= AI-NET Operators Reference

Generated from version `0.57.0` on 2026-09-24
Generated from version `0.58.0` on 2026-10-12

== Operators by Modality

Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/pages/reference/ops-status-matrix.adoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
= Operator Coverage Matrix
:description: Cross-backend status for every operator function in SKaiNET.

Generated from `operators.json` version `0.57.0` on 2026-09-24.
Generated from `operators.json` version `0.58.0` on 2026-10-12.

Rows are `Operator.function` pairs. The `Validated` column shows whether the function's documentation has been DARC-validated by a reviewer (see xref:contributing/darc-workflow.adoc[DARC workflow]). Remaining columns are backends that appear in any function's `statusByBackend` map — a missing entry means the backend makes no claim about the function (treat it as "unknown", not "not supported").

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,12 @@ An Android project with Kotlin. Add the SKaiNET modules:
[source,kotlin]
----
dependencies {
implementation("sk.ainet.core:skainet-lang-core:0.57.0") // tensors, DSL, training
implementation("sk.ainet.core:skainet-backend-cpu:0.57.0") // CPU ops
implementation("sk.ainet.core:skainet-compile-dag:0.57.0") // autograd (training context)
implementation("sk.ainet.core:skainet-data-api:0.57.0") // Dataset / DataBatch
implementation("sk.ainet.core:skainet-data-simple:0.57.0") // embedded Iris
runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.57.0") // NEON kernels (see below)
implementation("sk.ainet.core:skainet-lang-core:0.58.0") // tensors, DSL, training
implementation("sk.ainet.core:skainet-backend-cpu:0.58.0") // CPU ops
implementation("sk.ainet.core:skainet-compile-dag:0.58.0") // autograd (training context)
implementation("sk.ainet.core:skainet-data-api:0.58.0") // Dataset / DataBatch
implementation("sk.ainet.core:skainet-data-simple:0.58.0") // embedded Iris
runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.58.0") // NEON kernels (see below)
}
----

Expand Down
2 changes: 1 addition & 1 deletion gradle.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
GROUP=sk.ainet.core
VERSION_NAME=0.58.0-SNAPSHOT
VERSION_NAME=0.58.0
POM_DESCRIPTION=SKaiNET

POM_URL=https://github.com/SKaiNET-developers/skainet/
Expand Down
Loading