Skip to content

tokenizer: index added tokens by first char and length - #1443

Merged
michalharakal merged 2 commits into
developfrom
perf/sentencepiece-encode
Oct 24, 2026
Merged

michalharakal merged 2 commits into
developfrom
perf/sentencepiece-encode

Merge branch 'develop' into perf/sentencepiece-encode

cf0b487
Select commit
Loading
Failed to load commit list.
Sign in for the full log view

Annotations

3 notices
submit-gradle
succeeded Oct 24, 2026 in 2m 47s