Skip to content

Voice G2P hardening: misaki number bounds, shared eSpeak owner, CI labels - #71

Merged
lalalune merged 3 commits into
eliza/develop-pin-ea8b3f2from
claude/voice-g2p-hardening
Oct 14, 2026
Merged

lalalune merged 3 commits into
eliza/develop-pin-ea8b3f2from
claude/voice-g2p-hardening

Conversation

@lalalune

Copy link
Copy Markdown
Member

Follow-up to post-merge review of the misaki-g2p / Kokoro / Paradee voice stack. Three commits, one per concern.

1. misaki number reading: out-of-bounds read past the named scales

cardinal() (tools/misaki-g2p/src/numbers.cpp) indexed kScales[g] (14 entries) for every three-digit group. get_number only checked number_fits() in two branches. The ordinal, multi-dot, currency and float branches went straight to cardinal().

Evidence: with the pre-fix code under -fsanitize=address,undefined -fno-sanitize-recover=all, misaki-g2p-cli on It is <60 nines>.5 units., Pay $<60 nines> now. and The <60 nines>th one. each reports numbers.cpp:58:46: runtime error: index 19 out of bounds for type 'const char *[14]'. The multi-dot form It is 1.1<59 zeros>.5 hits the same line.

Fix:

  • cardinal() is now safe on its own. Past kMaxGroups it reads the number digit by digit.
  • Every call site applies the same number_fits() guard the two existing branches use. The ordinal and float branches check before the call. extend_num (currency and multi-dot) marks the word unreadable. The number is then rejected exactly as in the already-guarded branches, matching misaki/num2words, which raises there.
  • Regression tests in test_misaki_g2p.cpp cover the three sentences above, the multi-dot form and an oversized cents part, plus direct number_cardinal / number_ordinal / number_float cases. The pre-fix code fails the new tests under UBSan (index 14 out of bounds). With the fix, the ASan/UBSan unit run passes with eSpeak both OFF and ON, and the CLI cases report no sanitizer errors.

Also: lexicon.cpp used errno without <cerrno>. Added.

2. One libespeak-ng lock and init owner for misaki and Kokoro

libespeak-ng is process-global and not thread-safe. misaki (src/espeak.cpp) and Kokoro (src/kokoro-phonemes.cpp) each had their own mutex and their own "initialised" flag. In the fused libelizainference they ran unserialised against each other. Each could also end up on the other's voice: Kokoro selects en-us once, while misaki selects gmw/en-US only on (re)initialisation.

Fix: tools/espeak-shared/eliza-espeak-shared.h owns the one lock and the init record (data dir plus current voice). Under the lock, each component calls prepare(lock, data_dir, voice, err), which initialises libespeak-ng if needed and re-selects that component's voice whenever the other one changed it. Only then does it call espeak_TextToPhonemes.

  • Data path: each component keeps its own discovery. Kokoro asks for the default (empty: whatever the process already has, else libespeak-ng's compiled-in path or ESPEAK_DATA_PATH). misaki passes the explicit misaki_g2p_open directory. An explicitly configured path wins and later default-path callers keep it. This is documented in the header.
  • CMake layering: the helper is header-only. Its state lives in a C++17 inline function's static, which the linker folds to one instance per program or shared library. kokoro_lib and misaki_g2p therefore share it without a target dependency between them. misaki stays buildable standalone, and no extra static archive is added for the mobile slices to collect. Each component adds the include dir only when it links libespeak-ng, so builds without eSpeak are unchanged.
  • Test: misaki_g2p_espeak_shared runs misaki (out-of-lexicon words, so every call goes through eSpeak) and Kokoro phonemize_ipa concurrently from two threads, 200 iterations each, and compares every result with the single-threaded one. It exits 77 when either side lacks libespeak-ng.
    • Before (old per-component mutexes, same test, TSan build): misaki: 11/200 differ, kokoro: 74/200 differ (corrupted output; libespeak-ng itself is not TSan-instrumented, so the damage shows as wrong results).
    • After: misaki: 0/200 differ, kokoro: 0/200 differ under TSan (-DLLAMA_SANITIZE_THREAD=ON, halt_on_error=1, no reports), both with an explicit data dir and with the default one.
  • The agreement gates are unchanged against the prior build: reference 3751/3752 words and 226/230 sentences, agent-style 619/619 and 70/70.

3. CI labels

CI runs ctest -L main, and no kokoro or paradee test carried that label. Added LABELS main to test-kokoro-cancel, test-kokoro-layer-options, test-kokoro-istft, test-kokoro-layers-portable, test-paradee-phase-lock, misaki_g2p_mini_lexicon and misaki_g2p_unit. All are offline, weight-free and need no eSpeak. The misaki unit test builds its lexicon from committed tests/data. Excluded: the network-fetching misaki_g2p_lexicon, the agreement gates, test-paradee-api / -parity (GGUF), and the eSpeak-dependent kokoro tests. Release timings: layers-portable 17 s (microbench), all others under 1.5 s.

Verification (macOS arm64, Metal)

  • cmake -G Ninja -DLLAMA_BUILD_TESTS=ON -DLLAMA_BUILD_KOKORO=ON -DLLAMA_BUILD_OMNIVOICE=ON -DLLAMA_FATAL_WARNINGS=ON -DLLAMA_CURL=OFF -DLLAMA_BUILD_SERVER=OFF. Everything builds with -Werror except test-chat.cpp, which already fails on the base with this flag set (mtmd.h not found with the server off).
  • ctest -R "kokoro|misaki|paradee|eliza-inference|omnivoice": 21/21 pass. test-paradee-api / -parity skip (no GGUF).
  • ctest -L main -R "kokoro|misaki|paradee|eliza-inference": 9/9 pass.
  • Standalone misaki under -fsanitize=address,undefined -fno-sanitize-recover=all: misaki_g2p_unit passes with MISAKI_G2P_ESPEAK=OFF and =ON.
  • TSan build: test-espeak-shared passes as described above.

🤖 Generated with Claude Code

Shaw and others added 3 commits October 14, 2026 00:44
cardinal() indexed kScales[g] for every three-digit group, so a numeral
with more than 42 significant digits read past the 14-entry table. Only
two branches of get_number checked number_fits(); the ordinal, multi-dot,
currency and float branches did not. cardinal() now reads such a number
digit by digit, and every branch rejects it the way the guarded ones do.
Regression cases cover the float, currency, ordinal and multi-dot paths.
lexicon.cpp also includes <cerrno> for the errno it reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each guarded the process-global libespeak-ng with its own mutex and its
own init flag, so concurrent calls raced and each could run on the voice
the other selected. tools/espeak-shared/eliza-espeak-shared.h now owns the
lock and the init record (header-only, one instance per linked image); each
component re-selects its own voice under it before espeak_TextToPhonemes.
An explicitly configured data path wins over the default one.
test-espeak-shared runs both phonemizers from two threads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI runs ctest -L main, and no kokoro or paradee test carried the label.
Adds it to test-kokoro-cancel, -layer-options, -istft, -layers-portable,
test-paradee-phase-lock and the misaki mini-lexicon unit test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@lalalune
lalalune merged commit 79b9437 into eliza/develop-pin-ea8b3f2 Oct 14, 2026
0 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant