Repository navigation
Merge eliza/develop-pin-ea8b3f2 (Parakeet ASR, misaki hardening, CI fixes) into main - #77
Merged
Merged
Conversation
cardinal() indexed kScales[g] for every three-digit group, so a numeral with more than 42 significant digits read past the 14-entry table. Only two branches of get_number checked number_fits(); the ordinal, multi-dot, currency and float branches did not. cardinal() now reads such a number digit by digit, and every branch rejects it the way the guarded ones do. Regression cases cover the float, currency, ordinal and multi-dot paths. lexicon.cpp also includes <cerrno> for the errno it reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each guarded the process-global libespeak-ng with its own mutex and its own init flag, so concurrent calls raced and each could run on the voice the other selected. tools/espeak-shared/eliza-espeak-shared.h now owns the lock and the init record (header-only, one instance per linked image); each component re-selects its own voice under it before espeak_TextToPhonemes. An explicitly configured data path wins over the default one. test-espeak-shared runs both phonemizers from two threads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI runs ctest -L main, and no kokoro or paradee test carried the label. Adds it to test-kokoro-cancel, -layer-options, -istft, -layers-portable, test-paradee-phase-lock and the misaki mini-lexicon unit test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Voice G2P hardening: misaki number bounds, shared eSpeak owner, CI labels
…eliza_inference_asr_*, with streaming (#73) * tools/parakeet: native Parakeet-TDT ASR for Phonon-2 five-value weights Adds a CPU-only FastConformer encoder + TDT decoder for FermionResearch/Phonon-2 (five-value QAT re-quantisation of nvidia/parakeet-tdt-0.6b-v3, weights CC-BY-4.0) and serves it through the existing eliza_inference_asr_* surface, including the streaming session. - convert_phonon2_to_gguf.py: pinned archive (HF revision 7e153e40, sha256) to GGUF; five-value linears kept exact as 4-bit codes in kernel order, int6 tables as exact int8 + scales; --verify re-expands every record bit for bit against the container. - int8 kernels: SMMLA (i8mm) and SDOT (dotprod) tiers in their own translation units behind HWCAP/sysctl checks, plus a scalar reference; all tiers bit-identical (test-parakeet-kernels). - encoder/decoder in plain C++ with a persistent worker pool; fp16 relative-position key cache; per-token int8 activations. - streaming: pause-delimited windowed re-decode with local agreement. - libelizainference: ELIZA_ENABLE_PARAKEET fold; mmap_acquire("asr") picks a parakeet-tdt GGUF under <bundle>/asr/ (ELIZA_ASR_BACKEND overrides); real TDT word times; asr_stream_supported() is 1 while the backend is loaded. No new symbols, ABI stays v17. - tests: kernel tiers, reference parity (dump_parakeet_reference.py fixtures), FFI end to end; parakeet-bench + bench_parakeet.py for RTFx / first partial / RSS / WER. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tools/parakeet: pause-delimited streaming, MSVC portability, header notes Streaming windows now start in pauses between tokens (the model, the dense reference included, drops or invents words when a window starts mid-word); re-decoded committed tokens are matched case-insensitively. Portability: no M_PI / GCC builtins on MSVC, NEON paths off under MSVC, bench is POSIX only. Documents the parakeet backend semantics in the ABI header. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tools/parakeet: two-pass position-key cache, parity gates for both activation modes The relative-position keys are cached per layer, so they are now computed with residual (two-pass) int8 activations at no run-time cost. The parity test documents its two modes: the shipped int8-activation path (encoder relative L2 ~0.07-0.2, transcripts gated) and PK_DEBUG_F32_ACTS=1, which checks the graph itself against the dense reference (L2 <= 0.005). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tools/parakeet: yield in the pool's wait loops; exact fp32 diagnostic for the key cache When the CPU is oversubscribed (an LLM decoding next to ASR on a phone), a spinning caller or worker can starve a preempted peer; both now yield after a short spin. PK_DEBUG_F32_ACTS skips the second (residual) pass of the position-key cache, which the fp32 diagnostic already covers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tools/parakeet: pin the converted GGUF Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tools/parakeet: type annotations for ty Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tools/parakeet: re-pin the GGUF for the current converter Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tools/parakeet: indent the grouped GEMM loop Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tools/parakeet: run the weight-free kernel test in CI (label main) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ty: skip import resolution for the parakeet reference-env tools Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tools/parakeet: review fixes for streaming, FFI threading, loader and gates Streaming (pk-stream): - PK_STREAM_DEBUG logged raw token pieces and tentative text; it now logs window geometry and token counts only. - The last word before a pause waited for the next word, which pinned the segment start, so the window grew through silence (a 3 s utterance followed by 40 s of quiet re-decoded windows up to 41 s, 1.5 s each). The newest word now commits once nothing follows it for right_context. - With nothing uncommitted and only quiet audio since the last decode, the decode is skipped and the window keeps the newest 1.5 s of the pause. Same clip: 22 decodes / 2.2 s of compute instead of 32 / 13.5 s, and no words invented on short windows of near-silence. - A window past max_window with no token keeps its newest half. - Partial/finish truncate on a UTF-8 boundary. FFI (eliza-asr-parakeet.inc): - asr_stream_feed raced partial/finish on the stream's audio buffer when called from another thread (the JNI shape). Feed now only resamples into a pending buffer under a per-stream mutex; partial/finish drain it under asr_mutex. Feed after finish fails. - The model generation counter is atomic (contexts bump it under different mutexes). - eliza_free_asr checked ctx for NULL after dereferencing it. Loader: check GGUF value types before gguf_get_val_* (which abort on a mismatch), and reject empty/negative TDT durations (a negative duration looped forever) and blank_id != vocab. Position-key cache: grow by computing only the new rows. Rows are independent, so the cache is bit-identical to a full rebuild; a stream with a growing window no longer rebuilds all 24 layers at every 64-frame step. Parity gate: an empty fixture set now fails, and a pooled WER against the reference transcripts (PK_GATE_REF_WER, default 0.02) bounds how far the utterances that differ may drift. FFI test: transcripts are no longer printed; adds a feeder thread racing partial polls, feed-after-finish and a tiny-buffer partial. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Shaw <thefutureisaweirdplace@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
- tagger.cpp used std::all_of without <algorithm>; GCC and the Windows clang toolchains reject it. Include <algorithm> in every misaki-g2p and parakeet source that uses std::min/std::max/std::all_of. - GCC ignores [[maybe_unused]] on a non-static data member and -Werror=attributes turned that into an error in misaki-internal.h. Drop the attribute and reference data_dir_ in the no-espeak path so clang's -Wunused-private-field stays quiet. - lexicon.cpp munmap cast dropped const via a C cast (-Wcast-qual). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test-kokoro-layer-options compared the fast and libm Snake paths with a fixed 1e-6 bound on |diff| * |a|. Both paths round x + sin^2(a x) / a to float, so for |x| near 8-16 the output ulp alone (~9.5e-7) times |a| exceeds that bound. Every Linux, Windows and macOS-x64 job now reaches this test (misaki-g2p no longer stops the build first), and all of them failed it with 1.06e-06. Subtract one output ulp before scaling, so the check still bounds the fast_sin2 error (observed 2.4-2.8e-07) and no longer counts float rounding of the sum. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
x86_64 only had the scalar tier, which runs the encoder below real time and leaves ASR on x86_64 Android emulator images unusable. pk-kernels-avx2.cpp adds decode (PSHUFB nibble lookup), the 8-row tile (VPMADDWD on int16-widened straight and half-swapped token pairs, the same scheme as the dotprod tier) and the decoder matvec. Widening to int16 keeps every int8 product exact, so the tier is bit-identical to scalar for any input. The file is built with -mavx2 (/arch:AVX2) and entered only after CPUID AVX2 plus XGETBV YMM state; PK_KERNEL=avx2 / --kernel avx2 force it. The kernel test now also runs 200 random full-range int8 tile and matvec cases (including the all -128 corner) against scalar for every tier. CMake no longer passes the ARM -march flags to an x86_64 build on an Apple silicon host (CMAKE_OSX_ARCHITECTURES=x86_64). Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
misaki-g2p/parakeet: fix GCC and Windows CI build breaks
github-actions
Bot
added
documentation
Member
Author
|
Merging. The checks look the same as they did on #74. Every build and ctest job passes except macOS-latest-arm64 🤖 Generated with Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merges
eliza/develop-pin-ea8b3f2intomainwith a merge commit, so the pin branch history is kept. The merge had no conflicts.New on main:
tools/parakeet, FermionResearch/Phonon-2 five-value weights) behindeliza_inference_asr_*, with streaming.<algorithm>,[[maybe_unused]]on a member, cast-qual), plus a fix to thetest-kokoro-layer-optionsSnake tolerance.#59, #63, #64 and #69 were already on main.
Model and licence: the Phonon-2 GGUF is not published.
tools/parakeet/convert_phonon2_to_gguf.pyconverts it from HFFermionResearch/Phonon-2@7e153e4054c0a10db6d47d6fff5c062d4c77c154, and the output is byte-reproducible:phonon-2-fv5.gguf, 330,632,256 B, sha256dc552eb786486167dde3ac80bbe49a6088c8ef39c03c257d5a090fc57b2c47b3. The weights are CC-BY-4.0, so distributions must ship the attribution and NOTICE. That licence is flagged here, not a blocker.Local validation on Apple M4 Max, Apple clang,
-DGGML_METAL=OFF -DLLAMA_FATAL_WARNINGS=ON:ctest -L main: 63/63 pass.test-paradee-apiandtest-paradee-paritypass with the Paradee-8M f32 GGUF: SNR 58 dB, 40x realtime.test-parakeet-paritypasses with phonon-2-fv5.gguf: 20/20 utterances have identical text and token ids, WER 0.Not proven here: Android/device builds and any CUDA, Vulkan or Metal backend runs.
Coordination with #72 (codex/merge-65-main): this PR doesn't include #72. A trial merge of the pin branch into #72's head conflicts only in
tools/omnivoice/CMakeLists.txt. Keep both sides: the LiteRT/MLX options and blocks from #72, pluseliza_inference_fold_parakeetand itseliza_inference_fold_parakeet(elizainference)call from the pin.AI assistance disclosure: an AI coding assistant (Claude Code) prepared this merge and its validation. A maintainer-authorised agent reviewed it.
🤖 Generated with Claude Code