Skip to content

Tighten BreezeTTS streaming path - #483

Merged
0xShug0 merged 4 commits into
mainfrom
feature/breeze-streaming-pockettts-template
Sep 7, 2026
Merged

Tighten BreezeTTS streaming path#483
0xShug0 merged 4 commits into
mainfrom
feature/breeze-streaming-pockettts-template

Conversation

@0xShug0

@0xShug0 0xShug0 commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Summary

This tightens the BreezeTTS streaming implementation around the PocketTTS-style ownership boundary. The session now only drives stream events, while the generator owns AR streaming state and returns decoded audio events to the session. The speech decoder owns the incremental codec decode state with left context and pending frames.

Breeze streaming now uses the incremental codec-frame path by default instead of the old one-output-per-text-chunk path. stream_frames_per_event controls event granularity, and stream_lookahead_margin controls how many trailing codec frames are held before emission to reduce boundary artifacts.

The offline generation path is unchanged.

Validation

Case Result
Build audiocpp_cli and audiocpp_server build cleanly
Default streaming Passed, emitted 4 incremental audio events plus merged output without any opt-in subchunk flag
Offline A/B voice clone Passed, artifacts/text match; timing 4594.314 ms vs 4452.668 ms (+3.18%)
Offline A/B voice design Passed, artifacts/text match; timing 3375.125 ms vs 3375.804 ms (-0.02%)
Path-test compare summary 2 cases compared, 0 differences

Server TTFT

stream_frames_per_event Server TTFT Client first delta Events Wall
32 650.675 ms 663.389 ms 4 1844.125 ms
16 375.147 ms 375.717 ms 7 1836.698 ms
8 377.302 ms 377.861 ms 12 2062.941 ms

@0xShug0

0xShug0 commented Sep 7, 2026

Copy link
Copy Markdown
Owner Author

@Th-Underscore Credits to your PR. If you like, I can build on your PR directly instead of opening a separate PR.

@Th-Underscore

Copy link
Copy Markdown
Contributor

Looks good! No need to rebase on my PR, this is perfect as is. I was thinking of a full Moshi/Mimi StreamingConv1d+StreamingConvTranspose1d port, but this is much better.

A small thing I didn't include myself but may be useful, though, is in BreezeTTSSession::next_stream_event() (src/models/breeze_tts/session.cpp) the
next_stream_audio() AR step can throw, which leaves stream_chunk_active_ true (and the
generator's stream set). A try-catch that calls end_stream() would make it work cleanly.

@0xShug0

0xShug0 commented Sep 7, 2026

Copy link
Copy Markdown
Owner Author

Looks good! No need to rebase on my PR, this is perfect as is. I was thinking of a full Moshi/Mimi StreamingConv1d+StreamingConvTranspose1d port, but this is much better.

A small thing I didn't include myself but may be useful, though, is in BreezeTTSSession::next_stream_event() (src/models/breeze_tts/session.cpp) the next_stream_audio() AR step can throw, which leaves stream_chunk_active_ true (and the generator's stream set). A try-catch that calls end_stream() would make it work cleanly.

Will do later!

@0xShug0
0xShug0 merged commit 9c6a282 into main Sep 7, 2026
7 of 8 checks passed
@0xShug0
0xShug0 deleted the feature/breeze-streaming-pockettts-template branch September 7, 2026 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants