Repository navigation
docs(realtime-stt): recommend 20–40 ms audio messages - #294
Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
|
Important Review skippedReview was skipped as selected files did not have any reviewable changes.
 |
| Layer / File(s) | Summary |
|---|---|
Document message guidance and latency api-reference/asyncapi-asr-realtime.yml, api-reference/endpoint/websocket/asr-realtime.mdx, features/speech-to-text/realtime.mdx |
The documentation recommends 20–40 ms messages, states that 20–100 ms messages work, and explains that smaller messages reduce message-related latency. The WebSocket guidance adds TCP_NODELAY advice for clients that batch small writes. The 1 MiB message limit remains unchanged. |
Update streaming examples developer-guide/sdk-guide/cookbook/realtime-transcription.mdx, developer-guide/sdk-guide/cookbook/realtime-voice-agent.mdx |
Python, JavaScript, and browser AudioWorklet examples use 40 ms chunks. The voice-agent recipe changes its chunk size to 40 ms and recommends 20–40 ms messages. |
Estimated code review effort: 2 (Simple) | ~10 minutes
Change: Other
Suggested reviewers: kilerd
Merge Risk: 
3b5fc
-
No actionable merge-blocking risk was found in the realtime guidance or updated examples.
Pre-merge checks: 
-
Check name Status Explanation Description Check Check skipped - CodeRabbit’s high-level summary is enabled. Title check The title clearly and concisely describes the main change: recommending 20–40 ms audio messages for realtime speech-to-text documentation and examples. Docstring Coverage No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat… Linked Issues check Check skipped because no linked issues were found for this pull request. Out of Scope Changes check Check skipped because no linked issues were found for this pull request. Full details: Docstring Coverage
-
Explanation
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 0 files. (5 skipped: 5 unsupported.)
-
Finishing Touches
-
Fix failing CI checks
-
- Commit to this branch
- Create a new PR
🧪 Generate unit tests (beta)
-
- Commit to this branch
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Replace the 80–100 ms audio message recommendation. Any size from 20 to 100 ms works and gives the same results, but each result waits for the message that carries its audio (on average half a message), so smaller messages give lower latency; 20–40 ms is a good size for live audio. - WebSocket reference: the audio format paragraph says why, the audio message section recommends 20–40 ms and notes that clients which batch small writes should set TCP_NODELAY. - AsyncAPI: the message sequence and the audio message description. - Realtime overview: the stream-audio step, and the latency section says how much the message size adds. - Cookbooks: CHUNK_MS 100 -> 40 in the Python and Node file streamers and the voice agent, and the browser AudioWorklet posts 40 ms (sampleRate / 25). Every supported rate gives a whole number of samples per 40 ms message. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3b5fc79 to
02fa557
Compare
Summary
Replaces the "80–100 ms per message" recommendation for realtime STT audio. Message size does not change results, but a live client can only send a message once its audio has been captured, so every result (
speech_start, drafts,eager_end_of_turn,committed) waits on average about half a message longer: about 50 ms with 100 ms messages and 20 ms with 40 ms messages. The pages now say that anything from 20 to 100 ms works, that smaller messages give lower latency, and that 20–40 ms is a good size for live audio.audiomessage section recommends 20–40 ms for live audio, with one sentence saying that clients which batch small writes (Nagle's algorithm) should setTCP_NODELAY(browsers and common WebSocket libraries already send each message right away).audiomessage description use the same wording.CHUNK_MSgoes from 100 to 40 in the Python and Node file streamers and in the voice agent, with updated comments. The browserAudioWorkletnow posts 40 ms (sampleRate / 25). At every supported rate, 40 ms is a whole number of samples (320 at 8 kHz up to 1920 at 48 kHz), so the chunk math still works. The voice agent's capture tip says 20–40 ms.The 1 MiB limit, the "any number of bytes, an odd trailing byte waits" rule and the faster-than-real-time note are unchanged.
Checks: prettier (3.6.2, the repo version) on the changed files;
mint validateandmint broken-linksmatchmain(their only findings are existing ones outside the realtime pages, insnippets/,temp/and aLICENSElink in the Python SDK overview); the AsyncAPI YAML parses. The updated Python file streamer, Node file streamer and Python voice agent ran cleanly against production with a 57-second 16 kHz WAV (8 turns,session_done, close 1000), and a separate live test with 20 ms messages had 0 errors. No CJK characters and no internal names.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.Summary by CodeRabbit