liujiahua123123 · Oct 31, 2026 · Oct 28, 2026
diff --git a/api-reference/asyncapi-asr-realtime.yml b/api-reference/asyncapi-asr-realtime.yml
@@ -127,7 +127,7 @@ operations:
 
       **Message Sequence:**
       1. Client sends `config` once, as the first message, within 10 seconds of connecting
-      2. Client streams `audio` after `session_started`, 80–100 ms per message
+      2. Client streams `audio` after `session_started`, 20–40 ms per message for live audio
       3. Client can send `finalize`, `configure`, `context_add`, `keepalive` and `agent_speaking` at any time
       4. Client sends `audio_done` when there is no more audio
 
@@ -253,8 +253,10 @@ components:
               nothing in them is transcribed or counted as received audio.
               Any length; an odd trailing byte waits for the next message.
       description: |
-        Audio in the format from `config`. Send it after `session_started`;
-        80–100 ms per message is a good size. A message can be at most 1 MiB.
+        Audio in the format from `config`. Send it after `session_started`.
+        Anything from 20 to 100 ms of audio per message works, and smaller
+        messages give lower latency; for live audio, 20–40 ms per message is a
+        good size. A message can be at most 1 MiB.
       examples:
         - payload:
             type: audio

diff --git a/api-reference/endpoint/websocket/asr-realtime.mdx b/api-reference/endpoint/websocket/asr-realtime.mdx
@@ -1,6 +1,6 @@
 ---
 asyncapi: "/api-reference/asyncapi-asr-realtime.yml asrRealtime"
 title: "WebSocket Realtime Speech to Text"
 description: "Stream audio over a WebSocket and receive transcripts turn by turn, with turn detection, live drafts and conversation context"
 icon: "webhook"
 iconType: "solid"
@@ -11,7 +11,7 @@
  Connect to `wss://api.fish.audio/v1/asr/realtime` with a WebSocket client.
  Every message in both directions is one MessagePack map in a binary frame. A
  normal HTTP GET request does not start a session. For an introduction, see
  [Realtime Speech to Text](/features/speech-to-text/realtime).
 </Note>

 <Note>`transcribe-stream-1` is in preview.</Note>
@@ -20,7 +20,7 @@

 You need a [Fish Audio API key](/developer-guide/getting-started/api-key) and a WebSocket client that can set request headers. The API key travels in a request header, so connect from your server, not from a browser; to capture audio in a browser, relay it through your server as in [Transcribe a live audio stream](/developer-guide/sdk-guide/cookbook/realtime-transcription#capture-from-a-browser-microphone). The examples on these pages use Python with `websockets` and `msgpack`, and Node.js with `ws` and `@msgpack/msgpack`.

 This endpoint is separate from the realtime endpoints of the [compatible APIs](/developer-guide/compat/capabilities), which use manual commit and have no server-side turn detection.

 ## Connect

@@ -33,11 +33,11 @@
 | Header          | Required | Description                                                                                                                                   |
 | --------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
 | `Authorization` | Yes      | `Bearer <API key>`. Write the scheme exactly as `Bearer`, followed by a space and the key.                                                    |
 | `model`         | No       | `transcribe-stream-1` (Preview), the default and the only realtime model. The legacy name `transcribe-1-pro-realtime` reaches the same model. |

 Each open connection counts against the same concurrency limit as your other API requests, from the moment it opens until it closes, even while you send no audio. See [Concurrent request limits](/developer-guide/models-pricing/pricing-and-rate-limits#concurrent-request-limits). A session cannot be resumed: a new connection is a new session.

 Sessions are billed per second of session duration, as Realtime ASR. See [ASR pricing](/developer-guide/models-pricing/pricing-and-rate-limits#automatic-speech-recognition-asr-models).

 If the server refuses the upgrade, it answers with a plain HTTP response whose JSON body is `{"status": <int>, "message": <str>}`:

@@ -45,7 +45,7 @@
 | ------ | ----------------------------------------------------------------------------------------------- | ----------------------------------------------------- |
 | 401    | The API key is missing or invalid, or the `Authorization` header does not start with `Bearer `. | Check the key and the header.                         |
 | 402    | Insufficient API credit.                                                                        | Top up your balance.                                  |
 | 429    | Your team is at its concurrency limit, or the service is at capacity.                           | Retry with backoff. Honor `Retry-After` when present. |
 | 503    | The service is restarting.                                                                      | Retry.                                                |

 The `101 Switching Protocols` response carries an `x-request-id` header, and so do the capacity 429 and 503 refusals. It is the same value as `request_id` in `session_started`, `error` and `session_done`. Log it, and include it when you contact support. Refusals for the key, balance and your concurrency limit (401, 402 and concurrency 429) happen before a session is assigned and carry no `x-request-id`.
@@ -92,13 +92,13 @@
 {
  "type": "config",
  "audio": { "format": "pcm_s16le", "sample_rate": 16000, "channels": 1 },
  "turn_detection": { "mode": "auto", "eager": true },
  "results": { "drafts": true, "ignore_timestamps": false },
  "context": {
    "history": [{ "role": "assistant", "text": "Hi, how can I help?" }],
    "terms": ["Fish Audio", "transcribe-stream-1"]
  },
  "client_reference_id": "call-8f2c"
 }
 ```

@@ -126,7 +126,7 @@
 
 Send 16-bit little-endian PCM, mono, at one of the listed sample rates. Other encodings (such as μ-law, A-law or Opus) and stereo audio are `unsupported_audio`; decode phone audio in μ-law or A-law to 16-bit PCM first, and send it at 8000 Hz as is. Audio at 16000 Hz is used as is; every other rate is converted on the server, so you never need to resample. The conversion adds at most about 20 ms of delay. Every `*_ms` value is in milliseconds of your audio, whatever its rate.
 
-An `audio` message can hold any number of bytes; an odd trailing byte waits for the next message. Send 80–100 ms of audio per message. A message can be at most 1 MiB, which holds 65.5 seconds of audio at 8 kHz or 10.9 seconds at 48 kHz.
+An `audio` message can hold any number of bytes; an odd trailing byte waits for the next message. Anything from 20 to 100 ms of audio per message works and gives the same results, but each result waits for the message that carries its audio, so smaller messages give lower latency. For live audio, 20–40 ms per message is a good size. A message can be at most 1 MiB, which holds 65.5 seconds of audio at 8 kHz or 10.9 seconds at 48 kHz.
 
 #### Turn detection modes
 
@@ -174,7 +174,7 @@
 
 Audio in the format from `config`. `data` must be MessagePack `bin` holding the raw PCM bytes. Do not base64-encode the audio, and do not send it as a string or an array of numbers. If you do, you receive a recoverable `invalid_request`, and those messages are dropped: nothing in them is transcribed or counted as received audio.
 
-Send audio after `session_started` (see [`config`](#config) for audio sent earlier). For live audio, send it as it is captured; 80–100 ms per message is a good size. You can send faster than real time, for example from a file: the server accepts it and reads it at a bounded rate, and results keep following the audio.
+Send audio after `session_started` (see [`config`](#config) for audio sent earlier). For live audio, send it as it is captured, 20–40 ms per message (see [Audio format](#audio-format)). If your client batches small writes (Nagle's algorithm), turn that off with `TCP_NODELAY`; browsers and common WebSocket libraries already send each message right away. You can send faster than real time, for example from a file: the server accepts it and reads it at a bounded rate, and results keep following the audio.
 
 ### `finalize`
 
@@ -332,7 +332,7 @@
 | `turn_index` | integer | The open turn, or, if no turn is open, the index the next turn will get. |
 | `audio_ms`   | integer | Where the acknowledgement is in the audio.                               |

 `auto` mode only, while your agent is speaking (see [`agent_speaking`](#agent_speaking)): the user acknowledged ("mm-hm", "yeah", "right") without taking the turn. Keep talking. One message per acknowledgement. It always comes before the named turn's `committed`, and it can come before that turn's `speech_start`. A backchannel counts for the turn that contains it; that turn commits with `user_turn: false` and `turn_kind: "backchannel"` unless the user goes on to take the turn. A turn that opened in `vad` or `manual` mode never counts an acknowledgement. When your agent is not speaking, an acknowledgement takes the turn (`user_turn_taken`).

 ### `committed`


diff --git a/developer-guide/sdk-guide/cookbook/realtime-transcription.mdx b/developer-guide/sdk-guide/cookbook/realtime-transcription.mdx
@@ -1,5 +1,5 @@
 ---
 title: "Transcribe a live audio stream in realtime"
 description: "Stream audio to transcribe-stream-1 over a WebSocket and print drafts, turn events and final transcripts as they arrive"
 icon: "tower-broadcast"
 tag: "Preview"
@@ -13,7 +13,7 @@

 <Prerequisites />

 The SDKs do not wrap the realtime speech-to-text endpoint yet, so this recipe speaks the WebSocket protocol directly. Install a WebSocket client and a MessagePack library:

 <CodeGroup>

@@ -31,7 +31,7 @@
 
 ## Recipe
 
-The script connects to `wss://api.fish.audio/v1/asr/realtime`, sends a `config` message, waits for `session_started`, and then streams the file in 100 ms messages at real-time speed, as a live microphone would. It prints every message the server sends until the session ends.
+The script connects to `wss://api.fish.audio/v1/asr/realtime`, sends a `config` message, waits for `session_started`, and then streams the file in 40 ms messages at real-time speed, as a live microphone would. It prints every message the server sends until the session ends.
 
 <CodeGroup>
 
@@ -53,7 +53,7 @@
 import websockets
 
 URL = "wss://api.fish.audio/v1/asr/realtime"
-CHUNK_MS = 100  # 80-100 ms of audio per message
+CHUNK_MS = 40  # ms of audio per message; 20-40 ms suits live audio
 
 
 def pack(message):
@@ -134,7 +134,7 @@
 import WebSocket from "ws";
 
 const URL = "wss://api.fish.audio/v1/asr/realtime";
-const CHUNK_MS = 100; // 80-100 ms of audio per message
+const CHUNK_MS = 40; // ms of audio per message; 20-40 ms suits live audio
 
 // Read a 16-bit mono PCM WAV file: its sample rate and its samples.
 function readWav(path) {
@@ -236,7 +236,7 @@

 </CodeGroup>

 The audio goes in `data` as the raw PCM bytes, which must travel as MessagePack `bin`, not as base64 text: msgpack for Python packs `bytes` as `bin` by default (do not set `use_bin_type=False`), and `@msgpack/msgpack` packs a `Buffer` or `Uint8Array` as `bin`.

 The server closes a session after 30 seconds without any client message, but a WebSocket ping counts as one, and Python's `websockets` sends a ping every 20 seconds by default, so the session stays open even if your audio stops; see [Capture audio well](/developer-guide/sdk-guide/cookbook/realtime-voice-agent#capture-audio-well) for a live source that can stall.

@@ -378,10 +378,10 @@
     decode,
   } from "https://cdn.jsdelivr.net/npm/@msgpack/msgpack@3.1.2/+esm";
 
-  // Runs on the audio thread: float samples in, 16-bit PCM out, ~100 ms per message.
+  // Runs on the audio thread: float samples in, 16-bit PCM out, 40 ms per message.
   const WORKLET = `
 class Pcm16 extends AudioWorkletProcessor {
-  constructor() { super(); this.size = Math.round(sampleRate / 10); this.buf = new Int16Array(this.size); this.n = 0; }
+  constructor() { super(); this.size = Math.round(sampleRate / 25); this.buf = new Int16Array(this.size); this.n = 0; }
   process([input]) {
     const ch = input[0] || [];
     for (let i = 0; i < ch.length; i++) {
@@ -490,10 +490,10 @@

 Install `aiohttp`, put both files in one folder, run `python relay.py`, and open `http://localhost:8080`.
 
 - The page captures audio at the device's own sample rate and sends that rate in `config`; the server converts it, so the page does not resample. The rate must be one of the supported rates.
-- An `AudioWorklet` converts the samples to 16-bit PCM and posts about 100 ms per message.
+- An `AudioWorklet` converts the samples to 16-bit PCM and posts 40 ms per message.
 - **Stop** sends `audio_done`, so the server commits the last turn before it closes the connection.
 - The relay forwards the API's close code to the page. In production, add your own user authentication, per-user session limits and input checks to the relay.
 
 ## Related


diff --git a/developer-guide/sdk-guide/cookbook/realtime-voice-agent.mdx b/developer-guide/sdk-guide/cookbook/realtime-voice-agent.mdx
@@ -1,6 +1,6 @@
 ---
 title: "Build a realtime voice agent: streaming speech in, reply, speech out"
 description: "Use realtime speech to text to know when the user has finished, start replies early, keep talking through backchannels and stop on interruptions"
 icon: "headset"
 tag: "Preview"
 ---
@@ -8,7 +8,7 @@
 import Prerequisites from "/snippets/prerequisites.mdx";

 <Note>
  This guide is for building your own voice agent on the realtime speech-to-text
  API. To have Fish Audio run the whole agent for you, see
  [Agents](/agents/overview).
 </Note>
@@ -25,7 +25,7 @@
 pip install "websockets>=14" msgpack
 ```

 Read [Transcribe a live audio stream](/developer-guide/sdk-guide/cookbook/realtime-transcription) first if you have not used the realtime endpoint before.

 ## How a conversation flows

@@ -71,7 +71,7 @@
 import websockets
 
 URL = "wss://api.fish.audio/v1/asr/realtime"
-CHUNK_MS = 100
+CHUNK_MS = 40  # ms of audio per message; 20-40 ms suits live audio
 TAGS = re.compile(r"<\|[^|]*\|>|\[[^\[\]]*\]")  # speaker tags such as <|speaker:0|>, sound tags such as [laughter]
 T0 = time.monotonic()
 
@@ -260,7 +260,7 @@

 Without `agent_speaking`, every acknowledgement takes the turn, and you have to tell "mm-hm" from an interruption yourself.

 ## Handle interruptions and backchannels

 While your agent is speaking:

@@ -285,7 +285,7 @@
 {
  "type": "config",
  "audio": { "format": "pcm_s16le", "sample_rate": 16000, "channels": 1 },
  "turn_detection": { "mode": "auto", "eager": true },
  "context": {
    "history": [
      {
@@ -294,7 +294,7 @@
      },
      { "role": "user", "text": "Hi, I'm calling about my prescription." }
    ],
    "terms": ["Main Street Pharmacy", "Lisinopril"]
  }
 }
 ```
@@ -304,13 +304,13 @@
 - **Keep echo cancellation on.** Without it, the agent's own voice reaches the microphone, opens turns and can be taken for the user. In a browser, request `echoCancellation: true` from `getUserMedia`. Some echo can remain even with echo cancellation. It is usually committed as a non-user turn, but it still opens turns (`speech_start`) while your agent speaks, which is why ducks need a cap. If the start of most replies is ducked, echo is leaking: check echo cancellation, or test with headphones.
 - **Turn automatic gain control off where you can** (`autoGainControl: false` in a browser), so quiet background sound is not amplified between phrases.
 - **Test with headphones first.** They keep the agent's voice out of the microphone, so you can check turn-taking before you tune echo cancellation.
-- **Send audio as you capture it**, mono 16-bit PCM in 80–100 ms messages, at the device's own rate if it is one of the supported rates. The server converts the rate, so do not resample. Telephony audio at 8 kHz is supported; decode μ-law or A-law call audio to 16-bit PCM first.
+- **Send audio as you capture it**, mono 16-bit PCM in 20–40 ms messages, at the device's own rate if it is one of the supported rates. The server converts the rate, so do not resample. Telephony audio at 8 kHz is supported; decode μ-law or A-law call audio to 16-bit PCM first.
 - **Watch your audio source.** The server closes a session after 30 seconds without any client message (`idle_timeout`, close 4408), but a WebSocket ping counts as one, and Python's `websockets` sends a ping every 20 seconds by default. With that default, a session stays open even when your audio stops, and it counts against your concurrency limit until you close it or the 60-minute session limit ends it. If your audio source can stall, watch it in your own code and send `audio_done` or close the connection when it stops, or pass `ping_interval=None` to `websockets.connect` to leave idle detection to the server. Node's `ws` sends no pings by default.
 - **Keep keys on your server.** In a browser, capture audio and relay it through your server, as in [Capture from a browser microphone](/developer-guide/sdk-guide/cookbook/realtime-transcription#capture-from-a-browser-microphone). Send `agent_speaking` on that same connection so it stays in order with the audio.
 
 ## Languages

 Turn detection, including eager end of turn, backchannel detection and background awareness, works on speech in every language. In some languages, a turn can end a little later after the speaker stops. For faster turns, use `vad` mode (500 ms or more is a good start for `silence_timeout_ms`), or end turns yourself with `finalize`.

 ## Connect your LLM and speech


diff --git a/features/speech-to-text/realtime.mdx b/features/speech-to-text/realtime.mdx
@@ -1,7 +1,7 @@
 ---
 title: "Realtime Speech to Text"
 sidebarTitle: "Realtime"
 description: "Transcribe live audio turn by turn with transcribe-stream-1: turn detection that knows when the user has finished, backchannel and background awareness, and conversation context"
 icon: "microphone-lines"
 tag: "Preview"
 ---
@@ -30,7 +30,7 @@
    icon="headset"
    href="/developer-guide/sdk-guide/cookbook/realtime-voice-agent"
  >
    Turn-taking, interruptions and backchannels.
  </Card>
 </CardGroup>

@@ -64,7 +64,7 @@

 To transcribe a recording you already have, use [`POST /v1/asr`](/features/speech-to-text) with `transcribe-1-pro` instead:

 |                | Realtime (`/v1/asr/realtime`)                                  | File (`POST /v1/asr`)                                      |
 | -------------- | -------------------------------------------------------------- | ---------------------------------------------------------- |
 | Model          | `transcribe-stream-1` (Preview)                                | `transcribe-1-pro`                                         |
 | Input          | Live 16-bit PCM over a WebSocket                               | A complete audio file (WAV, MP3, Opus, and more)           |
@@ -72,11 +72,11 @@
 | Turn detection | Yes                                                            | No                                                         |
 | Session length | Up to 60 minutes                                               | Recordings up to 60 minutes                                |

 The realtime endpoints of the [compatible APIs](/developer-guide/compat/capabilities) are separate from this endpoint. They use manual commit and have no server-side turn detection.

 ## Quick start

 The SDKs do not wrap this endpoint; you connect with any WebSocket client and exchange MessagePack messages.

 <Steps>
  <Step title="Connect">
@@ -88,7 +88,7 @@
     `session_started`.
   </Step>
   <Step title="Stream audio">
-    Send `audio` messages with 16-bit mono PCM, 80–100 ms each, as you capture
+    Send `audio` messages with 16-bit mono PCM, 20–40 ms each, as you capture
     it.
   </Step>
   <Step title="Read the transcript">
@@ -105,7 +105,7 @@
 {
  "type": "config",
  "audio": { "format": "pcm_s16le", "sample_rate": 16000, "channels": 1 },
  "turn_detection": { "mode": "auto", "eager": true },
  "results": { "drafts": true }
 }
 ```
@@ -143,7 +143,7 @@

 With `turn_detection.eager: true`, you receive `eager_end_of_turn` as soon as the turn has probably ended, before the final transcript. Its text is exactly the text `committed` will have, unless `turn_resumed` follows because the user went on. A voice agent can start its LLM on the eager text and save that time.

 ### Backchannel detection

 Tell the server when your agent is speaking with `agent_speaking`. While it speaks, a short "mm-hm", "yeah", "right" or "uh-huh" from the user produces `user_backchannel`: the agent keeps talking. A real interruption produces `user_turn_taken`: the agent stops.

@@ -169,14 +169,14 @@
 | Mode             | A turn ends when                                                   | Use it for                             |
 | ---------------- | ------------------------------------------------------------------ | -------------------------------------- |
 | `auto` (default) | Turn detection judges that the user has finished.                  | Voice agents and conversations.        |
 | `vad`            | A fixed silence has passed (`silence_timeout_ms`, default 700 ms). | Simple silence-based endpointing.      |
 | `manual`         | You send `finalize`.                                               | Push-to-talk, or your own endpointing. |

 Eager end of turn, backchannel detection and background awareness are available in `auto` mode, when the session starts in `auto` (the default). In `vad` and `manual` mode, and after a later switch to `auto` from another mode, every turn is a user turn. See [Turn detection modes](/api-reference/endpoint/websocket/asr-realtime#turn-detection-modes) for details.

 ## Languages

 Smart Turn Detection, including eager end of turn, backchannel detection and background awareness, works on speech in every language. In some languages, a turn can end a little later after the speaker stops.

 `draft`, `eager_end_of_turn` and `committed` carry the language of their own text as `language_code` when it can be identified, one of 29 [supported codes](/api-reference/endpoint/websocket/asr-realtime#language).

@@ -188,13 +188,13 @@
 - `eager_end_of_turn` arrives about 0.6 seconds after the speaker stops.
 - `committed` arrives about 1.1 seconds after the speaker stops.
 
-Your network and how you send audio add to these times. Send audio as you capture it, in 80–100 ms messages.
+Your network and how you send audio add to these times. Each result also waits for the message that carries its audio, on average half a message: about 50 ms with 100 ms messages and 20 ms with 40 ms messages. Send audio as you capture it, in 20–40 ms messages.
 
 ## Implementation details
 
 ### Audio

 Send 16-bit little-endian PCM, mono, at 8000, 12000, 16000, 22050, 24000, 32000, 44100 or 48000 Hz. The server converts the sample rate, so you never need to resample. Compressed formats such as Opus or MP3 are not accepted on this endpoint.

 ### Speakers

@@ -210,7 +210,7 @@

 ### Pricing

 `transcribe-stream-1` costs $0.36 per hour, billed per second of session duration as its own line item, Realtime ASR. See [ASR pricing](/developer-guide/models-pricing/pricing-and-rate-limits#automatic-speech-recognition-asr-models).

 ## Going further