Skip to content

docs(realtime-stt): document results.tag_audio_events - #303

Merged
liujiahua123123 merged 1 commit into
mainfrom
docs/realtime-stt-tag-audio-events
Oct 31, 2026
Merged

liujiahua123123 merged 1 commit into
mainfrom
docs/realtime-stt-tag-audio-events

Conversation

@liujiahua123123

@liujiahua123123 liujiahua123123 commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Documents the new realtime STT option results.tag_audio_events (boolean, default true). With false, the server removes every sound tag, such as [laughter], from the text of a turn (draft and its settled_text, eager_end_of_turn, committed and its words) and collapses whitespace as it does with diarize: false. The default keeps today's output.

Merge only after realtime-asr 3.1.0 (https://github.com/fishaudio/realtime-asr/pull/50) is live. Until then the server refuses the field as an unknown key (invalid_config, close 4400).

  • WebSocket reference:
    • a results.tag_audio_events row in the config table;
    • a value that is not true or false is invalid_config (the strings "true" and "false" are not accepted, unlike diarize);
    • a configure that changes it applies to turns that open after configured, like ignore_timestamps and diarize;
    • a new guarantee: with the option false when the turn opened, no text of the turn contains an opening tag bracket ([, U+FF3B or U+3010);
    • Sound tags: tags can also come in full-width square brackets (U+FF3B/U+FF3D) or lenticular brackets (U+3010/U+3011), and the strip regular expression now matches all three kinds. A new paragraph covers tag_audio_events: false and its bracket rules: each bracket closes only with its own kind; a closing bracket with nothing open is text; an unclosed opening bracket hides the rest of a draft (the tag may still be growing), but in eager_end_of_turn and committed only the bracket is dropped. It also says that speaker tags still follow diarize alone, and what a turn that holds only tags commits;
    • Speakers, Language and Degraded commits each get a short note on how the option affects them. For example, the language label can reflect text that is not shown, and a degraded committed can show text after a stray bracket that its draft hid.
  • AsyncAPI: the property in the config (with default: true) and configure results schemas, tag_audio_events: true in the session_started config echo (the server always echoes the key), and the committed.text description.
  • Realtime feature page: the Speakers paragraph mentions the option next to the sound tags.
  • Cookbooks: the transcription quickstart mentions the option next to the sound tags. The browser strip regex and the voice agent's TAGS regex now also match full-width and lenticular tags, written as \uFF3B/\u3010 escapes. The voice agent's answer rule notes that the option already removes the tags.

The wording follows the realtime-asr source doc (docs/realtime-api-v1.md). Internal details (unchanged engine requests, settle state and history) are left out.

This conflicts with the open docs/realtime-stt-packet-size branch in realtime-voice-agent.mdx. That branch changes CHUNK_MS on the line next to TAGS. Whichever merges second keeps both changes.

Checks: prettier (3.6.2, the repo version) on the changed files; the AsyncAPI YAML parses; mint broken-links finds nothing in the changed files (its 25 findings are existing ones in .mintlify/skills/ and temp/). The installed mint CLI has no validate command. Both cookbook regexes were run on sample text with ASCII, full-width and lenticular tags in Python and Node, and the tags were removed as expected. Nothing was tested against production, because the option is not live yet. No CJK characters and no internal names.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

Summary by CodeRabbit

  • New Features
    • Added an option to disable sound tags in realtime transcription results. When disabled, sound tags are removed from turn text and associated word entries.
    • Expanded documentation for how sound tags are handled across drafts, final messages, language detection, and degraded commits.
  • Documentation
    • Updated realtime transcription and voice-agent guides to describe sound tags and show how to remove them in client examples.

@mintlify

mintlify Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
hanabiaiinc 🟢 Ready View Preview Oct 31, 2026, 4:31 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Walkthrough

The realtime ASR specification and documentation add the results.tag_audio_events setting, defaulting to true. They describe how the setting affects sound tags in transcript text and word entries, and update SDK examples and guidance.

Changes: Realtime sound-tag controls

Layer / File(s) Summary
Setting contract and turn configuration
api-reference/asyncapi-asr-realtime.yml, api-reference/endpoint/websocket/asr-realtime.mdx
The specification and WebSocket reference add tag_audio_events, document its default and validation, and describe when configuration changes apply. The specification example and committed.text description reflect the setting.
Sound-tag and language behavior
api-reference/endpoint/websocket/asr-realtime.mdx
The WebSocket reference documents bracket forms, removal from text and word entries, handling of unclosed brackets, and effects on language detection and degraded commits.
SDK examples and transcription guidance
developer-guide/sdk-guide/cookbook/realtime-transcription.mdx, developer-guide/sdk-guide/cookbook/realtime-voice-agent.mdx, features/speech-to-text/realtime.mdx
The cookbook examples expand tag-removal patterns and document the setting. The realtime speech-to-text guidance adds the setting as an option for removing sound tags.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Other

Possibly related PRs

  • fishaudio/docs#245: Documents removal of speaker markers from STT text; this PR documents sound tags and distinguishes the two settings.
  • fishaudio/docs#298: Documents disabling diarization and sound tags for plain transcripts; this PR defines the realtime tag_audio_events contract.

Suggested labels: python

Suggested reviewers: kilerd

Merge Risk: Low · up to 54265

Publish only after realtime-asr PR #50 is live; earlier servers reject the newly documented option and close the session. Otherwise, no new implementation defect is established.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

New realtime option results.tag_audio_events (boolean, default true).
false removes every sound tag from the text of a turn (draft,
eager_end_of_turn, committed and its words) and collapses whitespace as
with diarize false.

- WebSocket reference: the config row; invalid_config for a non-boolean
  value; a configure change applies to turns that open after configured;
  a guarantee that no text of such a turn holds an opening tag bracket;
  the Sound tags section says tags can also come in full-width
  (U+FF3B/U+FF3D) or lenticular (U+3010/U+3011) brackets, widens the
  strip regular expression to match, and gives the bracket rules for
  tag_audio_events false (an unclosed bracket hides the rest of a draft
  but only itself in a final); the Speakers, Language and Degraded
  commits sections note how the option interacts with them.
- AsyncAPI: the property in the config and configure schemas, the
  session_started config echo, and the committed text description.
- Realtime overview and cookbooks: mention the option next to the
  sound tags; the cookbook strip regexes also match the full-width and
  lenticular brackets; the voice agent's answer rule notes the option.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@liujiahua123123
liujiahua123123 force-pushed the docs/realtime-stt-tag-audio-events branch from adabab3 to 54265e8 Compare October 31, 2026 04:31
@liujiahua123123
liujiahua123123 marked this pull request as ready for review October 31, 2026 04:31
@liujiahua123123
liujiahua123123 merged commit cf975d3 into main Oct 31, 2026
5 of 6 checks passed
@liujiahua123123
liujiahua123123 deleted the docs/realtime-stt-tag-audio-events branch October 31, 2026 04:34

This branch was successfully deployed

1 active deployment
staging — 54265e85 Deployed Oct 31, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant