Skip to content

DRU-651 -- Voice notes reach the chat agent as text - #727

Merged
czpython merged 1 commit into
mainfrom
codex/dru-651-voice-notes-reach-the-chat-agent-as-text
Sep 26, 2026
Merged

czpython merged 1 commit into
mainfrom
codex/dru-651-voice-notes-reach-the-chat-agent-as-text

Conversation

@czpython

@czpython czpython commented Sep 26, 2026 •

Copy link
Copy Markdown
Owner

A person speaks a voice note in WhatsApp, or sends an audio clip on Slack, and the agent answers what they said, in text. Nothing shows on the channel before the reply.

  • A SpeechToText card in chat/services.py: url, key, and model, not required, no check on save. transcribe posts the audio as multipart to {url}/audio/transcriptions with the model and returns the answer's text. OpenAI, Groq, and a local server that speaks the same shape all work. An empty answer is a TranscriptionError, a new typed error in chat/exceptions.py.
  • Message.transcript, one text column, empty by default, with its migration. body stays what the person typed, file the audio.
  • In send_turn, before the blocks: for each pending message with an audio/* file, Druks calls the card, saves the transcript, and commits it with the delivered state, before the prompt.
  • The message's text block is the body, then a [Voice note] line, then the transcript, empty parts left out. VOICE_NOTE_MARKER lives in chat/constants.py beside the internal message templates. Audio adds no upload and no link.
  • When the call fails, no card is connected, the answer is empty, or the file is over MAX_UPLOAD_BYTES, the transcript stays empty and Druks adds an internal message from TRANSCRIPTION_FAILED_MESSAGE to the same turn, worded neutrally. The notes close the turn. The turn runs. A note over the cap makes no call.
  • The message response and the web page show the transcript under the typed text.
  • docs/configuration.md describes the Speech To Text card. docs/chat.md describes voice notes under WhatsApp and Slack, and what happens when Druks cannot transcribe.

@czpython
czpython force-pushed the codex/dru-650-other-files-reach-the-chat-agent-as-links branch from a44de84 to 14f0695 Compare September 26, 2026 19:59
@czpython
czpython force-pushed the codex/dru-651-voice-notes-reach-the-chat-agent-as-text branch 2 times, most recently from 994694a to c632874 Compare September 26, 2026 20:02
@czpython
czpython force-pushed the codex/dru-650-other-files-reach-the-chat-agent-as-links branch from 14f0695 to 12c91f0 Compare September 26, 2026 20:14
@czpython
czpython force-pushed the codex/dru-651-voice-notes-reach-the-chat-agent-as-text branch from c632874 to 84c2247 Compare September 26, 2026 20:14
Base automatically changed from codex/dru-650-other-files-reach-the-chat-agent-as-links to main September 26, 2026 20:18
@mintlify

mintlify Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
druks 🟢 Ready View Preview Sep 26, 2026, 8:26 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

@czpython
czpython force-pushed the codex/dru-651-voice-notes-reach-the-chat-agent-as-text branch from 84c2247 to e52bfec Compare September 26, 2026 20:19
@czpython
czpython force-pushed the codex/dru-651-voice-notes-reach-the-chat-agent-as-text branch from e52bfec to e60a925 Compare September 26, 2026 20:26
@czpython
czpython merged commit 84e6e99 into main Sep 26, 2026
4 checks passed
@czpython
czpython deleted the codex/dru-651-voice-notes-reach-the-chat-agent-as-text branch September 26, 2026 20:28

This branch was successfully deployed

1 active deployment
staging - docs — e60a925c Deployed Sep 26, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant