JoyVoice — Floating AI Voice Dictation & Real-Time Translation for Windows

Floating Mic Dictation — Speak Any Language, Get Clean Translations Instantly
Click mic → Speak in any of 10 languages → Clean translation pasted into any app
_{High-accuracy speech dictation with Google ASR & Gemini text processing — or switch on Free & Offline Mode (local Whisper, no API key).}

JoyVoice — Floating AI Voice Dictation & Real-Time Translation for Windows

Open-source Bangla voice typing and multilingual speech-to-text for Windows 11/10. Press F8, dictate in Bangla, English, Russian, Hindi, Spanish, Arabic, Chinese, Japanese, French or Portuguese, and auto-paste clean translated text into any app — zero GPU required in cloud mode.

Questions people ask

Does JoyVoice lose my words if a long recording fails halfway?

No. For long text the pipeline translates in chunks, and if one chunk fails after earlier chunks succeeded, it keeps the translated chunks so far instead of dropping the whole dictation (app/main.py). Usable partial speech is also preserved to the local history store for review instead of being discarded, so text is never silently lost.

Is my dictated text or API key written to any log file?

No. The crash guard (app/crash_guard.py) never writes the raw exception message to the log because that text can embed dictated speech or a key in a URL. Crash reports store only the exception type, the message length in characters, and redacted stack frames, with secrets, URLs, and Windows paths replaced by placeholders. Your API key itself lives only in local settings or the JV_API_KEY environment variable.

What does the conversation memory feature do, and is it on by default?

It is off by default and opt-in (prompt_memory_enabled: False in app/storage/settings_store.py). When enabled, it only affects the Prompt-for-AI text style: the compiler (app/transcription/prompt_compiler.py) sends your current request in full plus dated prior statements from the active conversation within a character budget, and the receiving agent is told to verify live state and ask when details are missing. Normal dictation paths are unchanged, and any failure falls back to your verbatim current request.

How do I dictate Bengali or Banglish into English on Windows?

Press F8, speak naturally in Bengali or Bengali-English mixed speech, and JoyVoice auto-detects the source and pastes clean English into the app under your cursor (single-line text is typed directly; multi-line text uses Ctrl+V). The transcription prompt explicitly handles Bangladeshi Bengali with English code-switching, and the text-style step translates Bengali or mixed Bengali-English input into natural English. No manual language switching or GPU is needed in cloud mode.

With the opt-in ElevenLabs instant engine (v2.6.0), Google Gemini 3.5 Transcribe Live hears Bengali and mixed speech while you speak. When you release the hotkey it finalizes the complete transcript in 0.3–0.6 s, a fast translator turns it into English, and the English is pasted about 1 s after release (2.3–3.5 s with the gateway text model alone; 3.6–4.3 s on the older route). It waits for the complete transcript, so endings are not cut off, and romanized Banglish is recognised so mixed speech is never mistaken for English. If translation fails, Gemini's live English is used only when it covers every sentence; then ElevenLabs' own transcript translation; then the Cloud engine.

How fast is JoyVoice?

With the ElevenLabs instant engine, audio streams to the speech services while you hold the hotkey, so most of the work is done before you let go. Measured on the developer's PC (2026-10-14 to 2026-10-16): English speech was pasted 0.41–0.56 s after release, including a 54-second dictation; Bengali or mixed Bengali-English speech about 1 s with a fast translator key (0.92–1.2 s; a 30-second fast clip took 1.44 s), 2.3–3.5 s without one; the ElevenLabs-only Bengali mode 3.4–4.3 s. The default Cloud engine takes longer and depends on recording length and network.

⚡ Quick Demo

Press F8 → speak in your language → the floating mic pulses with a live waveform → clean translated text appears wherever your cursor is.

That's it. No window switching. No copy-paste. No language selection. Just speak and keep typing.

  You say (Bengali):    "আমি কাল সকালে মিটিং এ যোগ দিতে পারবো না"
  You get (pasted):      "I won't be able to join the meeting tomorrow morning."

  You say (Russian):     "Я не могу присоединиться к встрече завтра утром"
  You get (pasted):      "I can't join the meeting tomorrow morning."

  You say (Chinese):     "我明天早上不能参加会议"
  You get (pasted):      "I can't attend the meeting tomorrow morning."

  You say (Arabic):      "لا أستطيع الانضمام إلى الاجتماع صباح الغد"
  You get (pasted):      "I can't join the meeting tomorrow morning."

Auto-detect means you never touch a language setting. Switch from Bengali to Russian to Chinese mid-conversation — JoyVoice figures it out. Or lock a source language if you prefer.

Step	What Happens	Time
🎙️	Record via your mic (16 kHz mono, float32)	—
🔢	Convert to signed int16 PCM	< 50 ms
🧠	Google Web Speech ASR transcribes speech (or optional Gemini native audio)	Dynamic
✨	Gemini LLM translates & applies text style prompts faithfully	Dynamic
📋	Clipboard-safe paste via `Ctrl+V` with exponential retry	~300 ms
✅	Done. Text is in your app.	Varies by length

Reliable pipeline. Google Web Speech ASR provides the zero-config fallback, while the optional native gateway route joyvoice-fast-audio performs transcription and translation in one request. The gateway's verified short-audio benchmark is approximately 1–2 seconds; latency depends on recording length and network conditions.

🖼️ App Preview

Floating glass-morphism mic widget over your workspace — always on top, never in the way.

Press F8 → Speak any language → Get clean translation. That's it.

📸 More screenshots

⚡ Direct Local Execution

If you have downloaded or cloned this repository into your project folder, you can run JoyVoice immediately:

:: 1. Open Command Prompt in the repository folder
cd joyvoice

:: 2. Activate virtual environment
.venv\Scripts\activate

:: 3. Set your API Key
set JV_API_KEY=your_api_key_here

:: 4. Launch JoyVoice!
run.bat

💡 Quick Desktop Launcher: Double-click run.bat or create a Windows Desktop shortcut to run.bat to launch JoyVoice in one click at any time.

📦 Installation & Setup Options

🪟 Option A: Download Pre-built EXE (recommended)

A single .exe — no Python, no venv, no dependency hell. Drop it on any Windows machine and start dictating. One consolidated executable supports both cloud and offline dictation:

Build	Best for	What's inside
JoyVoice.exe	All users — Cloud & Free/Offline Mode (recommended)	Single full bundle (~193 MB). Contains cloud mode pipeline and bundled offline libraries (faster-whisper / ctranslate2 / av / onnxruntime).

📁 JoyVoice/
   ├── JoyVoice.exe          ← Double-click to launch (supports Cloud & Free/Offline Mode)
   ├── assets/               ← Bundled icons & SVGs
   └── README.txt

Download from GitHub Releases · Standalone · Auto-update ready

💡 Single EXE convenience: JoyVoice.exe handles both Cloud Mode and Free & Offline Mode.

Cloud Mode: Requires an API key configured in Settings → API (stored locally) or set via JV_API_KEY.

Free & Offline Mode: Requires no API key. Click Set up Free Mode in Settings → Free Mode to download the speech model once (needs internet for initial model download only); after that it operates fully offline.

🐍 Option B: Run from Source

# 1. Clone
git clone https://github.com/MHJoy99/joyvoice.git
cd joyvoice

# 2. Create Python 3.11 venv
python -m venv .venv
.venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Set your API key
set JV_API_KEY=your_bdx_market_api_key

# 5. Launch!
python app\main.py

⚠️ Windows-only. PySide6 + global hotkeys + clipboard automation are deeply tied to Win32 APIs.

💡 Tip: Create a Desktop shortcut to run.bat for one-click launch without opening a terminal every time.

🎯 Features

🌐 10-Language Dictation + Translation

	Feature	Detail
🗣️	10 Languages	Bangla · English · Russian · Hindi · Spanish · Arabic · Chinese · Japanese · French · Portuguese
🔍	Auto-Detect Language	Speak naturally — JoyVoice detects your language. No manual switching.
🎯	Target Language Selection	Translate into any of the 10 supported languages, not just English.
🌐	Google ASR + Gemini LLM	Reliable Google Web Speech ASR paired with Gemini LLM for translation and style formatting.
🔄	Automatic Fallback & Salvage	Google ASR primary with Gemini fallback, plus transcript salvage if cloud translation fails.
🎛️	Dynamic Output Modes	Source transcript only · Target translation only · Both side-by-side. Labels adapt to your selected language pair.
📝	5 Text Styles	Clean English · Raw transcript · AI prompt · Formal email · Custom rewrite
🔌	Configurable API	Set the OpenAI-compatible base URL, API key, and audio/text models from the Settings UI — no env vars required.
🆓	Free & Offline Mode	Run totally free with no API key — local Whisper ASR, one-click setup, built-in offline Bangla → English.

⚡ ElevenLabs Instant Engine (v2.6.0)

An opt-in third engine (Settings → Free Mode → Engine → ElevenLabs instant). While the hotkey is held, microphone audio streams in 100 ms chunks to two services at once, so the text is nearly ready when you stop talking. Cloud stays the default engine.

	Feature	Detail
⚡	Instant English	ElevenLabs Scribe v2 Realtime. English text pasted 0.41–0.56 s after hotkey release (measured, including a 54 s dictation).
🌏	Bengali and mixed speech	Google Gemini 3.5 Transcribe Live (`gemini-3.5-transcribe-live`, `bn-BD`, your "Words to listen for" as custom vocabulary) hears Bengali or Banglish speech. On release it finalizes the full transcript in 0.3–0.6 s; a fast translator (any OpenAI-compatible API, default Groq) raced against the gateway text model turns it into English about 1 s after release. Romanized Banglish is detected, and when Gemini heard Bengali its English overrules the ElevenLabs text. `JV_BENGALI_EAR=translate` brings back the older Gemini Live Translate ear.
🔁	Translation backup chain	Non-English speech: translation of Gemini's transcript → Gemini's live English (only if it covers every sentence) → ElevenLabs transcript translation → Cloud engine. The Bengali speech option can also be ElevenLabs-only, local Banglish (Roman letters), or Cloud translation.
🗣️	Spoken target override	"…paste it in Russian" translates the dictation with the gateway text model (about 2.2 s).
📖	Words to listen for	Up to 50 names (≤20 characters each) sent to ElevenLabs as keyterms (Settings → Replacements). Adds about 20% to the ElevenLabs speech-to-text cost.
🔑	Your own keys	An ElevenLabs API key and a Gemini API key from Google AI Studio, entered in Settings → API (stored only in `%APPDATA%\JoyVoice\settings.json`). Without the ElevenLabs key the Cloud engine is used.

⚠️ Limits: About 1 s is the current floor for Bengali; under 0.5 s would need a paid streaming service. The older Live Translate ear is a preview model that paraphrases, so its live English is only a checked fallback. Without a fast translator key, Bengali takes 2.3–3.5 s, and the fast translator's free tier is rate-limited (it then falls back to the gateway). Mixed Bengali-English speech can be misheard when Google is unavailable, because ElevenLabs alone may write it as fake English words. Output mode both shows the English twice for Bengali speech. There is no live text preview on this engine yet. ElevenLabs transcript translation adds about 30% to the speech-to-text cost.

⌨️ Paste Changes (v2.6.0)

Direct typing: single-line results are typed into the focused app with SendInput Unicode, with no clipboard and no Ctrl+V (paste_method: auto). Multi-line text still uses the clipboard-safe Ctrl+V paste.
Enter after paste (opt-in): paste_press_enter sends Enter 0.12 s after the paste, only when focus is still on an app listed in paste_enter_apps (default: VS Code, Cursor, Windows Terminal, Claude, Antigravity). Off by default; never in browsers.
Replacements in Raw style: the replacement table now also runs in the Raw text style, longest phrase first.

🎨 Widget and Appearance (v2.6.0)

	Feature	Detail
🚀	GPU rendering	On NVIDIA the pill is drawn with shaders (moderngl). Quality Ultra / High / Balanced / Calm / Off, fps cap 30, 60 or 120. If OpenGL fails it falls back to Balanced. Ultra stays at about 180–200 MB of memory.
🌈	Aurora look	Aurora palette with real blurred glass and Dynamic-Island-style state morphs. Default theme: "Joy (signature)".
🎛️	Settings → Appearance	Live preview of opacity, tint, blur, glow, shadow, themes and custom colours, motion speed, wave style, sensitivity, scale 50–200%, text size, roundness, timer, badge and cancel chip, idle behaviour, snap and lock, light/dark/auto glass. Reset, Export and Import your look.
♿	Accessibility	High contrast, bold text, colour-blind palette, reduce motion (follows Windows), state word labels, screen-reader names, 44 px targets, proximity fade.
🛠️	Stability	Fixed a crash when switching quality and an 18 MB leak per Settings open; menus, toasts and popups stay on-screen across monitors; settings save atomically.

🆓 Free & Offline Mode (v2.3.0)

JoyVoice can now run totally free and offline — no API key, no cloud. A new Settings → Free Mode tab adds an engine switch: Cloud (uses API key) vs Free & Offline (local models, no API key). Cloud mode stays the default and is untouched.

	Feature	Detail
🆓	No API Key Required	Free Mode runs entirely on your computer using a small local Whisper model (faster-whisper). Zero cloud calls, zero cost.
⬇️	One-Click Setup	Set up Free Mode downloads the speech model once into `%LOCALAPPDATA%\JoyVoice\models\`. Needs internet once, then works fully offline.
🧪	Built-in Test	Test button loads the model and runs a test transcription with live status, so you can confirm offline ASR before dictating.
🌐	Built-in Bangla → English	Offline translation via Whisper's `translate` task — no extra model. Other targets are transcription-only in Free Mode for now.
🎚️	Model & Device Choices	Speech model Tiny / Base / Small (default Small); device Auto (GPU if available) or CPU only.

⚠️ Free Mode notes: the first model download needs internet once, then it's fully offline. Transcription quality depends on the chosen Whisper model (Small recommended). AI text styles (prompt_for_ai / professional_message / facebook_post) require Cloud mode — in Free Mode the cleaned text is pasted and a toast tells you so. Non-English translation targets are transcription-only in Free Mode for now (multilingual offline translation via NLLB and offline AI styles via Ollama are planned).

✨ 10 UI/UX Improvements (v2.0.0)

	Feature	Detail
🪟	Glass-Morphism Widget	Translucent frosted-glass mic pill with backdrop blur. Sits elegantly over any background.
📊	Live Waveform	5-bar animated audio visualization pulses while you speak — instant visual feedback.
⏱️	Recording Timer	Count-up display shows elapsed recording time on the widget.
🏷️	Language Badge	Compact pill badge shows current direction (e.g. `BN → EN`, `Auto → EN`). Changes live with auto-detect.
👁️	Live Preview	Transcription text streams onto the widget in real-time — see what's being captured.
📏	Confidence Bar	3px coloured bar at widget bottom: green (high confidence) / yellow (medium) / red (low). Auto-fades after 3s.
🔔	Floating Toast	Result appears in a toast bubble near your cursor — non-intrusive confirmation.
🔊	Audio Feedback	Tactile beeps at every lifecycle transition (start/stop/success/error) via `winsound.Beep`.
🔀	Quick Language Switcher	`Ctrl+Shift+L` — instantly switch language pair without opening settings.
📜	Right-Click History	Last 5 dictations in widget context menu. One click to re-copy any past result.

🛡️ Robustness — Never Lose Text

	Feature	Detail
👁️	Visibility Watchdog	2-second timer checks widget visibility. Auto-restores if Windows hides it (virtual desktops, UAC, sleep/wake).
⌨️	Hotkey Health Check	5-second timer verifies global hotkey registration. Auto-re-registers after sleep/wake or UAC elevation.
🔁	Paste Retry	Exponential backoff: 3 attempts with increasing delay. Handles focus-switch race conditions.
💾	History Guarantee	Text is saved to persistent history before paste is attempted. Even if paste fails, your text is safe.
📋	Clipboard-Safe Paste	Saves your clipboard → pastes result → restores original. No data loss.
🚀	Launch on Startup	Optional auto-start with Windows. Toggle in Settings.
🛡️	No GPU Required	All pure Python or prebuilt wheels. Runs on integrated graphics.

🏗️ Architecture

Pipeline Stages

┌──────────┐    ┌──────────┐    ┌─────────────────────┐    ┌──────────────────┐    ┌──────────┐
│   🎙️    │    │   🔢     │    │        🧠           │    │   🌐 + ✨        │    │   📋     │
│   Mic    │───▶│  PCM16   │───▶│  Google Web Speech  │───▶│  Gemini Text     │───▶│  Paste   │
│          │    │          │    │  ASR (Primary)      │    │  LLM             │    │          │
│ Any mic  │    │ float→   │    │                     │    │ Translate +      │    │ Ctrl+V   │
│ 16 kHz   │    │  int16   │    │ Transcribe →        │    │ Style Cleanup +  │    │ retry×3  │
│ float32  │    │ < 50ms   │    │ transcript          │    │ Transcript       │    │ restore  │
│          │    │          │    │                     │    │ Salvage          │    │ clipbrd  │
└──────────┘    └──────────┘    └───────┬─────────────┘    └──────────────────┘    └──────────┘
                                        │ (Optional native audio opt-in)
                                        ▼
                                 ┌─────────────────┐
                                 │  Gemini Audio   │
                                 │  (JV_NATIVE_    │
                                 │   AUDIO=true)   │
                                 └─────────────────┘

Robustness Layer

┌───────────────────────────────────────────────────────┐
│  🛡️ Defense-in-Depth                                  │
│                                                       │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐ │
│  │ Watchdog     │  │ Hotkey       │  │ Paste Retry  │ │
│  │ 2s interval  │  │ Health Check │  │ Exponential  │ │
│  │ Auto-restore │  │ 5s interval  │  │ Backoff ×3   │ │
│  │ widget vis.  │  │ Re-register  │  │              │ │
│  └──────────────┘  └──────────────┘  └──────────────┘ │
│                                                       │
│  ┌──────────────────────────────────────────────────┐ │
│  │ History-Before-Paste: save text FIRST,           │ │
│  │ then attempt paste. Never lose data.             │ │
│  └──────────────────────────────────────────────────┘ │
└───────────────────────────────────────────────────────┘

UI Architecture

┌──────────────────────────────────────────────────────────┐
│  🪟 Glass-Morphism Floating Widget (200×80)              │
│                                                          │
│  ┌─────────┐  ┌───────┐  ┌──────┐  ┌──────────────────┐ │
│  │ 5-Bar   │  │ Timer │  │ Badge│  │ Live Preview     │ │
│  │Waveform │  │ 00:03 │  │BN→EN │  │ "I won't be abl…"│ │
│  └─────────┘  └───────┘  └──────┘  └──────────────────┘ │
│  ┌──────────────────────────────────────────────────────┐│
│  │ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓  Confidence Bar (green) ││
│  └──────────────────────────────────────────────────────┘│
│                                                          │
│  State animations: idle (gray) → recording (orange       │
│  pulse) → transcribing (blue) → pasted (green scale-pop) │
│  → error (red). Smooth QPropertyAnimation transitions.   │
└──────────────────────────────────────────────────────────┘

State Machine

[Idle] ──F8──▶ [Recording] ──F8──▶ [Processing] ──done──▶ [Pasting] ──done──▶ [Idle]
                  │                    │
                  └── retry ──────────┘  (on API failure → fallback chain)

Widget states: idle (gray), recording (orange pulsing), transcribing (blue), pasted (green scale-pop), error (red).

Tech Stack

Layer	Technology	Why
UI Framework	PySide6 (Qt 6)	Native Windows look, system tray, global hotkeys
Audio Capture	`sounddevice`	Direct WASAPI access, float32 buffers, low latency
Primary ASR	Google Web Speech ASR	Highly reliable cloud ASR (native Gemini audio available via opt-in)
Translation	Gemini LLM (3.6 Flash)	Text translation & style prompts via OpenAI-compatible endpoint
API Gateway	OpenAI-compatible	Single endpoint for both audio and text models
Clipboard	`pyperclip` + `keyboard`	Clipboard save → paste → restore; safe for password managers
Persistence	JSON (`%APPDATA%\JoyVoice\`)	Settings + history. Human-readable, easy to debug

Audio Feedback

Tactile beeps at every lifecycle transition so you don't need to look at the widget:

Event	Beep	When
`play_start()`	800Hz / 80ms	Recording begins
`play_stop()`	600Hz / 80ms	Recording ends
`play_done()`	1000Hz / 100ms	Transcription succeeds
`play_error()`	300Hz / 150ms	Any error (ASR fail, LLM fail, generic)

Uses winsound.Beep (stdlib, no deps). Silent no-op on non-Windows or Terminal Services.

🌍 Supported Languages

10 languages for both source (auto-detect or locked) and target (translation destination):

Code	Language	Native	Google tag	Auto-Detect	Translation
`auto`	Auto Detect	🔍	—	✅	—
`bn`	Bangla	বাংলা	`bn-BD`	✅	✅
`en`	English	English	`en-US`	✅	✅
`ru`	Russian	Русский	`ru-RU`	✅	✅
`hi`	Hindi	हिन्दी	`hi-IN`	✅	✅
`es`	Spanish	Español	`es-ES`	✅	✅
`ar`	Arabic	العربية	`ar-SA`	✅	✅
`zh`	Chinese	中文	`zh-CN`	✅	✅
`ja`	Japanese	日本語	`ja-JP`	✅	✅
`fr`	French	Français	`fr-FR`	✅	✅
`pt`	Portuguese	Português	`pt-BR`	✅	✅

Auto-detect is the default. The Gemini prompt dynamically switches from "transcribe in {language}" to "detect the spoken language" when auto is active. Google ASR primary also supports native auto-detection.

Quick switcher: Press Ctrl+Shift+L anywhere to cycle language pairs without opening Settings.

🤔 Why JoyVoice?

	JoyVoice v2.0	Windows Dictation	Whisper Local	Google Translate
10 Languages	✅ With auto-detect	⚠️ Select few	⚠️ Model-dependent	❌ Typed text only
Auto-Detect Lang	✅ Default	❌ Manual	❌	❌
Translate to Any Lang	✅ 10 targets	❌ English-only	⚠️ Two-step (ASR + LLM)	❌ Typed only
Latency	Varies by length	~2–5s	10–30s (CPU)	N/A (not speech)
GPU Required	❌ No	❌ No	⚠️ Recommended	❌ No
Auto-Paste	✅ With retry	✅ Yes	❌ Manual	❌ N/A
Glass UI Widget	✅ Frosted glass	❌ OS-level only	❌ No UI	❌ No
Live Waveform	✅ 5-bar	❌	❌	❌
Confidence Indicator	✅ Colour bar	❌	❌	❌
Audio Feedback	✅ 4 beeps	❌	❌	❌
API Cost	~$0.001/call	Free (built-in)	Free (local)	Free
Offline	❌	✅	✅	❌
Setup	5 min	Built-in	30+ min (model download)	Web only
Output Modes	3 modes + 5 styles	1 mode	Raw transcript only	Raw text only
History	✅ Searchable + right-click	❌	❌	❌
Hotkey	✅ `F8` + `Ctrl+Shift+L`	`Win+H`	❌	❌

JoyVoice v2.0 is for the multilingual speaker who needs translation output now — in Slack, in Notion, in VS Code — without switching windows or breaking flow. It's not a general-purpose dictation tool; it's a translation pipeline disguised as a beautiful floating microphone.

⚙️ Settings

Stored at %APPDATA%\JoyVoice\settings.json:

Key	Default	Description
`language`	`auto`	Source language (`auto` to detect, or lock to `bn`/`ru`/`zh`/etc.)
`target_language`	`en`	Translation target language (any of the 10 supported codes)
`output_mode`	`translation`	`original` / `translation` / `both` — labels are dynamic per language pair
`text_style`	`clean_english`	`raw` / `clean_english` / `prompt_for_ai` / `formal_email` / `custom`
`hotkey`	`F8`	Global toggle key
`hotkey_mode`	`toggle`	`toggle` / `hold-to-record`
`audio_device_name`	—	Specific mic (null = system default)
`paste_mode`	`paste`	`paste` / `copy_only`
`paste_delay_ms`	`300`	Delay before `Ctrl+V`
`restore_clipboard`	`true`	Restore original clipboard after paste
`launch_on_startup`	`false`	Auto-start with Windows
`api_base`	`https://gpt.bdx.market/v1`	OpenAI-compatible endpoint root (ends in `/v1`)
`api_key`	(blank)	API key stored locally; blank falls back to `JV_API_KEY` env var
`audio_model`	`joyvoice-fast-audio`	Verified gateway alias for native speech transcription and translation
`text_model`	`gemini-3.6-flash`	Model for translation and AI text styles
`engine_mode`	`cloud`	Active engine: `cloud` (uses API key) / `free` (local models, no API key) / `elevenlabs` (instant engine, v2.6.0)
`elevenlabs_api_key`	(blank)	ElevenLabs key; blank falls back to `JV_ELEVENLABS_API_KEY` / `ELEVENLABS_API_KEY`
`elevenlabs_language`	`auto`	Spoken-language hint for ElevenLabs (`auto` is needed for Bengali)
`elevenlabs_bengali`	`live`	Non-English speech: `live` (Gemini Live heard transcript + translation, ElevenLabs backup) / `elevenlabs` / `banglish` / `translate` (Cloud)
`gemini_api_key`	(blank)	Google AI Studio key for the Bengali ear (Transcribe Live); blank falls back to `JV_GEMINI_API_KEY` / `GEMINI_API_KEY`
`fast_translate_api_key`	(blank)	Optional fast translator key (any OpenAI-compatible API, e.g. Groq) for Bengali in about 1 s; blank falls back to `JV_FAST_TRANSLATE_API_KEY` / `GROQ_API_KEY`, none = gateway only. Set in `settings.json`
`fast_translate_base` / `fast_translate_model` / `fast_translate_reasoning`	`https://api.groq.com/openai/v1` / `openai/gpt-oss-120b` / `low`	Fast translator endpoint, model and reasoning effort
`vocabulary_keyterms`	`[]`	Words for ElevenLabs to listen for (max 50, ≤20 characters each)
`paste_method`	`auto`	`auto` types single-line text directly (no clipboard) / `clipboard`
`paste_press_enter`	`false`	Press Enter after pasting, only in `paste_enter_apps`
`paste_enter_apps`	5 apps	`Code.exe`, `Cursor.exe`, `WindowsTerminal.exe`, `claude.exe`, `Antigravity.exe`
`mute_other_apps`	`off`	Call muting mode: `off` \| `hotkey` \| `virtual_device`
`call_mute_virtual_device`	`""`	Selected virtual audio device endpoint/name for `virtual_device` muting mode
`free_asr_model`	`small`	Free Mode local Whisper model: `tiny` / `base` / `small`
`free_device`	`auto`	Free Mode device: `auto` (GPU if available) / `cpu`
`free_translate_engine`	`auto`	Free Mode translation: `auto` / `whisper` / `none`

Access via right-click mic → Settings or system tray icon.

Settings Tabs

Tab	Contents
Output	Source language (10 langs + auto), target language (10 langs), dynamic output mode labels, text style, cloud note
General	Source language (mirrors Output), launch on startup
API	OpenAI-compatible base URL, masked API key (Show toggle), audio & text model dropdowns, Fetch models, Test connection, ElevenLabs key, Gemini key
Free Mode	Engine switch (Cloud / Free & Offline / ElevenLabs instant), Bengali speech option, speech model (Tiny/Base/Small), device (Auto/CPU), translation, Set up + Test
Hotkey	Preset + custom hotkey, toggle/hold mode
Audio	Input device picker + refresh
Paste	Paste/copy-only mode, delay, clipboard restore, wait-for-release, Enter after pasting
Replacements	Phrase → Replacement table, Words to listen for (ElevenLabs keyterms)
History	Dictation history list + copy

API Tab (v2.2.0)

The API tab configures JoyVoice's cloud connection entirely from the UI — no environment variables required:

Field	Purpose
API base URL	Any OpenAI-compatible endpoint root ending in `/v1` (e.g. `https://gpt.bdx.market/v1`, `https://api.openai.com/v1`). Default/placeholder: `https://gpt.bdx.market/v1`.
API key	Masked (password) field with a Show toggle. Stored locally in `settings.json`. If left blank, falls back to the `JV_API_KEY` env var.
Audio model	Editable dropdown — model used for native speech transcription. Default `joyvoice-fast-audio`, verified through `GET /models`.
Text model	Editable dropdown — model used for translation and AI text styles. Default `gemini-3.6-flash`.
Fetch models	Queries the endpoint's `GET /models` and populates both dropdowns with the live model list.
Test connection	Verifies the endpoint + key are reachable and reports how many models are available.

Config resolution precedence: settings.json value → environment variable → built-in default. Applied at startup and re-applied live whenever settings are saved.

🧪 Benchmark Results

Tested with Bengali audio sample, 2026-07-19:

Model	Time	Bengali Accuracy	Verdict
gemini-3.1-flash-lite ⭐	3.3 s	Best	✅ Default — fastest + cleanest
gemini-3.5-flash-extra-low	4.5 s	Correct	⚠️ Slightly slower
gemini-3.5-flash-low	5.1 s	Correct	⚠️ Slower
gemini-3-flash	5.1 s	Correct	⚠️ Slower
gemini-3.1-pro-low	10.3 s	Most faithful	❌ Too slow for dictation

Current native route: joyvoice-fast-audio is gateway-managed and routes to an audio-capable Flash Lite upstream. The developer's 3-second benchmark reported p95 around 1.58 seconds; longer recordings require their own latency measurement.

📁 Project Structure

joyvoice/
├── README.md                           ← You are here
├── run.bat                             ← Visible-console launcher (surfaces errors)
├── requirements.txt                    ← Python dependencies
├── icon.ico                            ← Tray icon
│
├── assets/
│   ├── logo.svg                        ← Dark-themed wordmark
│   ├── pipeline.svg                    ← Architecture diagram
│   ├── desktop-mockup.png              ← App screenshot
│   ├── how-it-works.png                ← Workflow visualization
│   ├── features_card.png               ← Feature highlights
│   ├── pipeline_infographic.png        ← Pipeline infographic
│   └── comparison_before_after.png     ← Before/after comparison
│
├── app/
│   ├── main.py                         ← Qt controller, state machine, workers
│   ├── audio/
│   │   └── recorder.py                 ← sounddevice InputStream (float32, 16 kHz)
│   ├── transcription/
│   │   ├── gemini_audio.py             ← Gemini native audio (optional opt-in via JV_NATIVE_AUDIO=true)
│   │   ├── cloud_asr.py                ← Google Web Speech primary ASR (10-language auto-detect)
│   │   ├── elevenlabs_realtime.py      ← ElevenLabs Scribe v2 Realtime websocket (instant engine)
│   │   ├── gemini_transcribe_live.py   ← Gemini 3.5 Transcribe Live websocket (default Bengali ear)
│   │   ├── gemini_live_translate.py    ← Gemini Live websocket (older ear, JV_BENGALI_EAR=translate)
│   │   ├── fast_translate.py           ← Fast translator (OpenAI-compatible API, raced with gateway)
│   │   ├── banglish.py                 ← Optional Bengali script → Banglish converter
│   │   ├── text_cleaner.py             ← Punctuation/capitalization cleanup
│   │   └── whisper_engine.py           ← Legacy local Whisper (repaired, inactive)
│   ├── storage/
│   │   ├── settings_store.py           ← JSON persistence (%APPDATA%\JoyVoice\)
│   │   └── history_store.py            ← Dictation history
│   ├── ui/
│   │   ├── floating_widget.py          ← Glass-morphism widget, waveform, toast, confidence
│   │   ├── widget_gpu.py               ← GPU (moderngl) shader renderer for the widget
│   │   ├── appearance.py / appearance_tab.py ← Look settings and Settings → Appearance tab
│   │   ├── placement.py                ← On-screen, multi-monitor placement
│   │   ├── tray.py                     ← System tray icon + menu
│   │   ├── settings_window.py          ← Tabbed settings dialog (9 tabs)
│   │   ├── benchmark_dialog.py         ← ASR speed benchmark
│   │   └── diagnostics_dialog.py       ← Device/connection diagnostics
│   └── system/
│       ├── hotkeys.py                  ← Global hotkey (F8) + quick language switcher (Ctrl+Shift+L)
│       ├── paste.py                    ← Direct Unicode typing; clipboard save → Ctrl+V (retry×3) → restore
│       ├── sounds.py                   ← winsound.Beep audio feedback (4 events)
│       └── startup.py                  ← Launch-on-startup toggle
│
├── docs/
│   ├── SETUP.md                        ← Step-by-step installation guide
│   ├── API.md                          ← Gateway config, model list, benchmarks
│   ├── TROUBLESHOOTING.md              ← Common issues and fixes
│   └── ARCHITECTURE.md                 ← Full project tree, design decisions
│
└── .venv/                              ← Python 3.11 virtual environment

🔧 API Gateway

Base URL:   https://gpt.bdx.market/v1   (default — change in Settings → API)
Auth:       Settings → API tab, or JV_API_KEY env var

Speech input:  `joyvoice-fast-audio` native audio (optional; Google Web Speech remains fallback)
Text model:    gemini-3.6-flash        (default — normal text route remains separate)

Both models are served through any OpenAI-compatible API gateway. The base URL, API key, and both models are configurable from Settings → API (see the API Tab section above); environment variables remain as optional fallbacks. Resolution precedence: settings.json → environment variable → built-in default.

Env Variable	Purpose	Required
`JV_API_KEY`	API gateway authentication	❌ No (optional if set in Settings → API)
`JV_API_BASE`	Override gateway URL	❌ No (defaults to `gpt.bdx.market`)
`JV_NATIVE_AUDIO`	`true` enables native gateway audio on the Cloud engine (`run.bat` sets `false`)	❌ No
`JV_ELEVENLABS_API_KEY` / `ELEVENLABS_API_KEY`	ElevenLabs key fallback for the instant engine	❌ No (optional if set in Settings → API)
`JV_GEMINI_API_KEY` / `GEMINI_API_KEY`	Gemini API key fallback for the Bengali ear	❌ No (optional if set in Settings → API)
`JV_BENGALI_EAR`	`translate` uses the older Gemini Live Translate ear instead of Transcribe Live	❌ No
`JV_FAST_TRANSLATE_API_KEY` / `GROQ_API_KEY`	Fast translator key fallback (Bengali in about 1 s)	❌ No (without it the gateway text model translates)
`JV_WIDGET_BLUR`	`0` turns off the floating widget's blurred-glass backdrop	❌ No
`JV_DATA_DIR`	Keep all JoyVoice data (settings, history, log) in another folder instead of `%APPDATA%\JoyVoice`	❌ No

Quick setup: Configure the key in Settings → API (stored locally), or setx JV_API_KEY "your-key" to persist it across reboots for Desktop shortcuts.

🚨 Critical Pitfalls

Read these before touching the codebase. Each one caused at least one hour of debugging.

🐍 PYTHONPATH Contamination

The Hermes profile venv leaks into the shell. pip sees packages in the Hermes venv and falsely reports them as installed for JoyVoice.

# ✅ Always install with:
env -u PYTHONPATH -u PYTHONHOME .venv/Scripts/python.exe -m pip install <pkg>

🔢 PCM Format Mismatch

Recorder produces float32 (-1.0 to +1.0). Cloud APIs expect signed int16 PCM. The conversion happens in app/main.py:

raw_bytes = (np.clip(audio, -1.0, 1.0) * 32767.0).astype(np.int16).tobytes()

💀 typing_extensions — Silent Killer

If typing_extensions is missing, SpeechRecognition silently disables recognize_google. No import error — it just returns None when called. No stack trace. No warning. Just silence.

🧵 QThread vs QTimer

LLM callbacks must use CloudLLMWorker(QThread) with Qt signals. QTimer.singleShot() from a plain Python thread has no event loop — the result is silently lost.

🪟 pythonw.exe Hides Errors

Always launch with run.bat (visible console) for debugging. pythonw.exe swallows startup exceptions. If JoyVoice doesn't start, run from terminal first.

💾 pycache After Secret Removal

If you ever remove a secret from source code, the compiled .pyc in __pycache__/ still contains the old code. Always run:

find . -name __pycache__ -type d -exec rm -rf {} +

after rotating secrets.

🐛 Debugging Checklist

Kill orphans: powershell "Get-Process python* | Stop-Process -Force"
Launch visible: Use run.bat (not pythonw.exe)
Check logs: %APPDATA%\JoyVoice\joyvoice.log
Verify venv: env -u PYTHONPATH -u PYTHONHOME .venv/Scripts/python.exe -I -c "import app.main"
Test ASR: Generate synthetic audio → verify cloud transcription
Verify API key: echo %JV_API_KEY% or curl -s https://gpt.bdx.market/v1/models -H "Authorization: Bearer %JV_API_KEY%"
Check settings: %APPDATA%\JoyVoice\settings.json — verify language, target_language, output_mode
Restart: Launch via Desktop shortcut after any config change

📦 Dependencies

PySide6 >= 6.7          → Qt 6 UI framework
sounddevice >= 0.5      → WASAPI audio capture
numpy >= 1.26           → Audio buffer math
pyperclip >= 1.9        → Clipboard read/write
keyboard >= 0.13        → Global hotkey hooks
SpeechRecognition >= 3.17 → Google Web Speech ASR (primary default)
typing_extensions >= 4.16 → Required by SpeechRecognition
websockets >= 14.0      → ElevenLabs and Gemini Live streaming (instant engine)
faster-whisper >= 1.0.0 → Free Mode local Whisper ASR (offline)
ctranslate2 >= 4.6.0    → Inference engine behind faster-whisper
av >= 11.0.0            → Audio decoding for Free Mode (onnxruntime comes transitively for VAD)

All pure Python or prebuilt wheels. Cloud mode needs no CUDA, no PyTorch, and no GPU. Free & Offline Mode adds an optional local Whisper model (faster-whisper/ctranslate2) that runs on CPU (or GPU if available).

📚 Documentation & Discoverability Index

Document	Purpose / Audience
`llms.txt`	AGO Standard: AI Model & LLM RAG Indexing Specification (llmstxt.org)
`llms-full.txt`	AGO Full Context: Complete concatenated repository reference for AI assistants
`schema.json`	Schema.org Data: Machine-readable JSON-LD software metadata for search engines
`docs/SEO_AND_DISCOVERY.md`	SEO & AEO Strategy: Tri-channel discoverability blueprint & distribution roadmap
`docs/FAQ.md`	AEO Direct Answers: Standalone Q&A index optimized for answer engines
`docs/SETUP.md`	Step-by-step: git clone → venv → pip install → `JV_API_KEY` → first launch
`docs/API.md`	Gateway config, model list, benchmark data, request/response shapes, fallback chain
`docs/TROUBLESHOOTING.md`	PYTHONPATH contamination, PCM float32→int16, typing_extensions, QThread, pythonw.exe
`docs/ARCHITECTURE.md`	Full project tree, pipeline flow, state machine, design decisions, extension points

🤖 Answer Engine & Search FAQ (AEO)

Q: What is JoyVoice and how does it work?

JoyVoice is an open-source Windows dictation tool that converts speech in 10 languages (Bangla, English, Russian, Hindi, Spanish, Arabic, Chinese, Japanese, French, Portuguese) into clean, translated text. Pressing F8 captures audio, transcribes it via Google Web Speech ASR, processes translation and formatting using Gemini 3.6 Flash, cleans up punctuation, and pastes the result directly into your focused application.

Q: Does JoyVoice need an NVIDIA GPU?

No. JoyVoice uses lightweight cloud endpoints for speech processing and translation. It requires 0% GPU resources and runs smoothly on integrated graphics under Windows 10 and Windows 11.

Q: How do I run JoyVoice from this directory?

Open Command Prompt in the cloned repository root, activate the environment with .venv\Scripts\activate, set your key with set JV_API_KEY=your_key, and launch run.bat.

🗺️ Roadmap

Keep what works. Ship only what is faster and better. Production stays on the proven cloud path until a challenger wins both.

✅ Shipped (v2.1.0)

10-language cloud dictation + auto-detect
Spoken one-shot target override (… Russian / বাংলায় দাও)
Cancel mid-record / mid-transcribe (Esc)
Dangling-end cleanup (cut open lines, don’t invent)
Durable usage telemetry (usage.jsonl — tokens + latency)
Windows portable EXE release
Tri-channel SEO, AEO, and AGO optimization (llms.txt, schema.json, FAQ.md)

🔜 Near term — make it feel instant

Faster model bake-off on BDX.market — same real BN/EN clips, pick winners only if ≤ half current latency and quality holds
Sentence-stream pipeline — VAD-split while speaking; translate completed sentences in the background; stop = only the tail remains
Usage dashboard — simple local report of per-day tokens, cost ballpark, p50/p90 paste time
Bare language cue hardening — trailing single word (Russian / Japanese) first-class, zero false positives mid-sentence

🧭 Medium term — smarter than a dictation box

Context-aware style — focused app picks style (Chat → prompt, Slack → casual, email → professional)
Personal lexicon — brands, people, BDX terms always correct without settings babysitting
Cross-platform — macOS / Linux hotkey + paste backends
Plugin outputs — custom formatters, post-hooks, optional Hermes/agent handoff

🚀 Long term — the hard problems

True simultaneous mode — continuous partials on screen while still talking; final paste is a polish, not a wait
Multi-speaker / meeting mode — diarize, clean, translate per speaker without losing who said what
Offline-capable dual path — local fallback that still feels good when the cloud is gone

🏁 Final boss — almost impossible, still achievable

The Ghost Interpreter
JoyVoice becomes an invisible OS layer: you speak in mixed Bangla/English/whatever, it understands intent + context + target app, and the right text (or action) lands before the sentence feels finished — sub-second, multi-language, personal-lexicon perfect, zero UI babysitting.
Not “another STT app.” A permanent simultaneous interpreter for your whole digital life.
Path: streaming VAD → edge partials → BDX fast audio models → personal memory/lexicon → app-aware paste/actions. Hard as hell. Not magic. Buildable.

_{Built with ❤️ by MH Joy · v2.6.0 · October 2026}
_{MIT License · GitHub · Changelog · Docs}

Name		Name	Last commit message	Last commit date
.github		.github
.go-local-workspace		.go-local-workspace
app		app
assets		assets
bin		bin
docs		docs
tests		tests
tools		tools
.gitignore		.gitignore
AGENTS.md		AGENTS.md
AI_STATUS.md		AI_STATUS.md
CHANGELOG.md		CHANGELOG.md
CONTRIBUTING.md		CONTRIBUTING.md
JoyVoice.spec		JoyVoice.spec
LICENSE		LICENSE
README.md		README.md
build_exe.bat		build_exe.bat
check_python.bat		check_python.bat
icon.ico		icon.ico
index.html		index.html
joyvoice.py		joyvoice.py
llms-full.txt		llms-full.txt
llms.txt		llms.txt
pyproject.toml		pyproject.toml
requirements.txt		requirements.txt
robots.txt		robots.txt
run.bat		run.bat
schema.json		schema.json
sitemap.xml		sitemap.xml

Latest commit

History

Folders and files

Repository files navigation

JoyVoice — Floating AI Voice Dictation & Real-Time Translation for Windows

Questions people ask

Does JoyVoice lose my words if a long recording fails halfway?

Is my dictated text or API key written to any log file?

What does the conversation memory feature do, and is it on by default?

How do I dictate Bengali or Banglish into English on Windows?

How fast is JoyVoice?

⚡ Quick Demo

🖼️ App Preview

⚡ Direct Local Execution

📦 Installation & Setup Options

🪟 Option A: Download Pre-built EXE (recommended)

🐍 Option B: Run from Source

🎯 Features

🌐 10-Language Dictation + Translation

⚡ ElevenLabs Instant Engine (v2.6.0)

⌨️ Paste Changes (v2.6.0)

🎨 Widget and Appearance (v2.6.0)

🆓 Free & Offline Mode (v2.3.0)

✨ 10 UI/UX Improvements (v2.0.0)

🛡️ Robustness — Never Lose Text

🏗️ Architecture

Pipeline Stages

Robustness Layer

UI Architecture

State Machine

Tech Stack

Audio Feedback

🌍 Supported Languages

🤔 Why JoyVoice?

⚙️ Settings

Settings Tabs

API Tab (v2.2.0)

🧪 Benchmark Results

📁 Project Structure

🔧 API Gateway

🚨 Critical Pitfalls

🐍 PYTHONPATH Contamination

🔢 PCM Format Mismatch

💀 typing_extensions — Silent Killer

🧵 QThread vs QTimer

🪟 pythonw.exe Hides Errors

💾 pycache After Secret Removal

🐛 Debugging Checklist

📦 Dependencies

📚 Documentation & Discoverability Index

🤖 Answer Engine & Search FAQ (AEO)

🗺️ Roadmap

✅ Shipped (v2.1.0)

🔜 Near term — make it feel instant

🧭 Medium term — smarter than a dictation box

🚀 Long term — the hard problems

🏁 Final boss — almost impossible, still achievable

About

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages