Skip to content

[Bug]: Local dictation loses the whole recording when the GPU fails on one piece #32673

Closed as not planned

Description

@TheSalaty

Before submitting

  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

  1. Settings → Voice: transcribe with the local environment, model Canary 1B v2, Hardware set to a specific GPU (gpu:["vulkan","0000:01:00.0"], RTX 3070 Ti, 8 GB).
  2. Keep other GPU apps open (browser, the T3 window itself), so VRAM is partly used.
  3. Dictate for 30–45 seconds and stop.

Expected behavior

A failure on one piece of audio is retried, and nothing I said is lost. At worst I can retry the same recording.

Actual behavior

The pill shows Transcription failed and the recording is gone: the text, and the audio as well, because live streaming keeps no client copy. The retry action records again from scratch.

In one evening, 8 of 22 recordings failed this way, mostly after 30 s or more. Recordings under ~30 s almost always worked.

Three causes in apps/server/src/speech/native.ts and the clients:

  • The worker only splits a piece and retries for OutputTruncated and InputTooLong, matched by exact constructor name. OutputRepetition extends OutputTruncated but has a different name, so it is not retried. OutOfMemory and BackendError are not retried either, although a smaller piece needs less memory.
  • A failure on a GPU the person chose is never retried on the CPU (only auto falls back).
  • browserPcmCapture keeps no copy of streamed audio, and the controller only retains failed recordings for batch transcription, so the audio cannot be retried.

Impact

Major degradation or frequent failure

Version or commit

0.1.8-nightly.20261106.4275 (desktop)

Environment

Manjaro Linux, desktop app, local Canary 1B v2 (Q8_0) on Vulkan, NVIDIA RTX 3070 Ti 8 GB

Logs or stack traces

speech.runSocket 45.6s  OutOfMemory: transcribe_run: out of memory (status 7)
speech.runSocket 33.7s  OutOfMemory: transcribe_run: out of memory (status 7)
speech.runSocket 33.2s  BackendError: transcribe_run: backend error (status 8)
speech.runSocket 45.8s  BackendError: transcribe_run: backend error (status 8)
speech.runSocket 26.6s  OutputRepetition: run stopped: output began repeating
speech.runSocket  0.4s  OutOfMemory: loading model .../canary-1b-v2-Q8_0.gguf: out of memory (status 7)
  at TranscribeModel.transcribe (transcribe-cpp/dist/index.js:1494:20)
  at transcribe ([eval1]:249:20) at decode ([eval1]:271:7) at feed ([eval1]:312:10)
"live transcription operation failed" → "Live transcription failed. Please try recording again."

Screenshots, recordings, or supporting files

Transcription failed

Workaround

Switching to CPU or keeping dictations under ~25 s makes it rarer, but on this machine the CPU is much slower than the GPU.

Activity

  1. juliusmarminge commented on Nov 12, 2026

    @juliusmarminge
    Member

    I tried to reproduce this on the current build and could not.

    Commit 5c6f2f1 (origin/main). The VM has no GPU (no /dev/dri, no Vulkan), so I could not run the reporter's Vulkan OOM case in the desktop app. Instead I added a test to apps/server/src/speech/native.test.ts that runs the real isolated speech worker (loadNativeSpeechModel) against a fixture model. The fixture throws OutputRepetition (a subclass of OutputTruncated, as in transcribe-cpp 0.4.1) or OutOfMemory for any piece over 8000 samples. The test transcribes 16000 samples and expects the worker to split and retry, giving '8000 8000'. On main it fails: the OutputRepetition error goes straight to the caller, because native.ts:74 compares error.constructor.name exactly against 'OutputTruncated'/'InputTooLong'. The other 9 tests in the file pass. From reading the code: loadNativeSpeechModel (native.ts:300-313) falls back to the CPU only when acceleration === 'auto'. I did not check the client-side audio retention claim.

    Closing as not planned. If it still happens for you, reply with the exact steps and the version you are on and it will be looked at again.

  2. TheSalaty commented on Nov 22, 2026

    @TheSalaty
    Author

    This reproduces without a GPU. Every failure of the live stream, not only a GPU error, ends the recording and deletes the audio.

    Version: desktop 0.1.11-nightly.20261118.4890 (Linux). Voice destination: a remote environment on 0.1.12-nightly.20261122.5071, CPU only (no GPU), Parakeet, reached over Tailscale.

    What happened (2026-11-22, times UTC):

    1. 19:16:38: a dictation starts. The server accepts the stream.
    2. 19:17:53–19:18:12: the client loses the connection to the environment. That recording is lost.
    3. The server never sees that socket close. It holds the only speech slot until 19:18:51.8 (speech.runSocket 133681.7 ms).
    4. 19:18:31–19:18:48: 10 new dictations. Status and media ticket return 200, then each stream is refused within ~1 ms:
    19:18:31.068 SpeechService.startStream 0.3ms Failure SpeechBusyError
    19:18:31.022 speech.runSocket 115ms  ERROR "live transcription operation failed"
                 { cause: { _tag: 'SpeechBusyError', operation: 'streaming transcription' } }
    … same at 32.7, 33.7, 34.2, 36.8, 41.1, 42.6, 43.1, 43.8, 48.0
    19:18:59.219 SpeechService.startStream 2.5ms Success
    

    The client shows Voice input failed · Record again. The audio is gone, and "Record again" is refused the same way until the slot frees.

    Why the audio is lost (main): browserVoiceInput.ts passes any stream error to onError(message) without a code. voiceInputSession.ts calls controller.interruptRecording(message), and discardRecording deletes the recording while the person is still speaking.

    Steps without a GPU:

    1. Choose an environment with a local streaming model as the voice destination.
    2. Make it refuse a stream: dictate from a second device to the same environment and keep that recording running (SpeechBusyError).
    3. Start a dictation on the first device and speak.
    4. Expected: the recording continues and can be transcribed or retried. Actual: it ends at once with "Voice input failed", and nothing is kept.

    Test: #32674 adds apps/web/src/speech/browserVoiceInput.browser.test.ts › "keeps recording when the environment refuses the live stream and transcribes the kept audio". It runs the real createBrowserVoiceInputPlatform and VoiceInputController against a socket that answers start with this refusal. On main it fails: while recording, the controller is already in { phase: "error", error: "Live transcription failed. Please try recording again." } with no error code. With #32674 the recording continues, and the retry transcribes all of the audio.

    Could this be reopened? (That the server holds the slot for a dead socket is a separate server problem; I'll report it on its own.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions