Skip to content
BranchesTags

Repository files navigation

SharpSpeech

.NET Vue.js Docker OpenAI Compatible

SharpSpeech is a high-performance Text-to-Speech and Speech-to-Text REST API with OpenAI-compatible endpoints, highly vibe-coded😊. Built with .NET 10, it features a Vue.js frontend, Docker containerization, and support for multiple state-of-the-art TTS models (Kokoro, Supertonic-3) and ASR models (Whisper, Nemotron). Which task types a given deployment serves — TTS, ASR, or both — is controlled by a single SERVER_MODE environment variable.

✨ Features

  • 🎯 OpenAI-Compatible API - Drop-in replacement for OpenAI's /v1/audio/speech and /v1/audio/transcriptions endpoints
  • 🚀 High Performance - Custom ONNX inference engine with direct OnnxRuntime integration
  • 🎙️ Speech-to-Text (ASR) - Whisper (whisper.cpp/GGML) and NVIDIA Nemotron 3.5 (cache-aware streaming FastConformer-RNNT, ONNX) transcription engines
  • 💾 Worker Process Architecture - OS-level memory isolation with automatic cleanup and guaranteed resource reclamation, for both TTS and ASR workers
  • 🌍 Multilingual - Support for 3+ TTS languages and 35+ ASR languages including English, Japanese, Chinese, Spanish, French, and more
  • 🎨 Multiple TTS Models - Kokoro (quantized & full precision) and Supertonic-3 models
  • 🎭 Rich Voice Library - 5+ speaker voices with diverse characteristics
  • 🔀 Configurable Server Mode - Run a single instance as TTS-only, ASR-only, or both via SERVER_MODE
  • 🔧 Production Ready - Built-in health checks, automatic model management, and graceful degradation
  • 📊 Interactive Docs - Swagger/OpenAPI documentation included
  • 🎨 Modern UI - Vue.js frontend that adapts to the running server's capabilities (TTS tab, ASR tab, or both)
  • 🐳 Docker Native - Multi-stage builds with optimized containerization, selectable per-mode via docker build --target
  • 🧵 Thread-Safe - Advanced concurrency handling for multi-tenant deployments

🚀 Quick Start

Using Docker Compose (Recommended)

# Clone the repository
git clone https://github.com/Fhrozen/SharpSpeech.git
cd SharpSpeech

# Start the service
docker compose up -d

# Access the web interface
open http://localhost:5768

# Or use the API directly
curl -X POST http://localhost:5768/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "kokoro-q4",
    "input": "Hello world, this is a test of SharpSpeech.",
    "voice": "af_bella",
    "speed": 1.0
  }' \
  --output speech.wav

The service will automatically download required models from Hugging Face on first startup.

By default the container only serves TTS. To enable Speech-to-Text as well, set SERVER_MODE:

# tts (default) | asr | both
SERVER_MODE=both docker compose up -d

# Transcribe an audio file (Whisper by default)
curl -X POST http://localhost:5768/v1/audio/transcriptions \
  -F file=@sample.wav \
  -F model=whisper-base

📚 Documentation

🎯 Core Technologies

  • Backend: .NET 10 with ASP.NET Core Minimal APIs
  • Frontend: Vue.js 3.5 with TypeScript
  • TTS Engine: Microsoft.ML.OnnxRuntime + espeak-ng
  • ASR Engines: Whisper.net (whisper.cpp/GGML) & Microsoft.ML.OnnxRuntime (Nemotron)
  • Models: Kokoro-82M & Supertonic-3 (TTS, ONNX); Whisper & Nemotron 3.5 (ASR, GGML/ONNX)
  • Containerization: Docker with multi-stage builds
  • Build Tool: pnpm for frontend dependencies

🔌 API Endpoints

Endpoint Method Description
/health GET Health check endpoint
/api/server-info GET Which task types (TTS/ASR) this server instance was started with
/api/models GET List available TTS models with details (gated by SERVER_MODE)
/api/asr-models GET List available ASR models with details (gated by SERVER_MODE)
/v1/models GET OpenAI-compatible model listing (merges TTS + ASR catalogs)
/v1/audio/speech POST OpenAI-compatible speech synthesis (gated by SERVER_MODE)
/v1/audio/transcriptions POST OpenAI-compatible audio transcription (gated by SERVER_MODE)
/swagger GET Interactive API documentation

🎭 Supported Models

Text-to-Speech

Model Description Languages Voices Quality
kokoro-q4 Quantized Kokoro 82M 3+ 5+ Fast, low memory
kokoro-full Full precision Kokoro 82M 3+ 5+ High quality
supertonic-3 Supertonic multilingual 3+ 10 styles Production grade

Speech-to-Text

Model Description Languages Notes
whisper-base OpenAI Whisper base (GGML, via whisper.cpp) Auto-detect + multilingual Whole-file batch transcription
nemotron-3.5 NVIDIA Nemotron 3.5 streaming ASR (FastConformer-RNNT, INT4 ONNX) 35+ Cache-aware chunked decoding

See Models Documentation for detailed information about each model.

🌍 Supported Languages

English (US/GB), Spanish, French, Hindi, Italian, Japanese, Portuguese (BR), Chinese (CN), Korean, German, Dutch, Arabic, Russian, Turkish, Polish, Swedish, Danish, Norwegian, Finnish, Greek, Czech, Romanian, Hungarian, Thai, Vietnamese, Indonesian, Hebrew, Ukrainian, and more.

⚙️ Configuration

Key environment variables:

HTTP_PORT=5768                        # HTTP port
SERVER_MODE=tts                       # tts (default) | asr | both
MODEL_CACHE_DIR=/cache                # Model cache directory
ESPEAK_DATA_DIR=/app/assets/espeak-ng-data  # espeak-ng data
MODEL_IDLE_TIMEOUT_SECONDS=60         # Model unload timeout

See Configuration Guide for all options.

🛠️ Development

Local Development (without Docker)

# Backend
cd src/SharpSpeech.Api
dotnet restore
dotnet run

# Frontend
cd frontend
corepack enable pnpm
pnpm install
pnpm dev

See Development Guide for detailed setup instructions.

🧪 Testing

# Run all tests
./tests/run-tests.sh

# Run specific test suite
dotnet test tests/SharpSpeech.Api.Tests
dotnet test tests/SharpSpeech.Api.IntegrationTests

# Opt-in: real-model circular TTS->ASR tests (downloads GB-scale weights, skipped by default and
# excluded from CI)
ASR_MODEL_TESTS=1 dotnet test tests/SharpSpeech.Api.IntegrationTests --filter "Category=AsrModelTests"
# or
./tests/run-tests.sh asr-model-tests

📦 Production Deployment

SharpSpeech is production-ready with:

  • Automatic model memory management
  • Graceful degradation on failures
  • Health check endpoints
  • Request validation and error handling
  • Thread-safe concurrent request processing

See Deployment Guide for production deployment strategies.

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

📄 License

See LICENSE file for details.

🙏 Acknowledgments

📞 Support

  • Documentation: docs/
  • Issues: GitHub Issues
  • API Docs: http://localhost:5768/swagger (when running)
  • Configure the timeout via the MODEL_IDLE_TIMEOUT_SECONDS environment variable
  • Set to 0 to disable automatic release and keep all loaded models in memory

Tests

Local Testing

Unit tests:

dotnet test tests/SharpSpeech.Api.Tests/SharpSpeech.Api.Tests.csproj

Integration tests with Docker:

# Run all tests (unit + integration)
./tests/run-tests.sh all

# Run specific test types
./tests/run-tests.sh unit
./tests/run-tests.sh integration

Or use Docker Compose directly:

docker compose -f docker-compose.test.yml up --abort-on-container-exit

Continuous Integration

GitHub Actions automatically runs tests on all pull requests:

  • Unit Tests: Fast, isolated tests (~2 minutes)
  • Integration Tests: Full API tests with model downloads (~5-15 minutes)
  • Test Reports: Automated test summaries in PR checks

The CI workflow:

  • Runs on PRs to main, master, or develop branches
  • Caches model files (~1-2GB) to speed up subsequent runs
  • Uploads test results and API logs as artifacts
  • Reports test failures directly in PR checks

See .github/workflows/README.md for details.

Architecture

HTTP Request → KokoroTtsSynthesizer
                    ↓
              KokoroTtsEngine
                    ↓
        ┌───────────┴───────────┐
        ↓                       ↓
  EspeakWrapper          OnnxRuntime
  (phonemization)        (inference)
        ↓                       ↓
  libespeak-ng.so.1      model.onnx
        └───────────┬───────────┘
                    ↓
            PCM16 WAV Output

ASR follows an analogous path (AsrTranscriberRouter → WhisperAsrTranscriber/ NemotronAsrTranscriber → WhisperAsrEngine/NemotronAsrEngine), running in its own worker process when SERVER_MODE enables ASR. See Architecture for the full diagram covering both TTS and ASR.

Requirements

  • .NET 10.0 SDK (preview)
  • Docker with Compose v2
  • Node.js 24+ with pnpm (for frontend development)
  • espeak-ng library (included in Docker image, TTS only)

About

Web & REST API for TTS/ASR

Topics

asrdotnetfastapikokorokokoro-ttsnemotrononnxonnxruntimeopenai-compatible-apiopenwebuisupertonicttstts-apiwhisper

Resources

Readme
Apache-2.0 license
Activity

Stars

0 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages