clap.cpp

C++20/ggml implementations of audio embedding inference features:

• Supports WAV, MP3, and FLAC audio input.
• Automatically resamples audio to the model's required.
• Automatically mixes stereo and multichannel audio down to mono.

Supported models

Model	Text	Audio	Embedding dimensions	Sample rate
CLAP	✅	✅	512	48 kHz
CLaMP3	✅	✅	768	24 kHz
MERT v1 95M	❌	✅	768	24 kHz
MERT v1 330M	❌	✅	1,024	24 kHz
MERT v2 FullSong / 30s	❌	✅	1,024	24 kHz
MuQ	❌	✅	1,024	24 kHz
MuQ-MuLan	✅	✅	512	24 kHz

GGUF models are available at Audio Embedding GGUF collection.

Bindings

Binding	CPU	CUDA	Vulkan	Install
Python	✅	❌	❌	pip3 install clap-cpp
Rust	✅	❌	❌	cargo install clap-cpp
Node.js	✅	❌	❌	npm install clap-cpp

Usage

Embeddings command

Load a converted GGUF and generate real native text or audio embeddings:

clap embedding --model model.gguf --text 'A jazz quartet.'
clap embedding --model model.gguf --audio music.flac

Embeddings server

Start the server with a GGUF supporting the requested modality, such as CLAP, CLaMP3 or MuQ-MuLan (default listen address: 127.0.0.1:8080):

clap serve --model model.gguf --host 127.0.0.1 --port 8080

The model is loaded once at startup. Its API identifier is the GGUF filename (model.gguf in this example), not its full path or a Hugging Face repository ID. Send an OpenAI-compatible text embeddings request:

curl http://127.0.0.1:8080/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"model.gguf","input":["hello","world"],"encoding_format":"float"}'

Audio inputs use one input_audio content block per item. Encode the complete WAV, MP3 or FLAC file as standard base64 (no whitespace or data-URL prefix):

python3 -c 'import base64,json,sys; print(json.dumps({"input":[{"content":[{"type":"input_audio","input_audio":{"data":base64.b64encode(open(sys.argv[1],"rb").read()).decode()}}]}]}))' music.flac |
  curl http://127.0.0.1:8080/v1/embeddings \
    -H 'Content-Type: application/json' --data-binary @-

License

This project is licensed under the Apache License 2.0. Third-party dependencies retain their respective licenses.