A text-to-speech and speech-to-text server compatible with the OpenAI API, powered by backend support from Whisper, FunASR, Bark, and CosyVoice.

• Python 3.10 or greater
• Support Nvidia GPU, requires the following NVIDIA libraries to be installed:
  • cuBLAS for CUDA 12
  • cuDNN 9 for CUDA 12

You can install the project using pip:

pip install vox-box

# For MacOS, you need to manually install `openfst`, `pynini`, and `wetextprocessing` after installing `vox-box` to make `cosyvoice` work:
brew install openfst
export CPLUS_INCLUDE_PATH=$(brew --prefix openfst)/include
export LIBRARY_PATH=$(brew --prefix openfst)/lib
pip install pynini==2.1.6
pip install wetextprocessing==1.0.4.1

vox-box start --huggingface-repo-id Systran/faster-whisper-small --data-dir ./cache/data-dir --host 0.0.0.0 --port 80

# Windows
vox-box start --huggingface-repo-id Systran/faster-whisper-small --data-dir C:\Users\michelia\AppData\Roaming\vox-box --host 0.0.0.0 --port 8082

• -d, --debug: Enable debug mode.
• --host: Host to bind the server to. Default is 0.0.0.0.
• --port: Port to bind the server to. Default is 80.
• --model: model path.
• --device: Binding device, e.g., cuda:0. Default is cpu.
• --huggingface-repo-id: Huggingface repo id for the model.
• --model-scope-model-id: Model scope model id for the model.
• --data-dir: Directory to store downloaded model data. Default is OS specific.

Endpoint: POST /v1/audio/speech

Generates audio from the input text. Compatible with the OpenAI audio/speech API.

Example Request:

curl http://localhost/v1/audio/speech \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cosyvoice",
    "input": "Hello world",
    "voice": "English Female"
  }' \
  --output speech.mp3

Response: The audio file content.

Endpoint: POST /v1/audio/transcriptions

Transcribes audio into the input language. Compatible with the OpenAI audio/transcription API.

Example Request:

curl https://localhost/v1/audio/transcriptions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: multipart/form-data" \
  -F file="@/path/to/file/audio.mp3" \
  -F model="whisper-large-v3"

Response:

{
  "text": "Hello world."
}

Endpoint: GET /v1/models

Returns the current running models.

Endpoint: GET /v1/models/{model_id}

Returns the current running model.

Endpoint: GET /v1/voices

Returns the supported voice for current running model.

Endpoint: GET /health

Returns the heath check result of the Vox Box.