Build AI Text-to-Speech API on GPU

What You’ll Build

In 30 minutes, you will have a production text-to-speech API that accepts text and returns natural-sounding audio with controllable speed, pitch, and voice selection. Running open-source TTS models on a dedicated GPU server, your API synthesises one minute of speech in under two seconds with support for 15+ languages, voice cloning from short reference clips, and real-time audio streaming for interactive applications.

Cloud TTS services charge $4-$16 per million characters and impose voice customisation limitations. At production volumes — generating audiobook chapters, IVR prompts, or in-app narration — costs escalate quickly. Self-hosted TTS on GPU delivers studio-quality synthesis with unlimited characters, custom voice models, and zero per-request fees.

Architecture Overview

The API wraps a neural TTS model behind FastAPI with streaming audio support. Text input passes through a preprocessing stage that handles SSML tags, number normalisation, and abbreviation expansion. The TTS model generates mel spectrograms from text, and a vocoder converts spectrograms to waveform audio. For streaming, the pipeline produces audio chunks as they are generated, enabling sub-second time-to-first-byte.

Voice cloning accepts a 10-30 second reference audio clip and extracts speaker embeddings for zero-shot synthesis. The API layer manages voice profiles, allowing clients to register custom voices and reference them by ID in subsequent requests. Pair with Whisper for a complete speech pipeline — transcription in, synthesis out.

GPU Requirements

Modern TTS models like XTTS-v2 use 3-5GB VRAM, leaving ample room for concurrent synthesis or co-hosting with other models. The same GPU can run TTS alongside an LLM for conversational AI with voice output. See our self-hosted model guide for multi-model deployments.

Step-by-Step Build

Deploy the TTS model on your GPU server and build the synthesis API with streaming support.

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from TTS.api import TTS
import io, torch, soundfile as sf

app = FastAPI()
tts = TTS("tts_models/multilingual/multi-dataset/xtts_v2").to("cuda")

@app.post("/v1/audio/speech")
async def synthesise(text: str, voice: str = "default",
                     language: str = "en", speed: float = 1.0):
    wav = tts.tts(
        text=text,
        speaker=voice,
        language=language,
        speed=speed
    )

    buf = io.BytesIO()
    sf.write(buf, wav, samplerate=24000, format="WAV")
    buf.seek(0)
    return StreamingResponse(buf, media_type="audio/wav")

@app.post("/v1/audio/speech/clone")
async def clone_voice(text: str, reference_audio: bytes,
                      language: str = "en"):
    # Save reference to temp file for speaker embedding extraction
    wav = tts.tts(
        text=text,
        speaker_wav="ref_speaker.wav",
        language=language
    )
    buf = io.BytesIO()
    sf.write(buf, wav, samplerate=24000, format="WAV")
    buf.seek(0)
    return StreamingResponse(buf, media_type="audio/wav")

Add SSML parsing for fine-grained control over pauses, emphasis, and pronunciation. The API accepts an OpenAI-compatible format so existing client code works by changing the base URL. See production deployment patterns for load balancing across concurrent synthesis requests.

Voice Management and Quality

Register custom voices by uploading reference clips through a management endpoint. The system extracts speaker embeddings and stores them for future synthesis requests. For production voice quality, use reference clips recorded in quiet environments with clear speech. XTTS-v2 achieves natural-sounding clones from as little as 15 seconds of reference audio.

Monitor synthesis quality by tracking mean opinion scores on sample outputs. Log request patterns to identify peak usage windows and pre-warm voice models accordingly. For audiobook or podcast generation, batch process chapters overnight and deliver finished audio by morning.

Deploy Your TTS API

A self-hosted TTS API eliminates per-character billing while giving you full control over voice models, languages, and audio quality. Power IVR systems, accessibility features, content creation, or conversational AI. Launch on GigaGPU dedicated GPU hosting and start synthesising. Browse more API use cases and tutorials in our library.