RTX 5070 for Whisper and Speech AI: Voice Pipelines on 12 GB GDDR7

Whisper on RTX 5070

OpenAI’s Whisper runs natively on CUDA and is trivially small relative to the RTX 5070’s 12 GB. Even the largest variant (Large-v3, 1.55B parameters) uses only ~3 GB of VRAM at FP16 — leaving 9 GB free for other models running simultaneously.

The recommended backend is Faster-Whisper with CTranslate2 (INT8 quantised), which delivers 70–90× real-time transcription speed on the RTX 5070. For production transcription pipelines, use WhisperX which adds word-level timestamps and speaker diarisation.

Whisper Model Sizes

For production transcription, large-v3 or large-v3-turbo are the recommended choices. Large-v3-turbo is ~4× faster with minimal quality loss — a strong default for real-time applications.

Faster-Whisper Setup

pip install faster-whisper

from faster_whisper import WhisperModel

# Load large-v3 in INT8 on CUDA (uses ~1.5 GB VRAM)
model = WhisperModel("large-v3", device="cuda", compute_type="int8")

# Transcribe an audio file
segments, info = model.transcribe("audio.mp3", beam_size=5)

print(f"Detected language: {info.language} (probability: {info.language_probability:.2f})")
for segment in segments:
    print(f"[{segment.start:.2f}s -> {segment.end:.2f}s] {segment.text}")

With compute_type="int8", the model is quantised to INT8 at load time — VRAM drops from ~3 GB to ~1.5 GB, and speed increases ~2× on CUDA. For the best accuracy, use compute_type="float16" (~3 GB).

For real-time streaming transcription (microphone input), use faster_whisper with a sliding window buffer or the RealtimeSTT library which wraps it:

pip install RealtimeSTT

from RealtimeSTT import AudioToTextRecorder

recorder = AudioToTextRecorder(model="large-v3", language="en", device="cuda")

def process_text(text):
    print(f"Transcribed: {text}")

recorder.text(process_text)

Full Voice Pipeline: STT + LLM + TTS

The RTX 5070’s 12 GB supports a complete voice AI pipeline running entirely on GPU:

# Component VRAM budget:
# Whisper Large-v3 INT8: ~1.5 GB
# Llama 3.1 8B Q4_K_M: ~5.2 GB
# XTTS-v2 TTS: ~2.5 GB
# Total: ~9.2 GB — fits comfortably in 12 GB

# 1. STT: Faster-Whisper
from faster_whisper import WhisperModel
stt = WhisperModel("large-v3", device="cuda", compute_type="int8")

# 2. LLM: llama-cpp-python or Ollama
import ollama
# ollama.pull("llama3.1:8b")  # run once

# 3. TTS: XTTS-v2 via TTS library
from TTS.api import TTS
tts = TTS("tts_models/multilingual/multi-dataset/xtts_v2").to("cuda")

def voice_pipeline(audio_file, output_file="response.wav"):
    # STT
    segments, _ = stt.transcribe(audio_file)
    user_text = " ".join(s.text for s in segments)
    print(f"User said: {user_text}")

    # LLM
    response = ollama.chat(model="llama3.1:8b",
                           messages=[{"role": "user", "content": user_text}])
    reply = response["message"]["content"]
    print(f"Response: {reply}")

    # TTS
    tts.tts_to_file(text=reply, speaker_wav="reference.wav",
                    language="en", file_path=output_file)
    return output_file

VRAM Budget for Multi-Model Pipelines

Transcription Speed

Faster-Whisper with CTranslate2 on the RTX 5070 (INT8, CUDA):

• large-v3 INT8: ~80–90× realtime — a 10-minute recording transcribes in ~7–8 seconds
• large-v3-turbo INT8: ~300–350× realtime — same 10-minute recording in ~2 seconds
• medium INT8: ~400× realtime — nearly instantaneous

For batch transcription jobs (processing audio archives), run Whisper directly from the CLI and process files in parallel across the GPU’s compute budget. The RTX 5070’s 6,144 CUDA cores handle batch workloads efficiently.