Table of Contents
"Real-time transcription" covers two distinct use cases. Live captioning of streaming audio is one (sub-second time-to-first-word). Transcribing pre-recorded audio faster than wall-clock is another (real-time factor >1×). The right GPU depends on which one you need.
For live captioning with sub-1s latency: RTX 5080 16 GB or larger using Whisper-Streaming. For batch transcription at 5-9× real-time: any 16+ GB Blackwell card. The cheapest credible real-time deployment is the RTX 5060 Ti 16 GB at £119/mo using faster-whisper-large-v3 INT8.
What "real-time" means for STT
Two definitions:
- Real-time factor (RTF): how many seconds of audio the model transcribes per second of wall time. RTF=1 = real-time. RTF=5 = 5× faster than real-time.
- Time-to-first-word (TTFW): how long after speech starts before the first word is emitted. The user-visible latency for live captioning.
For batch transcription you want high RTF. For live captioning you want low TTFW. The two have different optimal configurations.
Backends: faster-whisper, Whisper-Streaming, WhisperX
- faster-whisper (CTranslate2) — 4× faster than reference Whisper, half the VRAM. Default for batch.
- Whisper-Streaming — sliding-window streaming on top of Whisper. The right tool for live captioning. Sub-1s TTFW achievable.
- WhisperX — Whisper + Silero VAD + pyannote diarization. Adds ~200ms but gives speaker labels.
- Distil-Whisper — distilled small/medium variants. ~2× faster than Whisper Large-v3, English-only, <1% WER regression.
Real-time targets by GPU
| GPU | Whisper Large-v3 RTF | TTFW (Whisper-Streaming) | Concurrent streams |
|---|---|---|---|
| RTX 3050 6 GB | ~3× | ~1.2 s | 1-2 |
| RTX 3060 12 GB | ~4× | ~900 ms | 2-3 |
| RTX 5060 Ti 16 GB | ~5× | ~700 ms | 3-5 |
| RTX 5080 16 GB | ~7× | ~500 ms | 5-8 |
| RTX 5090 32 GB | ~9× | ~400 ms | 12-16 |
| RTX 6000 Pro | ~9× | ~380 ms | 20+ (with multi-instance) |
Latency budget breakdown
For a sub-1s TTFW live caption pipeline:
| Stage | Latency |
|---|---|
| Audio capture buffer | ~100 ms |
| VAD endpoint detection | ~100 ms |
| Whisper inference (sliding window) | ~250-500 ms |
| Network return | ~50 ms |
| Total TTFW | ~500-750 ms |
Verdict
- Live captioning, sub-1s: RTX 5080 or 5090 with Whisper-Streaming.
- Batch transcription, high throughput: any 16+ GB Blackwell card.
- Voice agent (STT + LLM + TTS): RTX 5090 32 GB. Stack all three on one card.
- Cheapest credible deployment: RTX 5060 Ti 16 GB.
Bottom line
Whisper is fast. Real-time is achievable on most modern GPUs. The right card depends on whether you need single-stream low-latency (5080/5090) or multi-stream high-throughput (5090/6000 Pro). See Whisper hosting for the deployment-side guide.
