Whisper Tiny vs Base vs Small Speed by GPU

Whisper Model Size Benchmark Overview

OpenAI Whisper comes in five sizes from Tiny (39M parameters) to Large (1.5B parameters). The smaller variants are dramatically faster at the cost of some accuracy. Running Whisper on a dedicated GPU server ensures consistent transcription speed for real-time and batch processing.

We benchmarked Whisper Tiny, Base, and Small on GigaGPU servers using faster-whisper with FP16 precision on a 10-minute English audio sample. Results are expressed as Real-Time Factor (RTF), where lower is faster (RTF 0.1x means 10x faster than real-time). For larger model benchmarks, see our Whisper Large-v3 RTF benchmark and Whisper Medium benchmark.

Real-Time Factor by GPU and Model Size

All three models run far faster than real-time on every GPU. Whisper Tiny on the RTX 5090 processes audio at 333x real-time, meaning a 1-hour recording completes in about 11 seconds. Even the RTX 3050 handles Tiny at 36x real-time. VRAM usage ranges from ~0.3GB (Tiny) to ~1GB (Small), so VRAM is never a constraint for these variants.

Accuracy vs Speed Trade-off

Smaller Whisper models trade accuracy for speed. Below we compare Word Error Rate (WER) on the LibriSpeech test-clean dataset.

Whisper Small at 3.4% WER is accurate enough for most production applications while being 2.5x faster than Large-v3. Tiny is best suited for real-time voice activity detection or applications where speed matters more than perfect transcription. For detailed VRAM analysis, see our Whisper VRAM requirements guide.

Cost Efficiency Analysis

The RTX 3090 and RTX 5080 offer nearly identical cost efficiency for Whisper Small. The RTX 3090 is cheaper overall, while the RTX 5080 provides more absolute speed.

GPU Recommendations

• Budget: RTX 4060 — Whisper Small at 18x real-time is more than enough for batch transcription.
• Best value: RTX 3090 — best cost efficiency with 38x real-time on Small.
• Real-time pipelines: RTX 5080 — 55x real-time on Small with VRAM left for an LLM alongside.
• Maximum speed: RTX 5090 — 83x real-time for high-volume transcription services.

For Whisper combined with LLM inference, see the RTX 5080 Whisper + LLM guide or RTX 5090 DeepSeek + Whisper guide. For larger Whisper models, check the Large-v3 benchmark. Browse all results in the Benchmarks category.

Conclusion

Whisper Tiny, Base, and Small all run far faster than real-time on modern GPUs, making them viable even on entry-level hardware. The choice between model sizes should be driven by accuracy requirements, not speed constraints. For most production transcription, Whisper Small offers the best accuracy-to-speed ratio, while Tiny is ideal for real-time streaming where minimal latency is critical.