Table of Contents
The 5090's Blackwell silicon needs the right vLLM flags to actually deliver its 2× FP8 throughput. Defaults leave money on the table.
TL;DR
5090 vLLM config: FP8 weights, FP8 KV cache, max-num-seqs=128, max-model-len=32768, prefix caching, optional speculative decoding. ~1,920 tok/s on Mistral 7B FP8.
Install
pip install vllm==0.6.3
# 5090 needs NVIDIA driver 555+ (Blackwell)Optimal config
vllm serve mistralai/Mistral-7B-Instruct-v0.3 \
--quantization fp8 \
--kv-cache-dtype fp8_e4m3 \
--max-model-len 32768 \
--max-num-seqs 128 \
--gpu-memory-utilization 0.92 \
--enable-prefix-cachingSpeculative decoding option
# Add for ~1.5× speedup on chat workloads
--speculative-model meta-llama/Llama-3.2-1B-Instruct \
--num-speculative-tokens 5Verdict
The 5090 is the best per-pound flagship for AI inference in 2026. This config gets you ~3× the throughput of naive defaults.
Bottom line
5090 + FP8 + spec decoding = best production config. See 5090 spec breakdown.
