Run DeepSeek on RTX 5090 (32GB VRAM Guide)

VRAM Check: DeepSeek on RTX 5090

The RTX 5090 is NVIDIA’s Blackwell flagship consumer GPU with 32 GB of GDDR7 VRAM and 1,792 GB/s bandwidth. That makes it the most powerful single consumer card for running DeepSeek models on a dedicated GPU server. Here is the VRAM breakdown for common DeepSeek variants:

At INT8, the 16B DeepSeek models fit comfortably with 12 GB to spare for KV cache and concurrent batching. FP16 is tight at 8K context but feasible with careful memory management. For complete sizing tables, see our DeepSeek VRAM requirements guide.

Setup with vLLM

vLLM with PagedAttention is the recommended serving framework for DeepSeek on the RTX 5090.

# Install vLLM
pip install vllm

# Launch DeepSeek-V2-Lite at INT8
python -m vllm.entrypoints.openai.api_server \
  --model deepseek-ai/DeepSeek-V2-Lite \
  --dtype float16 \
  --quantization gptq \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.90 \
  --port 8000

# Test the endpoint
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-ai/DeepSeek-V2-Lite",
    "messages": [{"role": "user", "content": "Explain mixture-of-experts architectures."}],
    "max_tokens": 512
  }'

For a breakdown of vLLM versus Ollama trade-offs, see our vLLM vs Ollama guide.

Setup with Ollama

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull and run DeepSeek
ollama run deepseek-v2:16b

# Serve as API
ollama serve &
curl http://localhost:11434/api/generate \
  -d '{"model": "deepseek-v2:16b", "prompt": "Write a Python quicksort implementation."}'

RTX 5090 Benchmark Results

Benchmarked with vLLM, 512-token input, 256-token output. The RTX 5090’s 1,792 GB/s GDDR7 bandwidth delivers class-leading inference speed. See the benchmark tool for current numbers.

At AWQ 4-bit, the RTX 5090 achieves 195 tok/s for a single user, nearly double the throughput of an RTX 3090 running the same model. The GDDR7 bandwidth advantage is most visible on decode-bound workloads.

Optimisation Tips

• Use INT8 for production to balance quality and throughput. The 32 GB VRAM comfortably fits INT8 DeepSeek with room for 16K+ context.
• Enable FP8 KV cache in vLLM to extend context length further without proportional VRAM growth.
• Use AWQ 4-bit when throughput matters more than quality, such as for batch processing or internal tooling.
• Set --gpu-memory-utilization 0.92 on the RTX 5090 to fully utilise the 32 GB pool.
• Pair with a second model such as Whisper for multi-modal pipelines, both fit in 32 GB at quantised precision.

Estimate costs with the cost-per-million-tokens calculator.

Next Steps

The RTX 5090 unlocks DeepSeek at higher precision and longer context than any other consumer GPU. For a head-to-head comparison, see our LLaMA 3 vs DeepSeek comparison. Compare the 5090 against other GPUs in our best GPU for LLM inference guide. For the full self-hosting walkthrough, read our self-host LLM guide.