DeepSeek V3 Distilled Models – Self-Hosted Options

DeepSeek V3 and R1 at full size require datacenter infrastructure. The distilled variants – where DeepSeek’s reasoning behaviour is trained into smaller base models like Llama 3 and Qwen – are the realistic self-hosting path on our dedicated GPU hosting.

Contents

• The distilled variants
• VRAM requirements
• Quality versus full model
• Deployment

Variants

The practical distilled models you can host:

• DeepSeek-R1-Distill-Qwen-1.5B
• DeepSeek-R1-Distill-Qwen-7B
• DeepSeek-R1-Distill-Qwen-14B
• DeepSeek-R1-Distill-Qwen-32B – best quality-to-size ratio
• DeepSeek-R1-Distill-Llama-8B
• DeepSeek-R1-Distill-Llama-70B

VRAM

Quality vs Full Model

Distilled models retain 70-90% of the reasoning quality of the teacher on benchmarks like MATH and GPQA. The 32B distill punches well above its weight on math and logic tasks. For most teams it is the right target – enough quality for production, small enough for a single card.

Deployment

32B Qwen distill on RTX 5090:

python -m vllm.entrypoints.openai.api_server \
  --model deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \
  --quantization awq \
  --max-model-len 16384 \
  --gpu-memory-utilization 0.92

Note that reasoning models emit long “thinking” traces before answers. Budget 2-4x the output tokens you would plan for a non-reasoning model.

For the 32B variant specifically see DeepSeek R1 Distill Qwen 32B. For coding see DeepSeek Coder V2.