DeepSeek R1 Distill Qwen 32B Deployment

DeepSeek-R1-Distill-Qwen-32B is the reasoning model most GigaGPU customers land on. It fits a single RTX 5090 at INT4 with headroom and delivers reasoning quality close to the full DeepSeek R1 on math, logic, and coding. Here is the real deployment.

Contents

• Hardware options
• Launch command
• Prompting for reasoning models
• Expected latency

Hardware

Launch

python -m vllm.entrypoints.openai.api_server \
  --model deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \
  --quantization awq \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.92 \
  --enable-prefix-caching \
  --served-model-name deepseek-r1-distill

32k max-model-len is important – reasoning models emit long thinking traces that consume context.

Prompting

Reasoning models work best with minimal system prompt interference. Use the model’s built-in template. The model emits <think>...</think> tags wrapping its reasoning before the final answer. Two handling patterns:

• Show thinking to the user – useful for debugging or trust
• Strip thinking tags before returning – cleaner UX

Client-side regex: /<think>[\s\S]*?<\/think>/g.

Latency

Reasoning models are slower because they generate 2-5x the tokens for the same final answer. A typical math problem on a 5090:

• Non-reasoning 32B: ~1.5 second response, 80 output tokens
• R1 Distill 32B: ~12 seconds, 700 output tokens (mostly thinking)

Plan SLAs accordingly. For latency-sensitive use cases run a non-reasoning model as the default and route to the R1 distill only when reasoning is needed.

See all distilled variants and Qwen 2.5 72B for the non-reasoning alternative.