Deploy DeepSeek on a Dedicated GPU Server

Why Self-Host DeepSeek?

DeepSeek’s open-weight models — including DeepSeek-R1 and DeepSeek-V3 — deliver reasoning and code generation quality that rivals closed-source alternatives. Running DeepSeek on a dedicated GPU server gives you full control over data privacy, zero per-token API fees, and consistent low-latency inference without cold starts. For teams processing sensitive data or operating under GDPR, self-hosted DeepSeek is the production-ready choice.

This guide walks you through every step: choosing the right GPU, installing your inference engine, downloading model weights, and exposing an OpenAI-compatible API. Whether you pick vLLM or Ollama, you’ll have DeepSeek running in under 30 minutes. Not sure which engine to use? Our vLLM vs Ollama comparison breaks down the trade-offs.

GPU & VRAM Requirements

VRAM is the primary constraint when deploying DeepSeek. The table below shows what you need for each model variant. All figures assume FP16 precision unless noted otherwise.

For single-GPU deployments, the distilled 7B and 14B variants offer the best balance of quality and efficiency. See our best GPU for LLM inference guide for detailed benchmarks across model sizes.

Server Selection & OS Setup

Start by provisioning a dedicated GPU server. For the 14B distilled model at full precision, an RTX 3090 with 24 GB VRAM is ideal. For quantised deployments, an RTX 4060 Ti works well for the 7B model.

Once your server is provisioned, SSH in and prepare the environment:

# Update system packages
sudo apt update && sudo apt upgrade -y

# Install NVIDIA drivers (if not pre-installed)
sudo apt install -y nvidia-driver-550
sudo reboot

# Verify GPU is detected
nvidia-smi

# Install Python 3.11 and pip
sudo apt install -y python3.11 python3.11-venv python3-pip

# Create a virtual environment
python3.11 -m venv ~/deepseek-env
source ~/deepseek-env/bin/activate

Confirm your GPU shows the expected VRAM in the nvidia-smi output before proceeding.

Option A: Deploy with vLLM

vLLM is the recommended engine for production deployments. It supports continuous batching, PagedAttention, and exposes an OpenAI-compatible API out of the box.

# Install vLLM
pip install vllm

# Launch DeepSeek-R1-Distill-Qwen-14B
vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-14B \
  --host 0.0.0.0 \
  --port 8000 \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.90 \
  --dtype auto

vLLM will automatically download the model weights from Hugging Face on first launch. For the 14B model, expect roughly 28 GB of downloads. The server is ready when you see Uvicorn running on http://0.0.0.0:8000.

To run the 7B distilled variant with 4-bit quantisation instead:

vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-7B \
  --host 0.0.0.0 \
  --port 8000 \
  --quantization awq \
  --max-model-len 4096 \
  --gpu-memory-utilization 0.90

Option B: Deploy with Ollama

Ollama is a simpler alternative that handles model downloading and serving in a single binary. It is best for development, prototyping, and single-user workloads.

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull and run DeepSeek-R1 14B distilled
ollama pull deepseek-r1:14b
ollama serve &

# Test with a quick prompt
ollama run deepseek-r1:14b "Explain the chain-of-thought in your reasoning."

By default, Ollama listens on localhost:11434. To expose it on all interfaces for remote access, set the environment variable before starting:

OLLAMA_HOST=0.0.0.0:11434 ollama serve

Configure & Test the API

Both vLLM and Ollama expose OpenAI-compatible endpoints. Here is how to test your deployment with curl:

# Test vLLM endpoint
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-ai/DeepSeek-R1-Distill-Qwen-14B",
    "messages": [
      {"role": "user", "content": "What are the benefits of mixture-of-experts models?"}
    ],
    "max_tokens": 512,
    "temperature": 0.7
  }'

# Test Ollama endpoint
curl -X POST http://localhost:11434/api/chat \
  -d '{
    "model": "deepseek-r1:14b",
    "messages": [
      {"role": "user", "content": "What are the benefits of mixture-of-experts models?"}
    ]
  }'

For production, add a reverse proxy (nginx or Caddy) with TLS, rate limiting, and API key authentication in front of your inference endpoint. You should also set up a systemd service so the model restarts automatically on reboot.

Performance Tuning & Next Steps

Once DeepSeek is running, optimise your deployment:

• Benchmark throughput — Use our tokens per second benchmark tool to compare your results against baseline numbers for your GPU.
• Estimate costs — Calculate your effective per-token cost with the LLM cost calculator. Self-hosting DeepSeek typically costs 5-10x less than API pricing at scale.
• GPU upgrades — If you need more throughput, review our RTX 3090 vs RTX 5090 comparison to decide whether upgrading is worth it for your workload.
• Quantisation trade-offs — 4-bit quantisation cuts VRAM usage by 60-70% with a modest quality drop. For most inference tasks, GPTQ or AWQ quantised models perform within 2-3% of full precision.
• Scaling — For the full 671B DeepSeek-R1 model, you’ll need multi-GPU tensor parallelism. vLLM handles this natively with --tensor-parallel-size.

For a broader overview of self-hosting options, read our complete self-host LLM guide. You can also explore more deployment walkthroughs in the model guides category.