Conda vs pip vs Docker for AI

You Picked the Wrong Dependency Tool and Now Nothing Works

Conda resolved CUDA 11.8 when you needed 12.1. pip installed a CPU-only PyTorch because the index URL was wrong. Docker pulled a 15GB image that duplicates drivers already on the host. Each tool manages AI dependencies differently, and the wrong choice creates hours of debugging on a dedicated GPU server. Here is when to use each and why.

pip + venv: Lightweight and Direct

Best for production inference servers where you control the base system:

# Create isolated environment
python3 -m venv /opt/envs/inference
source /opt/envs/inference/bin/activate

# Install PyTorch with specific CUDA version
pip install torch --index-url https://download.pytorch.org/whl/cu121
pip install vllm transformers accelerate

# Advantages:
# - Fast installs (wheels, no solver overhead)
# - Small disk footprint (~3-5GB for full vLLM stack)
# - Uses system CUDA drivers directly
# - Simple reproducibility with pip freeze

# Pin everything
pip freeze > requirements.lock

# Disadvantages:
# - Cannot manage CUDA toolkit or system libraries
# - Requires correct system CUDA pre-installed
# - No non-Python dependency management

# Verify
python3 -c "import torch; print(torch.cuda.is_available())"

Conda: Full Environment with CUDA Bundled

Best when you need specific CUDA toolkit versions without touching system packages:

# Install Miniforge (lighter than full Anaconda)
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh -b -p /opt/miniforge3
eval "$(/opt/miniforge3/bin/conda shell.bash hook)"

# Create environment with bundled CUDA toolkit
conda create -n training python=3.11 -y
conda activate training
conda install pytorch torchvision pytorch-cuda=12.1 -c pytorch -c nvidia -y

# Conda bundles CUDA toolkit inside the environment
du -sh /opt/miniforge3/envs/training/
# Expect 8-15GB (includes CUDA runtime)

# Advantages:
# - Bundles CUDA toolkit — no system CUDA dependency
# - Manages non-Python libs (NCCL, cuDNN, MKL)
# - Environment completely self-contained
# - SAT solver prevents conflicts

# Disadvantages:
# - Slow dependency resolution
# - Large disk usage (3-5x pip)
# - Duplicates CUDA libraries
# - Conda-forge packages lag behind PyPI

# Export for reproducibility
conda env export --no-builds > environment.yml

# Recreate on another server
conda env create -f environment.yml

Docker + NVIDIA Runtime: Full Isolation

Best for reproducible deployments and multi-tenant servers:

# Use NVIDIA's base images (CUDA + cuDNN pre-installed)
# Dockerfile for vLLM inference
FROM nvidia/cuda:12.1.1-devel-ubuntu22.04

RUN apt-get update && apt-get install -y python3 python3-pip && \
    rm -rf /var/lib/apt/lists/*

RUN pip3 install vllm torch --index-url https://download.pytorch.org/whl/cu121

EXPOSE 8000
CMD ["python3", "-m", "vllm.entrypoints.openai.api_server", \
     "--model", "/models/llama-3-70b", "--port", "8000"]

# Build and run with GPU access
docker build -t vllm-inference .
docker run --gpus '"device=0,1"' -p 8000:8000 \
    -v /opt/models:/models \
    vllm-inference

# Advantages:
# - Complete reproducibility across servers
# - Isolated from host system entirely
# - Easy rollback (just switch image tags)
# - Multi-tenant safe — resource limits built in

# Disadvantages:
# - Image sizes 10-20GB for AI stacks
# - Layer caching complexity with large models
# - Slight GPU overhead from container runtime
# - Requires NVIDIA Container Toolkit

# Resource limits
docker run --gpus '"device=0"' \
    --memory=32g --memory-swap=32g \
    --cpus=8 \
    -p 8000:8000 vllm-inference

Side-by-Side Comparison

# Disk usage comparison (vLLM + PyTorch + CUDA stack)
# pip + venv:    ~4 GB  (uses system CUDA)
# Conda:         ~12 GB (bundles CUDA toolkit)
# Docker image:  ~15 GB (full OS + CUDA + Python)

# Install speed (fresh environment)
# pip:    2-5 minutes
# Conda:  10-30 minutes (solver overhead)
# Docker: 5-15 minutes (pull or build)

# CUDA management
# pip:    Requires system CUDA pre-installed
# Conda:  Installs its own CUDA toolkit
# Docker: Uses NVIDIA base image CUDA

# Isolation level
# pip:    Python packages only
# Conda:  Python + system libraries
# Docker: Complete OS-level isolation

# Best for...
# pip:    Production inference (single-tenant, controlled base)
# Conda:  Research/training (need multiple CUDA versions)
# Docker: Multi-tenant, CI/CD, reproducible deployments

Practical Recommendation

# Production inference server: pip + venv
# Simple, fast, low overhead
python3 -m venv /opt/envs/vllm && source /opt/envs/vllm/bin/activate
pip install vllm && pip freeze > /opt/envs/vllm/requirements.lock

# Training workstation with experiments: Conda
# Need to test across CUDA versions
conda create -n exp-cuda121 pytorch pytorch-cuda=12.1 -c pytorch -c nvidia
conda create -n exp-cuda118 pytorch pytorch-cuda=11.8 -c pytorch -c nvidia

# Multi-tenant or CI/CD: Docker
# Different users, different models, need isolation
docker compose up -d vllm-service ollama-service comfyui-service

Choose the dependency tool that matches your GPU server workload. For production vLLM inference, follow the production deployment guide. Install PyTorch correctly using our GPU setup guide. Run Ollama with proper isolation. Check monitoring setup regardless of which tool you choose. More in our infrastructure articles and tutorials.