SSL/TLS for AI APIs: Let’s Encrypt + Nginx

Your AI API Is Transmitting Tokens in Plaintext

Every request to your inference endpoint travels unencrypted when you expose vLLM or Ollama on a raw port. API keys, prompt content, and model responses are visible to anyone sniffing the network path. Beyond security, most client libraries reject plaintext HTTP endpoints by default, and browsers block mixed content. A dedicated GPU server running production AI workloads needs TLS termination with automated certificate management.

Install Certbot and Obtain Certificates

Let’s Encrypt provides free, automated TLS certificates. Certbot handles issuance and renewal:

# Install Certbot with Nginx plugin
sudo apt update && sudo apt install -y certbot python3-certbot-nginx

# Obtain certificate for your inference domain
sudo certbot certonly --nginx \
    -d inference.example.com \
    --non-interactive \
    --agree-tos \
    -m [email protected]

# Verify certificate files
sudo ls -la /etc/letsencrypt/live/inference.example.com/
# fullchain.pem  privkey.pem  chain.pem  cert.pem

# Test automatic renewal
sudo certbot renew --dry-run

Certbot creates a systemd timer by default. Certificates renew automatically 30 days before expiry.

Nginx TLS Reverse Proxy for Inference

Terminate TLS at Nginx and proxy to your local inference process:

# /etc/nginx/sites-available/ai-api
upstream inference_backend {
    server 127.0.0.1:8000;  # vLLM
    keepalive 64;
}

server {
    listen 443 ssl http2;
    server_name inference.example.com;

    ssl_certificate /etc/letsencrypt/live/inference.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/inference.example.com/privkey.pem;

    # TLS 1.3 only — drop older protocols
    ssl_protocols TLSv1.3;
    ssl_prefer_server_ciphers off;

    # OCSP stapling for faster handshakes
    ssl_stapling on;
    ssl_stapling_verify on;
    resolver 8.8.8.8 8.8.4.4 valid=300s;

    # Security headers
    add_header Strict-Transport-Security "max-age=63072000" always;
    add_header X-Content-Type-Options nosniff;

    location /v1/ {
        proxy_pass http://inference_backend;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Connection "";
        proxy_read_timeout 300s;
        proxy_send_timeout 300s;
    }
}

# Redirect HTTP to HTTPS
server {
    listen 80;
    server_name inference.example.com;
    return 301 https://$host$request_uri;
}

# Enable the site and test
sudo ln -s /etc/nginx/sites-available/ai-api /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx

TLS Configuration for Streaming Responses

LLM inference uses server-sent events (SSE) for token streaming. Standard proxy buffering breaks this:

# Add inside the location /v1/ block for streaming support
location /v1/chat/completions {
    proxy_pass http://inference_backend;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
    proxy_set_header Connection "";

    # Disable buffering for SSE streaming
    proxy_buffering off;
    proxy_cache off;
    chunked_transfer_encoding on;

    # Long timeout for streaming completions
    proxy_read_timeout 600s;
}

# For Ollama streaming endpoint
upstream ollama_backend {
    server 127.0.0.1:11434;
    keepalive 32;
}

location /ollama/api/ {
    proxy_pass http://ollama_backend/api/;
    proxy_buffering off;
    proxy_read_timeout 600s;
}

Automated Renewal and Zero-Downtime Reload

Ensure certificates renew without dropping active inference connections:

# Check existing renewal timer
sudo systemctl list-timers | grep certbot

# Custom renewal hook to reload Nginx gracefully
sudo mkdir -p /etc/letsencrypt/renewal-hooks/deploy
cat <<'EOF' | sudo tee /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh
#!/bin/bash
nginx -t && systemctl reload nginx
logger "Let's Encrypt: certificate renewed and Nginx reloaded"
EOF
sudo chmod +x /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh

# Verify renewal works end-to-end
sudo certbot renew --dry-run

# Monitor certificate expiry with a simple check
echo | openssl s_client -servername inference.example.com \
    -connect inference.example.com:443 2>/dev/null | \
    openssl x509 -noout -dates

Verify TLS Configuration

# Test the endpoint over HTTPS
curl -v https://inference.example.com/v1/models

# Check TLS version negotiated
curl -sI https://inference.example.com/v1/models 2>&1 | grep -i tls

# Verify no plaintext access
curl -s http://inference.example.com/v1/models
# Should get 301 redirect

# Benchmark TLS overhead on inference latency
for i in $(seq 1 10); do
    curl -o /dev/null -s -w "connect: %{time_connect}s, ttfb: %{time_starttransfer}s\n" \
        https://inference.example.com/v1/models
done

TLS adds roughly 1-3ms per request, negligible compared to inference time. With certificates automated, your GPU server API stays encrypted without manual intervention. For the full vLLM proxy setup, see the production deployment guide. Secure Ollama endpoints the same way. Monitor certificate health alongside GPU metrics. More configuration in our infrastructure guides and tutorials.