Connect Ansible to Automate GPU Server Setup

What You’ll Connect

After this guide, your GPU server deployments will be fully automated through Ansible — no manual SSH sessions, no missed configuration steps. A single playbook command installs NVIDIA drivers, sets up Docker, deploys your vLLM or Ollama inference server, and configures the Nginx reverse proxy on dedicated GPU hardware.

The integration uses Ansible roles to modularise each deployment step. Whether you are setting up one server or twenty, the same playbook produces identical configurations — eliminating drift between environments and making disaster recovery a single command.

Ansible Playbook –> SSH to GPU Server(s) –> Configured Server (your laptop reads inventory executes tasks vLLM running, or CI runner) runs roles installs packages Nginx configured, deploys containers API secured | ansible-playbook –> Role: nvidia_drivers –> Role: vllm_deploy –> Role: nginx_proxy site.yml installs CUDA pulls model, TLS, reverse proxy configures GPU starts container to inference port –>

Prerequisites

• A GigaGPU server with SSH access and a user with sudo privileges
• Ansible 2.15+ installed on your control machine: pip install ansible
• SSH key-based authentication configured between your control machine and the GPU server
• Python 3 on the target GPU server (typically pre-installed on Ubuntu)

Integration Steps

Create an Ansible inventory file listing your GPU server(s) with connection details. Group servers by role (inference, training, etc.) for targeted playbook runs. Store sensitive values like API keys in an Ansible Vault-encrypted file.

Structure your playbook into roles: nvidia_drivers (installs CUDA toolkit and GPU drivers), docker (installs Docker and NVIDIA Container Toolkit), vllm_deploy (pulls the model and starts the inference container), and nginx_proxy (configures TLS and reverse proxy). Each role is idempotent — running the playbook again only changes what has drifted.

Create a main playbook (site.yml) that applies these roles in order. The playbook references variables for the model name, GPU API key, domain name, and other configuration that differs between environments.

Code Example

Ansible playbook and role tasks for deploying an AI inference server following our vLLM production guide:

# inventory.yml
gpu_servers:
  hosts:
    inference-1:
      ansible_host: gpu-server.gigagpu.com
      ansible_user: root
      model_name: "meta-llama/Llama-3-70b-chat-hf"
      gpu_api_key: "{{ vault_gpu_api_key }}"
      domain: "ai.yourdomain.com"

# site.yml
- hosts: gpu_servers
  roles:
    - nvidia_drivers
    - docker
    - vllm_deploy
    - nginx_proxy

# roles/vllm_deploy/tasks/main.yml
- name: Pull vLLM Docker image
  community.docker.docker_image:
    name: vllm/vllm-openai
    tag: latest
    source: pull

- name: Start vLLM inference container
  community.docker.docker_container:
    name: vllm-inference
    image: vllm/vllm-openai:latest
    state: started
    restart_policy: unless-stopped
    device_requests:
      - driver: nvidia
        count: -1
        capabilities: [["gpu"]]
    published_ports:
      - "127.0.0.1:8000:8000"
    command: >
      --model {{ model_name }}
      --api-key {{ gpu_api_key }}
      --max-model-len 8192

- name: Wait for vLLM to load model
  ansible.builtin.uri:
    url: "http://127.0.0.1:8000/v1/models"
    headers:
      Authorization: "Bearer {{ gpu_api_key }}"
  register: result
  until: result.status == 200
  retries: 30
  delay: 10

Testing Your Integration

Run the playbook in check mode first: ansible-playbook site.yml --check --diff. This previews all changes without applying them. Then run for real: ansible-playbook site.yml. Monitor the output for each task’s status — green (ok), yellow (changed), or red (failed).

After the playbook completes, test the inference endpoint: curl https://ai.yourdomain.com/v1/models. Verify the model is loaded and serving. Run the playbook a second time to confirm idempotency — everything should show “ok” with no changes.

Production Tips

Use Ansible tags to run specific roles independently: ansible-playbook site.yml --tags vllm_deploy updates the model deployment without reinstalling drivers. This speeds up model swaps and configuration changes.

For multi-server deployments, Ansible’s serial execution option lets you update servers in rolling batches — take one offline, update it, verify it, then proceed to the next. This maintains availability across your GPU fleet during updates.

Combine Ansible with a CI/CD pipeline (GitHub Actions or GitLab CI) so merging a change to the playbook repository automatically applies it to your infrastructure. For teams managing open-source AI deployments on dedicated GPUs, this creates a fully automated model deployment workflow. Secure your endpoints with our API security guide, explore more tutorials, or get started with GigaGPU to automate your GPU server management.