Connect Postman to Self-Hosted AI API on GPU

What You’ll Connect

After this guide, your Postman workspace will have a complete collection for testing your GPU-hosted AI API — no API costs, no rate limits. You will be able to send chat completions, test streaming, benchmark latency, and validate responses against your vLLM or Ollama endpoint running on dedicated GPU hardware.

The integration uses Postman’s environment variables, pre-request scripts, and test scripts to create a professional API testing workflow. Share the collection with your development team so everyone can debug and validate the inference API consistently.

HTTP Request –> GPU Server (vLLM/Ollama) (collection + POST/GET to OpenAI-compatible API environment) /v1/* endpoints on dedicated GPU | | Test results <-- Response viewer <-- JSON response with pass/fail + timing data completion data -->

Prerequisites

• A GigaGPU server with a running LLM behind an OpenAI-compatible API (vLLM production guide)
• Postman desktop app or web version (free tier works)
• HTTPS access to your GPU endpoint (Nginx proxy guide)
• Your API key for the inference server (security guide)

Integration Steps

Create a new Postman environment called “GPU AI Server.” Add two variables: base_url (set to https://your-gpu-server.gigagpu.com) and api_key (set to your GPU API key, marked as “secret” type to hide the value).

Create a new collection called “Self-Hosted AI API.” Add the Authorization header at the collection level: Bearer {{api_key}}. This applies to all requests in the collection automatically. Set the Content-Type header to application/json.

Add requests for each OpenAI-compatible endpoint: GET {{base_url}}/v1/models to list available models, POST {{base_url}}/v1/chat/completions for chat inference, and POST {{base_url}}/v1/completions for text completion. Each request includes a test script to validate response structure.

Code Example

Configure the chat completions request body and Postman test script for your FastAPI inference server:

// POST {{base_url}}/v1/chat/completions
// Request Body (raw JSON):
{
  "model": "meta-llama/Llama-3-70b-chat-hf",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the capital of France?"}
  ],
  "max_tokens": 100,
  "temperature": 0.1
}

// Postman Test Script (Tests tab):
pm.test("Status is 200", () => pm.response.to.have.status(200));

pm.test("Response has choices", () => {
  const json = pm.response.json();
  pm.expect(json).to.have.property("choices");
  pm.expect(json.choices).to.be.an("array").that.is.not.empty;
});

pm.test("Completion has content", () => {
  const content = pm.response.json().choices[0].message.content;
  pm.expect(content).to.be.a("string").that.is.not.empty;
});

pm.test("Response time under 10s", () => {
  pm.expect(pm.response.responseTime).to.be.below(10000);
});

// Pre-request Script (for latency benchmarking):
pm.variables.set("request_start", Date.now());

Testing Your Integration

Select your “GPU AI Server” environment and send the GET /v1/models request first to verify connectivity. The response should list the models your vLLM instance serves. Then send the chat completions request — all four test assertions should pass with green checkmarks.

Use Postman’s Collection Runner to execute all requests sequentially and generate a test report. This is useful for validating your GPU server after deployments or configuration changes. Export the collection to share with teammates.

Production Tips

Create multiple Postman environments — one for staging and one for production — each with different base_url values. Switch environments to test the same requests against different GPU servers without editing URLs.

Add a “Streaming” request that tests server-sent events. Set the response type to “Stream” in Postman and send a request with "stream": true in the body. This validates that your vLLM deployment streams tokens correctly for real-time applications.

Postman’s monitoring feature can run your collection on a schedule to detect API regressions. Set up a monitor that hits your GPU endpoint every 15 minutes and alerts if response times exceed your threshold. For teams managing open-source model deployments, this provides basic uptime monitoring alongside your test suite. Explore more tutorials or get started with GigaGPU to build your AI API testing workflow.