API Hosting — Your Own OpenAI-Compatible Endpoint
Drop-in replacement for the OpenAI API on a dedicated GPU server. Run vLLM, Ollama, TGI or Triton with an /v1/chat/completions route at a fixed monthly price — no per-token bill, no rate limit, no shared tenancy.
Why Self-Host Instead of Using a Hosted API
The cost-benefit shifts dramatically once you pass a few hundred pounds a month in API spend or have any data-residency obligation.
OpenAI / Anthropic / Together API
Self-hosted API on GigaGPU
What You Get
Everything you need to run production AI workloads on dedicated hardware in the UK.
OpenAI-compatible API
vLLM exposes /v1/chat/completions, /v1/completions, /v1/embeddings and streaming. Existing SDKs (openai-python, openai-node) work unchanged — just swap base_url.
Pre-installed inference stacks
vLLM, Ollama, Text Generation Inference (TGI) and NVIDIA Triton are all pre-built. Pick the one that suits your model and concurrency profile.
Production-ready defaults
Continuous batching, prefix caching, KV-cache reuse, prompt caching — all the throughput tricks vLLM ships with, enabled by default.
Auth your way
API keys via Bearer tokens, mTLS, IP allow-list, or a reverse proxy on Cloudflare Access / Tailscale. We don’t impose an auth model.
Multi-modal endpoints
Whisper (speech to text), TTS (Bark, XTTS, Piper), image (FLUX/SDXL via ComfyUI HTTP) all available on the same box. One server, many APIs.
Metrics + logs you keep
Prometheus exporter, Grafana dashboard, structured request logs to your local syslog. We don’t ship them anywhere.
Common Self-Hosted API Configurations
These are the GPU + model combinations our customers most often deploy as a self-hosted API.
| Model | Params | FP16 VRAM | INT4 VRAM | Recommended |
|---|---|---|---|---|
| Mistral 7B Instruct + vLLM | RTX 3090 | ~720 tok/s | OpenAI-compatible drop-in | |
| Llama 3.1 8B + vLLM | RTX 5080 | ~95 tok/s single-stream | Latency-sensitive chatbots | |
| Llama 3.1 8B + vLLM | RTX 5090 | ~1,200 tok/s aggregate | High-concurrency | |
| Mixtral 8x7B + vLLM | RTX 6000 Pro | ~280 tok/s | Frontier-quality on one card | |
| Llama 3 70B INT4 + vLLM | 2× RTX 5090 | ~150 tok/s | 70B-class on commodity GPUs | |
| Whisper Large-v3 + faster-whisper | RTX 3090 | ~3-4× real-time | Voice transcription API | |
| BGE-large + nomic-embed | RTX 3060 | ~ 50K embeddings/s | Embedding-only API | |
| FLUX.1 dev + ComfyUI | RTX 5090 | ~6 s / 1024×1024 | Image generation API |
Why People Move Off Hosted APIs
Real customer workloads we run on this hardware every day.
Cost predictability
When monthly token spend exceeds about £1,500 on a hosted API, a dedicated GPU is cheaper. See our cost per 1M tokens breakdown.
Data sovereignty
Same as private AI hosting — when your prompts contain sensitive data, you control where they go.
Latency control
Co-locate the GPU with your application servers. London → London is <5 ms. London → US-East is 80 ms.
Custom models
Fine-tuned, domain-adapted, or PEFT-merged models can’t be served on most hosted APIs. Self-hosting handles them natively.
Model pinning
Hosted APIs deprecate models on their schedule. Self-hosting pins you to a Hugging Face commit forever.
Frequently Asked Questions
The questions buyers actually ask before committing to a GPU server.
Does it work with the official OpenAI Python SDK?
Yes — vLLM, TGI, and Ollama all expose OpenAI-compatible routes. Set base_url to your server’s address and you’re done.
What about /v1/embeddings?
Supported on vLLM with sentence-transformer-style models, and standalone via TEI (Text Embeddings Inference). Both pre-installed.
Can I run multiple models on the same endpoint?
Yes — Ollama supports model multiplexing on a single port. With vLLM, run separate processes on separate ports and front them with a router (LiteLLM works well).
How do I add API key authentication?
vLLM supports --api-key flag for a static Bearer token. For multi-tenant key management, drop LiteLLM in front. We document both patterns.
What about streaming and function calling?
Streaming works out of the box. Function calling depends on the model — Llama 3.1 8B+, Mistral 7B v0.3+, and Qwen 2.5 all support tool use natively.
Can I migrate from OpenAI gradually?
Yes — LiteLLM lets you split traffic between providers based on model name or weight. Run 80/20 for a week, ramp to 100% self-hosted when you’re confident.
Related Pages
Pages our visitors typically read next.
Stop paying per token. Start paying per server.
If your hosted-API bill is over £1,500/mo, a dedicated GPU is already cheaper. Talk to us about migration — most teams flip over in an afternoon.
