Table of Contents
Picks: 5060 Ti / 4090 / 5090 / 6000 Pro as the consumer / workstation tiers. vLLM as the serving layer. Llama 3.1 / 3.3 / Qwen 2.5 / Mistral as the model defaults. FP8 as the precision default. Self-hosting beats hosted-API economics above ~30M tokens/month for most workloads.
Summary
- Hardware: 5060 Ti (£119), 4090 (£289), 5090 (£399), 6000 Pro (£899)
- Serving: vLLM (default), TensorRT-LLM (max), SGLang (structured)
- Models: Llama 3.1 / 3.3 (general), Qwen 2.5 (multilingual), Mistral (English), DeepSeek R1-Distill (reasoning)
- Precision: FP8 default, AWQ-INT4 fallback
- RAG: BGE-large + reranker + Qdrant
- Voice: Whisper + Llama 3.1 + Kokoro / XTTS v2
Production patterns
Hybrid stacks dominate: self-hosted for the bottom 90%, hosted frontier API (Claude, GPT-4o) for the hardest 5-10%. OpenAI-compatible endpoints everywhere so clients are agnostic. Eval harness in version control next to the prompts. Blue-green or rolling deploys with graceful drain. systemd plus nginx is the boring-and-reliable serving stack.
Trade-offs
Self-hosting trades operational simplicity for cost, residency, and customisation. Below ~30M tokens/month, hosted APIs are simpler and cheaper. Above that, dedicated wins decisively. UK / EU residency makes self-hosting the default rather than the optimisation. Reasoned guidance from observed 2025-26 patterns.
Verdict
Self-hosted AI in 2026 is mature, predictable, and cost-effective. The hard problems are evaluation, data quality, and operational discipline, not hardware.
Bottom line
Self-hosted is the production default. See dedicated GPU hosting.
