Table of Contents
Best LLMs for 12 GB in 2026: Qwen 2.5 14B Q4_K_M (strongest model that fits), Llama 3.1 8B FP8 (best quality-to-speed), Mistral 7B Q4_K_M (fastest), DeepSeek-R1 7B (reasoning), Qwen 2.5 Coder 7B (code). Anything above 14B at Q4 exceeds 12 GB. 7B+ at FP16 doesn’t fit.
The 12 GB Ceiling
12 GB of VRAM (as on the RTX 5070) imposes a hard ceiling on model size. The practical rules:
- FP16 (full precision): Maximum ~6B parameters. Llama 3.1 8B at FP16 is ~16 GB — doesn’t fit. Phi-3.5 Mini (3.8B) at ~7.6 GB fits.
- FP8 (Blackwell-native): Maximum ~11B parameters. Llama 3.1 8B at FP8 is ~8 GB — fits with 4 GB for KV cache.
- Q4_K_M (GGUF): Maximum ~14–15B parameters. Qwen 2.5 14B at Q4 is ~8.7 GB — fits with 3 GB for KV cache at moderate context.
- Q2_K (extreme compression): Technically allows larger models but quality degrades significantly — not recommended for production.
The sweet spot for 12 GB is 7B–14B models at Q4_K_M or FP8, which delivers excellent quality while fitting comfortably. The RTX 5070 at £139/mo is the entry Blackwell card optimised for this range.
Full Model VRAM Table
| Model | Params | FP16 | FP8 | Q4_K_M | Fits 12 GB? |
|---|---|---|---|---|---|
| Phi-3.5 Mini | 3.8B | 7.6 GB ✓ | 3.8 GB ✓ | 2.4 GB ✓ | All precisions |
| Qwen 2.5 3B | 3B | 6 GB ✓ | 3 GB ✓ | 1.9 GB ✓ | All precisions |
| Mistral 7B | 7B | 14 GB ✗ | 7 GB ✓ | 4.4 GB ✓ | FP8 or Q4 |
| Llama 3.1 8B | 8B | 16 GB ✗ | 8 GB ✓ | 5.2 GB ✓ | FP8 or Q4 |
| Gemma 2 9B | 9B | 18 GB ✗ | 9 GB ✓ | 5.8 GB ✓ | FP8 or Q4 |
| Qwen 2.5 7B | 7B | 14 GB ✗ | 7 GB ✓ | 4.4 GB ✓ | FP8 or Q4 |
| DeepSeek-R1 7B | 7B | 14 GB ✗ | 7 GB ✓ | 4.4 GB ✓ | FP8 or Q4 |
| Qwen 2.5 Coder 7B | 7B | 14 GB ✗ | 7 GB ✓ | 4.4 GB ✓ | FP8 or Q4 |
| Llama 3 13B | 13B | 26 GB ✗ | 13 GB ✗ | 7.9 GB ✓ | Q4 only |
| Qwen 2.5 14B | 14B | 28 GB ✗ | 14 GB ✗ | 8.7 GB ✓ | Q4 only — best fit |
| Gemma 2 27B | 27B | 54 GB ✗ | 27 GB ✗ | 16 GB ✗ | No — exceeds 12 GB |
| Llama 3.1 70B | 70B | 140 GB ✗ | 70 GB ✗ | 41 GB ✗ | No |
Top Picks by Use Case
Best overall for 12 GB: Qwen 2.5 14B Q4_K_M — the largest model that fits, with strong coding, reasoning, and multilingual capabilities. Uses ~8.7 GB leaving ~3 GB for KV cache at 4K context.
Best speed: Mistral 7B Q4_K_M — generates ~80–90 tok/s on RTX 5070, conversational with low latency. Uses only 4.4 GB leaving 8 GB for context and concurrent models.
Best quality for the size: Llama 3.1 8B FP8 — retains ~99% of FP16 benchmark quality, runs at ~85–95 tok/s via vLLM on Blackwell FP8 Tensor Cores, uses ~8 GB.
Best for reasoning: DeepSeek-R1 7B Q4_K_M — chain-of-thought reasoning model, ~4.4 GB, leaves 8 GB free for long reasoning chains in the KV cache.
Best for code: Qwen 2.5 Coder 7B Q4_K_M or Qwen 2.5 Coder 14B Q4_K_M — purpose-trained for code generation and completion. 7B at 4.4 GB or 14B at 8.7 GB.
Smallest useful model: Phi-3.5 Mini 3.8B Q4 — 2.4 GB, generates ~150+ tok/s, excellent for edge/latency cases where raw speed is critical.
Quantisation: Q4 vs Q8 vs FP8
| Precision | Quality | Speed (relative) | VRAM (8B model) | Best use |
|---|---|---|---|---|
| FP16 | Reference (100%) | 1× | 16 GB — doesn’t fit | — |
| FP8 | ~99.5% | 1.8–2× | 8 GB ✓ | vLLM production, Blackwell |
| Q8_0 (GGUF) | ~99% | 1.5× | 8.5 GB ✓ | Max quality GGUF |
| Q6_K (GGUF) | ~98.5% | 1.7× | 6.4 GB ✓ | Good balance |
| Q5_K_M (GGUF) | ~98% | 1.9× | 5.3 GB ✓ | Good balance, faster |
| Q4_K_M (GGUF) | ~97% | 2.2× | 4.8 GB ✓ | Recommended default |
| Q3_K_M (GGUF) | ~95% | 2.5× | 3.9 GB ✓ | Maximum speed, some quality loss |
On the RTX 5070 with Blackwell Tensor Cores, FP8 via vLLM is the recommended precision when using Hugging Face models. FP8 is computed natively on 5th Gen Tensor Cores and delivers better throughput than Q4 GGUF while retaining more quality. Use Q4_K_M GGUF when using llama.cpp/Ollama (which don’t yet have a vLLM-equivalent FP8 path).
Context Length and KV Cache
At 12 GB, context length management matters. KV cache grows with context:
- Llama 3.1 8B Q4 (5.2 GB weights) at 4K context: +~0.5 GB KV = 5.7 GB total ✓
- Llama 3.1 8B Q4 at 16K context: +~2 GB KV = 7.2 GB total ✓
- Llama 3.1 8B Q4 at 64K context: +~8 GB KV = 13.2 GB total ✗ (exceeds 12 GB)
- Qwen 2.5 14B Q4 (8.7 GB) at 8K context: +~2 GB = 10.7 GB ✓
- Qwen 2.5 14B Q4 at 32K context: +~8 GB = 16.7 GB ✗
For long-context workloads, prefer smaller models (7B Q4) to keep weights low and maximise KV cache budget.
Running Two Models at Once
12 GB allows paired loading of two small models with careful budgeting:
- Llama 3.1 8B Q4 + Mistral 7B Q4 = ~9.6 GB — fits, 2.4 GB for context
- Llama 3.1 8B Q4 + Whisper Large-v3 (~3 GB) = ~8.2 GB — comfortable
- Mistral 7B Q4 + Qwen 2.5 7B Q4 = ~8.8 GB — comfortable
Run these models on the RTX 5070
12 GB GDDR7 · Blackwell FP8 · Full CUDA · £139/mo · Order the RTX 5070 or see the LLM inference guide.
