Table of Contents
Llama 3 ships in 8B, 70B, and 405B sizes (3.1 / 3.3 refresh). VRAM math at each precision:
TL;DR
Llama 3 8B: 16 GB FP16, 8 GB FP8, 5 GB INT4. Llama 3 70B: 140 GB FP16, 70 GB FP8, 40 GB INT4. Llama 3 405B: multi-node only. Plus 2-20 GB KV cache depending on context.
VRAM by variant
| Variant | FP16 | FP8 | AWQ-INT4 | GGUF Q5 |
|---|---|---|---|---|
| Llama 3.2 1B | 2 GB | 1 GB | 0.7 GB | 0.7 GB |
| Llama 3.2 3B | 6 GB | 3 GB | 2 GB | 2 GB |
| Llama 3.1 8B | 16 GB | 8 GB | 5 GB | 5.5 GB |
| Llama 3.2 11B Vision | 22 GB | 11 GB | 6 GB | 7 GB |
| Llama 3.3 70B | 140 GB | 70 GB | 40 GB | 49 GB |
| Llama 3.2 90B Vision | 180 GB | 90 GB | 50 GB | 63 GB |
| Llama 3.1 405B | 810 GB | 405 GB | 230 GB | 283 GB |
KV cache scaling
Per concurrent request:
| Variant | 8K context FP16 KV | 32K context FP16 KV | 128K context FP16 KV |
|---|---|---|---|
| Llama 3.1 8B | ~2.5 GB | ~10 GB | ~40 GB |
| Llama 3.3 70B | ~5 GB | ~20 GB | ~80 GB |
| Llama 3.1 405B | ~12 GB | ~48 GB | impractical |
Hardware fit
| Variant | Recommended GPU |
|---|---|
| Llama 3.2 1B / 3B | Any 8+ GB GPU |
| Llama 3.1 8B FP16 | RTX 3090 24 GB or RTX 5090 |
| Llama 3.1 8B FP8 | RTX 5060 Ti 16 GB+ |
| Llama 3.3 70B FP8 | RTX 6000 Pro 96 GB single-card |
| Llama 3.3 70B INT4 | 2× RTX 5090 OR single RTX 6000 Pro |
| Llama 3.1 405B | Multi-node H100 / H200 cluster |
Verdict
For most teams the practical Llama 3 deployments are 8B (single 24-32 GB card) and 70B (6000 Pro or 2× 5090).
Bottom line
For 8B / 70B sizing see best GPU for Llama. For 405B contact sales.
