Microsoft’s Phi-3 family gives you three distinct points on the quality-cost curve: Mini at 3.8 billion parameters, Small at 7 billion, and Medium at 14 billion. Each doubles roughly in cost while offering diminishing returns on quality. Picking the right size for your workload on dedicated GPU hardware saves real money without sacrificing the output quality your application requires.
Specifications Side by Side
| Specification | Phi-3 Mini | Phi-3 Small | Phi-3 Medium |
|---|---|---|---|
| Parameters | 3.8B | 7B | 14B |
| Context Window | 4K / 128K | 8K / 128K | 4K / 128K |
| Architecture | Dense Transformer | Dense Transformer | Dense Transformer |
| Attention | GQA | GQA + Block Sparse | GQA |
| Vocabulary | 32K | 100K | 32K |
| FP16 VRAM | 7.6 GB | 14 GB | 28 GB |
| INT4 VRAM | 2.8 GB | 5.2 GB | 9 GB |
Note that Small uses a 100K vocabulary versus 32K for Mini and Medium. This larger vocabulary improves tokenisation efficiency for multilingual text and code, but adds a fixed VRAM overhead from the embedding layer.
Benchmark Comparison
| Benchmark | Phi-3 Mini | Phi-3 Small | Phi-3 Medium |
|---|---|---|---|
| MMLU | 68.8 | 75.3 | 78.0 |
| HumanEval | 58.5 | 61.0 | 62.2 |
| GSM8K | 75.7 | 87.4 | 86.2 |
| ARC-Challenge | 85.7 | 90.7 | 91.6 |
| MT-Bench | 7.22 | 7.81 | 7.96 |
The Mini-to-Small jump is larger than the Small-to-Medium jump on most benchmarks. GSM8K actually dips slightly from Small to Medium (87.4 to 86.2), suggesting that Small’s block sparse attention and larger vocabulary give it an edge on mathematical reasoning. This makes Small the best quality-per-parameter option in the family.
Hardware Mapping
| Model | Min GPU (INT4) | Recommended GPU | Throughput (tok/s) |
|---|---|---|---|
| Mini (3.8B) | Any 4GB+ GPU | RTX 3090 | ~145 |
| Small (7B) | RTX 3090 | RTX 3090 | ~92 |
| Medium (14B) | RTX 3090 (tight) | RTX 5090 | ~55 |
Mini at INT4 uses under 3 GB of VRAM, leaving massive headroom for other models on the same GPU. This makes it ideal for multi-model pipelines where Phi-3 Mini handles one stage of processing alongside a larger model for another. Check the benchmark tool for real-time throughput data.
Workload-Specific Recommendations
Mini (3.8B) — Best for: Classification, entity extraction, short-form Q&A, routing and triage, edge deployment experiments. When your task has a clear correct answer and the model just needs to pick it. Also excellent as a fast first-pass filter before a larger model.
Small (7B) — Best for: Code assistance, maths-heavy workloads, multilingual tasks, general-purpose chat. The sweet spot of the Phi-3 family — best benchmarks per VRAM invested. Ideal for teams that want one model handling diverse tasks.
Medium (14B) — Best for: Long-form content generation, complex reasoning chains, nuanced instruction following. The quality ceiling is higher but the marginal gains over Small are modest. Justify Medium only when Small demonstrably fails on your evaluation set.
Beyond Phi-3
For the latest improvements, see the Phi-3.5 upgrade guide which adds an MoE variant and improved multilingual support. Compare against Gemma 2 size selection for an alternative small-model family, or Qwen 2.5 for broader size options. The best GPU for inference guide covers hardware pairing across model families.
Run Phi-3 on Dedicated Hardware
Deploy any Phi-3 size on bare-metal GPU servers. From Mini on a single card to Medium with full VRAM headroom. No shared tenancy, no token fees.
Browse GPU Servers
