Table of Contents
The most-asked question for the 5060 Ti is "will my model fit?" This page is the precise answer.
TL;DR
Maximum model size on a 16 GB GPU: ~7B FP16, ~13B FP8, ~22B AWQ-INT4, ~30B GGUF Q4. Beyond that you need offloading (slow) or a bigger GPU.
Size ceilings by precision
| Precision | Max model params | Reason |
|---|---|---|
| FP16 (BF16) | ~7B | 14 GB weights leaves 2 GB for cache |
| FP8 | ~13B | 13 GB weights leaves 3 GB for cache |
| INT8 | ~14B | Slightly more efficient than FP8 |
| AWQ-INT4 | ~22B | 11 GB weights leaves 5 GB for cache |
| GGUF Q5_K_M | ~18B | Mostly INT5 |
| GGUF Q4_K_M | ~22B | INT4 with quantisation scales |
| INT3 | ~30B | Quality cost noticeable |
Concrete model examples
| Model | Params | Best precision on 5060 Ti | Fits? |
|---|---|---|---|
| Mistral 7B | 7B | FP16 or FP8 | Yes, easily |
| Llama 3.1 8B | 8B | FP8 (FP16 tight) | Yes |
| Qwen 2.5 14B | 14B | AWQ-INT4 (FP8 tight) | Tight at INT4 |
| DeepSeek-V2 16B Lite | 16B (MoE) | AWQ-INT4 | Yes |
| Codestral 22B | 22B | AWQ-INT4 only | Just fits |
| Mixtral 8x7B | 47B (MoE) | Does not fit | No |
| Llama 3 70B | 70B | Does not fit | No |
Verdict
For a 16 GB card the practical ceiling is ~14B FP8 or ~22B INT4. Models above that need a bigger GPU.
Bottom line
If your target model is >22B, skip the 5060 Ti and start with a 5090 32 GB or larger. See when to upgrade.
