Table of Contents
Self-hosted translation is one of the cheaper AI workloads — small models, predictable cost, no per-character API fees. The 5060 Ti is a strong host.
TL;DR
For 200-language translation: Meta NLLB-200 3.3B on the 5060 Ti at ~3,000 tok/s. For higher quality: Aya Expanse 8B at ~600 tok/s, multilingual. For specific language pairs: Qwen 2.5 7B or Llama 3.1 8B.
Translation model picks
- NLLB-200 (Meta): 200 languages, dedicated translation model. 3.3B params (~7 GB FP16) or 600M params (smaller variants).
- M2M-100 (Meta): 100 languages, older but battle-tested. 1.2B params.
- Aya Expanse 8B (Cohere): explicitly multilingual LLM, 100+ languages, CC-BY-NC license (non-commercial only).
- Qwen 2.5 7B: general LLM, strong on Chinese/Japanese/Korean + European.
- Llama 3.1 8B: general LLM, decent on 8 European languages.
Throughput on the 5060 Ti
| Model | VRAM | Tok/s on 5060 Ti | Best for |
|---|---|---|---|
| NLLB-200 3.3B | 7 GB FP16 | ~3,000 | Pure translation, 200 langs |
| NLLB-200 1.3B | 3 GB FP16 | ~5,500 | Fast, fewer langs |
| M2M-100 1.2B | 2.5 GB FP16 | ~5,800 | Older, well-tested |
| Aya Expanse 8B | 8 GB FP8 | ~600 | High-quality multilingual (NC only) |
| Qwen 2.5 7B | 7 GB FP8 | ~880 | Asian languages strong |
| Llama 3.1 8B | 8 GB FP8 | ~820 | Western European |
Verdict
- Pure throughput, broad language coverage: NLLB-200 3.3B
- Highest quality, multilingual: Aya Expanse 8B (NC only)
- Commercial multilingual: Qwen 2.5 7B or Llama 3.1 8B
Bottom line
Self-hosted translation on a 5060 Ti is dramatically cheaper than DeepL or Google Translate API at any meaningful volume. NLLB-200 is the workhorse; LLM-based translation for higher quality.
