Table of Contents
The RTX 4090 24 GB at £289/mo and Together AI’s per-token pricing cover overlapping use cases. This page is the precise comparison for the models you’d actually run on either.
Self-hosted 4090 wins on cost above ~2.5B output tokens/month for Mistral 7B-class models. Below that, Together AI is cheaper. For Llama 3 70B (which the 4090 cannot host single-card), Together always wins. The 4090 wins on data residency at any volume.
Models that fit a 4090 + are on Together
- Mistral 7B Instruct — both
- Llama 3.1 8B Instruct — both
- Qwen 2.5 7B / 14B (INT4) — both
- DeepSeek-Coder V2 Lite — both
- Mixtral 8x7B INT4 — fits 4090 (tight); Together hosts FP16
- Llama 3 70B — does NOT fit 4090; Together hosts
Cost-per-token comparison
| Model | 4090 self-hosted (60% util) | Together AI |
|---|---|---|
| Mistral 7B Instruct | £0.20/1M | £0.17/1M |
| Llama 3.1 8B | £0.21/1M | £0.17/1M |
| Qwen 2.5 14B INT4 | £0.34/1M | £0.27/1M |
| DeepSeek-Coder V2 Lite INT4 | £0.30/1M | £0.25/1M |
| Mixtral 8x7B INT4 | £0.62/1M | £0.46/1M |
| Llama 3 70B | Does not fit | £0.68/1M |
Important: Together’s per-token price is 100% utilisation by definition. The 4090 numbers are at 60% utilisation. At 100% util the 4090 is genuinely cheaper.
Break-even token volume
For Mistral 7B Instruct:
- 4090 monthly: £289.
- Together at £0.17/1M: break-even = 1.74B tokens/month.
- Below 1.74B tokens/month → Together is cheaper.
- Above 1.74B tokens/month → 4090 is cheaper.
Non-cost factors
Cost-per-token isn’t the only variable:
- Data residency — Together is US-hosted. UK/EU regulated workloads can’t use it.
- Custom models — Together supports limited fine-tunes; self-hosting allows any LoRA, QLoRA, merged model.
- Latency — UK → US adds 80–120 ms RTT. Local 4090 is ~5 ms.
- Reliability — Together has had outages; self-hosted is your uptime.
- Predictability — fixed monthly bill vs traffic-scaled bill.
Verdict
- Below 1.5B tokens/mo, no residency need: Together AI.
- Above 1.5B tokens/mo: RTX 4090 self-hosted.
- Need data residency: 4090 always wins.
- Need 70B-class: Together (4090 can’t host it).
Bottom line
For a steady production workload above ~1.5B tokens/mo of 7B–8B inference, the 4090 at £289/mo beats Together. For spiky or low-volume workloads, Together stays cheaper. The most resilient setup uses both: dedicated 4090 for steady traffic, Together for overflow.
