Together AI exposes Llama 3 70B and Qwen 14B as serverless per-token APIs at very competitive blended rates. The question for any team building a real product is whether the per-million-token meter beats the flat cost of a dedicated RTX 4090 24GB running the same model in AWQ INT4 form. The answer turns on volume, on the workload mix, and on a handful of hidden costs that never show up on the per-token sticker. The breakpoints sit around 280-830M tokens/month for 70B and around 1.8B tokens/month for Qwen 14B – and there are several reasons dedicated wins below those break-evens too. Background context lives in the wider dedicated GPU range.
Contents
- Together AI per-token rates
- Self-hosting throughput on a 4090
- Break-even token volume for 70B
- Qwen 14B comparison
- Hidden costs the per-token rate excludes
- Three deployment scenarios
- Production gotchas with serverless APIs
- Verdict and decision matrix
Together AI per-token rates
Together’s headline pricing for the open-weight Llama and Qwen families runs as follows. These are blended input/output rates – actual bills depend on prompt vs completion split, with system prompts and RAG context inflating input tokens disproportionately.
| Model | Input $/M tokens | Output $/M tokens | Blended (~70/30 in/out) | Context limit |
|---|---|---|---|---|
| Llama 3.1 70B Instruct Turbo | $0.88 | $0.88 | $0.88 | 128k |
| Llama 3.3 70B Instruct Turbo | $0.88 | $0.88 | $0.88 | 128k |
| Qwen 2.5 14B Instruct Turbo | $0.30 | $0.30 | $0.30 | 32k |
| Llama 3.1 8B Instruct Turbo | $0.18 | $0.18 | $0.18 | 128k |
| Qwen 2.5 72B Instruct Turbo | $1.20 | $1.20 | $1.20 | 32k |
| Llama 3.1 405B Instruct Turbo | $3.50 | $3.50 | $3.50 | 128k |
Self-hosting throughput on a 4090
An Ada AD102 with native 4th-gen FP8 tensor cores runs the following sustained rates under vLLM 0.6.x with continuous batching. Cross-reference with the 8B benchmark and the 70B INT4 benchmark for full configuration details.
| Model on 4090 24GB | Quantisation | t/s batch 1 | Aggregate at concurrency | Approx M tokens/day |
|---|---|---|---|---|
| Llama 3.1 8B | FP8 native | 198 | ~1,100 at conc 8 | ~95M |
| Qwen 2.5 14B | FP8 native | 120 | ~700 at conc 8 | ~60M |
| Llama 3.1 70B | AWQ INT4 | 22 | ~110 at conc 4 | ~9.5M |
| Mistral 7B | FP8 native | 220 | ~1,250 at conc 8 | ~108M |
| SDXL 1024×1024 | FP16 | 3.4s/image | ~25,000 images/day | n/a |
A typical production deployment running a single Llama 70B AWQ INT4 endpoint at sustained concurrency 4 produces ~285M tokens/month at full utilisation. That number sets the upper bound for the 4090’s “compete with Together” envelope on 70B traffic.
Break-even token volume for 70B
Take £550/month for a dedicated 4090 (~$700). At Together’s $0.88/M for Llama 70B, you would need to push 700/0.88 = 795M tokens/month through Together to spend the same as one dedicated card. The 4090 ceiling is 285M tokens/month at full utilisation. So the 4090 cannot saturate against the Together break-even on a single card – you need multiple cards to push past 795M, at which point the dedicated bundle starts winning on bulk economics.
| Monthly 70B tokens | Together cost | Dedicated 4090 cost | Cards needed | Cheaper |
|---|---|---|---|---|
| 10M | £7 | £550 | 1 | Together |
| 50M | £35 | £550 | 1 | Together |
| 200M | £141 | £550 | 1 | Together |
| 285M (4090 ceiling) | £200 | £550 | 1 | Together |
| 500M | £352 | £1,100 | 2 | Together |
| 800M | £563 | £1,650 | 3 | Together |
| 1,500M | £1,058 | £3,300 | 6 | Together |
| 3,000M | £2,116 | £6,050 | 11 | Together |
On pure compute economics, Together is cheaper across every realistic 70B volume. The break-even with bundled extras included shifts toward dedicated only when you account for fine-tuned models that Together cannot host, sub-200ms TTFT requirements, log retention for compliance, GDPR data residency, or multi-modal services where the same card also runs SDXL/Whisper.
Qwen 14B comparison
Qwen 14B at FP8 fits comfortably on one 4090 with KV cache headroom for ~16k context at concurrency 8. Together’s $0.30/M means one 4090 has to displace $700/0.30 = 2,330M tokens/month of API traffic to break even. The 4090 ceiling on Qwen 14B is ~1,800M tokens/month. So a single 4090 still cannot quite break even on Qwen 14B against Together.
| Monthly Qwen 14B tokens | Together cost | Dedicated 4090 cost | Cheaper |
|---|---|---|---|
| 500M | £120 | £550 | Together |
| 1,500M | £360 | £550 | Together |
| 1,800M (4090 ceiling) | £432 | £550 | Together |
| 2,500M (needs 2 cards) | £600 | £1,100 | Together |
| 5,000M (3 cards) | £1,200 | £1,650 | Together |
Hidden costs the per-token rate excludes
The per-token sticker hides several cost categories that flip the maths in dedicated’s favour for many production deployments.
Fine-tuned models and adapters
Together hosts the base instruct variants only. If your product depends on a LoRA adapter, a domain-specific fine-tune, or a quantised distillation that you produced, Together cannot serve it. You either host it yourself (4090) or pay Together’s dedicated endpoint surcharge ($1.20-2.50/hr per card, putting you above dedicated economics anyway).
Latency and TTFT variance
Together’s serverless TTFT for 70B sits around 400-700ms, with p99 spikes to 2-3 seconds during peak hours. A self-hosted 4090 with vLLM warm and continuous batching delivers sub-300ms TTFT consistently. For consumer-facing chat UIs, the gap matters.
Per-request rate limiting
Free-tier and pay-as-you-go accounts hit per-second token limits during traffic spikes. Burst above the limit returns 429s. Self-hosted endpoints scale only with your hardware – no per-account ceiling.
Logging, compliance, and data residency
Together’s API processes your prompts and completions in US datacentres. For UK NHS, financial services, or any GDPR-bound workload sending PII, that is often a deal-breaker regardless of cost. Self-hosted in the UK keeps the data on-shore.
Multi-workload sharing
A single 4090 simultaneously runs your LLM, an SDXL endpoint, a Whisper queue, and an embedding service if you size carefully. Together gives you token-metered LLM only – everything else lives on separate infrastructure. The dedicated bundle absorbs adjacent workloads without extra cost.
Three deployment scenarios
Scenario A: experimental low-volume B2B
You serve 30M tokens/month of Llama 70B output to 50 paying B2B users. Together: £21/mo. Dedicated: £550/mo. Together wins decisively. Use Together until volume justifies the switch, with a contingency plan for fine-tuned variants. Adjacent reading: for startup MVP.
Scenario B: steady production scale-up
You hit 200M tokens/month of 70B traffic with steady UK-business-hours patterns. Together: £141/mo. Dedicated: £550/mo on 70B alone. But the dedicated 4090 also runs your Llama 8B chat (60M tokens/mo, would cost £8 on Together), SDXL image gen (would cost extra), and a fine-tuned customer-specific 8B variant (Together cannot host). All-in dedicated saves £100-300/mo vs piecing together multiple Together products plus a separate dedicated SDXL host.
Scenario C: high-volume mature product
You’re at 1.5B tokens/month of 70B traffic, with a fine-tuned 8B for customer support and serving UK regulated entities. Together for 70B: £1,058/mo. Plus separate dedicated for the fine-tune (~£550/mo). Plus compliance overhead for transatlantic data flow. Vs three dedicated 4090s in tensor-parallel TP=2 pairs (one TP pair for 70B at higher throughput, one solo for 8B fine-tune): ~£1,650/mo total, fully on-shore, including image gen and embeddings. Dedicated wins on TCO and on compliance.
Production gotchas with serverless APIs
- Per-token rate is a floor, not a ceiling. System prompts, RAG context, retries, evaluation runs, and shadow-traffic A/B tests all multiply your token count beyond what user-facing flow consumed.
- Cold starts are invisible but real. Together’s serverless 70B can have 1-3 second first-request latency after idle periods. For low-traffic endpoints this destroys p95.
- Rate limits bite at the worst time. A traffic spike triggers 429s. Self-hosted endpoints saturate but do not 429.
- Fine-tunes require dedicated endpoints. The serverless rate is base models only. Custom adapters bump you to $1-3/hr per card on Together, often more than dedicated.
- No log retention guarantees by default. If you need audit logs for regulated workloads, you build retention yourself – Together does not promise it.
- Token-counting is opaque. Together’s tokeniser may report different counts than your local one for the same input. Reconcile billing carefully.
- Vendor concentration risk. A pricing change, model deprecation, or capacity event at Together hits all your traffic at once. Dedicated is independent infrastructure.
Verdict and decision matrix
For Llama 70B inference under ~280M tokens/month with no fine-tuning, no fixed-IP requirements, and no UK data residency constraint, Together is meaningfully cheaper. For 70B above that volume, or for any workload where you need fine-tuning, on-prem RAG with private indexes, sub-200ms TTFT in the UK, multi-workload bundling, or GDPR-bound infrastructure, dedicated 4090 wins on total cost of ownership and on operational control. For Qwen 14B the dedicated breakpoint is ~1.8B tokens/month on raw compute, lower with bundled extras factored in. The pragmatic path for many teams is to start on Together for early traction, then migrate to dedicated 4090 when token volume, fine-tunes or compliance demands justify the move.
Own the meter
Run Llama 70B AWQ INT4 on dedicated UK silicon at fixed cost. Native FP8 Ada AD102, 24GB VRAM, no per-token charges, no rate limits, no transatlantic data flow.
Order the RTX 4090 24GBSee also: Llama 70B monthly cost on 4090, vs OpenAI API cost, vs Anthropic API cost, Llama 70B INT4 benchmark, FP8 Llama deployment, Llama 70B INT4 VRAM, vs RunPod pricing, vs Lambda Labs, vs cloud H100, monthly hosting cost, ROI analysis, break-even calculator, 70B INT4 deployment, tier positioning 2026, for SaaS RAG, 5060 Ti vs Together.
