Cost per 1M Tokens by GPU: Full Breakdown

Why Cost per 1M Tokens Matters

If you run LLM inference at any meaningful scale, cost per million tokens is the single metric that determines whether your deployment is financially viable. A dedicated GPU server charges a flat monthly rate, so your effective cost per token drops as you push more volume through the hardware. API providers charge per token regardless of volume, which means costs scale linearly with usage.

This guide calculates the real cost per 1M tokens for each GPU tier available on open source LLM hosting platforms, then stacks those numbers against the major API providers. For an interactive version, use our cost per million tokens calculator.

How We Calculated the Numbers

Every cost figure below uses the same formula:

Cost per 1M tokens = Monthly server cost / (tokens per second x 86,400 seconds/day x 30 days / 1,000,000)

Tokens per second figures come from our tokens per second benchmarks, measured on LLaMA 3 8B with vLLM at batch size 1 on bare-metal servers. We assume 100% GPU utilisation across 30 days. Real-world utilisation varies, so your actual cost per token will be higher at lower utilisation rates.

Cost per 1M Tokens by GPU

Here is the full breakdown for every GPU tier, running LLaMA 3 8B inference at FP16 (or 4-bit quantised where VRAM requires it):

The RTX 3090 delivers the lowest cost per million tokens thanks to its combination of strong throughput and affordable monthly hosting. The RTX 5080 and RTX 5090 push more tokens per second, but their higher server cost raises the per-token rate. For a head-to-head GPU comparison, read our RTX 3090 vs RTX 5090 breakdown.

API Pricing: OpenAI, Anthropic, Together.ai

Here is what the major API providers charge per 1M output tokens for comparable models as of early 2025:

Even the cheapest API option — Together.ai at $0.20 per 1M tokens for LLaMA 3 8B — is already more expensive than a self-hosted RTX 3090 at $0.28 per 1M tokens at moderate utilisation. At high utilisation, the self-hosted option pulls further ahead. For a deeper dive into alternatives, see our guide on Together.ai alternatives.

Self-Hosted vs API: Side-by-Side

This table compares the cost of processing 1 billion tokens per month using self-hosted GPUs versus API providers, all running LLaMA 3 8B or equivalent:

At 1B tokens per month, a dedicated RTX 3090 costs roughly 86% less than Together.ai and 95% less than OpenAI’s GPT-4o mini. Use our GPU vs API cost comparison tool to model your own volume.

Best Value GPU for Token-Heavy Workloads

Highest throughput per dollar: RTX 3090. At 42 tokens/sec and the lowest monthly cost of the 24GB+ GPUs, the RTX 3090 remains the best value option for high-volume LLM inference. It handles LLaMA 3 8B, Mistral 7B, and most 13B models at 4-bit quantisation.

Best for larger models: RTX 5090. If you need FP16 inference on 13B+ parameter models, the RTX 5090’s 32GB VRAM is the only single-GPU option. The per-token cost is higher, but avoiding quantisation means better output quality.

Budget option: RTX 4060 Ti. For lighter workloads under 80M tokens per month, the RTX 4060 Ti provides decent throughput at a lower server price point. Ideal for internal tools or low-traffic chatbots.

For a full comparison of GPU performance across models, see our best GPU for LLM inference guide. If you want data privacy alongside low per-token costs, explore private AI hosting with full root access.

Key Takeaway

Self-hosted GPU inference is dramatically cheaper than API pricing at any volume above roughly 50M tokens per month. The RTX 3090 on a dedicated GPU server delivers cost per million tokens that no API provider can match — and the gap widens with scale. For teams processing billions of tokens monthly, the savings fund additional hardware within months.

Run the numbers for your workload with the LLM cost calculator, or explore more cost analysis in our cost and pricing guides.