Table of Contents
Most LLM cost calculators get the math wrong by ignoring utilisation rate or FP8 throughput. This is the formula we use.
TL;DR
Cost per 1M tokens = (monthly_cost) / (aggregate_tok/s × 86400 × 30 × utilisation_rate) × 1,000,000. The variables that decide accuracy: utilisation_rate (most-skipped), FP8 vs FP16 throughput, and prefix cache hit rate.
Variables that matter
- monthly_cost: server rental, fixed
- aggregate_tok/s: from vLLM benchmark, varies by precision and batching
- utilisation_rate: 0.15-0.80 in real workloads
- prefix_cache_hit_rate: 0-90%, multiplies effective throughput
- average output tokens per request: matters for cost-per-request not per-1M
The formula
def cost_per_1m_tokens(
monthly_gbp: float,
aggregate_toks_per_sec: float,
utilisation: float = 0.6,
prefix_cache_hit_rate: float = 0.0,
) -> float:
effective_throughput = aggregate_toks_per_sec * (1 + 0.4 * prefix_cache_hit_rate)
monthly_tokens = effective_throughput * 86400 * 30 * utilisation
return (monthly_gbp / monthly_tokens) * 1_000_000Examples:
- RTX 5090 (£399), Mistral 7B FP8 (1920 tok/s), 60% util, 70% prefix hit rate: £0.085/1M
- RTX 3090 (£159), Mistral 7B FP16 (720 tok/s), 30% util, no prefix cache: £0.28/1M
Verdict
Most cost calculators that show £0.30-0.50/1M for self-hosted are missing utilisation discipline and prefix caching. Real production deployments hit £0.08-0.15/1M.
Bottom line
Build the calculator with utilisation as a slider. The answer changes by 3× depending on the value. See cost per 1M tokens.
