RTX 4090 24GB or RTX 3090 24GB: Decision Guide

The RTX 4090 24GB and RTX 3090 24GB share the same VRAM ceiling, which makes them look interchangeable on a spec sheet. They are not. The 4090’s Ada AD102 die brings native 4th-gen FP8 tensor cores and roughly 3x the inference throughput of the Ampere GA102 in the 3090 on modern quantised workloads. The 3090 retains one trump card the 4090 lacks: NVLink-3, useful for multi-card setups. This guide compares both for the workloads buyers actually run on UK dedicated GPU hosting, with reference to the wider gigagpu range, and lays out where each card wins decisively.

Contents

• Spec sheet at a glance
• Inference throughput across workloads
• FP8 native vs emulated: the architectural gap
• Model fit and KV headroom
• Per-workload winner table (10 workloads)
• Cost-per-token and watts-per-token
• Production gotchas
• Verdict and when each card wins

Spec sheet at a glance

Inference throughput across workloads

The headline gap is biggest on FP8 workloads where the 4090 has dedicated silicon and the 3090 must fall back to FP16 or emulate FP8 in software via Marlin or bitsandbytes kernels. For straight FP16 work the gap narrows but stays decisive. For INT4 work (AWQ, GPTQ) both cards lean on shader-based dequantisation and the gap is smallest.

FP8 native vs emulated: the architectural gap

The 3090 can run FP8 weights through Marlin or bitsandbytes kernels but the matmul itself happens at FP16 with conversion overhead at every layer. That costs roughly 60-70% of theoretical FP8 throughput. The 4090 has hardware FP8 matmul, so vLLM, TensorRT-LLM, and SGLang all hit close to peak. For Llama 70B AWQ INT4 the difference is smaller because the dominant cost is INT4 dequantisation – both cards do that in shaders.

Practically, this means a 4090 deployment for FP8 chat APIs runs three times the user concurrency of a 3090 for the same model. If your roadmap is “FP8 everything” – and most modern inference stacks default to that – the 4090’s architectural advantage is decisive. Read the deeper analysis in FP8 tensor cores on Ada.

What FP8 emulation actually costs on a 3090

vLLM’s `–quantization fp8` flag on a 3090 dispatches Marlin kernels that pack FP8 weights but execute matmul at FP16. The kernel does the dequantisation per-layer, costing 30-40% of the saved memory bandwidth back as compute overhead. Net result: FP8 on 3090 is faster than FP16 on memory-bound layers but slower than FP8 on Ada by a factor of 2-3x.

Model fit and KV headroom

Both cards have 24GB so the maximum model footprint is identical. What changes is whether you can use the modern quantisation formats efficiently, and how much KV cache you have left after the weights load. The 4090’s native FP8 KV-cache support gives it more effective concurrency on the same VRAM.

Per-workload winner table (10 workloads)

Cost-per-token and watts-per-token

Assume £550/month for a dedicated 4090 and £275/month for a dedicated 3090. The 4090 costs ~2.0x more. For most LLM inference workloads it produces 2.5-3x the throughput, so the 4090 wins on cost-per-token despite the higher monthly fee. The exception is INT4-bound work where the gap closes.

Production gotchas

1. Marlin/bitsandbytes kernel availability on 3090. FP8 emulation requires specific vLLM/SGLang versions. Older deployments fall back to slow paths. Pin versions and benchmark.
2. 3090 cooling under sustained load. The 350W reference design throttles in poorly-ventilated 1U/2U chassis. Datacentre installs need active airflow. Cross-reference thermal performance.
3. NVLink-3 availability on 3090 dedicated. Many hosting providers do not bridge dual 3090s with NVLink. Confirm before you order if you intend to use the bandwidth.
4. FP8 KV cache on 3090. vLLM’s `–kv-cache-dtype fp8` works but with FP16 conversion overhead per access. Real win is smaller than on 4090.
5. Driver lifecycle for Ampere. The 3090 ships under the long-term-support driver branch. Newer features (Triton kernels, FA3) sometimes target Ada+ first.
6. Power budget for 2x 3090. 700W GPU + 200W host = 900W sustained. Many shared rack PDUs limit to 1500W per outlet; 2x 3090 + headroom is tight.
7. Resale value asymmetry. 4090 retains value strongly; 3090 has depreciated. Affects total cost of ownership if you’re buying rather than renting.

Verdict and when each card wins

Pick the 4090 for any FP8 workload (8B or 14B Llama/Qwen/Mistral chat APIs, modern quantised inference at high concurrency), for image and audio diffusion (SDXL, Flux, Whisper), for fine-tuning sprints, and for any deployment where cost-per-token at scale matters more than absolute monthly spend. Pick the 3090 if your budget is tight and your workload is Llama 70B AWQ INT4 batch jobs where INT4 dequantisation dominates and FP8 emulation does not matter, or if you need NVLink-3 for cheap multi-card setups (the 4090 has no NVLink at all). For most modern inference stacks running FP8 chat models, the 4090 is the better buy at UK dedicated rates; for cost-sensitive research labs grinding INT4 traffic, the 3090 is still defensible.

See also: RTX 4090 vs 3090 for AI, FP8 tensor cores on Ada, Llama 3 8B benchmark, Llama 70B INT4 benchmark, spec breakdown, tier positioning 2026, tokens per watt, vLLM setup, FP8 Llama deployment, thermal performance, power draw efficiency, or 5090 decision, or 5060 Ti decision, 5090 vs 3090, 70B INT4 VRAM, best GPU for fine-tuning.