Quick Verdict: FP16 vs FP8 vs INT4
FP16 (16-bit floating point) is the baseline full-precision format for LLM inference. FP8 halves memory usage with less than 1% quality loss on supported hardware. INT4 quarters memory usage with 1-3% quality loss. On an RTX 6000 Pro 96 GB, FP16 fits a 40B model. FP8 fits a 70B model on the same card. INT4 fits a 130B model. For most production workloads on dedicated GPU hosting, FP8 offers the best trade-off between quality and efficiency when your GPU supports it.
Precision Format Overview
FP16 stores each weight as a 16-bit floating-point number. This is the native training precision for most LLMs and produces the highest inference quality. It requires approximately 2 bytes per parameter (a 70B model needs 140GB VRAM).
FP8 (8-bit floating point) was introduced with NVIDIA’s Hopper architecture (RTX 6000 Pro) and is now supported on Ada Lovelace (RTX 5090, RTX 6000 Pro) GPUs. It provides hardware-accelerated 8-bit computation with dynamic range scaling, using 1 byte per parameter. vLLM supports FP8 natively.
INT4 (4-bit integer) quantises weights to 4 bits using techniques like AWQ or GPTQ. It uses 0.5 bytes per parameter, fitting 4x more model per GB of VRAM. Quality depends heavily on the quantisation method. See serving engine comparisons for runtime support.
Benchmark Comparison (Llama 3 70B)
| Metric | FP16 | FP8 | INT4 (AWQ) |
|---|---|---|---|
| VRAM Required | 140 GB (2x RTX 6000 Pro) | 70 GB (1x RTX 6000 Pro 96 GB) | 38 GB (1x RTX 6000 Pro) |
| Throughput (tok/s, batch=1) | 28 | 45 | 52 |
| Throughput (tok/s, batch=32) | 420 | 680 | 780 |
| Quality (MMLU score) | 79.2 (baseline) | 79.0 (-0.3%) | 78.1 (-1.4%) |
| GPU Support | All CUDA GPUs | RTX 6000 Pro, RTX 6000 Pro, RTX 5090 | All CUDA GPUs |
| First Token Latency | 85ms | 52ms | 42ms |
Quality vs Speed Trade-Off
FP8 is nearly lossless. Benchmarks consistently show less than 0.5% degradation across MMLU, HumanEval, and MT-Bench. The quality is close enough to FP16 that most production applications cannot distinguish them. FP8 should be the default on RTX 6000 Pro and RTX 6000 Pro deployments. Check token speed benchmarks for hardware-specific data.
INT4 shows measurable quality loss, typically 1-3% on standard benchmarks. For creative writing and code generation, the degradation is minimal. For mathematical reasoning and factual recall, the loss is more noticeable. On GPUs with limited VRAM, INT4 enables running models that simply would not fit at higher precision.
Hardware Requirements
FP8 requires Hopper or Ada Lovelace GPUs with dedicated FP8 tensor cores. Running FP8 on older GPUs (RTX 6000 Pro, RTX 6000 Pro) falls back to emulation with no speed benefit. Before selecting FP8, confirm your dedicated GPU server has compatible hardware. INT4 runs on any CUDA GPU through software dequantisation, making it universally compatible.
For multi-GPU clusters, FP8 reduces inter-GPU communication by half compared to FP16 during tensor parallelism, further improving multi-card throughput. See the benchmarks section for multi-GPU scaling data.
Recommendation
Use FP8 on RTX 6000 Pro or RTX 6000 Pro GPUs for the best quality-to-speed ratio. Use INT4 when VRAM is constrained or when fitting a larger model is more important than marginal quality. Use FP16 only when absolute quality is paramount and you have sufficient VRAM across multi-GPU setups. Deploy on GigaGPU dedicated servers with vLLM for native FP8 and INT4 support. Explore LLM hosting and private AI hosting for production deployment.
