Table of Contents
Mixture-of-Experts models (Mixtral 8x7B, DeepSeek V3 671B) have different deployment characteristics from dense models. Total params are large; active params per token are smaller. The result: VRAM requirements track total params, but compute throughput tracks active params. Plan accordingly.
MoE: total params determine VRAM (Mixtral 8x7B = 47B = ~25 GB INT4); active params determine compute (~13B per token). vLLM handles MoE serving natively; tune max-num-seqs lower than dense equivalents. Best fit: 5090 32GB for Mixtral, multi-GPU for DeepSeek V3. Expert load-balancing matters at scale.
How MoE differs
- Total params: all experts in VRAM. Mixtral 8x7B = 47B params; DeepSeek V3 = 671B
- Active params: only N of K experts route per token. Mixtral 2/8 active = 13B active; DeepSeek V3 ~37B active
- Throughput: roughly tracks active params, not total. So Mixtral feels like ~13B for throughput
- VRAM: tracks total, not active. So Mixtral needs ~13B-equivalent VRAM, not 47B
- Routing overhead: small per-token cost for expert routing
Hardware
| Model | Total params | Active params | Recommended GPU |
|---|---|---|---|
| Mixtral 8x7B | 47B | 13B | 5090 32GB AWQ-INT4 |
| Mixtral 8x22B | 141B | 39B | 6000 Pro 96GB or 2× A100 |
| DeepSeek V3 | 671B | 37B | Multi-GPU H100 cluster |
| Qwen 2.5 MoE (rumoured) | TBD | TBD | TBD |
Tuning
- Lower
max-num-seqsthan dense equivalents (expert load-balancing creates variable per-batch workload) - Watch expert-utilisation metrics; if some experts dominate, routing balance is off
- FP8 for active path (compute) + appropriate quantisation for stored experts
- Long-context performance varies; expert routing patterns shift with context length
Verdict
MoE models are a real production option in 2026 (Mixtral 8x7B specifically widely deployed). The deployment is similar to dense models with specific tuning differences. Choose MoE when you need stronger reasoning than dense 13B but want to avoid the throughput cost of dense 70B.
Bottom line
MoE for between-tier quality. See Mixtral benchmark.
