Table of Contents
Quick Verdict
An interesting reversal from what architecture alone would predict: Qwen 72B serves 19.2 requests per second compared to Mixtral 8x7B’s 16.7, despite having over 5x more active parameters per forward pass. On a dedicated GPU server, Qwen 72B delivers 15% more throughput at the cost of higher VRAM consumption and slightly worse tail latency.
Mixtral fights back with a tighter p50 latency (78 ms versus 103 ms), meaning individual requests complete faster even though total request throughput is lower. For APIs where per-request speed matters more than aggregate capacity, Mixtral takes the edge.
Full benchmarks follow. More at our GPU comparisons hub.
Specs Comparison
Mixtral’s Apache 2.0 licence offers more commercial flexibility than Qwen’s proprietary licence, which may factor into enterprise API deployments.
| Specification | Mixtral 8x7B | Qwen 72B |
|---|---|---|
| Parameters | 46.7B (12.9B active) | 72B |
| Architecture | Mixture of Experts | Dense Transformer |
| Context Length | 32K | 128K |
| VRAM (FP16) | 93 GB | 145 GB |
| VRAM (INT4) | 26 GB | 42 GB |
| Licence | Apache 2.0 | Qwen |
Sizing: Mixtral 8x7B VRAM requirements and Qwen 72B VRAM requirements.
API Throughput Benchmark
Tested on an NVIDIA RTX 3090 under sustained concurrent load with vLLM, INT4 quantisation, and continuous batching. Check our tokens-per-second benchmark for more data.
| Model (INT4) | Requests/sec | p50 Latency (ms) | p99 Latency (ms) | VRAM Used |
|---|---|---|---|---|
| Mixtral 8x7B | 16.7 | 78 | 374 | 26 GB |
| Qwen 72B | 19.2 | 103 | 347 | 42 GB |
Qwen 72B’s higher throughput under batching suggests it benefits more from vLLM’s continuous batching optimisations at scale. For single-request latency, Mixtral’s 78 ms p50 is hard to beat. Consult our best GPU for LLM inference guide for hardware advice.
See also: Mixtral 8x7B vs Qwen 72B for Chatbot / Conversational AI for a related comparison.
See also: LLaMA 3 70B vs Mixtral 8x7B for API Serving (Throughput) for a related comparison.
Cost Analysis
Qwen’s 15% higher throughput partially offsets its 60% higher VRAM footprint from a cost-efficiency standpoint.
| Cost Factor | Mixtral 8x7B | Qwen 72B |
|---|---|---|
| GPU Required (INT4) | RTX 3090 (24 GB) | RTX 3090 (24 GB) |
| VRAM Used | 26 GB | 42 GB |
| Est. Monthly Server Cost | £128 | £133 |
| Throughput Advantage | 8% faster | 12% cheaper/tok |
Calculate at our cost-per-million-tokens calculator.
Recommendation
Choose Qwen 72B if you need the maximum total requests per second per GPU and your API handles payloads that benefit from the 128K context window. Its higher throughput under batched load makes it the better choice for high-traffic endpoints.
Choose Mixtral 8x7B if individual request latency is the priority, or if your deployment needs to fit on a single 24 GB GPU without memory pressure. Its lower VRAM footprint also leaves room for co-located services.
Serve on dedicated GPU servers with vLLM for production-grade reliability.
Deploy the Winner
Run Mixtral 8x7B or Qwen 72B on bare-metal GPU servers with full root access, no shared resources, and no token limits.
Browse GPU Servers
