Yes, the RTX 5090 can run multiple LLMs simultaneously. With 32GB GDDR7 VRAM, the RTX 5090 has enough capacity to host two or three quantised language models at the same time. This enables multi-agent architectures, model routing, A/B testing, and specialised-model pipelines all on a single GPU.
The Short Answer
YES. Two 7B models in INT4 (~10GB total) or three with careful budgeting (~15GB) fit easily within 32GB.
The key is VRAM budgeting. Each 7B model in INT4 consumes approximately 5GB for weights plus 1-2GB for KV cache. Two models sit at roughly 12-14GB, three at 18-21GB. Even with three INT4 7B models loaded, the RTX 5090 still has 11-14GB free for larger KV caches and system overhead. You can also mix model sizes, for example one 14B model plus one 7B model. For individual model VRAM needs, see our LLaMA 3 VRAM requirements guide.
VRAM Analysis
| Multi-Model Configuration | Model 1 VRAM | Model 2 VRAM | Model 3 VRAM | Total | RTX 5090 (32GB) |
|---|---|---|---|---|---|
| 2x Mistral 7B INT4 | ~5.5GB | ~5.5GB | – | ~11GB | Fits easily |
| LLaMA 3 8B INT4 + DeepSeek 7B INT4 | ~5.5GB | ~5GB | – | ~10.5GB | Fits easily |
| 3x 7B INT4 models | ~5.5GB | ~5.5GB | ~5.5GB | ~16.5GB | Fits well |
| DeepSeek 14B INT4 + Mistral 7B INT4 | ~8.5GB | ~5.5GB | – | ~14GB | Fits well |
| LLaMA 3 8B FP16 + Mistral 7B FP16 | ~16GB | ~14GB | – | ~30GB | Tight |
INT4 quantisation is essential for multi-model setups. Two FP16 7B models would consume the entire 32GB, leaving nothing for KV cache. Stick with INT4 or INT8 for multi-model configurations.
Performance Benchmarks
| Configuration | Model A (tok/s) | Model B (tok/s) | Impact vs Solo |
|---|---|---|---|
| LLaMA 3 8B INT4 solo | ~115 | – | Baseline |
| LLaMA 3 8B INT4 + Mistral 7B INT4 (sequential) | ~108 | ~105 | ~6-9% slower |
| LLaMA 3 8B INT4 + Mistral 7B INT4 (concurrent) | ~72 | ~68 | ~35-40% slower |
| 3x 7B INT4 (round-robin) | ~65 | ~63 | ~43% slower |
Sequential access (one model at a time, both loaded) incurs minimal overhead of 6-9%. Concurrent inference (both generating simultaneously) drops throughput by 35-40% per model due to memory bandwidth contention. For most real-world use cases such as agent routing and A/B testing, sequential access is sufficient and fast. See our benchmarks page for single-model throughput comparisons.
Setup Guide
Ollama natively supports multiple loaded models:
# Load both models (they persist in VRAM)
ollama run llama3:8b-instruct-q4_K_M "Hello"
ollama run mistral:7b-instruct-q4_K_M "Hello"
# Both now loaded, query either via API
curl http://localhost:11434/api/generate \
-d '{"model": "llama3:8b-instruct-q4_K_M", "prompt": "..."}'For production with vLLM, run multiple instances on different ports with VRAM limits:
# Model A: LLaMA 3 8B
vllm serve meta-llama/Meta-Llama-3-8B-Instruct \
--quantization awq --gpu-memory-utilization 0.30 \
--host 0.0.0.0 --port 8000
# Model B: Mistral 7B (separate terminal)
vllm serve mistralai/Mistral-7B-Instruct-v0.3 \
--quantization awq --gpu-memory-utilization 0.30 \
--host 0.0.0.0 --port 8001Recommended Alternative
If you need more than three models loaded or want larger models (14B+) at higher precision, consider two RTX 3090 cards for 48GB combined, each hosting separate models without VRAM contention.
For single-model performance on the 5090, see the LLaMA 3 70B INT4 guide or Mixtral 8x7B guide. For multi-modal setups, check DeepSeek + Whisper on the 5090. For budget multi-model options, see our cheapest GPU for AI inference guide. Browse all servers on our dedicated GPU hosting page.
Deploy This Model Now
Dedicated GPU servers with the VRAM you need. UK datacenter, full root access.
Browse GPU Servers
