Creative AI workflows often need two brains: an LLM to write the prompt or caption, and an image model to generate the visual. We tested whether the RTX 3090 (24 GB VRAM) can run both Stable Diffusion XL and LLaMA 3 8B concurrently on a GigaGPU dedicated server. The short answer: it can, but you will feel the VRAM squeeze.
Models tested: Stable Diffusion XL + LLaMA 3 8B
Concurrent Benchmarks
| Component | Metric | Solo | Concurrent |
|---|---|---|---|
| SDXL 1.0 | Iterations/sec | 3.2 | 1.92 |
| LLaMA 3 8B (FP16) | Tokens/sec | 62 | 34.1 |
All models loaded simultaneously in GPU memory. Throughput figures reflect concurrent operation with shared VRAM and compute.
VRAM Reality Check
| Component | VRAM |
|---|---|
| Combined model weights | 25.5 GB |
| Total RTX 3090 VRAM | 24 GB |
| Free headroom | ~0.0 GB |
This is the tightest fit in our multi-model benchmark suite. SDXL alone needs around 9.5 GB, and LLaMA 3 8B at FP16 demands 16 GB. The 3090 manages through aggressive memory management, but there is zero headroom for extras. If you hit out-of-memory errors under load, switch the LLM to 4-bit quantisation — that drops its footprint to roughly 5 GB and gives you a much more comfortable ~14 GB buffer.
The Price of Consolidation
| Cost Metric | Value |
|---|---|
| Server cost (single GPU) | £0.75/hr (£149/mo) |
| Equivalent separate GPUs | £1.50/hr |
| Savings vs separate servers | 50% |
Even with reduced throughput, running both workloads on one GPU at £149/mo is a significant saving over two separate machines. This matters for indie developers and small teams building AI art tools that use an LLM for prompt refinement or image captioning alongside SDXL for generation. See all GPU options at our benchmark page.
Practical Guidance
The 3090 is viable for SDXL + LLM if your workflow alternates between the two models rather than hammering both simultaneously. Generate a batch of images, then use the LLM to caption them — that pattern works well because each model gets full GPU bandwidth during its turn. For truly concurrent use where both models must sustain high throughput at the same time, the RTX 5090 with its 32 GB VRAM is a much more comfortable option.
Quick deploy:
docker compose up -d # comfyui + llama.cpp containers with --gpus allSee our LLM hosting guide, GPU server options, and all benchmark results. Related benchmarks: LLaMA 3 8B on RTX 3090, SDXL on RTX 3090.
Deploy SDXL + LLM Pipeline on RTX 3090
Order this exact configuration. UK datacenter, full root access.
Order RTX 3090 Server
