NVIDIA RTX 6000 Pro Hosting — 96 GB on a Single GPU
The professional Blackwell card you can’t get from a hyperscaler at sane prices. 96 GB of GDDR7 with ECC, certified drivers, and the ability to serve a 70B FP16 model from one socket. Single-tenant, bare metal, UK-resident.
RTX 6000 Pro Server Specs
The hardware you actually rent.
| GPU model | NVIDIA RTX 6000 Pro Blackwell Workstation Edition |
|---|---|
| Architecture | Blackwell GB202 — 5th gen Tensor Cores |
| VRAM | 96 GB GDDR7 ECC @ 1,792 GB/s |
| CUDA cores | 24,064 |
| FP16 compute | ~ 117 TFLOPS |
| FP8 / FP4 throughput | ~ 936 / ~ 1,872 TOPS |
| TDP | 600 W (configurable down to 300 W) |
| Form factor | 2-slot blower, fits in 4U server chassis |
| Driver | NVIDIA RTX Enterprise + Studio (certified) |
| Host CPU | AMD Threadripper PRO / EPYC |
| Host RAM | Up to 256 GB ECC DDR5 |
| Storage | 3.84 TB NVMe + 16 TB SATA SSD |
| Network | 10 Gbps unmetered |
| Location | London, United Kingdom |
Models That Fit on a Single 6000 Pro
96 GB is the only single-GPU SKU we offer that runs a 70B model in FP16, a 27B at long context, or a Mixtral 8x7B without quantisation.
| Model | Params | FP16 | INT4 / FP8 | Notes |
|---|---|---|---|---|
| LLaMA 3.3 70B Instruct | 70B | 140 GB FP16 | Fits FP8 — split across 2 cards for FP16 | Tensor parallel optional |
| Qwen 2.5 72B | 72B | 144 GB FP16 | Fits FP8/INT4 single card | Production chatbot tier |
| Mixtral 8x7B Instruct | 47B (12.9B active) | 94 GB FP16 | Fits FP16 single card | MoE — full quality |
| Mixtral 8x22B | 141B (39B active) | 282 GB FP16 | Fits INT4 single card | Multi-card for FP16 |
| Gemma 2 27B | 27B | 54 GB FP16 | Fits with 64K context | Comfortable single card |
| DeepSeek V2 236B | 236B (21B active) | n/a | FP4/INT4 sharded multi-card | Cluster recommended |
| Whisper + Llama 3 + Bark stack | — | ~ 30 GB total | All three on one card concurrently | Voice agent stack |
| HunyuanVideo | 13B | 30 GB FP8 | Comfortable headroom for batch 4 | Video farm tier |
| FLUX.1 + SDXL ensemble | — | ~ 35 GB total | Both pipelines hot-loaded | Image API |
What the 96 GB Buys You
Real customer workloads we run on this hardware every day.
70B-class single-GPU serving
Llama 3.3 70B in FP8 fits comfortably on one card with vLLM. Throughput sits around 60-90 tok/s aggregate — enough for an internal copilot or a vertical SaaS product.
Long-context RAG
32B–72B models with 128K context windows pin the KV cache to several gigabytes. The 6000 Pro gives you the headroom to actually serve those windows in production.
Research & reproducibility
ECC memory eliminates a class of bit-flip bugs that bite long training and eval runs. NVIDIA Studio drivers are signed and pinned — exactly what compliance teams ask for.
Multi-model production stacks
One card serving an LLM, an embedding model, a reranker, and a TTS model concurrently — without any of them stealing VRAM from the others. Common for vertical SaaS platforms.
Full SFT of 13B+ models
A 13B full fine-tune at BF16 needs ~75 GB peak VRAM with optimiser states. The 6000 Pro is the cheapest way to do that without a multi-GPU setup.
Regulated workloads
ECC + certified drivers make it easier to pass infrastructure security reviews. The Pro line has a longer driver support lifecycle than consumer cards.
RTX 6000 Pro vs Other Single-Card Options
How this card stacks up against the rest of the GigaGPU catalogue for the workloads we benchmark.
| GPU | VRAM | Throughput / Notes | 70B INT4 fits? | Price |
|---|---|---|---|---|
| RTX 6000 Pro 96 GB | 96 GB GDDR7 ECC | 70B FP8 ✓ / 70B FP16 ✗ | Yes — 24K context | from £899 |
| RTX 5090 | 32 GB GDDR7 | 70B INT4 ✓ / 70B FP8 ✗ | No (tight) | from £399 |
| A100 80 GB | 80 GB HBM2e | 70B FP8 ✓ / 70B FP16 ✗ | Yes — 16K context | POA |
| RTX 4090 | 24 GB GDDR6X | 70B INT4 ✓ tight | No | from £289 |
| RTX 3090 | 24 GB GDDR6X | 70B INT4 ✓ tight | No | from £159 |
Deep Dive
Workstation card vs datacenter card vs consumer card
NVIDIA’s GPU lineup splits three ways. The professional "Pro" line (formerly Quadro) is the family the RTX 6000 Pro belongs to: full ECC, certified drivers, longer support, available through standard reseller channels. The datacenter line (A100 / H100 / H200) has SXM form factors, NVLink, and HBM — designed for racks of 8 cards talking to each other at >500 GB/s. The consumer line (5090, 5080, 4090) drops ECC and pro driver certification but offers the best price/perf for inference.
For a single-server, single-tenant deployment that needs more than 32 GB of VRAM, the 6000 Pro is the best card on the market. For a research cluster doing distributed BF16 training, A100/H100 still wins. For pure cost-per-token on 7B–14B chatbots, take the 5090.
ECC — does it actually matter?
For inference, rarely. A single bit flip in a model weight changes one logit slightly; the model usually recovers. For training, sometimes — especially on multi-week fine-tunes where one corrupted gradient propagates and quietly degrades quality. For research that needs reproducibility (publishing a paper, auditing a compliance pipeline), ECC removes one source of variance and one source of doubt.
If you’re shipping a regulated product, the ECC + certified-driver combination is often a non-negotiable line item from your security team. The 6000 Pro is the only single card we rent that ticks both boxes.
Why some teams pick 6000 Pro over 2× 5090
2× RTX 5090 has the same 64 GB combined VRAM as a 6000 Pro at <65% the price. So why do customers still choose the Pro?
- Single-card simplicity. No tensor parallel, no NCCL tuning, no multi-process orchestration. The Pro card just runs the model.
- 96 GB > 64 GB. If you actually need 70B FP8 with a 32K context, 64 GB is too tight.
- ECC and certified drivers. See above.
- Power and cooling. One 600 W card vs two 575 W cards. Easier in a 4U chassis.
If you’re cost-driven and your model fits in 32 GB, take the 5090. If you need the headroom or the regulatory features, the 6000 Pro earns its premium.
Frequently Asked Questions
The questions buyers actually ask before committing to a GPU server.
How is this different from the older RTX A6000 / 6000 Ada?
The Pro Blackwell is roughly 2× the FP16 throughput, 2× the VRAM (48 → 96 GB), and adds FP4 hardware. Software stack is the same — same drivers, same CUDA, same vLLM.
Can I run two 6000 Pros for 192 GB?
Yes — that’s our Cluster L+ SKU on the multi-GPU clusters page. 2× 6000 Pro is the cleanest way to serve Llama 3 70B at full FP16 with a 64K context.
Is the 600 W TDP a problem?
Not in our datacenter — N+1 power, 30°C cold aisle, validated thermal envelope. We do offer a 300 W power-capped mode if you want to run 24/7 at a lower draw.
What about the H100 / H200?
Available by request — custom build, 4–6 week lead time, "contact sales" pricing. For most production inference the 6000 Pro is the better economics.
Can I bring my own license for NVIDIA AI Enterprise?
Yes. We can also bundle it on a separate line item. Talk to sales.
Is the firmware the same as the consumer 5090?
Same Blackwell silicon, different VBIOS, different power management, ECC enabled, more aggressive thermal limits. Drivers are also different — Studio/Enterprise rather than Game Ready.
How long is the typical lead time?
Single 6000 Pro: ~5 working days. Pairs: 5–10 working days. We hold reasonable stock but cards do go in and out.
Can I run training on this?
Yes — full SFT of 13B–32B models is comfortable. For 70B BF16 training you’ll want 4× 6000 Pro or 4× A100 80 GB.
Related Pages
Pages our visitors typically read next.
Need 96 GB on a single GPU?
The RTX 6000 Pro is the only consumer-priced single card that runs Llama 3 70B in FP8 without splitting it. Talk to a specialist about your model and we’ll quote a delivery date.
