Most 7B-class models need quantisation to fit on mid-range GPUs. Phi-3 Mini is different. With only 3.8 billion parameters, it loads at full FP16 precision on the RTX 4060 Ti and still leaves half the card’s 16 GB free. Here is exactly what that buys you in practice, tested on GigaGPU dedicated hardware.
Performance at Full Precision
| Metric | Value |
|---|---|
| Tokens/sec (single stream) | 28 tok/s |
| Tokens/sec (batched, bs=8) | 44.8 tok/s |
| Per-token latency | 35.7 ms |
| Precision | FP16 |
| Quantisation | FP16 |
| Max context length | 8K |
| Performance rating | Good |
Measured with single-stream generation, 512-token prompt, 256-token completion. Backend: llama.cpp / ONNX Runtime.
VRAM Breakdown — Room to Spare
Running FP16 means zero quality loss from quantisation, and the memory picture is comfortable:
| Component | VRAM |
|---|---|
| Model weights (FP16) | 8.0 GB |
| KV cache + runtime | ~1.2 GB |
| Total RTX 4060 Ti VRAM | 16 GB |
| Free headroom | ~8.0 GB |
That 8 GB of headroom is genuinely useful. You can extend context to 8K tokens, run a second lightweight model alongside Phi-3, or handle several concurrent inference streams without swapping.
Cost Analysis
| Cost Metric | Value |
|---|---|
| Server cost | £0.50/hr (£99/mo) |
| Cost per 1M tokens | £4.960 |
| Tokens per £1 | 201,613 |
| Break-even vs API | ~1 req/day |
At £4.96 per million tokens single-stream, the 4060 Ti already undercuts most hosted APIs. Batching drops the effective rate to around £3.10/M. The flat £99/mo pricing on a GigaGPU RTX 4060 Ti server means costs stay predictable no matter how much traffic you push. Our cost-per-million-tokens calculator can help model your specific workload.
Verdict
Twenty-eight tokens per second is snappy enough for real-time chat and internal tooling. The FP16 advantage means you preserve the model’s full instruction-following fidelity — important if you are evaluating Phi-3 before deploying at scale. When you outgrow this tier, the RTX 3090 roughly doubles throughput.
Deploy in seconds:
docker run --gpus all -p 8080:8080 ghcr.io/ggerganov/llama.cpp:server -m /models/phi-3-mini.Q4_K_M.gguf --host 0.0.0.0 --port 8080 -ngl 99Dive deeper in our Phi-3 hosting guide, explore best GPUs for LLM inference, or browse the full benchmark library.
Get a Phi-3 Mini Server on the RTX 4060 Ti
Full FP16, no quantisation compromises. UK datacentre with root access.
Order RTX 4060 Ti Server
