Flux.1 Deployment Guide: dev, schnell, and pro on Dedicated GPUs

Flux.1 from Black Forest Labs is the strongest open-weights image model of the current generation, and the de facto replacement for SDXL in commercial pipelines that need typography, prompt fidelity and photoreal output. The trade-off is size: twelve billion parameters with a T5-XXL text encoder bolted on. This guide walks through deploying every variant – dev, schnell and pro – on dedicated GPU hosting, with concrete VRAM, throughput and licence numbers, so a team can pick a GPU, pick a runtime, and ship a Flux.1 endpoint without three weeks of trial and error. Pair this with the best GPU for Flux overview and the RTX 4090 24GB Flux schnell benchmark for single-SKU reference data.

Flux.1 Model Variants

Three official variants ship under the Flux.1 umbrella. They share an architecture – a rectified flow transformer with double-stream and single-stream blocks – but differ on weights, licence and number of denoising steps required for a usable image. Picking the right one is the most important deployment decision; everything else flows from it.

Variant	Parameters	Licence	Steps to converge	Self-hostable	Best for
Flux.1 dev	12B	Non-commercial (FLUX.1 [dev] Non-Commercial)	20-50	Yes	Internal tooling, research, demos
Flux.1 schnell	12B	Apache 2.0	1-4 (distilled)	Yes	Commercial production, high throughput
Flux.1 pro	~12B (proprietary)	API only, BFL-hosted	~25 (server-side)	No	Quality-first commercial use without infra

Pro is closed-weights and API-only, so the rest of this guide concentrates on dev and schnell. Dev is the highest-quality open variant; schnell is the four-step distilled version and the only one that is commercially clean. Most production deployments run schnell for the user-facing path and dev for high-quality batch jobs that stay inside the firewall.

VRAM Requirements by Precision

The transformer is the dominant memory cost, but the T5-XXL text encoder is what catches teams out. T5-XXL alone is 9 GB in FP16, and it is loaded every time you change a prompt unless you keep it resident. Plan for the encoder, not just the diffusion model.

Component	FP16	FP8 (e4m3)	GGUF Q8	GGUF Q5_K_M	GGUF Q4_K_S
Flux.1 dev/schnell transformer	23.8 GB	11.9 GB	12.7 GB	8.4 GB	6.6 GB
T5-XXL text encoder	9.1 GB	4.8 GB	5.2 GB	3.6 GB	3.0 GB
CLIP-L text encoder	0.25 GB	0.25 GB	0.25 GB	0.25 GB	0.25 GB
VAE (16-channel)	0.34 GB	0.34 GB	0.34 GB	0.34 GB	0.34 GB
Activations + KV (1024×1024)	~3.5 GB	~2.0 GB	~2.5 GB	~2.0 GB	~1.8 GB
Total resident peak	~37 GB	~19 GB	~21 GB	~14.5 GB	~12 GB
With T5 CPU-offloaded	~28 GB	~14 GB	~16 GB	~11 GB	~9 GB

Two takeaways. FP16 dev with T5 resident does not fit on a 24 GB card; you have to either offload T5 to CPU (adds 0.5-1.2 s of first-prompt latency) or step down to FP8. GGUF Q8 is functionally larger than FP8 because of dequantisation overhead – on Ada and Hopper, FP8 native wins. Q5 and Q4 GGUF only make sense on GPUs without FP8 tensor core support.

GPU Recommendations

The picture is cleaner than it was for SDXL. Flux is large enough that VRAM dominates – if the model and encoder do not fit, no amount of compute will save you.

GPU	VRAM	FP8 native	Best Flux config	Notes
RTX 4060 Ti 16GB	16 GB	No	GGUF Q5/Q4 with T5 offload	Slow (4-5x a 4090), but viable for low-volume internal use
RTX 5060 Ti 16GB	16 GB	Yes	FP8 schnell, Q5 dev	Blackwell FP8 makes 16 GB workable for schnell
RTX 3090 24GB	24 GB	No	FP16 dev with T5 offload	Strong value if you find them; no FP8, so capped at FP16 throughput
RTX 4090 24GB	24 GB	Yes	FP8 dev, FP16 schnell	The sweet spot – see spec breakdown
RTX 5090 32GB	32 GB	Yes (improved)	FP16 dev resident, batch 2-4	Headroom for ControlNet stacks; 4090 vs 5090 comparison
A100 80GB	80 GB	No (BF16 only)	FP16 dev, large batches, multi-tenant	Production multi-user; no FP8, but raw bandwidth wins
H100 80GB	80 GB	Yes	FP8 dev, batch 8+	Highest throughput per node; overkill for single-stream

For single-tenant, single-stream workloads the 4090 is the right answer for almost everyone. For multi-tenant SaaS where many users hit one GPU concurrently, the A100 80GB or H100 wins on cost-per-image because batched inference scales near-linearly until memory bandwidth saturates. The best GPU for inference guide covers the same trade-offs in more depth.

Throughput Numbers

Numbers below are end-to-end per-image latency at 1024×1024, including text encoding and VAE decode. Schnell defaults to 4 steps, dev to 28 steps unless noted. T5 is resident on cards with sufficient VRAM, offloaded otherwise (penalty of about 0.4-0.8 s on first generation per prompt change).

GPU	Schnell 4-step (FP8/FP16)	Dev 28-step	Dev 50-step	Schnell batch (img/min)
RTX 4060 Ti 16GB (Q5)	3.4 s	32 s	56 s	17
RTX 5060 Ti 16GB (FP8)	1.9 s	17 s	30 s	30
RTX 3090 24GB (FP16)	2.4 s	22 s	39 s	24
RTX 4090 24GB (FP8)	0.85 s	12.6 s	22.4 s	62
RTX 4090 24GB (FP16)	1.80 s	22 s	39 s	32
RTX 5090 32GB (FP16)	1.10 s	16.8 s	29.5 s	52
RTX 5090 32GB (FP8)	0.62 s	9.4 s	16.7 s	88
A100 80GB (BF16, batch 4)	4.6 s / batch	38 s / batch	67 s / batch	52
H100 80GB (FP8, batch 4)	1.9 s / batch	16 s / batch	28 s / batch	126

The single most important number is the schnell-FP8 line on the 4090: under one second per image at full 1024×1024. That latency band is what makes Flux.1 a viable replacement for the OpenAI Images API for self-hosters – end users do not perceive sub-second generation as a wait. A 5090 at FP8 cuts that to 620 ms and lets you batch four prompts inside a 2.5 s response window. For more on whether self-hosting beats API economics, the cost-per-1M-tokens vs OpenAI analysis covers the LLM side; the same break-even logic applies to diffusion, with the crossover usually around 5,000-15,000 images per day per GPU.

Inference Servers and Pipelines

Three runtimes dominate. Pick one, do not try to run two side by side – they fight over the same model files and the same GPU memory.

Runtime	Strengths	Weaknesses	Best for
ComfyUI	Most flexible, native Flux support, GGUF via custom nodes, fastest inference, lowest VRAM overhead	Node graph not API-friendly out of the box; needs wrapper for REST	Internal tooling, batch jobs, anything ControlNet-heavy
diffusers (Python)	Programmatic control, easy to wrap in FastAPI, Hugging Face ecosystem, best for fine-tuning	Slightly higher VRAM overhead than ComfyUI; FP8 needs torchao	Custom commercial APIs, integration into existing Python stacks
AUTOMATIC1111 / Forge	Familiar UI, wide extension ecosystem, web frontend ready	Newer Flux support; less mature than ComfyUI for advanced Flux workflows; more memory overhead	Existing A1111 shops migrating to Flux

ComfyUI install (Ubuntu 24.04) – covered in detail in the ComfyUI on RTX 4090 setup guide:

sudo apt update && sudo apt install -y python3.11 python3.11-venv git
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
python3.11 -m venv venv && source venv/bin/activate
pip install --upgrade pip
pip install torch==2.5.1 torchvision --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt
# GGUF support for low-VRAM cards
cd custom_nodes && git clone https://github.com/city96/ComfyUI-GGUF
cd .. && python main.py --listen 0.0.0.0 --port 8188 --highvram

diffusers minimal example – the FluxPipeline wraps the whole stack:

import torch
from diffusers import FluxPipeline

pipe = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-schnell",
    torch_dtype=torch.bfloat16,
)
pipe.enable_model_cpu_offload()  # only if <24 GB VRAM
# Or, on 4090+: pipe.to("cuda")

image = pipe(
    prompt="a glass of water on a wooden table, photoreal, 50mm lens",
    num_inference_steps=4,
    guidance_scale=0.0,         # schnell uses CFG 0
    height=1024, width=1024,
    max_sequence_length=256,    # schnell caps at 256
).images[0]
image.save("out.png")

For dev, change the model id to black-forest-labs/FLUX.1-dev, set num_inference_steps=28 and guidance_scale=3.5, raise max_sequence_length to 512. If you need PyTorch installed cleanly first, the PyTorch on a GPU server guide covers driver + CUDA + wheel pinning end to end.

Forge install is identical to A1111 with the Forge fork; Flux support arrived in 2024 and has stabilised. It is the right answer if you already have an A1111 estate and need Flux on the same UI; for greenfield deployments, prefer ComfyUI or diffusers.

Quantisation Paths

You have three viable quantisation strategies, each with a different cost/quality profile.

Path	VRAM saving	Quality loss	Hardware	Tooling
FP8 native (e4m3)	~50%	Near-lossless (PSNR drop <0.3 dB)	Ada (40-series), Blackwell (50-series), Hopper (H100)	torchao float8, ComfyUI native
GGUF Q8_0	~47%	Imperceptible	Any CUDA card	city96/FLUX.1-dev-gguf, ComfyUI-GGUF
GGUF Q5_K_M	~65%	Slight (small detail loss in dense scenes)	Any CUDA card	city96 GGUFs
GGUF Q4_K_S	~72%	Visible on portraits and typography	Any CUDA card, including 12 GB	city96 GGUFs
Schnell (distilled, not quantised)	0% but 7-12x faster	Slightly less detail than dev	Same as dev	Black Forest Labs FLUX.1-schnell

The decision tree is short. On Ada or newer (4060 Ti and up): use FP8 native. On Ampere or older (3090, A100, A6000): use BF16 if it fits, otherwise GGUF Q8. On 16 GB or less without FP8: GGUF Q5_K_M is the floor for commercial-quality output. For a deeper take on AWQ, GPTQ and the broader quantisation landscape that applies to diffusion as much as LLMs, see the RTX 4090 AWQ quantisation guide. Note that for high-throughput pipelines, switching from dev to schnell is a bigger throughput win than any quantisation – 4 steps versus 28-50 is a 7-12x speedup at almost identical visual quality for most prompts.

Production Deployment Recipe

The production pattern that has held up across our deployments is diffusers + FastAPI + Uvicorn behind nginx, packaged in a Docker image with a pinned PyTorch wheel. The model is warmed at boot so the first request does not eat the cold-start cost.

app.py:

import io, os, time
from contextlib import asynccontextmanager
import torch
from fastapi import FastAPI, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
from diffusers import FluxPipeline

MODEL = os.environ.get("FLUX_MODEL", "black-forest-labs/FLUX.1-schnell")
STEPS = int(os.environ.get("FLUX_STEPS", "4"))
CFG   = float(os.environ.get("FLUX_CFG", "0.0"))
pipe  = None

@asynccontextmanager
async def lifespan(app: FastAPI):
    global pipe
    pipe = FluxPipeline.from_pretrained(MODEL, torch_dtype=torch.bfloat16).to("cuda")
    # Warm-up: 256x256 throwaway to compile kernels
    _ = pipe("warmup", num_inference_steps=1, height=256, width=256,
             guidance_scale=CFG, max_sequence_length=256).images[0]
    yield
    del pipe; torch.cuda.empty_cache()

app = FastAPI(lifespan=lifespan)

class GenReq(BaseModel):
    prompt: str
    width: int = 1024
    height: int = 1024
    steps: int | None = None
    seed: int | None = None

@app.post("/generate")
def generate(req: GenReq):
    if pipe is None:
        raise HTTPException(503, "model loading")
    g = torch.Generator(device="cuda")
    if req.seed is not None: g.manual_seed(req.seed)
    img = pipe(req.prompt, height=req.height, width=req.width,
               num_inference_steps=req.steps or STEPS,
               guidance_scale=CFG, generator=g,
               max_sequence_length=256).images[0]
    buf = io.BytesIO(); img.save(buf, format="PNG"); buf.seek(0)
    return StreamingResponse(buf, media_type="image/png")

@app.post("/batch")
def batch(reqs: list[GenReq]):
    prompts = [r.prompt for r in reqs]
    imgs = pipe(prompts, num_inference_steps=STEPS,
                guidance_scale=CFG, max_sequence_length=256).images
    return {"count": len(imgs)}  # in production, upload to S3 and return URLs

@app.get("/healthz")
def healthz():
    return {"status": "ok" if pipe is not None else "loading",
            "model": MODEL, "vram_gb": torch.cuda.memory_allocated() / 1e9}

docker-compose.yml:

services:
  flux:
    image: nvidia/cuda:12.4.1-runtime-ubuntu24.04
    runtime: nvidia
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    environment:
      - FLUX_MODEL=black-forest-labs/FLUX.1-schnell
      - FLUX_STEPS=4
      - HUGGING_FACE_HUB_TOKEN=${HF_TOKEN}
    volumes:
      - ./app:/app
      - hf-cache:/root/.cache/huggingface
    working_dir: /app
    command: >
      bash -c "pip install torch==2.5.1 --index-url https://download.pytorch.org/whl/cu124 &&
               pip install diffusers==0.31.0 transformers==4.46.0 accelerate==1.1.1 fastapi uvicorn sentencepiece protobuf &&
               uvicorn app:app --host 0.0.0.0 --port 8000"
    ports: ["8000:8000"]
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/healthz"]
      interval: 30s
      timeout: 5s
      retries: 3
volumes:
  hf-cache:

Dependency pinning warning. The diffusers, torch and transformers triple is brittle around Flux. diffusers below 0.30 has no native Flux support; above 0.32 has changed pipeline argument names twice. Pin all three. Upgrade in a staging container, run a 50-image regression suite, then promote. Skipping this step is the single most common cause of “the same prompt produces different images this morning” tickets. If you are running multi-model fleets, the same vLLM-style production discipline from the vLLM production setup guide applies. Add GPU monitoring from day one – Flux is memory-hungry enough that a leaked tensor will OOM before you notice from latency alone.

Common Gotchas

Gotcha	Symptom	Fix
Flux.1 dev licence is non-commercial	Legal/compliance flags in audit	Use schnell (Apache 2.0) or pay BFL for a commercial dev licence
T5-XXL eats 9 GB FP16	OOM on 24 GB cards in FP16 dev	Use FP8/Q8 T5, or set enable_model_cpu_offload()
schnell quality degrades above 4 steps	Soft, washed-out output at 8-20 steps	Cap schnell at 4 steps; switch to dev if you need more
schnell needs CFG 0.0, dev needs CFG 3.5	Schnell at CFG 7 produces saturated noise	Read the model card; defaults differ from SDXL
ComfyUI vs Forge node compatibility	Workflows do not port between runtimes	Standardise on one runtime per environment
Aggressive attention slicing	2-3x latency penalty	Only enable slicing on cards that genuinely need it (16 GB or less)
ControlNet support is patchy	Only XLabs and InstantX ControlNets work; SDXL CNs do not	Use Flux-specific ControlNets; expect smaller selection than SDXL
max_sequence_length default is wrong	schnell silently truncates; dev wastes VRAM	schnell: 256 tokens. dev: 512 tokens. Set explicitly.
HF token gating	Model download fails with 401	Accept the licence on Hugging Face, set HUGGING_FACE_HUB_TOKEN
16-channel VAE in FP8 produces colour drift	Slight green/magenta cast	Keep VAE in FP16; only quantise the transformer

The licence gotcha is the one that has cost teams the most time. Flux.1 dev is explicitly not for commercial use without a paid licence from Black Forest Labs – including using it to generate marketing assets, product imagery or any output that ends up in a paid product. Schnell is Apache 2.0 and has none of these restrictions, which is why most production deployments end up running schnell even when dev quality is marginally higher. If your pipeline is for video generation rather than stills, the AI video generation hosting page covers the related model families with similar licence considerations.

Verdict

The right Flux.1 deployment depends almost entirely on whether the output is leaving your firewall or not.

• Hobby or internal tooling, no commercial use, low volume. Flux.1 dev in GGUF Q5 on a 16 GB RTX 4060 Ti, served through ComfyUI. Around 30 seconds per image at 28 steps; cheap to host, no licence drama because the output never leaves the office. Total cost of ownership is dominated by the GPU lease itself.
• Commercial small-scale, single-tenant API. Flux.1 schnell at FP16 (or FP8 if available) on an RTX 4090 24GB. Sub-2-second latency per 1024×1024 image, 32-62 images per minute at batch 1. Apache 2.0 licence, so output can ship in paid products. Wrap in the FastAPI recipe above. This is the default recommendation for nine out of ten deployments.
• Commercial production, multi-tenant SaaS. Flux.1 dev FP8 on an RTX 5090 32GB or A100/H100 80GB, served through diffusers in a batched serving mode. Buy the dev commercial licence from BFL. Use the headroom for ControlNet, IP-Adapter and LoRA stacks. Plan for at least one warm spare per region.
• Quality-first, no infra appetite. Flux.1 pro through the Black Forest Labs API. You will pay per call, but you skip every operational concern in this guide. Once volumes pass roughly 5,000 images/day, self-hosting schnell on a 4090 wins on cost – but until then, the API is the rational choice.

Flux.1 has reset the bar for self-hostable image generation. With the right GPU and a disciplined deployment recipe, it gives you OpenAI-Images-grade output at a fraction of the per-image cost, on hardware you control. The two decisions that matter most are which variant fits your licence (almost always schnell for commercial work) and which GPU fits your throughput target (almost always a 4090 for single-tenant, an A100/H100 for multi-tenant). Everything else is execution.

Ready to deploy? Spin up a Flux-ready dedicated GPU on the GigaGPU UK network with hourly or monthly billing and full root access. Browse dedicated GPU hosting plans – RTX 4090 24GB, RTX 5090 32GB and A100 80GB configurations are available with NVMe storage, 1 Gbps uplinks and the Ubuntu 24.04 image used throughout this guide.