InstantID Pipeline on a GPU Server

InstantID lets you generate stylised portraits that preserve a person’s identity from a single reference photo. It layers on top of SDXL as an adapter. On our dedicated GPU hosting it runs on any 16 GB+ card.

Contents

• Pipeline components
• Deployment
• Quality tips
• Ethics

Components

• SDXL base model
• InstantID face encoder + IP-Adapter (captures identity embedding)
• InstantID ControlNet (guides facial structure)
• Optional: LoRA for specific style

Deployment

from diffusers import StableDiffusionXLInstantIDPipeline, ControlNetModel
from insightface.app import FaceAnalysis

face_app = FaceAnalysis(name="buffalo_l", providers=["CUDAExecutionProvider"])
face_app.prepare(ctx_id=0, det_size=(640, 640))

controlnet = ControlNetModel.from_pretrained("InstantX/InstantID", subfolder="ControlNetModel")
pipe = StableDiffusionXLInstantIDPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    controlnet=controlnet,
    torch_dtype=torch.float16,
).to("cuda")
pipe.load_ip_adapter_instantid("InstantX/InstantID/ip-adapter.bin")

face_info = face_app.get(reference_image_np)[0]
face_emb = face_info["embedding"]
kps = draw_kps(reference_image, face_info["kps"])

image = pipe(
    prompt="portrait in the style of Van Gogh",
    image_embeds=face_emb,
    image=kps,
    controlnet_conditioning_scale=0.8,
    ip_adapter_scale=0.8,
).images[0]

Quality

• Reference photo should be clear, well-lit, single face
• ip_adapter_scale: 0.6-0.9 controls identity strength. Higher = more similar but stiffer
• controlnet_conditioning_scale: 0.6-0.9 controls pose match
• Add “high quality, detailed” to prompts – helps SDXL-family models

Ethics

InstantID makes it trivial to generate impersonating images. Do not use for photos of people without consent. Avoid generating recognisable public figures in misleading contexts. Most commercial terms of service forbid impersonation.

See IP-Adapter and ControlNet Union.