Medical Image Generation: Synthetic Data on GPU

The Challenge: Not Enough Data, Too Many Constraints

A Manchester-based medical AI startup is developing a skin lesion classification model targeting melanoma, basal cell carcinoma, and seborrhoeic keratosis. Their dermatologist advisors agree the model needs at least 50,000 annotated images across all classes and skin tones to achieve diagnostic-grade accuracy. The startup has secured data sharing agreements covering 8,000 real patient images from two NHS trusts — far short of the target. Expanding to additional trusts would take 12-18 months of ethics approvals and information governance negotiations.

Public datasets like ISIC exist but suffer from severe class imbalance and limited representation of darker skin tones — a known problem that leads to models performing poorly on non-Caucasian patients. The startup needs to generate synthetic medical images that augment their real dataset with controlled variation in lesion morphology, skin tone, lighting conditions, and anatomical location. Every synthetic image must be generated on infrastructure where the training data — real patient photographs — never leaves UK jurisdiction.

AI Solution: Conditional Diffusion Models for Medical Imagery

Diffusion models fine-tuned on medical image datasets can generate photorealistic synthetic pathology that is clinically useful for training downstream classifiers. The approach involves fine-tuning a Stable Diffusion variant on the 8,000 real images with conditioning labels for diagnosis, Fitzpatrick skin type, lesion size, and body location. The fine-tuned model then generates novel images that exhibit realistic morphological variation while statistically matching the characteristics of real clinical photographs.

Crucially, synthetic images trained this way do not contain identifiable patient information. A dermatoscopic image of a synthetic melanoma does not correspond to any real patient — it is a statistical interpolation learned from the training distribution. This makes synthetic data shareable across institutions without triggering data protection obligations, potentially unlocking collaborative model development that real patient data cannot support.

GPU Requirements: Generating Thousands of High-Resolution Images

Medical image generation demands higher fidelity than typical creative applications. Dermatoscopic images are typically 1024×1024 or higher, and each must pass clinical plausibility review. Fine-tuning the diffusion model requires significant VRAM, and generation throughput determines how quickly the 42,000-image synthetic supplement can be produced.

An RTX 6000 Pro on GigaGPU dedicated hosting generates the full synthetic dataset in roughly five days. More importantly, the same GPU handles the iterative cycle — generate a batch, have clinicians review a sample, adjust conditioning parameters, regenerate — that is essential for producing clinically useful synthetic data.

Recommended Stack

• Stable Diffusion XL or Stable Diffusion 3 as the base architecture, fine-tuned using LoRA or DreamBooth on the clinical image dataset.
• ControlNet for conditioning on structural attributes — specifying lesion shape, border irregularity, and colour distribution.
• AUTOMATIC1111 or ComfyUI for rapid prototyping during the conditioning design phase.
• FID and KID metrics computed against held-out real images to quantitatively validate synthetic image quality.
• A vision model (e.g., DINOv2) for computing embedding-space similarity between synthetic and real images — a stronger quality signal than pixel-level metrics.

Once synthetic images pass clinical review, the downstream classification model can be trained on the combined real + synthetic dataset using the same GPU infrastructure. An open-source LLM can generate the textual annotations (clinical descriptions, differential diagnosis notes) that accompany each synthetic image in the training set.

Cost vs. Alternatives

Acquiring additional real patient data is the primary alternative, with costs including ethics committee fees (£2,000-£5,000 per application), data sharing agreement legal costs (£3,000-£10,000 per trust), and the 12-18 month timeline that threatens the startup’s product roadmap. Commercial synthetic data services for medical imaging quote £30-£80 per image for clinically validated output, putting a 42,000-image order at £1.26M-£3.36M — clearly non-viable for a startup.

Generating synthetic data in-house on private GPU infrastructure costs a fraction of that. The startup maintains full ownership of the generative model, can produce unlimited additional images as needed, and controls the entire data pipeline within GDPR-compliant infrastructure.

Getting Started

Begin with a proof-of-concept: fine-tune Stable Diffusion XL on 2,000 images from a single diagnostic class. Generate 500 synthetic images and have two dermatologists independently assess clinical plausibility on a 5-point Likert scale. Benchmark a classifier trained on real-only versus real+synthetic data to quantify the accuracy uplift before committing to full-scale generation.

GigaGPU provides dedicated GPU servers with the VRAM and storage medical image generation demands. Patient data stays on UK soil throughout fine-tuning and generation, and the synthetic output is yours to share freely with collaborators worldwide.