Build AI Moderation API on GPU

What You’ll Build

In 30 minutes, you will have a production moderation API that screens text, images, and user-generated content for toxicity, hate speech, harassment, spam, and custom policy violations — returning category scores, flagged segments, and action recommendations in under 50 milliseconds per item. Running on a dedicated GPU server, your API handles 15,000 moderation requests per second, keeping pace with real-time user activity across your platform.

Cloud moderation APIs charge $1-$4 per 1,000 items and apply generic policies that do not match your platform’s specific content guidelines. A gaming community, a healthcare forum, and a marketplace each have different moderation needs. Self-hosted moderation on open-source models enforces your exact policies at unlimited volume while keeping all user content on your infrastructure.

Architecture Overview

The API combines a fast toxicity classifier for real-time screening with an LLM through vLLM for nuanced policy evaluation. Every piece of content passes through the classifier first — flagged items route to the LLM for detailed analysis explaining why the content violates policy and suggesting appropriate actions (remove, warn, restrict, or approve with annotation).

The API layer accepts text, images (via a vision model), and structured content (username + text + context). Output includes toxicity scores per category, flagged spans with offsets, a policy verdict, confidence level, and recommended action. A feedback loop lets moderators override decisions, and those overrides feed back into accuracy tracking. Pair with an AI chatbot for moderator tooling that explains flagging decisions.

GPU Requirements

The toxicity classifier uses under 1GB VRAM, leaving room to co-host an LLM for detailed review on the same GPU. Most content passes the fast classifier without needing LLM review — only 5-15% of items typically get flagged for deeper analysis. See our self-hosted LLM guide for multi-model deployment patterns.

Step-by-Step Build

Deploy a toxicity classifier and vLLM on your GPU server. Build the moderation API with tiered screening and custom policy enforcement.

from fastapi import FastAPI
from transformers import pipeline
import requests

app = FastAPI()
toxicity = pipeline("text-classification",
    model="unitary/toxic-bert", device=0, top_k=None)

VLLM_URL = "http://localhost:8000/v1/chat/completions"
PLATFORM_POLICY = """Flag content that contains:
- Hate speech targeting protected groups
- Explicit threats or harassment
- Spam or commercial solicitation
- Personal information exposure
- Misinformation about health or safety"""

@app.post("/v1/moderate")
async def moderate(texts: list[str], threshold: float = 0.7):
    results = []
    for text in texts:
        # Fast classifier screening
        scores = toxicity(text)[0]
        score_dict = {s["label"]: s["score"] for s in scores}
        max_score = max(s["score"] for s in scores)

        if max_score < threshold:
            results.append({"text": text[:100], "action": "approve",
                           "scores": score_dict, "flagged": False})
            continue

        # LLM review for flagged content
        resp = requests.post(VLLM_URL, json={
            "model": "meta-llama/Llama-3-8b-chat-hf",
            "messages": [
                {"role": "system", "content": f"You are a content "
                 f"moderator. Policy:\n{PLATFORM_POLICY}"},
                {"role": "user", "content": f"Review this content:\n"
                 f"{text}\n\nReturn JSON: {{verdict, violated_rules: "
                 f"[string], severity, recommended_action, explanation}}"}
            ],
            "max_tokens": 300, "temperature": 0.1
        })
        review = resp.json()["choices"][0]["message"]["content"]
        results.append({"text": text[:100], "action": "review",
                       "scores": score_dict, "flagged": True,
                       "llm_review": review})
    return results

Customise the platform policy in the system prompt to match your content guidelines exactly. The classifier catches obvious violations instantly while the LLM evaluates borderline cases with nuance. The OpenAI-compatible endpoint powers the LLM review stage. See production setup for real-time streaming integration.

Policy Customisation and Accuracy

Every platform has unique moderation needs. Define your policies in the system prompt, including specific examples of allowed and disallowed content for borderline cases. Track false positive and false negative rates by having human moderators review a sample of decisions weekly. Adjust classifier thresholds per category — lower thresholds for high-risk categories like threats, higher for subjective categories like mild profanity.

Build moderator dashboards showing flagged content with AI explanations, one-click approve/reject actions, and trend analytics. The feedback from moderator overrides improves your understanding of where the model and your policy definitions diverge, enabling iterative refinement.

Deploy Your Moderation API

A self-hosted moderation API enforces your exact content policies at platform scale without per-request fees or user content leaving your infrastructure. Protect your community with real-time screening and nuanced policy enforcement. Launch on GigaGPU dedicated GPU hosting and start moderating. Browse more API use cases and tutorials in our library.