What You’ll Build
In 30 minutes, you will have a production NER API that extracts named entities from text — people, organisations, locations, dates, monetary amounts, and custom entity types — with character-level offsets and confidence scores. Running on a dedicated GPU server, your API processes 10,000 documents per minute using specialised transformer models, with an LLM fallback for custom entity extraction that no pre-trained NER model covers.
Cloud NER services charge $1-$5 per 1,000 text units and limit entity types to their pre-defined categories. When you need to extract custom entities — drug names from clinical notes, contract clause types from legal documents, or product codes from invoices — cloud APIs fall short. Self-hosted NER on open-source models handles both standard and custom entity types at unlimited volume.
Architecture Overview
The API runs two extraction paths. A fine-tuned token classification model (spaCy transformer or GLiNER) handles standard entity types at maximum throughput. For custom entity types, an LLM through vLLM extracts entities using few-shot prompting — you define the entity types and examples, and the model finds them in new text without training data.
The API layer accepts single texts, document batches, and HTML payloads (stripping markup before extraction). Output includes entity text, type label, character offsets, confidence score, and optional entity linking to a knowledge base. A deduplication step merges entities that appear multiple times in different forms (e.g., “Microsoft” and “MSFT”).
GPU Requirements
| Extraction Mode | Recommended GPU | VRAM | Throughput |
|---|---|---|---|
| Standard NER (classifier) | RTX 5090 | 24 GB | ~10k docs/min |
| Custom entities (8B LLM) | RTX 5090 | 24 GB | ~400 docs/min |
| Complex extraction (70B) | RTX 6000 Pro 96 GB | 80 GB | ~150 docs/min |
NER classifier models are compact — under 1GB VRAM — and co-host easily with an LLM on the same GPU. Route standard entity extraction through the fast classifier and custom types through the LLM. See our self-hosted LLM guide for multi-model deployment.
Step-by-Step Build
Deploy a transformer NER model and vLLM on your GPU server. Build the API with both extraction paths and entity merging.
from fastapi import FastAPI
from gliner import GLiNER
import requests
app = FastAPI()
ner_model = GLiNER.from_pretrained("urchade/gliner_multi_pii-v1")
VLLM_URL = "http://localhost:8000/v1/chat/completions"
@app.post("/v1/ner")
async def extract_entities(texts: list[str],
entity_types: list[str] = None,
mode: str = "auto"):
default_types = ["person", "organization", "location",
"date", "money", "email", "phone"]
types = entity_types or default_types
if mode == "fast" or (mode == "auto" and not entity_types):
results = []
for text in texts:
entities = ner_model.predict_entities(text, types)
results.append({
"text": text,
"entities": [{"text": e["text"], "type": e["label"],
"start": e["start"], "end": e["end"],
"confidence": e["score"]}
for e in entities]
})
return results
# LLM path for custom entity types
results = []
for text in texts:
resp = requests.post(VLLM_URL, json={
"model": "meta-llama/Llama-3-8b-chat-hf",
"messages": [{"role": "user", "content":
f"""Extract these entity types from the text.
Entity types: {types}
Text: {text}
Return JSON array: [{{text, type, start_char, end_char}}]"""}],
"max_tokens": 500, "temperature": 0.1
})
results.append(resp.json()["choices"][0]["message"]["content"])
return resultsAdd entity linking by matching extracted entities against your knowledge base or Wikidata for disambiguation. GLiNER supports zero-shot NER for any entity type you specify, bridging the gap between fixed classifiers and full LLM extraction. The OpenAI-compatible endpoint handles the LLM path. See production setup for batch processing configuration.
Custom Entities and Domain Adaptation
Define domain-specific entity types without training data using the LLM extraction path. Legal documents need clause types, party names, and statutory references. Medical texts need drug names, dosages, conditions, and procedures. Financial documents need ticker symbols, account identifiers, and regulatory references. Specify your entity types in the API call and the LLM extracts them.
For high-volume custom extraction, fine-tune GLiNER on a small labelled dataset (100-500 examples) from your domain. This gives you classifier-speed extraction for your custom types. Track extraction accuracy with periodic human review and adjust confidence thresholds per entity type.
Deploy Your NER API
A self-hosted NER API extracts structured knowledge from unstructured text at any scale. Power document processing, compliance scanning, data enrichment, and AI chatbot context extraction. Launch on GigaGPU dedicated GPU hosting and start extracting entities. Browse more API use cases and tutorials in our library.
