Fine-Tuning an Embedding Model on a Dedicated GPU

General-purpose embeddings (BGE, Nomic, E5) work on almost any retrieval task, but a 2-hour fine-tune on your own data can lift recall@10 by 10-20 points in specialised domains (legal, medical, internal docs). On our dedicated GPU hosting any 16 GB+ card handles this comfortably.

Contents

• Data format
• Training
• Evaluation
• Serving

Data

You need triples: (query, positive, negative). Positives are documents that answer the query; negatives are similar-but-wrong documents. 5,000-20,000 triples is typically enough. Mine hard negatives from your own corpus using a base embedder – random negatives are too easy and teach little.

Training

sentence-transformers makes this straightforward:

from sentence_transformers import SentenceTransformer, losses, InputExample
from torch.utils.data import DataLoader

model = SentenceTransformer("BAAI/bge-base-en-v1.5")

train_examples = [
    InputExample(texts=[q, pos, neg])
    for q, pos, neg in triples
]
dataloader = DataLoader(train_examples, shuffle=True, batch_size=32)
loss = losses.TripletLoss(model=model)

model.fit(
    train_objectives=[(dataloader, loss)],
    epochs=3,
    warmup_steps=100,
    output_path="./finetuned-embedder",
)

Evaluation

Hold out 10-20% of triples. Evaluate recall@k on the holdout with a simple loop: encode queries and the entire document corpus, cosine-similarity rank, check if the positive is in the top-k. Compare against the base model – if fine-tune doesn’t improve, your data is the problem.

Serving

Deploy via Text Embeddings Inference (TEI) or a FastAPI wrapper. See BGE-M3 self-hosted for the serving pattern.

See embedding throughput benchmark for inference performance post-training.