RTX 5060 Ti 16GB Reranker Server

Rerankers re-score candidate passages for higher RAG quality. TEI serves them nicely alongside your embedding server on the RTX 5060 Ti 16GB at our hosting.

Contents

• Deploy
• API
• Integrate into RAG
• Model picks

Deploy with TEI

docker run --gpus all -p 8081:80 \
  -v $PWD/tei-rerank:/data \
  ghcr.io/huggingface/text-embeddings-inference:cuda-1.5 \
  --model-id BAAI/bge-reranker-base \
  --max-batch-tokens 32768

API

curl http://localhost:8081/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "query": "When did we launch the product?",
    "texts": [
      "We launched in June 2024.",
      "Our office is in London.",
      "Product design started Jan 2024."
    ]
  }'

Response contains scored candidates sorted by relevance.

Integrate into RAG

1. Embed query, retrieve top-100 from vector DB
2. POST query + top-100 to rerank endpoint
3. Take top-4 reranked candidates
4. Pass those to the LLM as context

Latency: ~30-60 ms to rerank 100 candidates on BGE-reranker-base. Worth every millisecond – NDCG@10 typically improves 10-15%.

Model Picks

Default: BGE-reranker-base for English RAG. Upgrade to BGE-reranker-large for quality-critical workloads.

See also: throughput numbers, embedding server, RAG stack, SaaS RAG.