Table of Contents
Embedding models age slowly but real. New model releases (BGE-large v1.5 → BGE-m3) and domain drift in your content can both reduce retrieval quality. The right cadence balances re-indexing cost against retrieval quality.
Three signals to re-embed: (1) measurable retrieval quality drop on eval set, (2) substantially better embedding model released (e.g., new architecture, larger context), (3) major content domain shift. Most teams re-embed every 6-18 months. Cost: ~£10-100 of GPU time for typical 1M-doc corpus.
When to re-embed
- Quality drift: eval set retrieval recall drops 5%+ over weeks/months
- Better model available: new SOTA model (e.g., BGE-m3 released 2024 over v1.5)
- Domain shift: corpus content has changed substantially (new product line, new language)
- Format changes: chunking strategy update (e.g., adopting contextual retrieval)
Don't re-embed routinely without signals — it's wasted compute.
Drift signals
Track these continuously:
- Retrieval recall@K on a held-out eval set: should be stable
- User "not helpful" feedback rate: rising indicates retrieval issue
- Query / document distribution shift: content changes that the embedding model wasn't trained for
Process
- Run eval harness against current embedding to baseline retrieval recall
- Stand up new embedding model in parallel (TEI on a different port)
- Re-embed corpus into new collection — typically ~1-3 hours of GPU time per 1M docs
- Run same eval harness against new collection
- If new collection scores higher: cut over via feature flag with gradual rollout
- Keep old collection warm for 30 days for rollback
- Decommission old collection after monitoring window
Verdict
Re-embed when signals justify it, not on a fixed schedule. For most production deployments, 6-18 months between re-embeds is the right cadence. Always run the eval harness comparison — sometimes the "newer" model isn't actually better for your specific corpus.
Bottom line
Re-embed by signal, not schedule. See embedding sizing.
