Table of Contents
For organisations of 50-500 people, internal search across knowledge silos (Notion, Confluence, SharePoint, Drive, Slack history) benefits dramatically from LLM augmentation. Self-hosted on dedicated GPU is the right architecture — data stays internal, cost predictable, brand-voice consistent.
Stack: ingest from Notion / Confluence / Drive into Qdrant + BGE-large embeddings + reranker + Mistral 7B for answer summary, all on a single 4090. Search query → hybrid retrieval → rerank → LLM-grounded answer with citations. £289/mo serves 200-500-person organisation. UK residency for sensitive internal data.
Workflow
- Periodic ingest: Notion / Confluence / Drive APIs → chunks → BGE-large embeddings → Qdrant
- User query at internal search bar
- Hybrid retrieval: BM25 + dense embedding lookup over Qdrant
- Reciprocal Rank Fusion combines scores
- BGE reranker rescores top 50 → top 10
- Mistral 7B generates answer summary with citations
- UI shows answer + clickable citations to source documents
Stack
- Ingestion workers: per-source connectors (Notion, Confluence, Drive, etc.)
- Embedding: BGE-large-en-v1.5 via TEI
- Vector store: Qdrant single-node (CPU + disk)
- Reranker: BGE-reranker-v2-m3 via TEI
- LLM: Mistral 7B FP8 via vLLM
- Auth: SSO via Auth0 / Okta; per-user permissions reflected in retrieval (filter by ACL)
Total VRAM: ~13 GB on 4090 — comfortable.
Integration
- Browser extension: trigger search from any tab via keyboard shortcut
- Slack bot:
/askcommand queries internal search - Slack home tab: search-as-you-type in dedicated tab
- API for in-app embedding: integrate into internal tools
Verdict
For internal search augmentation, single-4090 self-hosted is the right architecture for orgs of 50-500 people. £289/mo. UK residency for sensitive data. Replaces Glean / Notion AI / similar at fraction of subscription cost.
Bottom line
4090 = internal search AI. See 5060 Ti search.
