Reference glossary of self-hosted AI terms in 2026. Each entry: brief definition + link to in-depth post where applicable. Useful for onboarding and shared vocabulary.
Glossary covers ~50 key terms across hardware (FP8, FP4, KV cache), models (Llama, Mistral, Qwen, R1-Distill), serving (vLLM, TGI, TensorRT-LLM), quantisation (AWQ, GPTQ, GGUF), patterns (RAG, multi-LoRA, prefix caching), compliance (SOC 2, PCI-DSS, NHS DSPT, UK GDPR). Reference for shared team vocabulary.
A-H
- AWQ: Activation-aware Weight Quantisation; production INT4 quantisation method
- BGE: BAAI General Embeddings; the leading open-weight embedding family
- Blackwell: NVIDIA architecture (RTX 50-series, RTX 6000 Pro, B100, B200) with native FP8 + FP4
- Continuous batching: per-token request scheduling; vLLM's throughput foundation
- Cross-encoder: reranker architecture; cross-attention for relevance scoring
- DPO: Direct Preference Optimisation; alignment fine-tuning method
- FP4 / FP8: low-precision floating-point formats; native on Blackwell
- FlashAttention: memory-efficient attention algorithm; FA-3 latest
- HNSW: Hierarchical Navigable Small World; default vector index algorithm
I-P
- INT4: 4-bit integer quantisation
- KV cache: keys / values cached during attention to avoid recomputation
- LiteLLM: open-source LLM router; standard primitive for hybrid architectures
- LoRA / QLoRA: Low-Rank Adaptation; parameter-efficient fine-tuning
- MCP: Model Context Protocol; standard for LLM tool integration
- MoE: Mixture-of-Experts; sparse model architecture (Mixtral, DeepSeek V3)
- PagedAttention: vLLM's KV cache management algorithm
- Prefix caching: KV reuse for shared prompt prefixes
Q-Z
- Qdrant: leading open-source vector database
- RAG: Retrieval-Augmented Generation; the standard pattern for grounding LLM in your data
- RAGAS: RAG Assessment library; standard eval framework
- Reranker: second-stage relevance scoring after initial retrieval
- SFT: Supervised Fine-Tuning
- SGLang: alternative LLM serving framework with structured-output focus
- TEI: Text Embeddings Inference; HuggingFace embedding server
- TensorRT-LLM: NVIDIA's high-performance LLM serving library
- TGI: Text Generation Inference; HuggingFace LLM server
- vLLM: leading open-source LLM serving framework; production default
Verdict
Shared vocabulary matters for team coordination. This glossary serves as reference; new team members should familiarise themselves with these terms before depth-diving any specific area. The 2026 self-hosted AI vocabulary is reasonably stable; updates likely as FP4, MCP, multi-LoRA serving mature further.
Bottom line
Shared vocabulary; standard reference. See field guide.
