Embedding Generation: Cost at 100M Tokens/Month
What does it cost to run embedding generation at 100M tokens/month? Self-hosted dedicated GPU vs API provider pricing.
Monthly Cost Comparison at 100M tokens/month
| Provider | Monthly Cost | Pricing Model | vs GigaGPU |
|---|---|---|---|
| GigaGPU (RTX 5060 Ti) | £119/mo | Fixed | — |
| OpenAI text-embedding-3-small | £2/mo | Per-tokens | API is cheaper at this volume |
| Cohere Embed v3 | £10/mo | Per-tokens | API is cheaper at this volume |
| Voyage AI | £12/mo | Per-tokens | API is cheaper at this volume |
Embeddings Are Cheap via API — Until They Are Not
At 100M tokens/month, embedding APIs are genuinely affordable. OpenAI text-embedding-3-small costs just £2/month, and even Voyage AI tops out at £12. A dedicated RTX 5060 Ti at £119/month is significantly more expensive for embeddings alone at this volume.
So why would anyone self-host embeddings at 100M tokens? Because embeddings rarely exist in isolation. If you are already running an LLM, RAG pipeline, or search system on dedicated hardware, adding an embedding model to the same GPU costs nothing extra. The £119/month you are paying for your LLM server already includes the capacity for embedding generation as a side task.
Annual savings potential: Varies — at this volume, self-hosting is primarily justified by data privacy, latency, or bundling with other GPU workloads rather than cost alone.
Reasons to Self-Host Embeddings Despite API Pricing
- Data sensitivity: Every document you send to an embedding API is processed on external servers. For proprietary research, legal documents, or medical records, this is a non-starter.
- Bundled workloads: If you already run an LLM or search pipeline on a GPU server, embedding generation is essentially free — the hardware is already paid for.
- Custom models: Train domain-specific embeddings that outperform generic API models on your particular data distribution.
- Zero-latency co-location: Embeddings generated on the same server as your vector database eliminate network overhead entirely.
When Embedding APIs Are the Right Choice
- Standalone embedding workloads: If embeddings are your only GPU task, £2-12/month via API is far cheaper than £119/month for dedicated hardware.
- No infrastructure team: API calls require zero server management, monitoring, or capacity planning.
- Model evaluation: Quickly benchmark different embedding models via API before committing to a self-hosted deployment.
Hardware Recommendation
If running embeddings alongside other workloads, the RTX 5060 Ti at £119/month provides ample capacity. For embeddings as a standalone workload at 100M tokens/month, API providers offer better value.
Bundle Embeddings with Your GPU Workload
Already running inference on a GigaGPU server? Add embedding generation at zero marginal cost — no additional API fees.
