Migrate from Google Vertex to Dedicated GPU: Recommendation Engine Guide

Google Charges You Per Recommendation — Even When Users Don’t Click

An e-commerce platform with 2 million daily active users deployed their AI recommendation engine on Google Vertex AI. Each page load generated 20 recommendation predictions through Vertex’s Gemini Pro endpoint — product suggestions, “customers also bought” carousels, and personalised banners. That’s 40 million predictions per day. At Vertex’s pricing, the recommendation system alone cost $28,000 per month. Click-through rate on recommendations was 3.2%, meaning 97% of those predictions generated zero revenue. The cost-per-click for AI recommendations was higher than their Google Ads spend.

Recommendation engines are the quintessential self-hosting use case: massive prediction volume, low per-prediction value, and enormous waste at per-request pricing. A dedicated GPU handles 40 million predictions per day at a flat rate. Here’s the migration from Vertex.

Understanding Your Vertex Recommendation Stack

Migration Approach

Phase 1: Audit your architecture. Recommendations typically involve two stages: candidate generation (fast, approximate) and ranking (slower, precise). Determine which stage runs on Vertex AI and which uses custom models. Most Vertex recommendation setups use Gemini for the ranking stage, which is the expensive part.

Phase 2: Provision hardware. An RTX 6000 Pro 96 GB from GigaGPU handles both the embedding-based candidate generation and the LLM-based ranking simultaneously. For high-traffic sites (10M+ daily users), two RTX 6000 Pros provide ample capacity.

Phase 3: Self-host the ranking model. The biggest cost saver is replacing Vertex’s Gemini predictions with a self-hosted model. For recommendation ranking, you don’t need a 70B model — an 8B model fine-tuned on your click-through data outperforms a general-purpose large model. Deploy via vLLM:

python -m vllm.entrypoints.openai.api_server \
  --model your-finetuned-llama-8b \
  --max-model-len 2048 \
  --max-num-seqs 256 \
  --port 8000

The high max-num-seqs setting enables processing hundreds of ranking requests concurrently — critical for recommendation workloads where you rank 50-100 candidates per user request.

Phase 4: Migrate the candidate generation layer. Replace Vertex Matching Engine with FAISS (GPU-accelerated) or Qdrant. Both support approximate nearest-neighbour search at millions of queries per second on GPU hardware.

Phase 5: Validate with live traffic. Run an A/B test: 50% of users get Vertex-powered recommendations, 50% get self-hosted. Compare CTR, revenue per session, and recommendation diversity over two weeks.

Performance Optimisation for Recommendations

Recommendation latency directly impacts revenue — Amazon found that every 100ms of latency costs 1% of sales. Self-hosting gives you latency advantages Vertex can’t match:

• GPU-accelerated vector search: FAISS on GPU retrieves 1,000 nearest neighbours from 10 million items in under 5ms.
• Co-located ranking: The ranking model runs on the same GPU as the vector search. No network roundtrip to Vertex.
• Batch ranking: Rank all 50-100 candidates in a single GPU inference pass rather than individual API calls.
• Pre-computation: Generate recommendations during off-peak hours and cache them. Unlimited compute means you can pre-compute for every user.

Explore open-source model hosting for model selection options, or use Ollama for rapid prototyping of recommendation models.

Cost Comparison

Run your numbers through the LLM cost calculator for precise projections.

Recommendations That Scale With Your Business

Per-prediction pricing for recommendations is a tax on growth. Every new user, every additional page view, every product you add to your catalogue increases your Vertex bill. Self-hosting decouples recommendation quality from recommendation cost.

Related reading: the self-hosting breakeven analysis, the TCO comparison, and our self-host LLM guide. Compare provider options on the GPU vs API cost page, and explore more migration paths in our tutorials section. For data privacy needs, see private AI hosting.

Filed under: Tutorials