Migrate from Google Vertex to Dedicated GPU: Translation Pipeline Guide

Google Translate API Charges by the Character — and Your Users Type Millions

A SaaS company serving the European market had a multi-language content pipeline. Customer-facing content was authored in English and translated into 12 European languages through Google Vertex AI’s Gemini model, which produced more natural translations than the basic Translate API. The content team published 50 articles per week, each averaging 2,000 words. Across 12 languages, that’s 1.2 million words translated per week — approximately 7.2 million characters. At Gemini Pro’s pricing on Vertex, the monthly translation bill sat at $4,800. Then marketing wanted to translate the entire 5,000-article back catalogue. The quote: a one-time cost of $28,000.

Modern open-source LLMs are exceptional translators. Models like Qwen 2.5 72B handle European languages fluently, while NLLB-200 covers 200 languages at a fraction of the compute. Self-hosting on a dedicated GPU makes translation a fixed-cost operation, no matter how much content you push through. Here’s the migration path from Vertex.

Choosing Your Translation Architecture

For most European language translation, a general-purpose 70B model produces translations that professional translators rate as equal or superior to Google’s Gemini output — particularly for nuanced content where context matters.

Step-by-Step Migration

Phase 1: Benchmark your current quality. Take 200 translated pieces across your key language pairs and have bilingual reviewers score them on a 1-5 scale. This becomes your quality baseline. Save these pieces — you’ll compare the self-hosted translations against them.

Phase 2: Provision your server. An RTX 6000 Pro 96 GB from GigaGPU runs Qwen 2.5 72B for high-quality translation. For real-time chat translation, the NLLB-200 3.3B model runs on a tiny fraction of the GPU, leaving room for both workloads simultaneously.

Phase 3: Deploy via vLLM. Set up vLLM with translation-optimised settings. Translation benefits from longer output lengths:

python -m vllm.entrypoints.openai.api_server \
  --model Qwen/Qwen2.5-72B-Instruct \
  --max-model-len 16384 \
  --port 8000

Phase 4: Replace Vertex API calls. Swap each Vertex prediction call with an OpenAI-compatible request. Translation prompts are straightforward:

response = client.chat.completions.create(
    model="qwen-72b",
    messages=[{
        "role": "system",
        "content": "You are a professional translator. Translate the following English text to French. Maintain the original formatting, tone, and meaning."
    }, {
        "role": "user",
        "content": source_text
    }]
)

Phase 5: Quality validation. Run your 200 benchmark pieces through the self-hosted pipeline and have the same reviewers blind-score them. Most teams see equivalent or higher scores — LLMs often produce more natural-sounding translations than neural machine translation services.

Batch Translation Optimisation

Content translation is typically a batch workload — you don’t need real-time response for translating blog articles. Optimise for throughput:

• Parallel processing: Fire 10-20 translation requests simultaneously. vLLM’s continuous batching handles them efficiently.
• Chunking strategy: For long articles, translate in 1,000-word chunks with 100 words of overlap. This maintains context while maximising throughput.
• Terminology consistency: Include a glossary in the system prompt to ensure product names and technical terms are translated consistently across all content.
• Multi-target batching: Translate to all 12 languages in parallel rather than sequentially. On an RTX 6000 Pro, your 50-article weekly batch across 12 languages completes in under two hours.

Cost Comparison

The back-catalogue use case alone often justifies the migration. Use the LLM cost calculator for your specific volumes.

Translation Without Boundaries

Self-hosted translation doesn’t just save money — it removes constraints. Translate your entire back catalogue. Add new languages without calculating ROI per language. Let your team experiment with tone variations across markets. When translation is free at the margin, you optimise for quality rather than cost.

For related migrations, see the Vertex recommendation engine guide and the OpenAI content generation migration. The breakeven analysis covers the economics, while the GPU vs API cost tool models specific workloads. Read the self-host LLM guide for setup details, and browse more in our tutorials section.

Filed under: Tutorials