RVC Voice Cloning on a GPU Server

RVC (Retrieval-based Voice Conversion) is a voice-conversion technique: rather than text-to-speech, it takes speech in one voice and converts it to another. Training takes ~15 minutes of clean reference audio. On our dedicated GPU hosting training and inference both fit a modest card.

Contents

• Use cases
• Training a voice model
• Inference
• Ethics

Use Cases

RVC is useful when you have:

• A TTS voice you like but want to customise (run TTS through RVC for final voice)
• An archive of one speaker’s voice you want to extend with new speech
• Localisation workflow where a voice actor dubs once per language and RVC provides consistency

Training

Data requirement: 10-30 minutes of clean audio from the target speaker. Clean means one speaker, minimal background noise, consistent recording conditions.

python train.py \
  --model_name my_voice \
  --dataset_path ./my_voice_samples \
  --epoch 100 \
  --batch_size 8

On a 4060 Ti, training 100 epochs on 15 minutes of audio takes ~2 hours.

Inference

python infer.py \
  --model my_voice \
  --input source_speech.wav \
  --output converted.wav \
  --pitch_shift 0

Inference is real-time on any GPU – ~0.1-0.3x real-time factor.

Ethics

Same principles as Fish Speech cloning: document consent, avoid impersonation, consider watermarking. RVC specifically has been misused for deepfake audio – handle accordingly in your product.

See Fish Speech (zero-shot alternative).