InternLM 2.5 20B Deployment

InternLM 2.5 from Shanghai AI Lab punches above its size on Chinese and mathematical reasoning. The 20B variant fits a 24 GB 3090 on our dedicated GPU hosting at INT8 with usable concurrency, filling a gap between 14B and 32B class models.

Contents

• VRAM
• Deployment
• Strengths and weaknesses

VRAM

Deployment

python -m vllm.entrypoints.openai.api_server \
  --model internlm/internlm2_5-20b-chat \
  --quantization awq \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.92 \
  --trust-remote-code

InternLM uses custom architecture code – --trust-remote-code is required. Read the model card before deploying in untrusted environments.

Strengths

InternLM 2.5 is strong on:

• Chinese reasoning tasks
• Mathematics (MATH, GSM8K)
• Tool use and function calling
• 1M token context in the extended variant

Weaker on:

• Pure English creative writing versus Llama
• Coding versus Qwen Coder

Compare against Qwen 2.5 14B and Mistral Small 3 24B for alternatives in the same size bracket.