Solar 10.7B from Upstage is a depth-upscaled model that achieves performance competitive with 13-15B models at smaller size. On our dedicated GPU hosting it fits a 16 GB 4060 Ti at INT8 with decent concurrency.
Contents
VRAM
| Precision | Weights |
|---|---|
| FP16 | ~22 GB |
| FP8 | ~11 GB |
| AWQ INT4 | ~6.5 GB |
Deployment
python -m vllm.entrypoints.openai.api_server \
--model upstage/SOLAR-10.7B-Instruct-v1.0 \
--quantization awq \
--max-model-len 4096 \
--gpu-memory-utilization 0.92Note the 4096 context limit – Solar was trained on shorter context than Llama 3 or Mistral Nemo. For long-context workloads pick a different model.
Strengths
Solar 10.7B excels at:
- Korean-English bilingual tasks
- Cost efficiency – good English performance in a small footprint
- Decode speed – smaller than 13B so faster per token
Weakness: shorter context and aging training cutoff. For English-first 2026 workloads Qwen 2.5 14B or Mistral Nemo 12B are usually better choices.
Compact Production LLM Hosting
Solar and other ~10B-class models on UK dedicated GPU servers.
Browse GPU ServersSee the broader tier comparison at best GPU for LLM inference.
