Molmo 7B Self-Hosted Vision-Language Model

Molmo 7B from the Allen Institute for AI is a vision-language model trained from scratch (not fine-tuned from a text LLM), with a focus on spatial reasoning – pointing at things in images, counting, and describing exact locations. On our dedicated GPU hosting it fits a 16 GB card at FP16.

Contents

• VRAM
• Deployment
• What Molmo does well

VRAM

Deployment

python -m vllm.entrypoints.openai.api_server \
  --model allenai/Molmo-7B-D-0924 \
  --dtype bfloat16 \
  --trust-remote-code \
  --max-num-seqs 4 \
  --limit-mm-per-prompt 'image=1'

Molmo’s architecture requires --trust-remote-code. Review the model card before production deployment.

Strengths

Molmo excels at:

• Pointing: “where is the cat?” returns coordinates
• Counting: accurate object counting in crowded scenes
• Precise spatial descriptions
• UI element identification

It is weaker than Llama 3.2 Vision and Qwen VL on general Q&A and long reasoning. Use Molmo for specific spatial tasks, not as a generalist VLM.

For generalist VLMs see Llama 3.2 Vision 11B, Pixtral 12B, and Qwen VL 2.