Running Ollama on Intel Arc Pro B60: Local LLMs on 24 GB ECC at £129/mo

Does Ollama Support Intel Arc?

Ollama uses llama.cpp under the hood, and llama.cpp has a production-ready SYCL backend for Intel GPU hardware. Ollama’s Intel Arc support was added in 2024 and has matured through 2025–26. On Intel Arc Pro B60 hardware, the path is:

1. Ollama detects the Intel GPU via the Level Zero driver
2. Routes inference through llama.cpp’s -DGGML_SYCL=ON build
3. Dispatches matrix operations to the B60’s 160 XMX engines (INT8/BF16)

The result: GGUF models run GPU-accelerated on the B60’s 24 GB ECC GDDR6, with generation speeds comparable to Ollama on an NVIDIA GPU of similar VRAM at the same quantisation level. The main limitation is that CUDA-only features (NVIDIA flash attention, TensorRT) don’t apply, but Ollama’s core inference path is SYCL-native and doesn’t require them.

For a full overview of LLM options on Intel Arc, see the B60 LLM inference guide. To compare GPUs for Ollama workloads see best GPU for LLaMA.

Which Models Fit on 24 GB

Token throughput estimates assume GGUF Q4_K_M models fully resident on the B60’s 24 GB with the SYCL backend. Real throughput depends on context length (longer = slower due to KV cache reads), batch size, and the exact model architecture. Models with grouped query attention (Llama 3, Gemma 2) tend to outperform models with full multi-head attention at the same parameter count on bandwidth-bound hardware.

Setting Up Ollama on Intel Arc Pro B60

GigaGPU Arc Pro B60 servers ship with Ubuntu 22.04/24.04 and Intel GPU drivers pre-installed. The setup:

# Verify Intel GPU is visible
ls /dev/dri/

# Set the SYCL device selector to use the Arc GPU
export ONEAPI_DEVICE_SELECTOR=level_zero:0

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Start the Ollama service
ollama serve &

# Pull and run a model
ollama run llama3.1:8b

# Or pull a larger quantised model
ollama run qwen2.5:14b

The ONEAPI_DEVICE_SELECTOR=level_zero:0 environment variable is the key step — it tells the Level Zero driver to use GPU index 0 (the Arc Pro B60). Without it, Ollama may default to CPU inference. Add it to /etc/environment or your ~/.bashrc to make it persistent.

To run Ollama as a background service with the GPU selector set persistently:

# Create a systemd override for the Ollama service
sudo mkdir -p /etc/systemd/system/ollama.service.d/
sudo tee /etc/systemd/system/ollama.service.d/override.conf <<'EOF'
[Service]
Environment="ONEAPI_DEVICE_SELECTOR=level_zero:0"
EOF

sudo systemctl daemon-reload
sudo systemctl restart ollama
sudo systemctl status ollama

LlamaCPP SYCL: The Manual Route

For more control over GPU layers, context size, and batch parameters, build llama.cpp directly with the SYCL backend:

# Ensure oneAPI is sourced
source /opt/intel/oneapi/setvars.sh

# Clone and build llama.cpp with SYCL
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
mkdir build && cd build
cmake .. -DGGML_SYCL=ON -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx
make -j$(nproc)

# Run inference (all 33 layers on GPU, 4096 context)
ONEAPI_DEVICE_SELECTOR=level_zero:0 \
  ./bin/llama-cli \
  -m /path/to/models/llama-3.1-8b-instruct-q4_k_m.gguf \
  -ngl 33 \
  -c 4096 \
  -p "You are a helpful assistant." \
  --interactive

Key parameters for production inference:

• -ngl 33: number of GPU layers (set to match the model’s total layer count to keep everything on GPU)
• -c 4096: context window size — increase to 8192 or 16384 if your VRAM budget allows
• --threads 8: CPU threads for layers not offloaded (not needed if all layers fit on GPU)

For the llama.cpp HTTP server (OpenAI-compatible API):

ONEAPI_DEVICE_SELECTOR=level_zero:0 \
  ./bin/llama-server \
  -m /path/to/model.gguf \
  -ngl 33 \
  -c 8192 \
  --host 0.0.0.0 --port 8080

Throughput Estimates

The B60’s 160 XMX matrix engines handle INT8 and BF16 matrix multiply-accumulate efficiently. For GGUF inference via SYCL, the bottleneck switches between memory bandwidth (for large context KV cache reads) and compute (for the attention and FFN layers themselves):

These are single-stream estimates (one user at a time). For multi-user serving at higher concurrency, throughput per user drops — consider the IPEX-LLM guide which covers batched serving.

Running Multiple Models Simultaneously

24 GB lets you keep multiple small models resident at the same time — useful if you’re switching between a chat model, a coding model, and a vision model in a workflow:

Ollama handles model loading and unloading automatically, but with 24 GB you can use OLLAMA_KEEP_ALIVE=-1 to keep models resident permanently and avoid reload latency between requests.

Using the Ollama REST API

Once Ollama is running, it exposes an OpenAI-compatible REST API on port 11434:

# Standard Ollama endpoint
curl http://localhost:11434/api/generate \
  -d '{"model": "llama3.1:8b", "prompt": "Explain XMX matrix engines in one paragraph.", "stream": false}'

# OpenAI-compatible chat endpoint
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.1:8b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

The OpenAI-compatible endpoint means you can drop the B60 into any application that already talks to the OpenAI API — just change the base URL and model name. LangChain, LlamaIndex, and Continue.dev all support Ollama out of the box.

B60 vs CUDA Cards for Ollama

The RTX 3090 generates tokens about 25–30% faster than the B60 on the same model at the same quantisation — its 936 GB/s memory bandwidth vs the B60’s 456 GB/s is the primary driver. If raw generation speed per pound is the goal, the RTX 5060 Ti at £119/mo is faster at 16 GB (but limited to smaller models). If you need 24 GB for larger models and the price matters, the B60 at £129/mo is the cheapest option in the catalogue.