Using OpenClaw with Ollama (Local LLMs)

Run OpenClaw entirely offline using Ollama local LLMs. 100% private, zero API costs. Supports NVIDIA (CUDA), Apple Silicon (Metal), and AMD (ROCm) GPU acceleration.

Integration8 min read

Prerequisites

  • OpenClaw v1.0.0 or later installed (Installation Guide)
  • At least 8 GB RAM for 7B models; 16 GB for 13B models
  • A supported OS: Linux, macOS (Apple Silicon or Intel), or Windows (with WSL2)
  • Optional: NVIDIA GPU with CUDA for faster inference

Step 1: Install Ollama

macOS / Linux

curl -fsSL https://ollama.com/install.sh | sh

Windows

Download the installer from ollama.com/download and run it. Ollama starts automatically as a background service.

Verify Ollama is Running

ollama --version
curl http://localhost:11434/api/tags

Step 2: Pull Models

# Pull popular models
ollama pull llama3.2          # Meta's Llama 3.2 (3B, fast)
ollama pull mistral           # Mistral 7B (balanced)
ollama pull codellama         # Code-focused Llama
ollama pull qwen2.5-coder     # Alibaba coding model

# List downloaded models
ollama list

Step 3: Configure OpenClaw

openclaw config set provider ollama
openclaw config set ollama.model llama3.2
openclaw config set ollama.host http://localhost:11434
# config.yaml
provider: ollama
ollama:
  host: http://localhost:11434
  model: llama3.2
  temperature: 0.7
  num_ctx: 8192

Step 4: Test

openclaw run "What is 2+2?" --provider ollama
openclaw run "Write a Python hello world" --provider ollama --model codellama

GPU Acceleration

NVIDIA (CUDA)

# Install CUDA drivers then verify Ollama uses GPU
ollama run llama3.2
# Look for "GPU" in Ollama's log output

Apple Silicon (Metal)

Ollama automatically uses Metal on M1/M2/M3 Macs. No configuration needed.

AMD (ROCm)

ROCM_PATH=/opt/rocm ollama serve

Custom Modelfile

Create a custom model with a system prompt baked in:

FROM llama3.2
SYSTEM """
You are a helpful AI assistant integrated with OpenClaw.
Always respond with structured, actionable output.
When writing code, always include error handling.
"""
PARAMETER temperature 0.3
PARAMETER num_ctx 8192
ollama create openclaw-custom -f Modelfile
openclaw config set ollama.model openclaw-custom

Hardware Requirements

Ollama supports GPU-accelerated inference and runs on CPU as a fallback. The table below shows approximate RAM requirements per model size:

Model SizeMin RAM (CPU)Recommended RAMGPU VRAMExample Models
3B parameters4 GB8 GB4 GBllama3.2, phi3-mini
7B parameters8 GB16 GB6–8 GBmistral, llama3, gemma7b
13B parameters16 GB32 GB10–12 GBllama2-13b, vicuna-13b
34B parameters32 GB64 GB24 GBcodellama-34b
70B parameters64 GB128 GB48 GB+llama3-70b, mixtral
📋
Tip: Start with a 7B model for daily automation tasks. Use 3B models for fast, lightweight queries on low-spec hardware.

Performance Tuning

Increase Context Window

# Larger context = more memory, but enables longer documents
ollama:
  num_ctx: 16384   # default: 2048
  num_gpu: 1       # GPU layers to offload (higher = faster, more VRAM)
  num_thread: 8    # CPU threads for inference

Run Multiple Models Concurrently

# Keep multiple models warm in memory
OLLAMA_MAX_LOADED_MODELS=3 ollama serve

Benchmark a Model

time ollama run mistral "Write a 500-word essay on AI"

Switching Between Local and Cloud Models

OpenClaw supports routing tasks between local (Ollama) and cloud (OpenAI, Claude) models based on task type or flag:

# Default: use local Ollama
openclaw run "Draft a short email"

# Override per-run to use cloud model
openclaw run "Analyze this 150-page legal PDF" --provider claude
openclaw run "Generate production code" --provider openai

# Define routing rules in config
# config.yaml
routing:
  default: ollama
  fallback: openai       # use cloud if local fails
  rules:
    - pattern: "*.legal*"
      provider: claude
    - pattern: "*production*"
      provider: openai

LM Studio as Alternative

LM Studio is a GUI application for running local LLMs. It exposes an OpenAI-compatible API on port 1234, which OpenClaw can connect to:

# config.yaml — connect OpenClaw to LM Studio
provider: openai
openai:
  api_base: http://localhost:1234/v1
  api_key: lm-studio   # any non-empty string
  model: local-model   # name shown in LM Studio

LM Studio is a great choice for Windows users who prefer a visual interface over command-line model management.

Building Private, Air-Gapped Automation

Ollama + OpenClaw is the only setup that works completely offline. Use cases include:

  • Corporate environments where data cannot leave the network
  • Medical / legal document processing without data exposure
  • Development on flights or remote locations without internet
# Disable all external requests in OpenClaw config
# config.yaml
privacy:
  offline_mode: true
  disable_telemetry: true
  disable_update_checks: true

# Run a fully offline task
openclaw run "Analyze /data/contracts/*.pdf and extract key clauses" --offline

Troubleshooting

ProblemCauseFix
Connection refused on port 11434Ollama service not runningRun ollama serve or check system services
Model not foundModel not pulled yetRun ollama pull <model-name>
Out of memory (OOM)Model too large for RAM/VRAMUse a smaller model variant (e.g., 3B instead of 7B)
Slow inference (<5 tok/s)CPU-only, no GPUInstall CUDA drivers or use Apple Silicon; set num_gpu
Context length exceedednum_ctx too smallIncrease num_ctx in config (e.g., 8192 or 16384)

What's Next

💡
See Also
← Claude Integration Telegram Bot →