LLM Integration Patterns

Advanced patterns for integrating multiple LLM providers, building fallback chains, managing context windows, and implementing RAG pipelines with OpenClaw AI.

Use Case9 min read

Multi-Provider Routing

Route different task types to the most cost-effective model:

# Cheap model for classification
openclaw run "Classify this email as: spam, important, or routine" \
  --provider openai --model gpt-4o-mini

# Powerful model for complex reasoning
openclaw run "Analyze the business implications of this 50-page report" \
  --provider claude --model claude-3-opus-20240229

# Free local model for private data
openclaw run "Summarize this confidential HR document" \
  --provider ollama --model llama3.2

Provider Fallback Chain

# config.yaml — try providers in order
provider: openai
fallback_providers:
  - claude
  - ollama
fallback_on:
  - rate_limit
  - timeout
  - api_error

If OpenAI hits a rate limit, OpenClaw automatically retries with Claude, then falls back to local Ollama.

Context Window Management

# Automatically chunk large inputs
openclaw run "Summarize this 300-page PDF" \
  --file large-document.pdf \
  --chunk-size 30000 \
  --chunk-overlap 500 \
  --aggregate-strategy "hierarchical"
# config.yaml — context settings
context:
  max_tokens: 100000
  chunking: auto
  overlap_tokens: 500
  aggregation: hierarchical  # or: sequential, map-reduce

RAG Pipeline

Retrieval-Augmented Generation: ground responses in your own documents:

# Index your knowledge base
openclaw index ./docs/ --output ./my-kb.index

# Query with RAG
openclaw run "How do I configure the webhook timeout?" \
  --rag ./my-kb.index \
  --provider openai

Prompt Templates

# ~/.openclaw/templates/code-review.yaml
name: code-review
template: |
  You are an expert {{ language }} developer.
  Review the following code for:
  1. Security vulnerabilities (OWASP Top 10)
  2. Performance issues
  3. Code style ({{ style_guide }} guidelines)
  4. Missing error handling
  
  Code:
  {{ code }}
openclaw run --template code-review \
  --var language=Python \
  --var style_guide=PEP8 \
  --file src/app.py

Benchmarking Models for Your Use Case

Not all models perform equally on every task. Use OpenClaw's built-in benchmark tool to evaluate latency, cost, and output quality for your specific prompts:

# Run a task on multiple models and compare
openclaw benchmark \
  --providers openai:gpt-4o,anthropic:claude-3-5-sonnet,ollama:llama3.2 \
  --task "Classify this support ticket as: bug / feature / question" \
  --file sample-ticket.txt \
  --output benchmark-results.json

# View results
cat benchmark-results.json | jq '.[] | {provider, latency_ms, cost_usd, output}'
Use CaseRecommended ModelReason
Code generationGPT-4o / Claude 3.5 SonnetStrong coding benchmarks, function calling
Long document analysisClaude 3 Opus / 3.5 Sonnet200K context window, nuanced reasoning
High-volume classificationGPT-4o-mini / Claude 3 HaikuLow cost, fast latency
Private / offline tasksOllama llama3 / mistralNo data sent, zero API cost
Reasoning / mathGPT o3-mini / Claude 3 OpusBest-in-class reasoning benchmarks

Advanced RAG: Dynamic Context Injection

Retrieval-Augmented Generation (RAG) allows the AI to answer questions using your private documents without exposing everything to the model at once:

# config.yaml — configure RAG vector store
rag:
  enabled: true
  embedding_model: text-embedding-3-small  # OpenAI
  vector_store: chroma                      # or: pinecone, weaviate, faiss
  chunk_size: 512
  chunk_overlap: 64
  top_k: 5                                 # retrieve top 5 relevant chunks
# Index your knowledge base
openclaw rag index ./docs/

# Query with RAG
openclaw run "How do I configure Redis caching?" --rag
# OpenClaw retrieves relevant docs, injects them as context, then answers

What's Next

💡
See Also
← Data Analysis Use Cases Overview →