Response Caching in OpenClaw

How LLM Response Caching Works

When the agent makes an LLM call, OpenClaw hashes the input (system prompt + user message + model params) and checks the cache before hitting the API. A cache hit returns the stored response instantly — no API call, no cost, no latency.

Request → Hash input → Cache lookup
                              ↓            ↓
                          HIT: return   MISS: call LLM → store → return
                          cached result  API result  in cache

Enable Caching

In-Memory Cache (default, zero setup)

# config.yaml
cache:
  enabled: true
  backend: memory       # Stores in process RAM
  ttl: 3600             # Cache entries expire after 1 hour
  max_entries: 1000     # Evict LRU when full

In-memory cache is lost when the process restarts. Use for short-lived agents or development.

Disk Cache (persists across restarts)

cache:
  enabled: true
  backend: disk
  path: ~/.openclaw/cache/
  ttl: 86400            # 24 hours
  max_size_mb: 500      # Max disk usage

Redis Cache (production, multi-instance)

cache:
  enabled: true
  backend: redis
  redis_url: "redis://localhost:6379/0"
  ttl: 3600
  key_prefix: "openclaw:"  # Namespace for multi-app Redis

# Install Redis backend
pip install openclaw[redis]
# Start Redis (Docker)
docker run -d -p 6379:6379 redis:7-alpine

Semantic Caching (Advanced)

Regular caching only matches exact inputs. Semantic caching matches similar inputs using embedding similarity — so "What is Python?" and "Tell me about Python" both hit the same cache entry.

cache:
  enabled: true
  backend: semantic
  similarity_threshold: 0.92   # 0.0-1.0; higher = stricter matching
  embedding_model: text-embedding-3-small
  base_backend: redis          # Store vectors in Redis
  redis_url: "redis://localhost:6379/0"

Install the semantic cache extras:

pip install openclaw[semantic-cache]

Cache Hit/Miss Metrics

Monitor cache effectiveness with built-in metrics:

openclaw cache stats

# Output:
# Cache backend: redis
# Total requests:  1,847
# Cache hits:        702  (38.0%)
# Cache misses:    1,145  (62.0%)
# Avg hit latency:   3ms
# Avg miss latency: 4.2s
# Cost saved:      $1.24 (est.)

# Clear the cache
openclaw cache clear

# Clear only expired entries
openclaw cache prune

# Export cache to inspect
openclaw cache export --output cache-dump.json

Cache Invalidation Strategies

# Skip cache for a specific run
openclaw run "What's the current BTC price?" --no-cache

# Invalidate all entries tagged "pricing"
openclaw cache invalidate --tag pricing

Caching Best Practices

• Always cache in production — even 20% hit rate pays for the Redis instance
• Set ttl based on data freshness requirements, not arbitrarily long
• Use key_prefix in Redis to avoid collisions between environments (dev/staging/prod)
• Monitor hit rate weekly — a falling hit rate signals workload drift or too-short TTLs
• Don't cache tasks with user-specific private data unless you namespace by user ID

Caching Strategies by Use Case

Different parts of an agent workflow benefit from different caching approaches. System prompt caching (prefix caching) is handled by the LLM provider for long system prompts — it's free performance improvement that requires no configuration change beyond having a long, stable system prompt. Tool result caching is managed by OpenClaw and caches the output of expensive tool calls (web fetches, API calls, file reads) across agent runs. Response caching stores complete agent outputs for identical inputs — useful for FAQ-style agents where many users ask the same questions.

Implement cache warming for predictable workloads. If your agent runs scheduled tasks at 9am every day and always fetches the same external data sources, pre-fetch and cache that data at 8:55am before the first task runs. Warmed caches mean the first tasks of the day see cache hit rates close to steady-state rather than starting cold and paying full latency for every fetch. Cache warming is particularly valuable for data sources with strict rate limits where cache misses are expensive.

Cache Invalidation Design

Stale cache data is often worse than no cache — an agent confidently citing outdated information causes user trust issues that are harder to recover from than a slower but accurate response. Design your TTL values around the staleness tolerance of each data source. A company's contact page might be fine with a 24-hour TTL; a stock price API should have a 1-minute TTL. When in doubt, err toward shorter TTLs. Monitor cache hit rates — a cache hit rate below 30% indicates either the TTL is too short or the workload lacks sufficient repetition to benefit from caching, and the caching overhead may outweigh the benefit.

Implement explicit cache invalidation for data you know has changed. When your documentation is updated, invalidate the document cache for modified pages rather than waiting for TTL expiry. When a tool's API changes its response schema, invalidate all cached results from that tool. Explicit invalidation keeps your cache fresh without requiring artificially short TTLs on data that changes infrequently.

Measuring Cache Effectiveness

Track cache hit rate, cache-miss latency, and cache-hit latency separately. The most important metric is the latency reduction per hit — if cached responses are 2x faster and you're hitting the cache 60% of the time, your average latency is 0.6x + 0.4x2 = 1.4x relative to no-cache, a 30% improvement. As hit rate improves, the benefit compounds. Log cache events at debug level so you can diagnose unexpected miss patterns — a sudden drop in hit rate often indicates an upstream data source changed its response format, causing all cached keys to become invalid.