Profiling & Performance Debugging

Enable Profiling

Per-Run Profiling

# Profile a single run
openclaw run "your task" --profile

# Save detailed profile data for later analysis
openclaw run "your task" --profile --profile-output profile.json

Always-On Profiling (Production)

# config.yaml
profiling:
  enabled: true
  level: basic           # "basic" | "detailed" | "trace"
  output_dir: ~/.openclaw/profiles/
  retention_days: 30

Reading Profiler Output

openclaw run "research async Python patterns" --profile

# ════════════════════════════════════════════
# OpenClaw Profile Report
# Task: research async Python patterns
# ════════════════════════════════════════════
# Total time:      12.4s
# ├── Planning:     0.9s  (7%)
# ├── Tool calls:   6.2s  (50%)  ◄ MAIN BOTTLENECK
# │   ├── web_search:   3.1s
# │   ├── read_url[0]:  1.4s
# │   └── read_url[1]:  1.7s
# ├── LLM synthesis: 4.8s  (39%)
# └── Output format: 0.5s  (4%)
#
# Tokens:  1,847 input + 412 output = 2,259 total
# Cost:    $0.0004 (GPT-4o-mini)
# Cache:   0/2 LLM calls hit cache
# ════════════════════════════════════════════

In this example, tool calls dominate (50%). The fix is to enable parallel execution so the two read_url calls run simultaneously instead of sequentially (saves ~1.4s).

Identifying Performance Bottlenecks

Slow Tool Calls

# Find the slowest tools across all recent runs
openclaw profile analyze --last 100 --sort tool_duration

# Output:
# Tool          | Calls | Avg    | P95    | Failures
# web_search    |  342  | 2.1s   | 5.8s   | 12 (3.5%)
# read_url      |  891  | 1.4s   | 4.2s   | 8 (0.9%)
# file_read     |  203  | 0.02s  | 0.1s   | 0 (0.0%)

LLM Latency Breakdown

openclaw profile analyze --last 100 --sort llm_duration

# Output:
# Model          | Calls | Avg TTFT | Avg Total | Tokens/call
# gpt-4o-mini    |  891  | 0.4s     | 1.2s      | 1,847
# gpt-4o         |   47  | 1.2s     | 4.8s      | 3,200

TTFT = Time To First Token — a high TTFT with low total time suggests the prompt is large; a low TTFT with high total time suggests a long response.

Memory Profiling

# Check memory usage per agent instance
openclaw profile memory --watch

# Output (refreshes every second):
# Agents running: 3
# Total RSS:      428MB
# ├── agent-1:    142MB  (task: research...)
# ├── agent-2:    156MB  (task: analyze...)
# └── agent-3:     88MB  (task: format...)
# Cache (in-memory): 34MB
# Peak today: 612MB

# Reduce memory footprint
memory:
  backend: disk          # Move context to disk instead of RAM
  max_context_tokens: 4096  # Limit context window size
  gc_interval: 60        # Run garbage collection every 60s

Production Performance Monitoring

For long-running deployments, export metrics to your monitoring stack:

# config.yaml — export to Prometheus
metrics:
  enabled: true
  exporter: prometheus
  port: 9090
  labels:
    service: openclaw
    environment: production

# config.yaml — export to Datadog
metrics:
  enabled: true
  exporter: datadog
  api_key: "${DD_API_KEY}"
  tags: ["env:prod", "service:openclaw"]

Key metrics to alert on:

• openclaw_task_duration_p95 — alert if > 30s
• openclaw_task_failure_rate — alert if > 5%
• openclaw_token_cost_daily — alert on budget thresholds
• openclaw_cache_hit_rate — alert if drops below 20%

See Monitoring Overview for a complete observability setup guide.

Profiling in Practice

Before profiling, form a hypothesis. "I think the web_fetch calls are slow" or "I think the LLM is generating too many tokens for simple tasks" gives you a specific measurement target. Without a hypothesis, you'll collect data without knowing what to do with it. Measure your hypothesis, validate or refute it, form the next hypothesis based on the data, and repeat. This hypothesis-driven approach is 5-10x more efficient than exploratory profiling where you measure everything hoping to find the bottleneck.

Profile with production data. Profiling with test inputs that are smaller or simpler than production often points to bottlenecks that don't exist in production while missing the real ones. Export a sample of 50-100 representative production inputs (excluding any PII), run the agent against this sample with profiling enabled, and analyze the results. The bottlenecks you find this way are the ones that actually matter for your users' experience.

Reading the Profiling Output

OpenClaw's --profile flag produces a per-step timing breakdown. Look for: total LLM time (usually 60-80% of wall time in well-optimized agents), total tool call time, step count, and any steps that took dramatically longer than the median. Outlier steps — one step taking 10x longer than average — indicate intermittent issues: a slow API endpoint, a timeout that triggered, or an unusually large LLM output for one particular input. Fix outliers before trying to optimize the average case, as outliers degrade p95 latency disproportionately.

Compare profiles across different configurations systematically. Create a spreadsheet with columns for each configuration option you're testing and rows for each metric. Run each configuration against the same benchmark inputs and fill in the measurements. This side-by-side view makes trade-offs visible: configuration A is 20% faster on average but has 40% higher p95 latency; configuration B is slightly slower on average but has consistent latency. Spreadsheet analysis of profiling data is more reliable than pattern-matching from memory across multiple runs.

Continuous Performance Monitoring

One-time profiling during development is a starting point, not a complete strategy. Add lightweight performance tracking to production: log per-task timing, step count, and token usage for every run. Set up alerts for regressions: if median task latency increases by more than 25% compared to last week's baseline, trigger an investigation. Performance often degrades gradually through accumulation of small changes — a new tool call here, a longer system prompt there — that each seem insignificant but together cause meaningful regression. Continuous monitoring catches accumulation before it reaches users.