Token Cost Optimization

Why Token Costs Matter

At scale, token costs add up fast. An agent that uses 3,000 tokens per task costs:

Model selection alone is your biggest lever. But even within a model choice, you can cut token usage by 40-60% with the techniques below.

Model Selection Strategy

Use the cheapest model that meets quality requirements for each step:

# agent.yaml — routing by task type
routing:
  classify_intent:
    model: gpt-4o-mini    # Fast classification (cheap)
  retrieve_facts:
    model: gpt-4o-mini    # Simple lookup (cheap)
  complex_reasoning:
    model: gpt-4o         # Only frontier model for hard steps
  format_output:
    model: gpt-4o-mini    # Formatting is trivial (cheap)

Prompt Compression Techniques

1. Trim System Prompts

Every token in the system prompt is charged on every LLM call. Audit your system prompt and remove verbose explanations — the model follows concise instructions equally well:

BEFORE (48 tokens):
"You are a helpful AI assistant. Your job is to help users with their
questions. Please be polite and provide clear, helpful answers. Always
structure your response with clear headings."

AFTER (12 tokens):
"Answer concisely with MD headings."

2. Response Length Control

llm:
  max_tokens: 1024       # Hard limit on response length
  response_format: |     # Tell the model to be concise
    - Bullet points only
    - Max 5 bullets
    - No preamble

3. Use Structured Output Formats

output:
  format: json           # JSON is ~40% more token-efficient than prose
  schema:
    result: string
    confidence: number
    sources: array

Context Window Management

Long multi-turn conversations accumulate tokens exponentially. Use these strategies:

Auto-Summarization

memory:
  summarize_after: 15    # Summarize conversation after 15 turns
  summary_model: gpt-4o-mini  # Use cheap model for summarization
  keep_last: 5           # Always keep last 5 turns verbatim

RAG Instead of Context Stuffing

Instead of putting an entire knowledge base in the system prompt, use Retrieval-Augmented Generation to fetch only the relevant chunks:

# Without RAG: 8,000 tokens (full docs in prompt)
# With RAG: 800 tokens (only top-3 relevant chunks)
rag:
  enabled: true
  index: ./my-docs/
  top_k: 3
  chunk_size: 512

See RAG Guide for full setup instructions.

Token Budget Alerts

Set spending limits to prevent runaway costs:

budget:
  max_tokens_per_task: 5000     # Abort task if it exceeds this
  max_cost_per_task: 0.10       # Abort if estimated cost exceeds $0.10
  daily_token_limit: 1000000   # Alert if you exceed 1M tokens/day
  alert_email: "ops@yourco.com"

# Check current spend
openclaw budget status

# Output:
# Today: 284,521 tokens ($1.22)
# This month: 6,432,100 tokens ($27.62)
# Budget remaining: 715,479 tokens / $198.78

Cost Tracking and Attribution

# Cost breakdown by task type
openclaw analytics cost --group-by task_type --period last-30-days

# Cost breakdown by user
openclaw analytics cost --group-by user_id

# Export for accounting
openclaw analytics cost --format csv --output cost-report.csv

System Prompt Optimization

Your system prompt is included in every single LLM call, making it the highest-leverage text for token optimization. Every token in the system prompt is multiplied by the total number of LLM calls in the agent's lifetime. A 500-token reduction in your system prompt, for an agent that makes 10 LLM calls per task and processes 1000 tasks per day, saves 5 million tokens per day. At typical API pricing, this is real money. Audit your system prompt for redundancy: are there sections that describe behavior the model would exhibit anyway from training? Are there repetitive phrasings? Eliminating true redundancy reduces cost without affecting output quality.

Use structured formats (bullet points, headers) rather than verbose prose in system prompts. "Respond concisely. Avoid filler phrases. Use markdown headers for multi-part responses." (10 tokens) conveys the same instruction as a paragraph-long description of desired output style (50+ tokens). Brevity in your instructions often produces better model behavior too — dense, specific instructions are easier for the model to follow than verbose descriptions that bury the key directives in explanatory text.

Context Window Management

Multi-step agents accumulate context with each step — tool results, previous reasoning, intermediate outputs. Without active management, context grows linearly until it hits the model's context window limit, at which point older information gets truncated. Design your agent's prompting to actively manage context: after a tool call returns results, the agent should extract the relevant information and summarize it rather than keeping the full raw response in context. A web page fetch might return 10,000 tokens of HTML and text; the relevant extracted content might be 200 tokens. Summarizing tool results before adding them to context keeps the context window focused and costs manageable.

For very long-running agent sessions, implement periodic context compression. After every N steps, have the agent produce a concise summary of what has been accomplished and what's known so far, then replace the accumulated context with this summary. This "rolling summary" pattern keeps context size bounded regardless of how many steps the agent takes, enabling truly long-running agents without hitting context limits or paying for context that is mostly redundant historical detail.

Output Length Control

Unconstrained output length creates both cost and quality problems. Models tend to pad responses when no length guidance is given, producing verbose outputs that cost more and are harder to process downstream. Add explicit length instructions to your prompts when length is predictable: "Respond with a single sentence," "Provide a 3-5 bullet summary," "Write 200-300 words." These constraints focus the model and directly reduce output token usage. For structured outputs (JSON, tables), specify the exact schema — a schema constraint is the strongest length control available because it eliminates all prose variation.