Chain-of-Thought Prompting Techniques for Agents

Zero-Shot CoT

The simplest CoT trigger — append "Let's think step by step" to any prompt:

agent = Agent(
    model="gpt-4o",
    system_prompt="""
Solve problems step by step.
Always reason through your answer before giving it.
End with: "Therefore, the answer is: [answer]"
"""
)
result = agent.run("A factory makes 1,200 units/day. How many in 3.5 working weeks (5 days/week)?")
print(result.output)

Few-Shot CoT

Provide examples with explicit reasoning traces to demonstrate the expected thinking style:

examples = """
Q: If a train travels 60 mph for 2.5 hours, how far does it go?
A: Step 1 — identify values: speed=60 mph, time=2.5 h
   Step 2 — apply formula: distance = speed × time = 60 × 2.5 = 150
   Therefore, the answer is: 150 miles

Q: {question}
A: """

Self-Consistency

Run the same prompt multiple times, then vote on the most common answer. Improves accuracy on ambiguous tasks by ~10–15%:

from collections import Counter
from openclaw import Agent

agent = Agent(model="gpt-4o-mini")

def self_consistent_answer(question: str, n: int = 5) -> str:
    answers = [agent.run(question).output.strip() for _ in range(n)]
    return Counter(answers).most_common(1)[0][0]

Step-Back Prompting

Ask the model to first retrieve relevant principles before solving. Reduces hallucination on knowledge-heavy tasks:

step_back_prompt = """
Before answering, identify:
1. What type of problem is this?
2. What general principles apply?
3. Are there any edge cases to consider?

Then solve the specific problem.

Problem: {problem}"""

Least-to-Most Decomposition

Break complex problems into sub-problems, solve each, then combine:

system_prompt = """
When given a complex task:
1. List sub-tasks from simplest to hardest.
2. Solve each sub-task in order.
3. Use earlier answers to solve later tasks.
4. Combine into a final answer.
"""

When CoT Helps vs Hurts

CoT helps	CoT hurts or is neutral
Multi-step arithmetic	Simple classification
Logical deduction	Fact retrieval
Planning sequences	Short, direct lookups
Debugging code	Token-constrained tasks

CoT With Tools (ReAct Pattern)

OpenClaw uses the ReAct pattern internally: Thought → Action → Observation → repeat. You can expose this explicitly:

system_prompt = """
For each step:
THOUGHT: what you know and need to find out
ACTION: which tool to call and with what arguments
OBSERVATION: what the tool returned
Repeat until you have enough info to answer.
FINAL ANSWER: your conclusion
"""

Debugging CoT Chains

Chain-of-thought reasoning is transparent by design — the model shows its work, making failures easy to diagnose. Common CoT failure modes and how to address them:

Symptom	Root cause	Fix
Correct reasoning, wrong final answer	Model ignores its own chain when producing the last step	Add: “Your final answer must follow directly from your reasoning.”
Reasoning loop (repeats same step)	No termination signal in the instruction	Use least-to-most decomposition with explicit done condition
Fabricated intermediate facts	CoT alone can’t access ground truth	Add retrieval steps (ReAct) or ground the chain in provided context
Very long chain, no answer	Task is too complex for in-context reasoning alone	Break into sub-tasks; use tool calls to offload computation

Cost Implications of CoT

CoT substantially increases output token count — sometimes 3–10× compared to a direct answer. This directly multiplies both latency and cost. Mitigate with three strategies:

Use CoT only when accuracy justifies the cost. For simple lookups or format conversions, direct prompting is cheaper and equally accurate.
Model downgrade. o1-mini and similar reasoning-optimised smaller models often match o1 quality on well-structured CoT tasks at a fraction of the price.
Extract and discard the chain. If only the final answer is needed by downstream components, parse it from the output and discard the chain text. Don’t store or forward thousands of reasoning tokens you don’t use.

import re

def extract_final_answer(cot_output: str) -> str:
    """Extract text after 'Final answer:' label."""
    match = re.search(r'(?i)final\s+answer[:\s]+(.+)', cot_output, re.DOTALL)
    return match.group(1).strip() if match else cot_output.strip()

CoT With OpenClaw Agents

OpenClaw’s agent loop integrates chain-of-thought reasoning natively through the think_before_act option. When enabled, the agent produces an internal reasoning trace on each step before deciding which tool to call.

from openclaw import Agent

agent = Agent(
    model="gpt-4o",
    think_before_act=True,   # enables internal CoT on each loop step
    max_steps=10,
)

result = agent.run(
    "Analyse last week's Slack exports and summarise the three most discussed topics."
)

# Access the reasoning trace
for step in result.steps:
    print(f"--- Step {step.index} reasoning ---")
    print(step.thought)
    print(f"Tool called: {step.tool_name}({step.tool_args})")