Unit Tests

Unit Test Structure

Unit tests live in tests/unit/ with a mirror structure to the source code: tests/unit/test_agent_runner.py tests openclaw/agent_runner.py, for example. Each test file focuses on one module, keeping test files small and navigation easy.

Tests use pytest's standard def test_* naming convention. Fixtures are defined in conftest.py files at each directory level. Shared fixtures (mock LLM provider, sample config objects, temp directories) are defined in tests/conftest.py and available everywhere.

Mocking the LLM

The MockLLMProvider class lets you script exact LLM responses for deterministic testing:

from openclaw.testing import MockLLMProvider, AgentTestCase

class TestAgentSelectsTool(AgentTestCase):
    def test_agent_calls_web_search(self):
        mock_llm = MockLLMProvider(responses=[
            # First call: agent decides to use web_search
            {"type": "tool_call", "tool": "web_search", "args": {"query": "Python asyncio tutorial"}},
            # Second call: agent synthesizes final answer
            {"type": "text", "content": "Here is what I found about asyncio..."},
        ])
        result = self.run_agent("Find me a good Python asyncio tutorial", llm=mock_llm)
        assert mock_llm.call_count == 2
        assert "web_search" in mock_llm.calls[0]["tool"]
        assert "asyncio" in result.output

The mock provider records every request made to it, so you can also assert on what context was passed to the LLM — verifying system prompt construction, tool result formatting, and memory assembly.

Testing Tool Implementations

Custom tools should be tested in isolation using the ToolTestHarness:

from openclaw.testing import ToolTestHarness
from my_plugin.tools import database_query_tool

class TestDatabaseQueryTool:
    def test_returns_results_for_valid_query(self, mock_db):
        harness = ToolTestHarness(database_query_tool, context={"db": mock_db})
        result = harness.call(query="SELECT name FROM users LIMIT 5")
        assert result.success
        assert len(result.data) <= 5

    def test_handles_invalid_sql_gracefully(self, mock_db):
        harness = ToolTestHarness(database_query_tool, context={"db": mock_db})
        result = harness.call(query="INVALID SQL !!!")
        assert not result.success
        assert "invalid" in result.error.lower()

Testing Memory Operations

Memory operations can be tested using the in-memory backend that avoids any external dependencies. Set memory.backend: in_memory in your test config, or use the InMemoryMemoryManager fixture directly. Verify that semantic retrieval returns expected chunks by seeding test memories, then asserting on retrieval results for known queries.

Async Test Patterns

OpenClaw's core is async. Tests for async code use @pytest.mark.asyncio and declare test functions as async def:

import pytest
from openclaw.testing import AsyncAgentTestCase

@pytest.mark.asyncio
async def test_streaming_output():
    agent = AsyncAgentTestCase(llm=mock_llm)
    chunks = []
    async for chunk in agent.stream("Write a haiku"):
        chunks.append(chunk)
    assert len(chunks) > 0
    assert "".join(chunks)  # non-empty final output

Common Assertion Helpers

The openclaw.testing.assertions module provides helpers for common agent test assertions: assert_tool_called(result, tool_name), assert_tool_not_called(result, tool_name), assert_output_mentions(result, keyword), assert_completed_under(result, seconds), and assert_within_token_budget(result, max_tokens). These helper functions produce clear failure messages that identify exactly what went wrong.

Test File Organization Best Practices

Good test file organization makes the test suite navigable and prevents tests from becoming unmaintainable. Each test class should test one class or one coherent cluster of functionality. Test method names should read like sentences describing what the test verifies: test_tool_registry_raises_when_tool_name_conflicts is more useful than test_registry_3.

Group related tests using pytest's class-based organization, even for tests that don't share fixtures — the grouping itself communicates which tests are related and makes it easy to run a subset: pytest tests/unit/test_memory.py::TestSemanticMemory -v. Use pytest marks to categorize tests that need special conditions: @pytest.mark.slow, @pytest.mark.network, @pytest.mark.gpu.

Parameterized Tests

pytest's parametrize decorator lets you run the same test logic against multiple inputs, dramatically increasing coverage without duplicating test code:

import pytest

@pytest.mark.parametrize("input_tokens,model,expected_cost", [
    (1000, "gpt-4o",       0.005),
    (1000, "gpt-4o-mini",  0.00015),
    (1000, "claude-haiku", 0.00025),
])
def test_cost_estimation(input_tokens, model, expected_cost):
    cost = calculate_cost(input_tokens=input_tokens, model=model)
    assert abs(cost - expected_cost) < 0.0001

Parameterized tests appear as separate test cases in the CI report, making it easy to identify exactly which parameter combination failed. Use them for boundary conditions, multiple providers, different configuration values, and any scenario where you'd otherwise copy-paste a test with minor variations.

Fixtures and Dependency Injection

pytest fixtures are the primary mechanism for setting up test state. They support dependency injection — a fixture can depend on other fixtures. Write fixtures at the lowest possible scope: prefer function scope (a fresh instance for every test) over session scope (one shared instance for all tests) unless the setup is genuinely expensive. Function-scoped fixtures prevent subtle test-interdependency bugs where test A's side effects break test B.

Define your most reusable fixtures in conftest.py at the appropriate directory level. A fixture in tests/unit/conftest.py is available to all unit tests. A fixture in tests/conftest.py is available to all tests. Avoid defining fixtures in individual test files unless they're specific to that file's needs — scattered fixture definitions are hard to discover and maintain.

Covering Edge Cases and Error Paths

The most common gap in test suites is insufficient coverage of error paths and edge cases. It's tempting to write tests that verify the happy path works and call coverage done. But the most dangerous bugs live in the code paths that handle unexpected inputs, network failures, and misconfigured states — exactly the paths that tests tend to skip.

For every function with a non-trivial error path, write at least one test that exercises it. For functions that accept user-supplied input, test boundary values: empty strings, very long strings, strings with special characters, null values, negative numbers, zero, and maximum values. For functions that call external services, test what happens when the service returns an error. These tests are cheap to write and catch bugs that would otherwise only surface in production under rare conditions.

Use parameterized tests to cover multiple boundary conditions in a single, readable block rather than writing separate test functions for each boundary. The visual conciseness makes it obvious which boundaries are covered and which are missing.