Code Review Agent

Build an agent that automatically reviews GitHub pull requests: fetches the diff, analyzes code quality and security issues, and posts a structured review comment with severity ratings.

Tutorial25 minIntermediate

What We'll Build

  • Triggered automatically by a GitHub webhook on every PR opened/updated
  • Fetches the PR diff via GitHub API
  • Reviews each changed file for bugs, security issues, and code style
  • Posts a structured review comment to the PR

Agent YAML

# code-review-agent.yaml
name: code-reviewer
description: "Reviews GitHub PR diffs for bugs, security, and style"

llm:
  model: gpt-4o          # Use a capable model for code analysis
  temperature: 0.1

system_prompt: |
  You are a senior software engineer conducting a code review.
  Analyze the provided diff and return a structured JSON review with:
  - summary: one-paragraph overall assessment
  - issues: list of {file, line, severity, category, message, suggestion}
    - severity: critical | high | medium | low | info
    - category: security | bug | performance | style | test_coverage
  - score: integer 1-10 (10 = excellent)
  
  Focus on real issues. Do not flag stylistic preferences as bugs.
  For security issues, cite the OWASP category when applicable.

tools:
  - github_get_pr_diff
  - github_post_comment
  - github_set_pr_status

output:
  schema:
    type: object
    required: [summary, issues, score]

Webhook Trigger

# triggers/pr-review-trigger.yaml
triggers:
  - type: webhook
    path: /webhooks/github
    secret: "${GITHUB_WEBHOOK_SECRET}"  # HMAC-SHA256 verification
    filters:
      event: pull_request
      action: [opened, synchronize, reopened]
    run:
      agent: code-reviewer
      input:
        pr_url: "{{ payload.pull_request.url }}"
        repo: "{{ payload.repository.full_name }}"
        pr_number: "{{ payload.number }}"
# Start OpenClaw with webhook listener
openclaw serve --triggers ./triggers/ --port 8080

GitHub Tools Setup

# tools/github.yaml
tools:
  github_get_pr_diff:
    type: rest
    url: "https://api.github.com/repos/{{ repo }}/pulls/{{ pr_number }}/files"
    method: GET
    headers:
      Authorization: "Bearer ${GITHUB_TOKEN}"
      Accept: "application/vnd.github.v3+json"
    output: "{{ response.items }}"
    
  github_post_comment:
    type: rest
    url: "https://api.github.com/repos/{{ repo }}/issues/{{ pr_number }}/comments"
    method: POST
    headers:
      Authorization: "Bearer ${GITHUB_TOKEN}"
    body:
      body: "{{ comment_markdown }}"

Review Comment Template

# 🤖 OpenClaw Code Review — Score: {{ score }}/10

{{ summary }}

## Issues Found

{% for issue in issues | sort(attribute='severity') %}
### {{ '🔴' if issue.severity == 'critical' else '🟠' if issue.severity == 'high' else '🟡' if issue.severity == 'medium' else '🔵' }} `{{ issue.file }}` (line {{ issue.line }})
**{{ issue.category | upper }}** — {{ issue.message }}

💡 _Suggestion:_ {{ issue.suggestion }}
{% endfor %}

Test Locally with a Sample Diff

openclaw run --config code-review-agent.yaml \
  --input '{"diff": "--- a/app.py\n+++ b/app.py\n+query = f\"SELECT * FROM users WHERE id = {user_id}\""}' \
  "Review this diff"

# Expected: critical security issue flagged (SQL injection)

What This Agent Does

This tutorial builds a code review agent that analyzes code for bugs, security vulnerabilities, style issues, and improvement opportunities. The agent can review individual files, compare diffs, or analyze entire directories. It produces structured review comments similar to what you'd receive in a pull request review, making it suitable for pre-commit checks, CI pipeline integration, or developer tooling.

Step 1 — Agent Configuration

Create code-review-agent.yaml:

agent:
  name: code-reviewer
  description: Reviews code for bugs, security issues, and style
  system_prompt: |
    You are an expert code reviewer with deep knowledge of software
    engineering best practices, security vulnerabilities (OWASP Top 10),
    and idiomatic patterns in major programming languages.

    When reviewing code, structure your output as follows:
    ## Summary
    Brief overall assessment (2-3 sentences).

    ## Critical Issues
    Bugs or security vulnerabilities that must be fixed before merge.
    Each issue: line number, issue description, suggested fix.

    ## Warnings
    Non-critical but important issues: performance concerns,
    error handling gaps, deprecated API usage.

    ## Suggestions
    Style improvements, better approaches, documentation gaps.

    ## Verdict
    APPROVE / REQUEST_CHANGES / NEEDS_DISCUSSION

llm:
  model: claude-3-5-sonnet-20241022
  temperature: 0.2
  max_tokens: 4096

tools:
  - name: file_read
    enabled: true
  - name: file_list
    enabled: true
    config:
      allowed_dirs:
        - ./src
        - ./tests

Step 2 — Review a Single File

Review a specific file by passing it directly in the task:

openclaw run --config code-review-agent.yaml   --task "Review the file src/auth.py for security vulnerabilities and code quality issues"

The agent reads the file, analyzes it, and produces a structured review. For security-sensitive code like authentication handlers, it will highlight injection risks, improper error handling that leaks information, insecure credential storage, and missing input validation.

Step 3 — Review a Git Diff

For PR-style reviews, generate a diff and have the agent review it:

git diff HEAD~1 > changes.diff
openclaw run --config code-review-agent.yaml   --task "Review this diff for issues: $(cat changes.diff)"

Or create a task file to keep the command clean:

echo "Review the following git diff for bugs, security issues, and style problems:" > review-task.txt
echo "" >> review-task.txt
git diff HEAD~1 >> review-task.txt
openclaw run --config code-review-agent.yaml --task-file review-task.txt

Step 4 — Integrate with Pre-commit Hooks

Add automatic code review to your git workflow. Create .git/hooks/pre-commit:

#!/bin/bash
# Get list of staged Python files
STAGED=$(git diff --cached --name-only --diff-filter=ACM | grep '\.py$')
if [ -z "$STAGED" ]; then exit 0; fi

echo "Running AI code review on staged files..."
RESULT=$(openclaw run --config ~/code-review-agent.yaml   --task "Review these files for critical issues only: $STAGED"   --output-format json)

VERDICT=$(echo $RESULT | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('verdict',''))")
if [[ "$VERDICT" == "REQUEST_CHANGES" ]]; then
  echo "AI review found critical issues. Fix them or use git commit --no-verify to bypass."
  echo $RESULT | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('critical_issues',''))"
  exit 1
fi
echo "AI review passed."

Step 5 — Language-Specific Review Profiles

Create specialized review profiles for different languages by customizing the system_prompt. A Python profile emphasizes type hints, exception handling, and PEP 8 compliance. A JavaScript profile focuses on async/await correctness, prototype pollution risks, and dependency hygiene. A SQL profile looks for injection vulnerabilities, missing indexes on query predicates, and N+1 query patterns. Maintain a library of review profiles and select the appropriate one based on the files being reviewed.

Troubleshooting

If the agent produces overly verbose suggestions for trivial issues, add a directive to the system_prompt: "Focus on actionable issues only. Do not comment on formatting or trivial style preferences." If the agent misses language-specific patterns, add explicit instructions: "Pay special attention to SQL injection via string concatenation in database query functions." The more specific your system prompt, the more focused and useful the reviews become.

Calibrating Review Strictness

A code review agent that flags everything including trivial style issues creates alert fatigue — reviewers stop paying attention to its output. Calibrate strictness by defining what matters for your codebase. In the system prompt, explicitly tell the agent which issue categories to report and which to ignore. If your codebase has an enforced linter for style, tell the agent not to comment on style and focus exclusively on logic, security, and correctness. This targeting makes every comment actionable.

Use severity tiers consistently. Reserve CRITICAL for "this code has a security vulnerability or will produce incorrect results." Use WARNING for "this may cause problems in edge cases or under load." Use SUGGESTION for "this could be cleaner or more idiomatic." When a reviewer sees CRITICAL, they know they must address it before merge. When they see SUGGESTION, they know they can use judgment. Consistent semantics for severity tiers make the review actionable rather than requiring the reviewer to interpret the agent's level of concern for each comment.

Building a Review History

Store review outputs with the commit hash and timestamp they were generated for. Over time, this history reveals patterns: which files get the most critical findings, which developers frequently receive warnings about the same issues, which code areas have never been reviewed. Use these patterns to target your human review effort — reviewers should spend the most time on high-risk areas that the agent consistently flags, and can move quickly through code that consistently receives clean reviews.

Feed recurring agent findings back into your team's coding standards. If the agent repeatedly flags the same anti-pattern across multiple reviews, that pattern should be added to your linting rules or coding standards document so it's caught automatically on every commit without requiring agent analysis. The agent is most valuable for novel issues and complex logic — repetitive mechanical checks should be automated away so the agent's time (and cost) is spent on higher-value analysis.

Multi-Language Review Strategies

For repositories with multiple languages, create separate review profiles and select the appropriate one based on the file extensions being reviewed. A shell script review prioritizes injection prevention and quoting issues. A Dockerfile review checks for unnecessarily large base images, secrets baked into image layers, and missing security best practices. An infrastructure-as-code review looks for overly permissive IAM policies, unencrypted storage, and resources missing required tags. Specialization makes each review profile far more useful than a generic review that covers all languages shallowly.