Fine-Tuning with OpenClaw

When Fine-Tuning Is Worth It

Fine-tuning is not always the right tool. Before investing in a fine-tuning project, consider whether these alternatives already solve your problem:

• Better system prompts: 80% of quality problems can be solved with improved prompting. Always exhaust prompt engineering before fine-tuning.
• RAG: If the problem is that the model lacks domain knowledge, a RAG knowledge base is faster, cheaper, and easier to update than a fine-tuned model.
• Few-shot examples: Include 3-5 examples of desired input/output in the prompt. Often produces fine-tuning-level improvements for simple tasks.

Fine-tuning is worth it when: you need consistent output format that prompting alone can't reliably produce; you're running thousands of API calls daily and a cheaper fine-tuned small model can replace a larger model; or your task requires specialized vocabulary or reasoning patterns that base models handle poorly.

Dataset Preparation

Fine-tuning quality is entirely determined by training data quality. OpenClaw provides a dataset pipeline to help collect, clean, and format training examples from your existing agent runs.

# Collect examples from production agent runs (with consent/permissions)
openclaw finetune collect   --agent my-agent   --min-quality-score 0.85   --output dataset/raw.jsonl

# Review and curate examples interactively
openclaw finetune review dataset/raw.jsonl

# Validate dataset format and quality metrics
openclaw finetune validate dataset/curated.jsonl

// Fine-tuning dataset format (JSONL)
{"messages": [
  {"role": "system", "content": "You are a code review assistant..."},
  {"role": "user", "content": "Review this Python function: def calc(x): return x*2"},
  {"role": "assistant", "content": "This function multiplies the input by 2. Consider: 1) Add type hints..."}
]}

You typically need 50-500 high-quality examples for meaningful improvement. More examples help, but quality matters far more than quantity — 100 excellent examples outperform 1,000 mediocre ones.

Launching a Fine-Tuning Run

# Fine-tune via OpenAI's API (most common)
openclaw finetune train   --dataset dataset/curated.jsonl   --base-model gpt-4o-mini   --provider openai   --suffix "code-reviewer-v1"   --epochs 3

# Monitor training progress
openclaw finetune status --job-id ftjob-abc123

# Fine-tune a local model via Axolotl/Unsloth (for open-source models)
openclaw finetune train   --dataset dataset/curated.jsonl   --base-model mistralai/Mistral-7B-Instruct-v0.2   --provider local   --framework axolotl   --config finetune-config.yaml

Evaluating Fine-Tuned Models

OpenClaw's evaluation harness runs your fine-tuned model against a held-out test set and compares performance metrics against the base model. This quantifies improvement before you commit to switching production traffic.

openclaw finetune eval   --model ft:gpt-4o-mini:your-org:code-reviewer-v1:abc123   --test-dataset dataset/test.jsonl   --compare-to gpt-4o-mini   --metrics [exact_match, semantic_similarity, format_compliance]

Deploying Fine-Tuned Models

# Update agent config to use fine-tuned model
llm:
  provider: openai
  model: ft:gpt-4o-mini:your-org:code-reviewer-v1:abc123
  temperature: 0.3   # fine-tuned models often work well at lower temperature

For local fine-tuned models, convert to GGUF format and serve via Ollama using the same integration path as other local models. This gives you the cost and privacy benefits of local inference combined with the quality improvements from fine-tuning on your domain data.

Prompt Engineering vs Fine-Tuning: Decision Framework

Before committing to fine-tuning, systematically evaluate whether prompt engineering alone can solve your quality problem. The following decision framework helps determine which approach to try first:

Start by measuring your baseline: what percentage of responses meet your quality bar with your current prompts? If you're below 60%, focus on prompt improvement first — fine-tuning mediocre prompts produces a more expensive mediocre model. Once your best prompt achieves 70-80% quality, fine-tuning can close the remaining gap to 90-95% more efficiently than further prompt iteration.

Fine-tuning is most impactful for: output format consistency (the model reliably generates JSON, structured tables, or specific response templates), style consistency (responses always match your brand voice without style instructions in every prompt), and domain vocabulary (the model uses your organization's specific terminology naturally).

Continuous Improvement Loop

Fine-tuning is not a one-time operation — the most effective deployment uses fine-tuning as part of a continuous improvement cycle. Production runs generate new examples; the best examples augment the training set; periodic retraining keeps the model current as requirements evolve.

# Weekly fine-tuning improvement cycle
# 1. Collect production examples rated highly by quality checks
openclaw finetune collect --min-quality 0.9 --since "7 days ago" --output new_examples.jsonl

# 2. Merge with existing training data
cat dataset/v1.jsonl new_examples.jsonl > dataset/v2.jsonl
openclaw finetune validate dataset/v2.jsonl

# 3. Retrain and evaluate
openclaw finetune train --dataset dataset/v2.jsonl --base-model gpt-4o-mini --suffix v2
openclaw finetune eval --model ft:...:v2 --compare-to ft:...:v1

Teams running continuous fine-tuning improvement loops typically see steady quality gains over months — the model improves week over week as it trains on more and better production examples. Budget approximately 1-2 hours of engineering time per weekly cycle once the pipeline is set up.

Common Fine-Tuning Pitfalls

Catastrophic forgetting: Fine-tuning on narrow domain data can cause the model to lose general capabilities. If your fine-tuned model performs great on your specific task but worse on related tasks, your training data is too narrow. Include diverse examples that cover the full range of tasks the model will be asked to perform.

Overfitting to training examples: A model trained on too few examples memorizes them rather than learning the underlying pattern. Symptoms include perfect performance on training examples but poor performance on new inputs. Use at least 100 diverse examples, and always hold out 15-20% as a validation set that you never train on.

Label noise: Low-quality training examples teach the model bad patterns. A common source of label noise is using the LLM itself (via GPT-4) to generate training labels without human review. Always review at least a sample of generated training data before using it for fine-tuning — automated quality is rarely as good as it appears.