# Prove every change before you ship it

> Canonical: https://www.overmindlab.ai/product/agent-testing

*Agent testing*

Generate evaluations from your agent’s context, compare models and prompts on the same data, and test code changes in your own repository. Inspect the scores, evidence and winning diffs before deciding what to ship.

## Measure the result. Inspect the change.

Generated evaluators and reproducible experiments connect production failures to changes in your code.

### Evaluators grounded in the capability

Overmind generates checks from the capability’s purpose, prompt, tools and available trace evidence. Inspect the generated library, edit your own evaluators, and organize them into eval sets. Generative testing and live trace scoring use the roles appropriate to each evaluator.

`Generated evaluators` `Eval sets` `Trace scoring`

### Compare on the same evidence

Evaluate recorded outputs or generate fresh responses from model and prompt variants. Each run pins its dataset version and evaluator snapshots. Compare against a baseline, inspect per-metric scores and judge reasoning, and add human annotations to samples that need review.

`Pinned data` `Baseline comparison` `Human annotations`

### Improve prompts, tools and code

Run /overmind optimize from your agent’s repository. Your coding agent writes candidate diffs and replays them locally; Overmind scores the results against your chosen eval set. Inspect iteration scores and candidate patches, then apply the winning change through your normal code review.

`Local execution` `Candidate diffs` `Measured improvements`

### Test the model before you change it

Use /overmind backtest to replay the same dataset through candidate models while keeping the agent’s prompts and logic fixed. Compare quality alongside latency and cost, and inspect the samples behind the averages before choosing the next model or training experiment.

`Model comparison` `Shared eval set` `Cost & latency`

## What you get

### Evaluation

- **Reusable evaluator library** — Combine LLM judges and deterministic checks in capability-scoped or project-wide eval sets.
- **Reproducible comparisons** — Pin data, prompts and evaluators so results remain tied to what the experiment actually tested.
- **Inspectable results** — Read score rationales and sample outputs, with incomplete generations distinguished from scored results.

### Optimization

- **Repository-level changes** — Test edits to prompts, tool definitions and application logic in the environment where your agent runs.
- **Model backtesting** — Compare models over a shared dataset and evaluation set with your existing harness.
- **Bounded experiments** — Choose iteration and candidate budgets, follow progress, and inspect the selected candidate and its diff.

## Stop guessing, start measuring

From a production failure to a tested change in your repository.

- [Generate your evals](https://console.overmindlab.ai)

- Book a call: https://www.overmindlab.ai/contact

## How evaluation works, in the documentation

- [How do I run an evaluation against a baseline model?](https://docs.overmindlab.ai/agent-testing/eval)
- [How does the optimizer score a change to my agent?](https://docs.overmindlab.ai/agent-testing/optimisers)
- [How do I turn production traces into a dataset?](https://docs.overmindlab.ai/core/datasets)