# Prove every change before you ship it

> Canonical: https://www.overmindlab.ai/product/agent-testing

*Agent Testing*

Catch regressions before your users do. Overmind writes the evals from your agent's own context graph, tests every candidate against your baseline, and turns the winning change into a GitHub PR.

## Evals you did not have to write

The context graph knows what each agent does. That knowledge becomes the test suite.

### Eval generation from context

The repo scan builds a capability card for each agent: what it is designed to do, the tools it can call, and how it can fail. Rubrics are written from that card and compiled into weighted pass/fail checks you review and edit before anything is scored.

`Grounded in your code` `Failure modes covered` `You review, then run`

### Scoped to each agent

A triage agent is not scored on a planner’s criteria. Rubrics are scoped per agent, and every run reports per-metric scores you can compare across models, prompts, and releases.

`Fair per-agent scores` `Every score explained` `Comparable over time`

### The Optimiser improves the whole agent

Candidates are real git diffs across prompts, tool definitions, and agent logic, written by a coding agent and run against your actual repo through the Overmind CLI. Every candidate is scored on the same eval set as your baseline, so a gain is a measured result, not a hunch.

`Whole-agent diffs` `Runs on your repo` `Same eval set as baseline`

- **52% → 78%** — Accuracy lift from a single optimiser run, shipped as a PR

### The winning change is a PR

When a candidate beats the baseline, one click opens the GitHub PR: the winning diff with the baseline-versus-candidate scores in the description. If gains flatten across experiments, Overmind says so and recommends training instead of more prompt surgery.

`Merge-ready PR` `Scores in the description` `Says when to train`

## What you get

A test suite generated from context and an optimisation loop that ends in a reviewable change.

### Evals

- **Rubric generation** — Rubrics grounded in each agent's purpose, tools, and failure modes, straight from the repo scan.
- **Multiple evaluator kinds** — Judge, trajectory, deterministic, statistical, and agentic evaluators, scored at output, turn, or trajectory level.
- **Per-metric scores with rationale** — Each metric has a pass threshold, and every judge score carries its reasoning. A low mark is an explanation.

### Optimisation

- **Same-set benchmarking** — Baseline and every candidate scored on the identical eval set. Differences come from the change alone.
- **GitHub PR output** — One click turns the winning diff into a pull request with its scores attached. Review, comment, merge.
- **Plateau detection** — When best scores stop improving across experiments, Overmind says so and recommends training instead.

## Stop guessing, start measuring

Evals from context. Improvements as PRs.

- [Generate your evals](https://console.overmindlab.ai)

- Book a call: https://www.overmindlab.ai/contact