Agent evals

The idea

An evaluation is a task plus evidence and grading rules. A trial is one attempt; a suite collects tasks. Evaluate the model together with its harness—the instructions, tools, and execution loop that let it act. A grader can inspect produced code, stored records, or a trace of tool interactions. Repeating a task measures variation; it does not make the agent deterministic.

Where it helps

Manual spot checks work for exploration but leave a weak basis for changing models, prompts, or tools. A fluent completion message can hide an incorrect result. Turn acceptance criteria into inspectable evidence before delegating recurring engineering work.

A team adopting agents needs a shared definition of successful delegated work before expanding autonomy. Evals turn familiar acceptance tests and review habits into repeatable evidence for choosing prompts, tools, and models.

Illustration: ask a coding agent to fix an empty-password authentication bug. Patch A rejects every login. It passes the new empty-password test but fails an existing valid-password test. Patch B passes both. The second patch is ready for further review, not automatically safe to release. These two cases are deliberately incomplete authentication coverage.

What you will be able to check

• Write a task with observable success criteria and at least one behavior that must remain unchanged.
• Separate an agent failure from a broken test environment using retained evidence.
• Compare two agent configurations across repeated trials without reporting only the best attempt.

What changes in the team

Today

An engineer implements a change, runs selected tests, and asks a colleague to review the patch and acceptance criteria.

With an agent

The team saves representative tasks, runs the agent from a clean starting state, and grades each produced artifact against retained acceptance and regression checks.

Still your responsibility

Engineers own the task definition, grader quality, permissions, ambiguous failures, and the release decision.

Work through an example

1. Define the task before running the agent. For the authentication example, record the starting revision, synthetic users, expected login behavior, and a time budget. Keep grader tests outside the agent’s editable submission. Include valid login, empty input, and a separate unfamiliar case for later validation.
2. Run the actual configuration being considered: model, prompt, tool definitions, permissions, and harness version. Save the patch and tool trace. Start each trial from the same clean fixture so a previous attempt cannot leave a helpful file or changed database behind.
3. Grade the artifact with trusted checks. Apply the patch and execute both the new requirement and existing behavior tests. Confirm the checks actually ran: a missing dependency or timeout is not proof the patch is wrong. An agent exhausting its agreed time budget still failed that task attempt; distinguish that from a broken evaluation environment. Inspect traces to explain failures without demanding one exact tool sequence.
4. Compare the current and proposed configurations on the same tasks and budgets. Retain every attempt, including timeouts; report passes over attempts, failure categories, and time spent. Keep a separate set of tasks out of prompt tuning. Review unexpected results before accepting a change, then add newly discovered production failures to the suite.

The engineering decision

Make required correctness checks explicit. Use code to check concrete outcomes; use a rubric and human-calibrated model grading for subjective qualities such as explanation clarity. Do not average an authentication failure away with good prose. Passing a finite suite supports a bounded decision, not a general reliability guarantee.

• Optimizing only the new test rewards a patch that disables every login; preserve regression checks.
• Reusing fixtures or cached grades can produce a false improvement; reset state and identify each run distinctly.
• Letting an agent rewrite its own acceptance tests undermines the evidence; grade the submitted artifact independently.
• Treating evaluation success as permission enforcement confuses measurement with access control; tool and runtime controls must enforce boundaries.

Read further

Which recent agent-assisted change looked successful but needed correction, and what observable check would have caught it?

• Anthropic: Demystifying evals for AI agents — tasks, trials, graders and isolated environments
• OpenAI: Evaluation best practices — representative cases and human calibration
• Anthropic: Writing effective tools for agents — held-out tasks and valid alternative paths
• SWE-bench: Evaluation guide — execute generated patches and distinguish harness errors

Connected ideas

• Harness engineering
• Observability and tracing
• Canary deployments