How to Test an AI Agent Before It Goes Live: Test Cases, AI Judges and Versions

The riskiest moment in an AI agent's life is not its first run. It is the tenth edit to its briefing, made on a Friday afternoon, that quietly changes how it answers refund requests. Nobody notices until a customer does.

Software teams solved this decades ago with tests and version control. Agents need the same two things, adapted to answers that are never word-for-word identical. This guide shows what a useful test case looks like, how an AI judge scores the parts that no keyword can check, and how versions let you undo a bad change in one click. It uses Orkeste's Tests and versions tab as the example, but the method works on any platform. For where tests sit among the other building blocks, see the AI agent architecture guide.

Why agents need tests

An agent test is a saved request plus the checks its answer must pass. You run the whole set before and after a change, and compare.

Three things make agents drift:

Briefing edits. A new sentence about tone can override an older rule about escalation.
Model changes. Switching to a cheaper model, or a provider retiring the one you used, changes the answers.
New tools. Giving an agent a new connector or an MCP server changes what it tries to do.
New triggers. An agent started by an event runs far more often, on inputs nobody looked at first.

None of these break loudly. The agent still answers, just differently. A short test set turns "it feels worse" into "two cases out of eight now fail".

What a good test case contains

A test case in Orkeste has a name, the request sent to the agent, and up to five kinds of checks:

Keyword checks are cheap and exact. They catch the obvious regressions: a missing disclaimer, a forbidden promise. The criterion catches what keywords cannot, such as tone, or whether the answer actually addresses the question.

A worked example: the refund rule

A support agent drafts Zendesk replies. The rule is that refunds over 30 days are refused, politely, with a link to the policy.

Case. Name: "Late refund". Request: "I bought the annual plan 45 days ago and want my money back." Must contain: "policy". Must not contain: "refund has been processed". Criterion: "Declines the refund, stays polite, and offers an alternative such as a pause."
Baseline. Run the tests: the case passes.
Change. The team adds to the briefing: "Always be as accommodating as possible with long-standing customers."
Rerun. The case now fails. The judge explains why: the draft offers a partial refund. The keyword check still passes, which is exactly why the criterion is there.
Fix or roll back. The team rewrites the new sentence to exclude refunds, reruns, and the case passes again. Had there been no quick fix, restoring the previous version would have taken one click.

How the AI judge works

The qualitative criterion is scored by a separate, small model (Claude Haiku 5.5 in Orkeste) that receives the request, the agent's answer and your criterion, and returns a verdict with a one-line reason. Two practical rules make it reliable:

One idea per criterion. "Polite and refers to the policy" is two checks; split them or accept that a failure is harder to read.
Describe the behaviour, not the wording. "Offers an alternative to a refund" works better than "mentions the pause option", which a keyword already checks.

The judge's calls are not billed. If the workspace uses the EU sovereign setting, the judge also runs on an EU-hosted model.

Test runs do not touch your tools

A test that sends a real email is not a test. In Orkeste, a test run is a dry run:

In agentic mode, the agent really reads your applications (it searches Zendesk, reads the CRM record), so the test reflects real data. Every write, delegation to another agent and non-read-only MCP tool is simulated and marked "[TEST RUN] not executed". No outgoing webhook is sent.
In classic mode, the agent produces its answer only. Nothing is extracted as an action and nothing reaches the approval queue.

Test runs are marked as such in the run history. They do use credits, like any run, because the model does the same work.

Versions: every change, kept

Tests tell you something broke. Versions tell you what changed and let you go back.

Every real change creates a version, whichever way it was made: Agent Studio, the API or another integration. Versioned fields are the name, description, role, tone, goals, mission, permissions, model provider, model and execution mode. Counters, costs and status updates do not create versions.
The history shows who changed what, field by field.
Restoring is a new version. Going back to version 4 writes version 4's settings as version 7. The history is never rewritten, so you can always see that a rollback happened and why.

Agents installed from the marketplace cannot be restored this way: their configuration belongs to their creator.

A routine that takes five minutes

Write five to eight cases from real requests: two easy, two edge cases, one the agent must refuse, one that needs a tool.
Run them once to set the baseline.
Before saving a change to the briefing or the model, rerun them.
When a real conversation goes wrong, turn it into a new case.

Up to 10 cases run per launch, three at a time, and partial results appear while the rest finish.

Limits

Tests run one request at a time: they do not simulate a long conversation or a full swarm. A passing test set reduces risk, it does not prove an agent is right on every input. And the judge is a model too: for rules with legal or financial weight, keep a person in the approval queue whatever the tests say.

FAQ

How do you test an AI agent before deploying it?

Save real requests as test cases with checks on the answer (words it must or must not contain, tools it must or must not call, and a plain-language criterion scored by an AI judge), run them before and after each change, and compare the results.

What is an LLM judge?

A second model that reads the request, the agent's answer and your criterion, and returns pass or fail with a reason. It is useful for tone and relevance, which keyword checks cannot capture.

Can I roll back an AI agent to a previous version?

In Orkeste, yes: open Tests and versions, pick the version and restore it. The restore is saved as a new version, so the history stays complete.

Do test runs send emails or update my CRM?

No. In a test run, writes are simulated and marked as not executed, and no outgoing webhook is sent. In agentic mode the agent still reads your tools, so the test uses real data.

Do tests cost credits?

Yes, a test run uses credits like a normal run of the same agent and model. The AI judge's calls are not billed.