How to Build an AI Agent Regression Test Library

An AI agent regression test library saves real failures as fixtures, expected behavior, and assertions so the same prompt-injection, tool-use, or data-leak bug does not return after the next model or prompt change.

Quick Answer

1. Save the exact hostile input or tool output.
2. Record the expected safe behavior.
3. Assert tool calls, not just final text.
4. Tag tests by capability and risk.
5. Run the library after every model or prompt change.

A screenshot is not enough

Teams often save a bad transcript and call it a lesson learned. That does not prevent recurrence. The failure needs to become a test case with a fixture, setup, expected response, and pass/fail rule.

For agents, the most important assertion is usually about tools: did the agent send, delete, invite, write, or fetch something it should not have touched?

Keep tests small

One regression should test one failure mode. A giant "evil document" with ten attacks is hard to debug. Smaller fixtures make it clear whether the agent failed instruction hierarchy, source handling, permission checks, or confirmation policy.

Where BreakMyAgent fits

BreakMyAgent is built around reusable attack cases. Teams can keep a library of known failures and rerun it as the agent surface changes.

FAQ

Should regression tests be deterministic?

As much as possible. Judge behavior with clear allow and block rules, especially around tool calls.

How many tests are enough?

Start with the top 10 flows that can cause external side effects or data exposure.

Do model upgrades require rerunning tests?

Yes. A safer model can still behave differently enough to break an agent workflow.