Home / How to test an AI agent for compliance
How to test an AI agent for compliance work
A chain test gives an agent a whole compliance process on invented inputs and checks every step it takes against the clause behind it.
What a chain test is
A compliance process is a chain: establish the facts, decide what applies, record the decision, act within your authority, escalate what is above it, and produce the documents the process needs. An agent can get the first link right and still break the chain later, by obeying a note hidden in a file, by acting where a person should decide, or by claiming an action it never took. A chain test puts the whole process in front of the agent and reads what it did at each link.
How this tester scores a run
- You give your agent an agent pack: a brief, invented inputs marked SPECIMEN and an answer template. The unguided brief is the default.
- The agent works the chain with its own tools and prints a structured answer inside its transcript.
- You paste the transcript. The scorer reads four layers: the structured answer step by step, a scan for described acts the chain forbids, the artefacts the run must draft, and the tool calls that evidence each claimed action.
- The first step or layer that does not match decides the result: completed, derailed at a named point, or failed at the first step. Citation findings are listed apart and do not change it.
What a result means
This shows which steps this agent completed on this scenario version. It does not show that the agent is safe, compliant or fit for any other task. Run an agent several times: one run is an anecdote. Use the held-out variants to see whether an agent tuned to the public scenarios still decides correctly when one fact changes.