Home / How to run a test
How to run an AI agent test, step by step
For an insured, a vendor or a colleague who did not build the scenarios. About thirty minutes for the first one.
- Pick the scenario that matches the work the agent will do, from the scenario list. Note its id (for example KC-02) and version (for example 1.0).
- Download its agent pack from the scenario page. It holds a brief, an answer template and the input files. Every input is invented and its first line reads SPECIMEN; never add real records to it.
- Give the agent only the pack. Not the scenario page, not this site: the scenario page shows the expected answers.
- Paste the instruction from the answer format page into the agent, with the pack's folder, and run it the way you normally run it, with your own prompt and tools. We never connect to it and never need its credentials.
- Keep the whole transcript, including the tool calls if your agent shows them.
- Write down the run's identity: the agent's name, its model or version, the configuration (a prompt version, for example), the date and who ran it. You type these yourself; this site never fills them in.
- Paste the transcript into the score page, choose the scenario and version, type the identity, and press Score it. The score is computed in your browser.
- Read the sheet: completed, derailed at a step, or failed at the first step, with each step's four checks and the reason the first failing step failed.
- Export the score sheet as CSV, as an A4 print, or as the scorecard JSON. Send the scorecard JSON with the transcript: the person receiving it can re-verify the two match.
- Run it again. Agents give different answers on different runs; ten runs pasted together on the Many runs tab show how many runs passed each step.
This shows which steps this agent completed on this scenario version. It does not show that the agent is safe, compliant or fit for any other task.