AI agent compliance testing: the questions underwriters and risk managers ask
What is AI Agent Compliance Workflow Tester?
It gives you end-to-end compliance workflows to run your own AI agent through, and scores its answer step by step: completed, derailed at a named step, or failed, with the rule behind every step. It is for risk managers, underwriters and compliance leads deciding whether to trust an agent with compliance work.
Does it run or connect to my agent?
No. The tester never runs, calls or connects to your agent. It scores the transcript and answer you paste, and it is not live testing of an agent or its API.
What do I paste in?
You paste your agent's answer, or its whole transcript, after running your own agent on the specimen inputs. You also enter the agent name, model or version, configuration label, run date and who ran it, as you entered them.
How does the scoring work?
Four layers are read: the structured answer, with citation, decision, record and actions on every step; a scan for described acts the chain forbids; the artefacts the run must produce; and the tool calls that evidence each claimed action. The sheet carries a fingerprint anyone can re-check.
What are the scenarios?
There are eight scenarios: vendor onboarding with a bank detail change, a phished mailbox under the Privacy Act, an NDIS worker barred mid-shift, a ransomware payment report, an APRA insurer's provider incident, an AI claims agent's impact assessment, a material service provider, and an auditor's access review request. They can be filtered by process, framework, jurisdiction and risk.
Is AI Agent Compliance Workflow Tester free?
Yes, every scenario page, every agent pack, the scorer and the specimen runs are free, including one run, many runs, compare, re-verify and evidence. Solo is USD 99 a month to save runs and add the held-out variants, the full pack and the committee summary.
What do the paid plans add?
Solo is USD 99 a month for saved runs, the held-out variants, the full pack and the committee summary. Team is USD 299 and adds five people and the change log, and Underwriter is USD 990 and keeps up to 25 insureds or clients apart.
What is a held-out variant?
Six held-out variants, one each of six of the scenarios, are on the paid plans: one fact is changed so the expected decision changes, and they are kept out of every public file. That way an agent tuned to the public answer key shows it.
Does it show that my agent is safe or compliant?
No. It shows which steps this agent completed on this scenario version; it does not show that the agent is safe, compliant or fit for any other task. Whether a step passed is scored against the rule behind that step.
Do I need to give credentials or real records?
No. Nothing connects to your agent, and the work never asks for credentials or real records. Nothing you paste leaves your browser unless you save it, and you can choose to save the transcript with a confirmation that it contains no client data.
What is the score sheet for?
The score sheet is the record an underwriting file or a model risk committee asks for: the scenario and version, the agent as you entered it, every step with its four checks, the rule behind each step, and the reason the first failing step failed. It exports as a CSV, a printable A4 sheet and scorecard JSON with both fingerprints.
What do I give my agent to run?
Each scenario has a free pack: a brief, the inputs, every one marked SPECIMEN, and an answer template. You give your agent the pack, never the scenario page, because the page is the answer key.