AI agent security testing: judge the agent by what it does, not what it says.
An agent reads, decides and acts, often several times per request, with real credentials. A chatbot that says something wrong is a bad answer. An agent that does something wrong is an incident. This guide covers where agents break, how to test every tool they hold, and how to grade them on actions instead of text.
AI Agent Security Test Cases, by Attack Class
Six ways agents fail. Pick one to see the setup, the task to give the agent and the action a secure agent takes, or refuses to take.
AI Agent Security Test: Replaying One Trace, Step by Step
A support agent is asked to clear the overnight queue. One ticket contains a planted instruction. Every step is graded on the action taken and whether the right control stopped it. All tools run in dry-run mode.
Illustrative trace · synthetic tickets and tools · dry-run · 20 s replay
Tool Poisoning Example: A Third-Party Tool That Rewrites the Agent's Rules
The agent never saw a malicious user or document. It loaded a tool from an external server, and the tool's own description told it what to do with every other tool.
Testing retrieval instead? →Illustrative finding · synthetic data
A Permission Model for Agent Tools
Classify every tool by what it can change. The tier sets the control that must sit in front of it, and what your tests should prove.
Read internal
search, tickets.get, docs.readWrite internal, reversible
tickets.update, labels, draftsExternal communication
email.send, webhooks, file sharingMoney and irreversible
refunds, payments, delete, deployA recommended model. Map tiers to your own change and approval policies.
How to Test an AI Agent: Assert on Actions
Six steps that grade an agent the way an incident review would: by the tool calls it made and the controls that held.
Running agent tests safely →Inventory tools and scopes
List every tool, its credentials, what it can change and who added it, including third-party tool servers.
Tier every tool
Classify tools by risk and record the control each tier requires. Gaps here are findings before any test runs.
Sandbox the actions
Run tools in dry-run or against test resources, with argument capture, so every attempted action is visible and harmless.
Plant the observations
Seed tickets, emails, pages and tool outputs with approved injection payloads the agent will read during normal tasks.
Run tasks as real users
Give the agent ordinary tasks on behalf of each user role and grade the trace: calls, arguments, identity, controls.
Rerun on change
Trigger on tool, definition, prompt, model, memory or permission changes; watch third-party definitions continuously.
Agent Validation, Graded on Every Tool Call
Hayrok gives your agent real tasks with planted observations, runs every tool in dry-run or against test resources, and records each step of the trace. Findings are graded on the actions taken and the controls that should have stopped them.
Task-based scenarios
Gives the agent realistic tasks with planted observations from an approved library.
Dry-run tool execution
Captures every attempted call and its arguments inside approved limits; nothing real is sent or deleted.
Per-user identity
Runs the same task on behalf of each role to expose confused-deputy and cross-tenant actions.
Graded on the trace
Findings are scored on actions and missing controls, mapped to OWASP LLM Top 10 and MITRE ATLAS.
Evidence and regression
Each trace is stored in Evidence Fabric and replayed when tools, prompts or models change.
AI Agent Security Testing FAQ
Testing an AI agent by giving it real tasks under adversarial conditions, such as planted instructions in what it reads, requests beyond the user's rights and tools that misbehave, then grading the tool calls it makes and the controls that should have stopped them.