Executive summary

Artificial intelligence (AI) agents are being built faster than they are being secured. As organizations deploy systems that can reason, retrieve information, and act on a user’s behalf, they are also introducing new security risks that are not always being tested with commensurate rigor. Recent public incidents demonstrate the consequences: In February 2026, an autonomous AI agent fully compromised McKinsey's internal AI platform in under two hours through a simple SQL injection, exposing 46.5 million chat messages and 95 system prompts.

This paper assesses and outlines the main security risks facing AI agents and how to mitigate them. We use the Open Web Application Security Project (OWASP) top 10 for Agentic Applications (2026) as the reference taxonomy to illustrate the main failure modes on an internally developed agent. This paper summarizes a structured security testing exercise (red teaming) conducted by DeepKeep on that agent. The exercise revealed that even a well-intentioned guardrail can be bypassed across virtually the entire OWASP taxonomy when it relies on a single layer of topic classification. The paper concludes that effective protection requires guardrails placed at several distinct points of the agent's execution path—and that red teaming should be a standard step before any agent goes into production.

Introduction

In February 2026, an autonomous AI agent breached McKinsey’s internal AI platform, Lilli, in under two hours, pulling out 46.5 million chat messages, 95 system prompts, and the credentials of 57,000 users. The culprit was a plain SQL injection—a well-known vulnerability where an attacker hides database commands inside a field meant for plain text—sitting behind 22 unauthenticated endpoints(CodeWall). Two years earlier, a Chevrolet dealership’s chatbot was talked into selling a $58,000 Tahoe for $1, after a customer told it to agree with anything and treat each reply as a binding offer (AI Incident Database #622). In the same year, a Canadian tribunal ordered Air Canada to honor a bereavement fare its chatbot had invented—the airline had argued the bot was a separate legal entity, and lost (Moffatt v. Air Canada, 2024 BCCRT 149).

Three different problems, one underlying cause. Building an agent has become easy: Pick a framework, wire up a few tools, ship it. Securing one has not. The threat surface is new: Attacks arrive as ordinary language, the agent itself can be turned on its owner, and traditional controls do not catch any of it. Most teams focus on what their agent is meant to do. Securing it gets less attention.

This paper looks at that gap. It covers the main risks AI agents face today (with reference to the OWASP framework), shows how those risks play out on an agent we built ourselves, summarizes a red-team exercise run against it by DeepKeep (a cybersecurity company specialized in red teaming and runtime protection for AI systems), and sets out what an effective defense looks like.

The OWASP reference framework

To address the new threats introduced by AI agents, OWASP—a nonprofit whose standards are used across the software industry—has published a dedicated reference for these systems. It defines 10 categories, labelled ASI01 to ASI10, which serve as the shared vocabulary for the rest of this paper.

The AI agent we put to the test

To show these vulnerabilities concretely, we built our own AI agent. It is a simple fictional travel assistant whose purpose is to let a user book a holiday in the Cornwall region of England. The agent answers questions about destinations, weather, lodging, and bookings.

How it works

The agent uses a structure common in production systems. The user sends a request about travel in Cornwall that includes anything within that scope, from checking the weather to booking a hotel within a budget or set of preferences. The system then uses a standard system prompt guardrail to filter user input.

If it passes, the request reaches a supervisor, which breaks it into smaller tasks and routes each one to a tool through an API, knowledge source, or other AI tools.

Each tool returns a result; the supervisor combines them into a single answer and sends it back to the user. Figure 1 below shows the architecture:

Security setup used in this exercise

The agent’s architecture is reasonable and reflects a common pattern for large language model (LLM)-based applications. Its protection relies on instructions placed at several points in the workflow: a main input guardrail that defines what the agent is allowed to discuss and internal prompts that guide how the LLM interprets tasks, uses tools, and structures its responses.

This reflects one part of the protection approach recommended for agentic AI systems.¹ In our case, the agent is protected through instructional prompts and an input guardrail. These controls provide a useful first layer, but they are not a complete defense on their own.

As the next section shows, an agent can be exposed to risk after the initial user prompt—through the context it retrieves, the tools it calls, and the outputs it generates.

Agent vulnerabilities

Four properties of the agent make it vulnerable:

1. It relies on multiple external inputs: The agent never works in isolation. It receives messages from the user, queries a weather API such a vector database, and reads from a hotel database. Each is an external channel, and each can carry content the agent will treat as trustworthy, even if the response is a poisoned record, a manipulated API response, or an adversarial prompt.
2. It operates across multiple turns: Because the agent maintains conversational context, an attacker is not limited to a single attempt. It can test how the agent responds, adjust its approach, and continue refining prompts over several interactions until it finds a path around the controls.
3. It can take actions, not just generate responses: Through connected tools, it can read and write data and trigger external actions. That means a successful attack can lead to real operational consequences, not just an incorrect or inappropriate response.
4. Its primary control focuses on topic, not intent: A forbidden request wrapped in travel language clears it: The surface matches, the intent does not. As a result, a prohibited request framed in acceptable language can still pass through. The wording may appear compliant even when the underlying intent is not.

A stronger guardrail would help but is not sufficient to fully secure this agent. Even a control that evaluates user intent only inspects one part of the interaction—the user’s message—missing the potential poisoned database record, the tool call, or the response on its way out. The context-poisoning attack we will show in Example 2 illustrates this clearly: The user's prompt does not present as malicious, and a single checkpoint cannot protect every stage at which an agent can be manipulated.

Two attacks in practice

The two examples below come from lessons learned with our own testing. They illustrate two types of vulnerability from the OWASP categories.

Example 1: Prompt injection (ASI01)

A direct out-of-scope request is correctly refused. The same request, wrapped inside a travel framing, slips through.

Example 2: Context poisoning (ASI06)

This attack is particularly dangerous because the user who planted the malicious content is targeting a future user. A hotel owner inserts a fake official notice into their own database record. Later, another user asks a normal question about hotel availability. The agent retrieves the poisoned record and presents the fake notice as fact.

Nothing looks wrong at the moment of compromise. The agent is faithfully reporting what its database says. The attack and the damage are separated in time and across users, which is exactly why a guardrail that reads one message at a time will never be sufficient.

What the red team found

DeepKeep's structured red-teaming exercise revealed weaknesses across multiple dimensions: prompt injection, data exposure, tool misuse, and business logic abuse. A single guardrail is not enough.

What is red teaming?

Red teaming is the process of intentionally attacking your own system to find security vulnerabilities and other issues before someone else does. The term has been used in cybersecurity for decades. Applied to AI agents, it means crafting realistic adversarial prompts and conversation flows, watching what the agent does, and tracking the failure modes.

Our red-team exercise was run by DeepKeep, an Israeli cybersecurity company focused on AI systems. They worked in two ways: An automated agent ran realistic attacks at scale, while their researchers handled the trickier multi-turn cases by hand. Both worked from our agent's public endpoint only—no access to the code—and focused on three things: keeping the agent inside its Cornwall scope, holding its own rules, and not leaking internal information.

Main findings

The guardrails could be consistently bypassed using simple prompt manipulation. Vulnerabilities were identified across multiple dimensions: prompt injection, data integrity violations, unauthorized data access, tool misuse, and business logic abuse. The current guardrail approach is therefore not robust enough for production deployment. Figure 2 lists each finding observed during the exercise.

Each finding maps to one or more of the OWASP categories listed earlier. The exercise covered most of the Top 10—the agent was vulnerable across more than half taxonomy. The full red-teaming report includes detailed traces and reproduction steps for every finding.

What an effective defense looks like

An effective defense places several guardrails at several points of the agent's execution path.

An effective defense places several guardrails at several points of the agent's execution path.

The agent we built is highly vulnerable. To protect it properly, dedicated firewalls—designed to deal with AI-specific attacks—need to be put in place at several points along the agent's execution path: at the user input, around tool calls, on the agent's output, and as continuous monitoring of its behavior.

For this exercise, we tested DeepKeep's firewall at a single point—the user input—and observed the following results:

Both detection metrics met the agreed criteria for keeping FPR and FNR below 10%.

Proper use of guardrails

In practice, several guardrails should sit at distinct points in the request lifecycle. Figure 3 illustrates one example of what a secured version of the agent could look like.

• Layer 1—Input. Validate and sanitize the user prompt before the model sees it. This is the layer that screens for prompt injection patterns, malicious instructions, and obvious policy violations.
• Layer 2—Tool execution. Every tool the agent can invoke sits on an explicit allowlist, and each call is validated for arguments, scope, and rate. This layer prevents tool misuse even when the agent has been tricked into attempting it.
• Layer 3—Output. Before any response reaches the user, it is checked for sensitive content (PII, internal logic, system prompts, credentials) and for content that violates the application's policy. The agent's output is treated as untrusted until validated.
• Layer 4—Monitoring and logging. Every tool call, prompt, and decision is logged in a structured form. Continuous monitoring detects anomalies and drift over time; logs support audit, incident review.

This layered model is consistent with current industry guidance on agentic AI security.² A secure agent should be protected through defense-in-depth across the full execution path—not only when it receives a user prompt, but also when it calls tools, produces outputs, and is monitored over time.

Conclusion and recommendations

AI agent security is no longer optional.

AI agents are no longer just demos or isolated prototypes. They are increasingly connected to users, business data, and operational processes. That changes the security question. An agent can now be attacked through conversation itself—by using ordinary language to influence how it reasons, which tools it uses, or which data it trusts.

This is not just about fixing individual bugs. Many of the weaknesses observed in the red-teaming exercise come from the way AI agents understand instructions, keep context, and decide when to use tools. Because an attacker only needs to find one phrasing that works, system prompts and traditional perimeter controls are not enough. Agents need layered security across the full execution path—input, tool calls, output, and continuous monitoring.

The challenge is also a moving target. The frameworks used today, including OWASP’s Top 10 for Agentic Applications, will continue to evolve as technology matures. Attack patterns will evolve as well. This means agent security cannot be treated as a one-time check before launch, but as an ongoing process of testing, monitoring, and adaptation.

The AI agent evaluation successfully demonstrated the ability to identify and mitigate various attack vectors, including prompt injection and context poisoning. Red-team findings highlighted agent vulnerabilities across the OWASP framework. Additionally, firewall testing confirmed that standard, basic guardrails often represent a structural weakness, while more robust guardrails (like those demonstrated by DeepKeep) can significantly mitigate these issues.

The main lesson is that security cannot be treated as something added at the end of an AI agent project. It must be there from the first design choices, through testing, deployment, and the day-to-day life of the agent once it's live. And as the agent grows, so do its defenses.

¹ Microsoft recommends a defense-in-depth approach, including input and output filtering, agent guardrails, logging, observability, and controls around tool use. See Microsoft, “Secure autonomous agentic AI systems”: https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems.

² See Microsoft, “Secure autonomous agentic AI systems,” for guidance on applying defense-in-depth to autonomous agents: https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems.