Prompt injection attacks: real examples, why they work and the defences that hold

Prompt injection is an attack in which text fed to a large language model overrides the instructions its developer gave it. The text can be typed by a user, or hidden in a web page, email, document or tool result the model reads on someone’s behalf. For a chatbot, the damage is usually an embarrassing answer. For an AI agent with access to a mailbox, a code repository or a payment API, a successful prompt injection attack can read private data and send it to a stranger without the user clicking anything.

This explainer is for engineers and security leads who are putting agents into production. It covers the two forms of the attack, documented prompt injection examples from 2023 to 2025, why every major lab still describes the problem as unsolved, and the layered defences that limit what a fooled model can do. Attacks are described at the level of mechanism, with example payloads defanged; the aim is to help you recognise and contain the pattern, not to reproduce it.

The short version: you cannot filter your way out of prompt injection. You can design so that when the model is fooled, it does not hold the keys, cannot reach the exit and leaves a trail.

What prompt injection is, and why it is not SQL injection

The term dates from September 2022, when Simon Willison named the attack after Riley Goodside posted examples of GPT-3 prompts being hijacked by inputs that told the model to ignore its previous instructions. Willison drew the obvious parallel: like SQL injection, it comes from concatenating trusted instructions and untrusted input into one string. He also pointed out the uncomfortable difference. SQL injection has a fix, parameterised queries, that keeps data in a separate channel from code. Language models have no equivalent channel.

That difference is still the heart of the problem three years later. In December 2025 the UK National Cyber Security Centre published a post titled Prompt injection is not SQL injection (it may be worse). Its argument is that current models do not enforce any boundary between instructions and data inside a prompt: everything is tokens, and the model predicts the next one. The NCSC describes a model as an “inherently confusable deputy” and concludes that prompt injection may never be fully mitigated in the way SQL injection can be. Its advice is to reduce the likelihood and the impact, and to accept that some systems should not use an LLM at all if they cannot tolerate the residual risk.

The OWASP Top 10 for LLM Applications puts prompt injection first, as LLM01 in the 2025 edition. OWASP’s definition is broad: user prompts alter the model’s behaviour or output in unintended ways, and the malicious input does not even need to be visible to a human, provided the model parses it. OWASP also notes that retrieval-augmented generation and fine-tuning, often assumed to make models more grounded, do not fully mitigate the risk.

Where it sits in the NIST taxonomy

The US National Institute of Standards and Technology covers the attack in NIST AI 100-2 E2025, its March 2025 taxonomy of adversarial machine learning. NIST splits it into two entries in its generative AI attack taxonomy: direct prompting attacks (NISTAML.018) and indirect prompt injection (NISTAML.015). The distinction it draws is useful. Because generative models “combine the data and instruction channels”, an attacker who controls a resource the system reads can inject instructions “without directly interacting with the application”. And unlike direct attacks, indirect ones are mounted by a third party, so in many cases the person harmed is the model’s legitimate user.

Direct vs indirect prompt injection

The two forms need different defences, so it is worth being precise about them.

Direct prompt injection is the version most people picture. The attacker is the user. They type something into the application that tries to override the system prompt: change the persona, reveal hidden instructions, skip a policy, or call a tool the developer did not intend them to reach. A defanged example of the pattern is a message that claims a fake system update and tells the assistant it now operates “without restrictions”. OWASP’s first example scenario is an attacker telling a support chatbot to ignore its guidelines, query private data and send emails.

Indirect prompt injection is the version that matters for agents. The attacker never talks to the model. They place instructions in content the model will process later: white-on-white text on a web page, an HTML comment, a calendar invite, a code comment, a GitHub issue, a PDF, an image, a tool description, or the output of another tool. When a user asks the agent to summarise the page or triage the inbox, the planted text arrives in the same context window as the user’s request and the developer’s system prompt, and the model may treat it as an instruction.

The research paper that made the indirect form concrete was Greshake and colleagues’ 2023 study, which showed planted prompts working against Bing’s GPT-4-powered chat and code-completion systems. The authors wrote that LLM-integrated applications “blur the line between data and instructions”, and catalogued impacts that are now familiar: data theft, worm-like self-propagation, and contamination of the information a user sees.

OWASP also lists a case that is neither malicious nor obvious: a hiring manager hides an instruction in a job description to flag AI-written applications, and an honest applicant who uses an LLM to polish their CV triggers it. The lesson is that any text a model reads is effectively code that runs with the model’s privileges, whoever wrote it and whatever they meant.

Figure 1: In an indirect prompt injection, the attacker’s only access is the ability to write somewhere the agent will read. Everything after that runs with the user’s permissions.

Prompt injection examples from documented incidents

The incidents below were disclosed publicly by the vendor or by the researchers, and fixed or mitigated before publication. They are summarised at the level of mechanism. What they share matters more than the details of any one exploit.

Microsoft 365 Copilot: CVE-2025-32711

In June 2025 Microsoft published CVE-2025-32711, an information disclosure vulnerability in Microsoft 365 Copilot that public write-ups call EchoLeak. Microsoft rated it critical with a CVSS 3.1 base score of 9.3 and described it as “AI command injection in M365 Copilot” allowing “an unauthorized attacker to disclose information over a network”. The vector string records no privileges and no user interaction required. Microsoft says the issue was fully mitigated on its side and customers had nothing to do.

The pattern described publicly is the textbook indirect case. An attacker sends an ordinary-looking email containing instructions addressed to the assistant. Later, when the user asks Copilot a routine question, retrieval pulls that email into context alongside internal documents, and the model follows the planted instructions to place sensitive content into a link or image reference. The output is rendered, the client fetches the URL, and the data leaves inside the request. No click is needed, which is why the case is often called zero-click.

GitHub’s MCP server and a public issue

In May 2025, researchers at Invariant Labs disclosed an attack against GitHub’s official MCP server, which Simon Willison summarised. An attacker files an issue in a public repository. The issue text asks whichever agent reads it to gather information about the repository owner, including the names of their other repositories, and to put it in a pull request. A developer whose coding agent has a token covering both public and private repositories asks the agent to look at open issues. The agent reads the planted text, reads the private repositories it has access to, and publishes what it found in a public pull request.

Nothing in that chain is a bug in the usual sense. The token worked as issued, the tools did what they were designed to do, and the model followed instructions. That is why this example is a fixture of every MCP gateway and MCP server security discussion: the vulnerability is the combination of tools, not any one of them.

Perplexity Comet and a Reddit comment

In August 2025 Brave published research on Perplexity’s Comet browser, whose assistant could summarise the current page. Brave found that the assistant passed page content to the model without separating it from the user’s request. In their proof of concept, instructions hidden in a Reddit comment caused the agent, when asked to summarise the thread, to retrieve the user’s account email, trigger a one-time password login, read the code from the user’s Gmail session and post both values back. Brave’s timeline shows the report on 25 July, an initial fix two days later, a retest showing the fix was incomplete, and a note at publication that the class of attack was not fully mitigated.

Brave’s proposed mitigations read like a summary of this article’s later sections: separate user instructions from page content, check planned actions against what the user actually asked, require confirmation for sensitive actions, and isolate agentic browsing from the user’s normal logged-in sessions.

ChatGPT memory and persistent injection

In September 2024, security researcher Johann Rehberger described SpAIware, a chain against the ChatGPT macOS app that combined two weaknesses. Prompt injection from an untrusted website or document could instruct ChatGPT to write attacker instructions into its long-term memory, and those instructions told it to render an image hosted on an attacker’s server with conversation content in the URL. Because memories persist, the instruction carried into later, unrelated chats. OpenAI closed the image exfiltration channel in app version 1.2024.247. Rehberger noted that a website or document could still invoke the memory tool to store arbitrary memories, and advised users to review their stored memories.

The persistent variant matters because it breaks a common assumption: that a prompt injection lasts one session. Any agent with writable memory, notes, a vector store or a shared scratchpad can be given a long-lived instruction.

What the examples have in common

Read across the rows and three ingredients recur every time: the agent could read something private, it processed text an attacker wrote, and it had a way to put data somewhere the attacker could see. That recurrence has a name.

The lethal trifecta: why agents changed the stakes

In June 2025 Simon Willison described the lethal trifecta for AI agents: “access to your private data”, “exposure to untrusted content” and “the ability to externally communicate”. Any system that combines all three in one context can be tricked into stealing the user’s data, because the attacker can use the second leg to instruct the model to move data from the first leg out through the third. Willison’s list of affected products is long and crosses vendors: Microsoft 365 Copilot, GitHub’s MCP server, GitLab Duo, ChatGPT and its plugins, Google Bard and NotebookLM, Amazon Q, Slack, Mistral Le Chat, xAI’s Grok, Anthropic’s Claude iOS app and ChatGPT Operator among them.

He also explains why MCP raised the stakes. The protocol “encourages users to mix and match tools from different sources”, and many tools that read private data also return untrusted content. A team that connects a mail server, a browser and a web-fetch tool to the same assistant has assembled the trifecta without writing a line of code. That is the context for the safety notes in our guide to the top MCP servers of 2026: the risk sits in the combination, not in any single server.

The trifecta is useful because it turns an unbounded problem into a design review. You cannot predict every phrasing an attacker will use, but you can list, for each agent, which of the three legs it has. Remove one leg and the data-theft path closes, even if the model is fooled. The NCSC makes the same point in different words, quoting Willison’s rule that when a model processes information from a party, its privileges should drop to those of that party.

Figure 2: Two legs of the trifecta are often acceptable. All three in one context is the configuration behind most documented data-theft cases.

The trifecta is about confidentiality. Agents with write tools face a parallel problem: an injected instruction does not need to exfiltrate anything to cause harm if it can delete a branch, send an email in the user’s name, change a setting or move money. OWASP’s scenarios include exactly that. For write-capable agents, the second leg (untrusted content) combined with any consequential action is enough to need controls.

Why prompt injection is still not solved

Every major model developer has published on prompt injection defences, and every one of them stops short of claiming a fix. That consensus is the most important fact in this article, because it decides where the defences have to live.

Anthropic. In November 2025 Anthropic published results for its browser agent. It described three lines of defence: reinforcement learning on simulated web content containing injections, classifiers that scan untrusted content entering the context for hidden text and deceptive interface elements, and internal and external red teaming. Against an internal adaptive “Best-of-N” attacker given 100 attempts per environment, the attack success rate fell to 1%, which Anthropic called a significant improvement that still represents “meaningful risk”. The post says prompt injection “is far from a solved problem”.

OpenAI. OpenAI’s developer guide, Safety in building agents, recommends keeping untrusted input out of developer messages, constraining data flow between steps with structured outputs, keeping tool approvals switched on (including for read operations through MCP), running input guardrails, and scoring agent behaviour with trace graders and evals. It says these measures reduce the risk rather than eliminate it.

Google. Google’s layered defence post from June 2025 lists five layers for Gemini: prompt injection content classifiers, security thought reinforcement, markdown sanitisation and suspicious URL redaction, a user confirmation framework, and end-user security mitigation notifications. A companion paper from Google DeepMind, Lessons from Defending Gemini Against Indirect Prompt Injections, describes continuous evaluation with adaptive attack techniques against past, current and future Gemini versions. The framing is ongoing hardening, not a finished product.

Microsoft. Microsoft’s security response centre describes a defence-in-depth approach of hardened system prompts and Spotlighting to mark untrusted input, Prompt Shields classifiers for detection, and impact mitigation through data governance, deterministic blocking of exfiltration techniques such as markdown image injection, and human consent before actions such as sending email. The post states plainly that Microsoft’s approach “does not rely on our ability to block all prompt injections”.

Why detection alone falls short

The benchmark literature explains the caution. AgentDojo, an evaluation environment from ETH Zurich and Invariant Labs researchers, contains 97 realistic agent tasks (email, banking, travel booking) and 629 security test cases. Its authors found that state-of-the-art models fail many tasks even with no attack present, and that existing attacks break some security properties but not all. That makes it a useful yardstick and a reminder that utility and security trade against each other.

Classifier-based defences face an arithmetic problem that Willison put bluntly: a guardrail vendor claiming to catch 95% of attacks is advertising “very much a failing grade” by web security standards, because an attacker gets to try many times and only needs one success. Adaptive attackers, the kind Anthropic and Google DeepMind now test against, rephrase, encode, split payloads across documents and switch languages, which are exactly the evasion tricks OWASP lists. The NCSC adds a practical warning: do not rely on deny-lists of known attack phrases, and be sceptical of any product that claims to “stop” prompt injection.

None of this makes classifiers useless. Lowering an attack success rate from double digits to one per cent matters, especially against opportunistic attacks. It means classifiers are one layer that buys time and visibility, and the layers that decide what a successful injection can reach have to be deterministic.

Defence in depth: the layers that cap the damage

The defences fall into five layers. The first two decide the blast radius before any prompt is processed. The middle one lowers the odds. The last two catch what gets through and keep it from leaving.

Figure 3: Detection lowers the odds; design, permissions and egress control decide what a fooled model can actually do.

Layer 1: design the agent so injection cannot steer it

The strongest defences change the architecture so that untrusted text cannot influence which actions run. Two pieces of research define the state of the art.

CaMeL, from Google DeepMind and ETH Zurich researchers, wraps the model in a system layer that extracts the control flow and data flow from the user’s trusted query before any untrusted data is read. Untrusted content retrieved later can fill in values but cannot change the program’s flow, and capabilities attached to data block private values from reaching tools they are not allowed to reach. On AgentDojo, CaMeL solved 77% of tasks with provable security, against 84% for an undefended system: a measurable cost in utility for a guarantee no classifier offers.

The June 2025 paper Design Patterns for Securing LLM Agents against Prompt Injections, with authors from IBM, Invariant Labs, ETH Zurich, Google and Microsoft among others, generalises the idea into six patterns. Its guiding principle is that once an agent has ingested untrusted input, it “must be constrained so that it is impossible for that input to trigger any consequential actions”.

Most teams will not implement CaMeL. But a simpler version of the idea is available to anyone: split agents so that the one reading the open web or inbound email has no access to private systems and no write tools, and passes only structured, validated results to the agent that does.

Layer 2: least privilege and tool allow-lists

If the model is fooled, it can only misuse the tools and data it can reach. Every source in this article recommends shrinking that set. OWASP lists “enforce privilege control and least privilege access” as a core mitigation. The MCP specification requires servers to implement access controls and rate limits, and tells clients to treat tool annotations as untrusted unless they come from trusted servers.

In practice that means three things. Give each agent or caller an explicit allow-list of tools rather than everything a server exposes. Prefer read-only variants of tools where a task does not need writes. And scope credentials to the task: the GitHub MCP case would not have leaked private repository details if the agent’s token had covered only the public repository it was asked to triage. Our ranking of MCP servers for coding agents scores servers partly on how finely they let you scope that access.

Layer 3: human approval for consequential actions

A person confirming an action is the one control an injected instruction cannot argue with, provided the person sees what is about to happen. The MCP tools specification says there “SHOULD always be a human in the loop with the ability to deny tool invocations”, and asks clients to show tool inputs to the user before calling the server “to avoid malicious or accidental data exfiltration”. OpenAI’s agent guide says to keep tool approvals on. Microsoft’s Copilot in Outlook drafts a message but leaves sending to the user.

The weakness of approval is fatigue. An agent that asks permission forty times an hour trains its user to click yes. The workable compromise is risk-tiered: auto-run idempotent reads inside a narrow scope, require confirmation for anything that writes, sends, pays, deletes or reaches a new domain, and display the full arguments rather than a summary written by the same model that may have been injected.

Layer 4: input and output guardrails

Guardrails are the classifiers and rules that inspect text on the way in and on the way out. They are the most visible defence and the most over-sold, so it helps to be specific about what each kind does.

• Injection classifiers on input. Microsoft’s Prompt Shields detects two attack types: user prompt attacks, and document attacks in content such as emails and web pages. For agents it can also scan tool responses, a preview feature at the time of writing. Google, Anthropic and AWS Bedrock Guardrails run comparable detectors.
• Spotlighting and delimiting. Marking untrusted content so the model can distinguish it, from simple XML tags (an NCSC suggestion) to Microsoft’s base64 Spotlighting, which costs extra tokens and works only on some APIs.
• Guardrails on tool arguments and results. The tool boundary is where an injection becomes an action, so it is the most valuable place to inspect. Checking the arguments of a send_email or create_pull_request call catches exfiltration attempts that look harmless in the prompt.
• Output checks. Secrets detection, PII redaction, and stripping or rewriting links and images in model output to close the rendering channel used in the Copilot and SpAIware cases. Google’s “markdown sanitisation and suspicious URL redaction” layer exists for this reason.
• LLM-as-judge policies. A second model evaluates content against a natural-language policy. Useful for organisation-specific rules; itself an LLM, so in principle injectable, which is why it belongs alongside deterministic checks.

OWASP’s “define and validate expected output formats” belongs here too. An agent step that must return an enum or a fixed JSON schema leaves an attacker much less room than one that returns free text, which is the reasoning behind OpenAI’s advice to use structured outputs between agent nodes.

Layer 5: egress control, sandboxing and monitoring

The exit channel is the third leg of the trifecta and often the easiest to close deterministically. Block the client from fetching images or links from domains outside an allow-list. Route agent network traffic through an egress proxy. Run code-execution tools in a sandbox with no network by default. Treat a tool that can post to an arbitrary URL as a write tool, because it is one. Microsoft’s “deterministic blocking of exfiltration techniques” is this layer.

Monitoring is the layer that tells you whether the others are working. The NCSC recommends logging enough to spot suspicious activity, potentially including full model inputs and outputs, tool use and API calls, and notes that failed tool calls can reveal an attacker probing. Add red-team runs against every new release (AgentDojo is a reasonable starting harness), and, for agents with memory, periodic review of what has been stored.

The gateway layer: guardrails, tool filtering and logs in one place

Several of those layers share a property: they are easier to enforce once, at a point every model call and tool call passes through, than separately in every application. That point is an AI gateway, the proxy between applications and model providers that our AI gateway explainer describes, and for agents it increasingly includes MCP traffic as well. A gateway cannot fix prompt injection; it is the model, not the proxy, that gets confused. What it can do is apply permissions, guardrails, egress rules and logging consistently, so that a team with twelve agents does not have twelve different ideas of what “least privilege” means.

The products in this space handle the security layer differently, and the details matter.

• LiteLLM’s proxy supports guardrails in several modes: pre_call on the input, post_call on input and output, during_call in parallel with the model call, and logging_only. Its documentation frames the feature around prompt injection detection and PII masking through third-party providers.
• Cloudflare AI Gateway offers guardrails that evaluate prompts and responses against content-safety categories such as violence and hate, and can log or block flagged content. Its guardrails page does not describe prompt injection detection.
• Microsoft’s Azure stack puts Prompt Shields, including the document and tool-response scanning described above, into Foundry guardrails assigned to model deployments and agents.

How Bifrost applies these controls

Bifrost, the open-source AI gateway written in Go by Maxim AI, is worth a closer look here because its documentation covers four of the five layers at the gateway: guardrails on model traffic, guardrails on MCP tool execution, tool allow-lists with approval defaults, and request logging. It also acts as an MCP gateway, so the tool boundary and the model boundary pass through the same process.

Guardrails on prompts, responses and tool calls. The Bifrost guardrails documentation describes rules written in CEL (Common Expression Language) that decide which requests to check, linked to profiles that decide how. A rule can target LLM traffic or MCP tool execution and apply to input, output or both; for MCP rules, input means the tool’s arguments before it runs and output means its result before the model sees it. A block on the input side stops the tool from executing. The docs give the example of a rule that inspects arguments before a GitHub tool creates an issue, which is the exact boundary the GitHub MCP case crossed. Bifrost-managed profiles include an LLM-as-judge Prompt Guardrails provider, custom regex and Gitleaks-backed secrets detection. External providers listed as of October 2026 include AWS Bedrock Guardrails (whose integration lists prompt attack detection), Azure Content Safety, Google Model Armor, CrowdStrike AIDR, Gray Swan Cygnal, Patronus AI, Check Point’s AI Agent Security, Repello Argus, Singulr AI, A10 Guardrails, Microsoft Presidio and Azure AI Language PII. A single rule can chain several profiles, so a team can pair a vendor injection classifier with its own regex and secrets checks. Maxim’s overview of AI guardrails at the gateway describes the same model at a higher level.

Tool allow-lists, deny by default. Bifrost’s tool filtering applies at three stacked levels: the MCP client configuration, per-request headers and the virtual key, and a tool must pass every level to reach the model. An empty or omitted tool list means no tools from that client. On a virtual key with no MCP configuration, no MCP tools are available unless a client is explicitly marked allow-by-default, and a caller-supplied header can only narrow the key’s allow-list, never widen it. That is least privilege expressed as configuration, per caller.

Approval as the default. By default Bifrost does not execute tool calls the model proposes; the application calls a separate execution endpoint, which is where an approval step goes. Its agent mode can run tools automatically, but only those listed in tools_to_auto_execute, which starts empty, and anything not on that list is returned to the application for approval. The documentation’s own guidance is that operations such as write_file, delete_file and execute_command should typically require human approval, which matches the risk-tiered approach above.

Logs for the monitoring layer. Bifrost’s built-in observability records inputs, outputs, tokens, cost and latency for each request, written asynchronously. When guardrail redaction is enabled, logs store the redacted form, with reveal limited to users holding a specific permission, which addresses the awkward fact that the logs you need for incident response are themselves sensitive.

For latency-sensitive agents, the gateway hop itself is small: Bifrost’s published benchmark reports 11 µs of added overhead per request at 5,000 requests per second on an AWS t3.xlarge. External guardrail providers add their own round trip, which is why Bifrost exposes sampling and per-rule timeouts. Teams that want the tool-side picture in more depth can read our MCP gateway explainer or Maxim’s page on running an MCP gateway for agent traffic.

Enterprise tier. Guardrails, along with SSO, RBAC, audit logs and clustering, are part of Bifrost Enterprise; tool filtering, agent-mode approval defaults and request logging are in the open-source gateway.

What a gateway does not cover is the first layer. No proxy can decide that your email-triage agent should not also hold a token for the finance system. That is a design decision, and it comes first.

A worked example: hardening an inbox assistant

Take a common first agent: an assistant that reads a shared support inbox, looks up customers in a CRM and drafts replies. Run the trifecta check. It reads private data (the CRM and the mailbox), it processes untrusted content (every inbound email is written by a stranger), and if it can send email or render links, it can communicate externally. All three legs are present, so this is the configuration behind the Copilot case.

A defensible version, layer by layer, looks like this:

1. Design. Split it in two. A quarantined step reads each inbound email and returns a fixed schema: customer identifier, intent category from an enum, a short summary with links stripped. A second step, which never sees raw email, queries the CRM and drafts a reply.
2. Permissions. The CRM tool is read-only and limited to the fields support staff need. The drafting step has no send tool at all; it writes drafts.
3. Approval. A human sends every reply. For high-volume categories the team can later auto-send templated answers, but never ones containing free text the model generated from an email.
4. Guardrails. An injection classifier runs on inbound email as a signal (flag, do not silently drop, since false positives on angry customers are common). Secrets and PII checks run on drafts. A rule on the CRM tool’s arguments rejects bulk queries.
5. Egress and monitoring. Drafts are rendered with external images and unknown links disabled. Every model call and tool call is logged with the email it came from, and a weekly job reviews blocked tool calls.

None of these steps requires the model to resist the injection. If a planted email still talks the first step into misclassifying itself, the worst case is a wrong summary in a draft that a person reads before sending.

What to do this quarter

Prompt injection is not going to be patched away by the next model release, and every developer quoted here says so. The teams that avoid the next CVE will be the ones that treat model output as untrusted input to the rest of their system. A practical sequence:

• Inventory agents against the trifecta. For each agent, write down which of private data, untrusted content and outbound communication it has. Redesign any that have all three before adding more tools.
• Turn default-allow into default-deny. Give every caller an explicit tool allow-list and scope its credentials to the task. Remove write tools that no workflow uses.
• Tier approvals by consequence. Reads in scope can run; writes, sends, payments, deletions and new domains need a human who sees the full arguments.
• Put guardrails on the tool boundary, not only the prompt. Inspect tool arguments before execution and results before the model sees them, and strip links and images from rendered output.
• Centralise enforcement where you can. A gateway that sees every model and tool call makes allow-lists, guardrails and logs consistent across teams; our piece on shadow AI governance covers the traffic that never reaches it.
• Measure. Run AgentDojo-style tests on each release, review blocked tool calls weekly, and track the attack success rate the way you track latency.

The honest framing for leadership is the NCSC’s: prompt injection is a residual risk. The engineering job is to decide, agent by agent, how much damage a fooled model is allowed to do, and to make that number small.