Subscribe
Opinion
5 min read

Opinion: Prompt Injection Is a Design Problem, Not a Bug

AI agents follow instructions hidden in what they read. OWASP says it is unclear if fool-proof prevention exists. My view: limit what a hijacked agent can do.

Olivetti-style cover reading Prompt Injection Is a Design Problem and Opinion, beside a charcoal panel holding teal and mustard bars, a red-outlined bar and a red dot, on cream paper
Olivetti-style cover reading Prompt Injection Is a Design Problem and Opinion, beside a charcoal panel holding teal and mustard bars, a red-outlined bar and a red dot, on cream paper

This is an opinion piece. The facts come from the linked sources; the argument is mine.

AI assistants can be tricked by instructions hidden in a web page, an email or a document. Simon Willison, who says he coined the term "prompt injection," wrote in June 2025 that "Researchers report this exploit against production systems all the time."

Here is where I land. Prompt injection is not a bug that the next filter will finally squash. It comes from how language models work. So the useful question is not "how do we block every bad instruction?" It is "what can this agent do if it obeys one?" I think companies and users should design around that question, and treat any promise of complete protection with suspicion.

Why models follow instructions they shouldn't

Willison puts the core issue plainly: "LLMs follow instructions in content. This is what makes them so useful." The catch is that "they don't just follow our instructions."

When you ask an assistant to summarize a page or read your inbox, that text goes into the same input as your request. In his words, "LLMs are unable to reliably distinguish the importance of instructions based on where they came from. Everything eventually gets glued together into a sequence of tokens and fed to the model."

The OWASP Gen AI Security Project lists prompt injection as LLM01, the first entry in its list of risks for AI apps. It separates two kinds. Direct injection comes from what a user types. Indirect injection happens when a model "accepts input from external sources, such as websites or files." OWASP also notes that prompt injections "do not need to be human-visible/readable, as long as the content is parsed by the model." So you cannot always spot the trap by looking at the page.

If you want a hands-on view of how this plays out, our guide on keeping AI agents safe from prompt injection walks through practical steps.

Why I don't expect a filter to fix it

OWASP itself is careful here. It says prompt injection is possible "due to the nature of generative AI" and that "it is unclear if there are fool-proof methods of prevention for prompt injection." It adds that techniques like retrieval and fine-tuning "do not fully mitigate prompt injection vulnerabilities."

Willison goes further: "we still don't know how to 100% reliably prevent this from happening." He says he is "deeply suspicious" of guardrail products that claim to catch "95% of attacks" or similar, because "in web application security 95% is very much a failing grade."

I agree with that framing. A filter that catches most attacks is useful as one layer. But an attacker only needs the one phrasing that gets through, and there is no fixed list of phrasings to block. Treating a detection score as a safety guarantee is, in my view, the main mistake in how this risk gets discussed.

The real risk is the combination

Willison's most useful idea is what he calls the "lethal trifecta." An agent is exposed when it has all three of these: "Access to your private data," "Exposure to untrusted content," and "The ability to externally communicate in a way that could be used to steal your data."

His warning is blunt: "If your agent combines these three features, an attacker can easily trick it into accessing your private data and sending it to that attacker."

The lethal trifecta: three overlapping circles labeled private data, untrusted content and external communication, with risk where all three meet

What I like about this is that it moves the conversation from the model to the setup. You may not be able to stop a model from reading a malicious sentence. You can decide whether the same agent also holds your private files and a way to send data out. Email is the clearest example. As Willison notes, an email tool is "a perfect source of untrusted content: an attacker can literally email your LLM and tell it what to do".

He also points out that the risk grows when people connect tools themselves. When vendors fix a reported attack, they usually close the path data leaks out through. But "once you start mixing and matching tools yourself there's nothing those vendors can do to protect you".

Design so a hijacked agent can't do much

If prevention is not guaranteed, the goal becomes limiting the damage. OWASP's own mitigation list leans that way. Among its steps:

Design limits for AI agents from OWASP: least privilege, ask for human approval, separate untrusted content, test the model as an untrusted user

Willison cites a research paper with a summary I would put on the wall of every team building agents: "once an LLM agent has ingested untrusted input, it must be constrained so that it is impossible for that input to trigger any consequential actions."

This lines up with an argument I have made before: start with a fixed workflow before reaching for a fully open agent. A workflow with set steps has fewer surprise doors for injected text to open. The more freedom an agent has, the more an attacker's sentence can steer it.

What I'd ask of companies and users

For companies, I think the honest pitch is "here is what this agent can reach, and here is what it will ask you before doing." That is more useful than a claim that attacks are blocked. Show the permissions. Default to asking before sending, deleting or paying. Keep tools that read the open web away from tools that hold private data, unless there is a strong control in between.

For users, the check is short. Before you connect a new tool to an assistant, ask three questions. Can it see my private data? Can it read things strangers write? Can it send anything out? If the answer is yes to all three, that setup deserves real caution. Our explainer on browser agents is a good example of a tool type that reads a lot of outside content.

I am criticizing a way of thinking about the problem, not anyone building these tools. Agents are useful, and the people warning about this risk want them to work. The point is to stop treating prompt injection as a bug on a fix list and start treating it as a limit to build around.

The bottom line

Language models follow instructions in the text they read, and OWASP's guidance says it is unclear if any method fully prevents that. So the safest agent is not the one with the best filter. It is the one that cannot do much harm when the filter misses.

Sources

OpinionSecurityAgents