Prompt injection is an attack on applications built on large language models in which text the model reads contains instructions, and the model follows them as if they came from the developer or the user. The name echoes SQL injection, and the root cause is similar: instructions and data travel through the same channel, so the system cannot reliably tell them apart.
The developer Simon Willison coined the term in September 2022, a day after Riley Goodside showed that a language model told to translate text could be talked into ignoring its instructions. Prompt injection has topped the OWASP Top 10 for LLM Applications, a widely used security checklist for software built on language models, since the list's first edition in 2023.
Direct and indirect injection
In a direct injection, the user types the instructions themselves, for example asking a support chatbot to reveal its hidden system prompt or to drop its rules. In an indirect injection, the instructions arrive through content the model processes on someone's behalf: a web page an assistant summarizes, an email an AI agent reads, a document in a RAG index, or white text on a white background that a human never sees. Indirect injection is the more dangerous of the two, because the person using the tool is not the attacker and may never notice.
Why it matters
The damage depends on what the model is allowed to do. A model that only writes text can be pushed into wrong or embarrassing answers. A model connected to tools through function calling or MCP can be pushed into actions: sending data to an attacker, changing records, or making purchases. The combination of access to private data, exposure to untrusted content and a way to send data out is what turns a curiosity into a breach.
How to limit it
There is no complete fix, because the model has no reliable way to separate trusted instructions from untrusted text. Practical defenses reduce what a successful attack can achieve:
- Give the model only the permissions and data the task needs.
- Require human confirmation before actions that send, delete, publish or pay.
- Treat everything the model produces as untrusted input, and escape or validate it before it reaches HTML, SQL or a shell.
- Keep untrusted content out of contexts that hold secrets or powerful tools.
Filters that look for phrases like "ignore previous instructions" catch the clumsy attempts and miss the rest. The reliable defense is the architecture around the model, not the wording of its system prompt.