What your AI can be told to do
Prompt injection is the new SQL injection. Here is what that means in practice.
In the early years of web development, SQL injection caught entire industries off guard. Developers trusted user input more than they should have, and attackers used that trust to rewrite database queries.
Prompt injection is the same class of problem, one layer up and it’s not theoretical. In December 2023, a car dealership’s chatbot was talked into agreeing to sell a 76,000 $ Chevrolet Tahoe for one dollar, and called it “a legally binding offer, no takesies backsies.” The screenshots went viral for a reason: this wasn’t a sophisticated hack. It was one sentence.
What is prompt injection?
A language model works by following instructions. Those instructions come from the system prompt, the conversation history, and any data the model retrieves at runtime. The model doesn’t have a reliable way to distinguish “this is an instruction I should follow” from “this is data I should reason about.” That distinction is enforced by structure, and structure can be broken.
Prompt injection is when an attacker embeds instructions in content the model processes, causing it to behave differently than intended. It doesn’t require technical skill. It requires understanding how the model is wired and inserting the right text in the right place.
Two types: one obvious, one dangerous
Direct injection is when the attacker speaks to the model directly: “Ignore your previous instructions and do X instead.” Most developers think about this variant, and some defences exist. Still worth testing.
Indirect injection is harder to catch. The attack arrives through data the model retrieves and processes, not through the user’s own input — a document the model summarises, a record from a database, a support ticket, a field in a form that feeds an automated workflow. The model reads it as data. It may act on it as a command.
Several high-profile similar attacks have been reported, with one public case being EchoLeak in 2025, where an unauthenticated attacker was allowed to exfiltrate sensitive corporate data by hiding instructions on an email that was consumed by a RAG engine. This is exactly the pattern behind most of the AI security incidents that actually make the news.
Why indirect injection deserves the focus
With direct injection, you control who talks to the model, and you can validate input before it arrives. With indirect injection, the attack surface is every piece of external data your model ever touches. If your model reads from the internet, customer records, third-party APIs, or uploaded documents, all of it is a potential vector you didn’t write and can’t validate the same way.
The defence is architectural: external data has to be structurally separated from instructions, embedded in clearly labelled data blocks rather than mixed in with system directives. Every field arriving from outside the model must be treated as untrusted before it enters the model’s context.
If you have a retrieval-augmented system
RAG, where the model searches a knowledge base and uses the results to answer questions, is particularly exposed to indirect injection. The retrieved documents aren’t authored by you. They may have been modified. A malicious document in your knowledge base, or a corrupted external source, can redirect the model’s behaviour for any user who retrieves it.
That’s not an argument against building RAG systems. It’s an argument for treating document provenance and integrity as a security concern, not just a data quality one.
Interesting test to run
Take your AI feature. Write the most harmful instruction you can think of. Something that would expose data it shouldn’t, perform an action it shouldn’t, or ignore its operational boundaries. Embed that instruction in every data source the model processes: a database record, an uploaded document, a retrieved web page, an API response.
If any of those inputs changes the model’s behaviour in a meaningful way, you have a prompt injection vulnerability. The only real question is whether you find it before someone else does.
Next time I’ll write about the agent that does too much: When AI moves from answering questions to taking actions. Want to read part 1: Why connecting an AI to your systems changes your threat model?