The architecture is the risk
Why connecting an AI to your systems changes your threat model
Typically, you spend months or even years hardening your APIs: input validation, access controls, rate limits, data schemas. Then you put a language model in front of those systems, and every one of those controls now sits downstream of something that can be talked into misbehaving by a well-written sentence.
That’s the real shift in the threat model: it’s not the model you need to worry about, it’s everything you’ve wired up around it. Suddenly the risk is no longer an exposed port or a misconfigured API, its actually written language amplifying any ill intention by attackers.
The model cannot do anything on its own
A language model, on its own, is text in and text out. It cannot reach your database, send an email, or delete a record. The risk starts the moment you give it tools — functions it can call, APIs it can invoke, data it can retrieve.
At that point, the model becomes a decision-making layer between the outside world and your infrastructure. Attackers don’t need to break the model. They need to convince it to do something harmful using its own capabilities. Natural language is now the attack surface, because natural language is what the model runs on.
Traditional security thinking doesn’t disappear — it just moves
Everything you already know about input validation, least-privilege access, and audit logging still applies. What’s new is a layer above all of it that doesn’t behave like software.
Traditional software does what you tell it to do. An AI system does what it interprets you to have told it, based on its training, the context it’s given, and the inputs it receives at runtime. That interpretation can be manipulated, wrong, or deliberately misled, and the defences need to account for that.
This is exactly the risk OWASP formalised when it made Prompt Injection the number one entry on its 2025 top 10 for LLM applications, with Excessive Agency close behind at number six. Two years into building production LLM systems at scale, the industry’s own risk register confirms what this post is arguing: the danger isn’t the model’s intelligence, it’s the permissions and inputs sitting around it.
Four questions to ponder if you are working with AI systems
Before any AI feature goes near production, four questions should have clear answers:
- What can it access? Every data source, API, and tool the model can reach is attack surface. The list should be minimal and explicit.
- What can it do? Read-only is a different risk profile from read-write. Irreversible actions require human confirmation — that’s an architecture decision, not a model setting.
- What does it know? Data flowing into the model, from users, databases, or external services, is untrusted input and needs to be treated that way.
- What record does it leave? If something goes wrong, can you reconstruct what the model saw, what it decided, and why? Without that, incident response is guesswork.
The hardening is structural, not cosmetic
A common mistake is treating AI security as a prompt problem: write better instructions, add a line saying “don’t reveal sensitive information,” tell the model to be careful. These help at the margins. They don’t replace structural controls.
The model shouldn’t be any line of defence. Access controls belong in the code that executes tool calls. Audit logs belong in the infrastructure. Data sensitivity belongs in the schema. The model’s instructions sit on top of all of that — they don’t substitute for it.
What changes, and what doesn’t
What changes is the attack surface. Natural language is an input vector now. Data the model reads is a potential injection point. The model’s outputs can reach operational systems with real consequences.
What doesn’t change is the underlying principle: every component that can be influenced by an external actor has to be designed as if it will be. The model is external. Its inputs are external. The threat model follows from that.
Teams that treat it as a prompt-wording problem tend to find out why that distinction mattered the hard way.
Next part in our series is coming soon. What your AI can be told to do: Prompt injection explained.