What AI Actually Changes in Your Threat Model

Phishing. The old tells, poor grammar and generic greetings, are gone. Generated messages can be fluent, personalised from public information, and produced at volume. Training that relies on spotting sloppy emails is weaker than it was.

Impersonation. Voice cloning and synthetic video make it cheaper to pose as an executive or an employee. The practical target is the process that resets credentials or approves payments, which is usually a person on a help desk or in finance following a script.

Reconnaissance. Collating public information about staff, infrastructure, and exposed services is faster when a model summarises it.

Exploit development. Analysing what a patch fixed, and therefore what the unpatched version exposes, takes less expert time with capable models. The window between a fix shipping and exploitation in the wild was already short for high-value targets; it is not getting longer.

Organisations are deploying AI assistants and agents that read email, tickets, documents, and web pages, and can take actions: query systems, open tickets, send messages, run code.

That combination is the risk. A model does not reliably separate instructions from its user from instructions hidden in the content it reads. An agent that processes inbound email and can also call internal tools can be steered by a crafted message. The risk scales with the permissions the agent holds and the channels it can use to send data out.

Add data exposure: staff pasting sensitive material into external AI tools, and AI features in SaaS products gaining access to data they did not have before.

Most intrusions still start with stolen credentials, an unpatched internet-facing system, a misconfiguration, a third party, or a person talked into doing something. AI changes the speed and quality of those paths, not their existence. Organisations with strong identity controls, fast patching of exposed systems, least privilege, and good detection are still hard targets. Organisations without them were already easy ones.

• Phishing-resistant authentication. Hardware keys and passkeys defeat credential phishing regardless of how convincing the lure is.
• Help desk verification that does not rely on voice or video. Call-backs to numbers on record, manager approval, and out-of-band confirmation for resets and privilege changes.
• Payment and access changes with a second channel. Any request to change bank details, grant access, or bypass process gets verified through a separate, known route.
• Patch internet-facing systems on a clock measured in days. Track exposure, not just CVE counts.
• Treat AI agents like privileged service accounts. Least privilege, no unrestricted outbound channels, confirmation for destructive actions, and logging of every tool call.
• An AI usage policy people can follow. Approved tools for sensitive data are better than a ban that pushes use underground.

Threat model changes are only real if they show up in what you track. Useful measures are the share of accounts, and especially privileged accounts, on phishing-resistant authentication; the share of credential resets and access changes that went through out-of-band verification; the time from a fix for an internet-facing system being available to it being applied; and, for AI agents, an inventory of what each can read, what it can do, and where it can send data.

None of these depend on predicting which AI technique an attacker will use next. That is the point. They measure whether the controls that hold regardless are actually in place.

Expect social engineering incidents where the artefact is a recording or a convincing message rather than malware. Keep evidence of the request itself, the channel it came through, and the verification steps that were or were not followed. For incidents involving AI agents, preserve prompts, retrieved content, and tool call logs: they are the equivalent of command history, and without them the incident cannot be reconstructed.