Led by a founder with more than 21 years of experience

contact@guilda.io

AI Security

MITRE ATLAS: how AI gets hacked, and how to catch it in your logs

MITRE ATLAS is MITRE's public knowledge base of how attackers target AI systems, built on the same model as MITRE ATT&CK. It lists their tactics, their techniques and real case studies, from prompt injection to poisoned plugins. Most of these attacks only show up if you log what the AI was asked, what it read and what it did.

Diagram: the five steps of the LLMjacking attack, from breaking into a server to reselling AI access, split between MITRE ATT&CK and MITRE ATLAS.
LLMjacking, named by Sysdig in May 2024, started as an ordinary intrusion and ended as an attack on the AI.

MITRE ATLAS (Adversarial Threat Landscape for AI Systems) is MITRE's public knowledge base of how attackers go after AI systems: the tactics they use, the techniques behind each one, and case studies of real attacks. It is built on the same model as MITRE ATT&CK. The current release, v2026.10 of 28 October 2026, lists 16 tactics, 129 techniques and 77 case studies. ATLAS tells you what to look for. Catching it is up to you, and most of these attacks only show up if you log what the AI was asked, what it read and what it did.

Four attacks that sound made up

Each of these is a case study in ATLAS.

A calendar invite that took over Gemini. Researchers at SafeBreach sent a Google Calendar invitation with instructions hidden in the event details. Later, the victim asked Gemini what was on their calendar, and the hidden text slipped into the conversation. When they replied "thanks", Gemini carried out the instructions: it opened the smart windows, switched on the boiler, started a Zoom call streaming video, deleted calendar events, and leaked data by opening a web address. The researchers reported it to Google in February 2025, and Google added defences before the research was published in August.

Stolen keys running someone else's AI. In the attack Sysdig named LLMjacking in May 2024, attackers broke into a server through an old web framework flaw, took the cloud credentials stored on it and used them to run Anthropic's Claude models through Amazon Bedrock, apparently to resell the access. Before anything else, they checked whether the victim logged the model's prompts. Sysdig estimated the bill could pass $46,000 a day.

An email tool that copied every message. A package on npm called postmark-mcp offered AI agents a way to send email through Postmark. Postmark never built it. After fifteen clean versions earned it users, version 1.0.16 quietly added the attacker's address as a blind copy on every email the agent sent, in September 2025.

"Summarize with AI" buttons that rewrite your assistant's memory. Microsoft found websites whose "Summarize with AI" buttons also told the visitor's assistant to remember the site as a trusted source in future conversations. In 60 days of email traffic it counted 50 such prompts from 31 companies in 14 industries, some on health and financial advice sites.

None of these broke the model itself. Each one abused what the AI could read, what it was allowed to do, or who held its keys. That is where detection has to look.

What MITRE ATLAS is, and how it relates to MITRE ATT&CK

MITRE ATT&CK, now at version 20, describes how attackers work against ordinary IT: email, endpoints, identities, cloud accounts. ATLAS applies the same structure to attacks where an AI system is the target or the tool. Tactics say what the attacker is trying to achieve, techniques say how, and each technique comes with mitigations and the case studies where it was seen. Identifiers start with AML: AML.TA for a tactic, AML.T for a technique, AML.CS for a case study. Where the OWASP Top 10 for LLM applications ranks the risks for the people building an AI application, ATLAS describes the attack from the attacker's side.

Fourteen of the sixteen ATLAS tactics carry the names of ATT&CK tactics and link to them, from Reconnaissance and Initial Access to Exfiltration and Impact. Two exist only in ATLAS: AI Model Access, gaining some level of access to a model, from its public API to its internals, and AI Attack Adaptation, turning general methods into an attack fitted to a particular AI system.

Real attacks use both. LLMjacking starts as an ordinary intrusion, an exploited server and stolen credentials, which ATT&CK covers, and ends as an AI attack, which ATLAS covers. We tag our detections the same way: the AI-specific behaviour with its ATLAS technique, the intrusion around it with ATT&CK, side by side in the same SIEM. ATLAS also changes fast. MITRE published a new release every month from July to October 2026, and the October one added techniques such as Exfiltration via AI User Interface and LLM Prompt Propagation.

The attacks on AI that ATLAS describes

The 129 techniques fall into a handful of families most teams need to know:

  • Prompt injection and jailbreaks. LLM Prompt Injection (AML.T0051), typed by the user or hidden in content the AI reads, as in the calendar invite, and LLM Jailbreak (AML.T0054), talking a model out of its safety rules.
  • Agents doing the attacker's work. AI Agent Tool Invocation (AML.T0053), making an assistant use its own tools, and AI Agent Context Poisoning (AML.T0080), planting instructions in its memory so they persist, as the "Summarize with AI" buttons did.
  • Data leaving through the AI. Data from AI Services (AML.T0085), collecting what an assistant can reach, such as its document store, its system instructions or past conversations, and exfiltration through the model's API (AML.T0024) or its chat window (AML.T0136).
  • A poisoned supply chain. Supply Chain Compromise (AML.T0010), through a model, a dataset or a package such as the fake Postmark tool, and Training Data Poisoning (AML.T0020).
  • Stolen access and runaway bills. Valid Accounts (AML.T0012), which in ATLAS includes API keys for AI services, and Cost Harvesting (AML.T0034), driving up the cost of a victim's AI services.
  • Copying the model. Replicate AI Model Behavior (AML.T0005), training another model on a target's answers. Recent case studies describe such campaigns against Anthropic and Google.

Why your current monitoring misses them

The four gaps we see most often:

  • Existing rules don't cover it. SIEM and endpoint rules watch logins, processes and network traffic. None of them reads a prompt, a retrieved document or an agent's tool call. To them, a calendar invite with hidden instructions is just a calendar invite.
  • The vendor's filter is treated as detection. A model provider's content filter decides whether to answer. It doesn't tell your team who tried, how often, or what the agent did next. The LLMjacking attackers checked first whether anyone was logging.
  • Nobody owns the alert. Without an owner and a runbook, an alert such as "agent emailed a new outside address" gets closed as noise.
  • Nobody knows what AI is running. Assistants inside software you already pay for, API keys in code, plugins a developer installed. You can't watch what was never listed. In IBM's Cost of a Data Breach Report 2025, 13% of organizations reported breaches of AI models or applications, and 97% of those had no proper AI access controls. One in five reported a breach involving shadow AI, tools staff used without approval.

What to log

Four records make attacks on AI visible. Most companies we work with are missing at least one:

  1. Prompts and replies, with the user and the session, kept in your own storage, not only in the vendor's console.
  2. What the AI read. The documents, emails and web pages pulled into each answer. Without it, an indirect prompt injection can't be traced back to the invite or page that carried it.
  3. Every tool and agent call: which tool, with what parameters, on whose behalf, and what it returned.
  4. The AI service's own logs: which key called which model, how often, and any change to its settings, such as whether prompt logging is on.

Then send all four to the SIEM, where your other security events already are. Prompts often hold personal data, so give these logs the same retention and access rules as any other sensitive log.

LLM monitoring for security: what to watch first

Much of what is sold as LLM monitoring tracks quality, latency and cost. The same data can answer a security question: is someone misusing this system? We start with four signals:

  • Usage and cost spikes per key or user: stolen keys, someone copying the model, or an agent stuck in a loop.
  • Agent actions out of pattern: a tool it has never called, an unusual volume or hour, or data sent to an outside address, as the blind copy in the Postmark case would have shown.
  • Sensitive data in the output: secrets, personal data, internal links, or a web address with data packed into it, as in the Gemini case.
  • Changes outside the release process to model files, system prompts, plugins, connected tool servers or training data.

Putting ATLAS to work next to ATT&CK

Search for "AI threat detection" and most results are about security tools that use AI. This article is about the other meaning: detecting attacks on your AI. For that, ATLAS is useful in three ways:

  • As a checklist when you build a threat model for an AI system. Walk it through the sixteen tactics and ask, for each, whether you would see it.
  • As a source of scenarios. The case studies make good tabletop exercises, and a purple team exercise can replay one to test whether the alerts fire.
  • As a shared language. An alert tagged AML.T0051 means the same thing to every analyst who reads it.

ATLAS is a map, not a rule set. It ships no detection rules: every rule still has to be written for your system and your logs.

Where GUILDA fits

Our AI threat detection work adds the logging above to the AI systems you build or buy, writes detection rules mapped to MITRE ATT&CK, with the ATLAS technique alongside where the behaviour is specific to AI, and a runbook for each alert. Your team operates it, in the same SIEM as the rest of your security events. If you don't have a SIEM yet, our threat detection and SIEM service sets one up and hands it over the same way.

Frequently asked questions

Is MITRE ATLAS part of MITRE ATT&CK?

No. It is a separate knowledge base, also from MITRE, built on the same model. Fourteen of its sixteen tactics carry the names of ATT&CK tactics and link to them, so an attack can be followed from one to the other. The techniques are its own, written for attacks where an AI system is the target or the tool.

How is MITRE ATLAS different from the OWASP Top 10 for LLM Applications?

The OWASP list ranks the kinds of weakness developers should fix in an application built on a language model. ATLAS describes what attackers do, step by step, against any kind of AI system, with real cases. Builders tend to start from OWASP; the people who watch for attacks tend to start from ATLAS. Most teams need both.

Do we need new tools to detect attacks on AI?

Usually not to begin with. The SIEM you already run can hold the alerts. What is missing is the data. Prompts and replies, what the AI read, what its tools did and how its keys were used. Once those logs reach the SIEM, the detection rules can be written there like any other.

Will GUILDA monitor our AI systems?

No. We add the logging, write the detection rules and the runbooks, and hand them to your team, who operate them in your own SIEM. We don't run a security operations centre or watch your systems around the clock.

Sources

  1. MITRE ATLAS, MITRE, 2026
  2. ATLAS data, release v2026.10, MITRE, 2026
  3. MITRE ATT&CK, version 20, MITRE, 2026
  4. Invitation Is All You Need: Hacking Gemini, SafeBreach, 2025
  5. LLMjacking: Stolen Cloud Credentials Used in New AI Attack, Sysdig, 2024
  6. Information regarding the malicious postmark-mcp package, Postmark, 2025
  7. Manipulating AI memory for profit: the rise of AI Recommendation Poisoning, Microsoft, 2026
  8. Cost of a Data Breach Report 2025, press release, IBM, 2025

AI Security

Agentic AI security: your newest employee has admin rights and no manager

AI agents do more than answer. They call tools, read data and act in other systems, often with a person's keys. Five public incidents from 2025 show what goes wrong, and each points to a control, from an identity of the agent's own to a named owner who can stop it.

7 min read

Tell us which AI systems you need to govern

List the tools and systems you use, built or bought. We reply within two business days.