Led by a founder with more than 21 years of experience

contact@guilda.io

AI Security

WormGPT and FraudGPT: ChatGPT's evil twins are writing your next phishing email

WormGPT and FraudGPT are AI chatbots sold to criminals without the safety limits of ChatGPT. They mostly write phishing and fraud messages that read perfectly. Phishing-resistant MFA, trained staff and a second check on every payment change still beat them.

Diagram: four AI-written fraud scenarios and the defence that stops each: verifying outside the message for three, phishing-resistant MFA for the fake login.
AI removed the old clues, such as bad spelling and odd phrasing; teaching staff to stop at the pattern of the request still works.

WormGPT is a chatbot like ChatGPT, built for criminals: it writes whatever it is asked, with none of the safety limits mainstream AI assistants have. FraudGPT is its better-marketed sibling. Both appeared on underground markets in July 2023, sold by subscription, and both are mostly used for one thing: writing phishing and fraud messages that read as if a native speaker who knows your business wrote them. The good news is that the defences that stop those messages do not care who wrote them. Phishing-resistant MFA, trained staff and a second check on every payment still work.

What WormGPT and FraudGPT are

In July 2023, researchers at SlashNext reported a tool called WormGPT advertised as an AI without limits for phishing and business email compromise. They asked it for an email pressuring an account manager to pay a fraudulent invoice and called the result "remarkably persuasive but also strategically cunning". Two weeks later, Netenrich described FraudGPT, sold at $200 a month or $1,700 a year and advertised for scam pages, phishing messages and malicious code.

The original WormGPT closed later in 2023, but the brand did not. In 2025 Cato Networks found two tools sold as WormGPT that were not new AI models at all. They were mainstream models, xAI's Grok and Mistral's Mixtral, wrapped in a hidden instruction telling them to ignore their safety rules. In November 2025 Palo Alto Networks' Unit 42 reported WormGPT 4, on sale since September 2025 from $50 a month, and a free alternative called KawaiiGPT. Both wrote convincing phishing emails on request.

The lesson is that the name matters less than the trend. These tools are cheap, they are easy to replace and anyone can rent one. What used to take a skilled fraudster an afternoon now takes a beginner a minute.

What attackers use them for: four emails your team could get this week

Unit 42 put the change plainly: these tools remove the grammatical errors and awkward phrasing that used to give phishing away, and can convincingly impersonate a CEO or a trusted supplier. These are the four scenarios we see most often at clients, and how AI makes each one harder to spot.

1. The supplier with new bank details

Your accounts payable clerk receives an email from a supplier you have paid for years. Same signature, same tone, the right invoice number. It says the supplier has changed banks and asks that this month's payment go to the new account. Before AI, the giveaway was often the English: a stiff phrase, a wrong idiom. Now the email is flawless, and it may even be written in the supplier's own style, copied from a real email the attacker stole earlier.

2. The CEO who needs it done now

On a Friday afternoon your assistant gets a message from the CEO, who is travelling: a confidential acquisition, a wire transfer that has to go today, or a bag of gift cards for a client. The FBI's 2025 Internet Crime Report names this exact pattern, saying chat generators "can quickly create official-sounding emails mimicking a company's CEO". Losses to business email compromise reported to the FBI reached $3.05 billion in 2025, and more than $30 million of it came from cases where victims reported AI.

3. The login page that also steals the code

An email says a shared document is waiting, or your mailbox is nearly full. The link opens a perfect copy of your Microsoft 365 sign-in page. Your employee types the password, then the code from the authenticator app, and the page passes both to the real site in real time. The attacker is now signed in as them. AI writes the email and builds the page; the trick of relaying the code is older, and it is why a text message or app code alone no longer protects an account.

4. The voice on the phone sounds exactly like your CFO

The careful employee calls to confirm, and the call goes to a number the attacker supplied, where a cloned voice answers. In the best-known case, an Arup employee in Hong Kong sent about $25 million in 2024 after a video call in which the chief financial officer and other colleagues were all deepfakes. The FBI's report says voice cloning is now used to request wire payments too.

The defences that still work

None of these attacks needs a new kind of defence. They need the old ones applied without exceptions, because the old clues (bad spelling, odd greetings, a strange tone) are gone.

Verify every payment change outside the message. A new bank account, an urgent transfer or gift cards is confirmed by calling a number you already had, from your supplier file or the company directory, never one in the email or the call. Two people approve any change of bank details. This one rule defeats scenarios 1, 2 and 4, however good the writing or the voice.

Use phishing-resistant MFA. Text message and app codes can be relayed by a fake login page; security keys and passkeys built on FIDO2 and WebAuthn cannot, because they only answer the real website. CISA calls phishing-resistant MFA the gold standard and urges every organisation to adopt it. Start with administrators, finance and executives. Attackers go after those accounts first, as we described in our look at identity-based attacks.

Train people on the request, not the spelling. Old awareness training taught staff to look for typos. That advice is now harmful, because it teaches them that a well-written email is safe. Teach them to stop at the pattern instead: urgency, secrecy, a change to how money moves, or a login reached from a link. Then test it. Our cybersecurity training runs phishing exercises with messages as polished as the real ones, and reports the totals, never the names.

Make reporting easy and thank people for it. A suspicious email reported in the first minutes can be pulled from every inbox before the next person clicks. A team that is afraid to admit a mistake reports nothing.

Check your email domain settings. SPF, DKIM and DMARC set to reject make it much harder for an attacker to send email that appears to come from your own domain. They do not stop a look-alike domain, which is why the payment check still matters.

If your company builds with AI too

WormGPT is the criminal side of the same technology your teams may be adopting. If you are deploying AI assistants or agents internally, the same question of who can make them misbehave applies to yours. Our AI security service reviews and hardens the AI systems you build or buy.

These tools will change names again; the ones in this article will likely be gone within a year. We review this article when they do. The defences above will not need to change.

Frequently asked questions

Is WormGPT still available?

The original WormGPT shut down in 2023, soon after researchers reported it. The name lives on. Cato Networks found two tools sold as WormGPT in 2024 and 2025 that were mainstream AI models with their safety rules stripped by a hidden instruction, and Unit 42 reported a paid WormGPT 4 on sale from September 2025. Expect the names to keep changing while the messages they write stay the same.

Can ChatGPT be used the same way?

Mainstream assistants refuse to write a fraud email when they recognise one, and their providers ban accounts that abuse them. Criminals get around both by renting an uncensored model, which is what WormGPT and FraudGPT sell. For your defences it makes no difference which tool wrote the message, because the checks in this article do not depend on spotting AI.

Can we detect an email written by AI?

Not reliably, and it is the wrong question. A well-written email from a fraudster and a well-written email from a colleague look the same. Check the request instead of the writing. A change of bank details, an urgent payment or a login page reached from a link is checked the same way, whoever or whatever wrote the message.

Does a small company need to worry about this?

Yes, and arguably more than a large one. These tools make a convincing message cheap, so attackers no longer save their effort for big targets. A small finance team that pays invoices from email, with no second check, is exactly what a fake supplier email is written for.

Sources

  1. WormGPT: a generative AI tool to compromise business emails, CSO Online, 2023
  2. FraudGPT: The Villain Avatar of ChatGPT, Netenrich, 2023
  3. WormGPT variants powered by Grok and Mixtral have emerged, Cato Networks, 2025
  4. The Dual-Use Dilemma of AI: Malicious LLMs, Unit 42, Palo Alto Networks, 2025
  5. 2025 Internet Crime Report, FBI Internet Crime Complaint Center, 2026
  6. British engineering giant Arup revealed as $25 million deepfake scam victim, CNN, 2024
  7. Implementing Phishing-Resistant MFA, CISA, 2022

AI Security

Agentic AI security: your newest employee has admin rights and no manager

AI agents do more than answer. They call tools, read data and act in other systems, often with a person's keys. Five public incidents from 2025 show what goes wrong, and each points to a control, from an identity of the agent's own to a named owner who can stop it.

7 min read

Tell us who needs training

The teams, the number of people and the language in a few lines. We reply within two business days.