AI Phishing Tells: What Actually Gives It Away Now

AI-written phishing has eliminated the tell security training spent two decades emphasizing: bad grammar. What replaced it is a different set of tells, leftover instructions the model never meant to send, code comments that document their own malicious purpose, and a growing reliance on AI website builders that hand attackers a legitimate platform’s reputation for free. None of these show up in training that still says “watch for typos.”

The Old Heuristic Is Actually Gone

The claim that AI-written phishing beats the old heuristic is not a training program being cautious. It is a measured result. Researchers at Harvard and MIT, including security researcher Bruce Schneier, tested phishing emails generated automatically by GPT-4 against a control group of generic phishing emails sent to 112 real participants. The study, published in IEEE Access in 2024, found the AI-generated emails achieved a 30 to 44% click-through rate, well above the 19 to 28% the generic control group produced. Grammar and polish were never the point of the old heuristic anyway. They were a proxy for effort, and AI removed the correlation between effort and detectability.

Detection tooling built for the old signal has its own documented blind spot. Independent research by email security firm Egress found that most automated AI-content detectors need a minimum sample of roughly 250 characters to work reliably, and a large share of real phishing emails fall short of that threshold entirely. A detector built to flag “AI-sounding” text is of limited use against a short, targeted lure that never gives it enough text to analyze.

The Tells That Replaced Bad Grammar

Security researchers across multiple vendors have converged on the same finding: AI-generated attacks leave their own signatures, just not the ones training programs have spent years looking for. The anatomy of a phishing email has not changed at the level of intent, only in the specific artifacts it leaves behind.

• Leftover generation artifacts. A model asked to write a phishing email sometimes prefaces its output with a structural summary of what it is about to produce, and an attacker who copies the response without trimming that preamble sends it along with the lure. To a recipient it reads as odd filler. To anyone trained to look for it, it is a direct signature of AI authorship.
• Self-documenting malicious code. Phishing kits increasingly ship with JavaScript or page logic that includes unusually thorough comments explaining what each block does, including its own evasion logic. This is a byproduct of how language models are trained to be helpful and explain their work, applied without editing to code that was never meant to explain itself to a defender.
• Unicode homoglyph substitution at scale. Swapping standard characters for visually identical Unicode equivalents to defeat text-pattern matching predates AI, but generating and testing homoglyph variants across an entire campaign used to take real effort. A model automates the substitution as easily as it writes the original sentence.
• Hidden filler content built to fool scanners, not people. Blocks of plausible-sounding text, hidden from the recipient with CSS properties like display:none or color:transparent, padding out an otherwise sparse, suspicious-looking email so automated heuristic scanners see what looks like a normal, text-rich message.
• Over-engineered formatting with no functional purpose. Language models generate code and markup based on patterns in their training data rather than contextual judgment, which produces small tells: styling properties with no relevance to the medium, or formatting choices no human author would make for a one-off email.
• Emoji and visual-flourish overuse. Language models are measurably more likely than human authors to reach for emoji and decorative visual elements in generated text and code comments, a pattern documented across AI-content research well beyond phishing specifically. A credential-harvesting page or an internal script comment dressed up with emoji is a small tell, but a consistent one.

How This Maps to a Standard Threat Framework

None of these tells require inventing new vocabulary. Each one lines up with an existing, named technique in the MITRE ATT&CK framework, the industry-standard catalog of adversary behavior, which matters because it means detection engineering teams can build rules against a known category rather than a vague sense that “something about this looks AI-generated.”

• T1566.001 and T1566.002 (Phishing: Spearphishing Attachment and Link) cover the delivery mechanism itself, unchanged in category even though AI has changed the content’s quality.
• T1027 (Obfuscated Files or Information) covers Unicode homoglyph substitution and hidden CSS filler content, both pre-existing evasion categories that AI has made faster and cheaper to generate at scale rather than invented outright.
• T1583.001 (Acquire Infrastructure: Domains) covers the practice of standing up hosting through a legitimate platform, which is exactly what makes a no-code AI website builder attractive: the infrastructure acquisition step becomes a text prompt instead of a domain registration.
• T1598 (Phishing for Information) covers the credential-harvesting page itself, regardless of whether it was hand-coded or generated from a prompt.

Framing AI-generated phishing against an existing standard, rather than as a wholly new threat category, keeps detection engineering grounded in rules and signatures that already exist and only need retuning for a faster, cheaper method of production.

Why Content Provenance Standards Won’t Solve This Yet

A natural question is whether a technical provenance standard could simply label AI-generated content the way a nutrition label labels food. The Coalition for Content Provenance and Authenticity, backed by Adobe, Microsoft, Google, and a large industry coalition, has built exactly this for images, video, and audio: a cryptographically signed record of what created a file and what has touched it since.

It does not yet reach plain-text email. The standard was built for media files, not for the body of a message, and even where it applies, most distribution platforms strip embedded credentials during normal processing before a viewer ever sees them. A provenance label on the phishing page’s hero image would not label the AI-generated phishing email that delivered it, and would not survive most email clients’ handling even if it did. Until that gap closes, the specific AI-generated phishing tells this piece documents, leftover generation artifacts, self-documenting code, homoglyph substitution, hidden filler content, are what defenders actually have to work with, not a trust label.

Why Domain Reputation Filtering Stops Working

The most consequential shift is not in the email at all. It is in what the link behind it points to.

Proofpoint’s research team documented large-scale abuse of Lovable, an AI-powered website builder that generates a fully functioning site from a text prompt, for exactly this purpose. Campaigns impersonating Microsoft, UPS, and DeFi platforms were built and hosted entirely on Lovable’s own infrastructure, with researchers noting they could recreate convincing fakes of major enterprise login pages without encountering any guardrails from the platform itself. Similar abuse has been documented on comparable no-code builders, including fake CAPTCHA pages designed specifically to look legitimate to automated scanners.

The mechanism is structural, not a gap in one vendor’s filtering. A URL scanner checks a domain’s reputation, and a page hosted on a legitimate, widely used platform inherits that platform’s good standing by default. The attacker did not compromise the platform. They used it exactly as designed, for a purpose the design never anticipated. Email spoofing and impersonation has always exploited a gap between what a system verifies and what it should verify, and this is the same gap wearing an AI-shaped disguise.

What Actually Replaces “Look for Typos”

The old “look for typos” heuristic has to be retired outright when facing AI-generated phishing, not supplemented. A polished, error-free email is no longer evidence of anything.

• Verify the URL bar, every time, on every login page. A genuine Microsoft page shows microsoft.com. A pixel-perfect replica hosted on a no-code builder’s domain is not the real service, regardless of how convincing the branding looks.
• Treat unexpected precision as a signal, not reassurance. Perfectly formatted, flawlessly written messages are now the baseline for attacks, not evidence of legitimacy the way they once were treated.
• Move detection from reputation to behavior. Since a malicious page can inherit a legitimate platform’s clean reputation, phishing simulation training has to test whether employees check the actual destination of a link, not just whether the surrounding email looks professional. Phishing awareness training programs built before this shift need an explicit update, not just a refresher.
• Confirm out of band for anything requesting credentials or payment. A phone call or a separate, known-good channel catches what visual inspection of an AI-polished page cannot, and a one-click way to report the attempt matters as much as catching it in the moment, since the next recipient benefits from the first person who noticed.
• Understand how the attacker got the target in the first place. Knowing how phishing scammers acquire an email address helps explain why a lure feels so specifically targeted, which is itself often an AI-scaled personalization step rather than evidence of a uniquely sophisticated attacker.

None of this replaces email authentication at the infrastructure level. DMARC configured correctly still stops a large share of direct domain spoofing before a polished AI-written email ever reaches an inbox, which is why the two defenses work as layers, not substitutes for each other.

The Bottom Line

AI did not make phishing harder to catch. It made a specific, widely taught heuristic obsolete while leaving a different set of tells in its place, ones that live in code comments, hidden CSS, and inherited domain reputation rather than in a misspelled word. Training and tooling that have not updated past “look for typos” are testing employees against a threat model that stopped being accurate months ago, which is exactly the gap phishing awareness and simulation programs need to close before the next refresher cycle rather than after it.

Frequently Asked Questions

Arpit Rao

Arpit Rao is a Product Manager at Kratikal, bringing a strong technical foundation and experience in building and managing cybersecurity products. His work spans product strategy, technology, user experience, and solving complex customer challenges. With a focus on translating technical capabilities into practical solutions, Arpit is interested in cybersecurity, AI, product innovation, and user-centric technology. He works on creating products that address evolving security and business needs.

Arpit Rao

Arpit Rao is a Product Manager at Kratikal, bringing a strong technical foundation and experience in building and managing cybersecurity products. His work spans product strategy, technology, user experience, and solving complex customer challenges. With a focus on translating technical capabilities into practical solutions, Arpit is interested in cybersecurity, AI, product innovation, and user-centric technology. He works on creating products that address evolving security and business needs.