13 minutes reading
AI social engineering attacks the trust between an employee and the AI tools they work with, rather than the employee alone. The shift matters for measurement. Click rate scores whether a person clicked a link, and the leading attacks in this class need no link, no click, and sometimes no action from the employee at all.
That leaves most awareness programs reporting a number that improves while the exposure it was built to track moves somewhere else.
What Does AI Social Engineering Actually Target?
AI social engineering targets the trust relationship between a person and their tools rather than the person’s judgment about a message. Classic social engineering works by manipulating human psychology directly, and awareness programs were built to interrupt that moment of manipulation. An employee who has learned to distrust an unexpected email has not learned to distrust the assistant summarizing that email, and the assistant has no suspicion of its own.
Alongside the established types of social engineering attacks, three AI-specific forms are in circulation, and they behave differently:
- Instruction smuggling. Hidden text inside a document, web page, calendar invite, or email that the AI reads as a command rather than as content. The employee sees an ordinary file.
- Poisoned input. An attacker gets a helpful employee to supply a tainted dataset, reference document, or knowledge base entry, which then shapes every output the tool produces from it.
- Confidence exploitation. The attacker relies on the AI’s authoritative tone to move a person past a control they would otherwise question, because output arrives formatted, fluent, and without visible uncertainty.
None of the three fit the shape awareness programs were built around. There is no sender to inspect, no domain to hover over, and often no moment where the employee is asked to decide anything. The decision was already made when the organization connected a tool to a mailbox.
The direction of travel is visible in breach data. Verizon’s 2026 Data Breach Investigations Report, covering more than 22,000 confirmed breaches, puts the human element in 62% of them, and found that 41% of social engineering breaches now involve vectors other than email. The human layer is still where breaches run, and the channel it runs through keeps moving away from the inbox that awareness programs were designed to defend.
Why Email Susceptibility Still Needs Measuring
Email susceptibility still needs measuring, because phishing remains the leading initial access vector in most breach datasets. A program that dropped click rate entirely would lose sight of a real and large exposure, and would deserve the criticism it got. The case for what phishing simulations contribute to enterprise security has not weakened.
The claim is narrower. Click rate should not stand alone as the number that represents human risk, because it covers one delivery mechanism and this attack class routinely uses others. Keep it, report it, and stop treating an improvement in it as evidence that the whole human layer got safer.
Why Zero-Click Attacks Break the Click Rate Metric
Click rate stopped covering this class the moment a working attack needed no click. Conventional email security practices assume a user decision somewhere in the chain, whether that is opening, clicking, or replying. The clearest case where that assumption fails is EchoLeak, catalogued as CVE-2025-32711 with a CVSS score of 9.3 and documented in a 2025 research paper as the first real-world zero-click prompt injection exploit in a production LLM system. A single crafted email reached a Microsoft 365 Copilot user. The user never opened it. Copilot processed the mailbox during routine summarization and organizational data left the tenant.
The pattern repeated. Reprompt, tracked as CVE-2026-24307, achieved single-click exfiltration from Copilot Personal through a crafted URL parameter. GitHub Copilot carried a remote code execution flaw at CVSS 9.6 under CVE-2025-53773. Published analyses put prompt injection success rates between 50% and 84% depending on system configuration, and NIST has called indirect prompt injection the greatest security flaw in generative AI. OWASP has kept prompt injection at the top of its LLM risk list through the 2026 edition.
Set those against what click rate measures. A simulated phishing email lands, a person either clicks or does not, and the resulting percentage is reported as human risk. That metric was a reasonable proxy when the delivery mechanism was a link and the decision point was a human one. For an attack that runs inside a tool the business deployed and trusts, it is measuring an unrelated behavior and reporting the result as coverage.
People Security Management
Book a Free
Demo Call
with Our Expert
Discover how Threatcop protects your workforce from modern cyber threats.
Your Details
How Do Employees Become the Delivery Route?
Employees become the delivery route by moving content, not by clicking anything. That breaks the assumption behind most countermeasures against social engineering, which place the control at a decision point the employee is aware of. Microsoft’s security research on prompt abuse in AI tools, published in March 2026, describes hidden instruction attacks as instructions buried inside documents, web pages, emails, or chats that an AI interprets as genuine input, producing information leaks or altered summaries without the user typing anything malicious.
The employee’s part in that chain is ordinary work:
- Uploading a supplier PDF into an assistant to extract the payment terms
- Pasting a competitor’s web page into a tool for a summary
- Forwarding a thread to an agent that has access to the file store
- Connecting an assistant to a shared drive so it can answer questions faster
- Adding a document to a knowledge base that other people’s agents will retrieve
Each action is reasonable in isolation and none of them looks like a security decision. What makes them consequential is the privilege on the other side. An agent that uses access it legitimately holds to serve an attacker is a confused deputy, which is behaviorally close to the profile security teams already study in insider threat detection, with the difference that this insider has API-speed access and no motive to investigate.
The training implication is narrow and teachable. Employees cannot patch prompt injection, since instructions and data share one channel inside a model’s context window. What they can learn is which content is untrusted, which tools hold enough access to make untrusted content dangerous, and that the combination of the two is the thing to avoid.
Why 66% of Employees Do Not Check What AI Tells Them
Employees do not check AI output because verification was never built into the workflow, and confidence exploitation depends on exactly that gap. A global study by KPMG and the University of Melbourne found that 66% of employees trust the output of large language models without checking it, and that more than half reported work-related mistakes caused by over-reliance. The EY AI Sentiment Index puts the share of users who verify AI-generated content at fewer than 1 in 3.
Workplace surveys through 2026 land in the same place from different angles. Resume Now’s December 2025 survey of 1,012 employed US adults found 35% rarely or only occasionally review AI output before using it, and 15% use AI tools for work without telling their manager. GoTo’s Pulse of Work 2026, covering 2,500 global employees and IT leaders, found 50% say they rely on AI too much and 28% say they have started trusting AI more than their own judgment.
An attacker does not need to defeat a verification step that most people are skipping. This is the same dynamic that makes human error a leading driver of security risk, with one difference: the error is trusting a tool the organization chose and deployed. That is the whole mechanism of confidence exploitation, and it explains why an authoritative-sounding instruction inside an AI response can move someone past a control that a plainly worded email never would.
It also sets the ceiling on what training can achieve here. Telling people to verify everything produces verification of nothing, because the time cost is unbounded. Naming four or five decision types that require an independent check, and the source that check runs against, produces four or five checks.
What a Culture That Questions AI Looks Like in Practice
A culture that questions AI is built on whether questioning is safe generally, not on whether the tool is AI. Organizations have spent years teaching people to comply with instructions that appear to carry authority. If challenging a director’s request is career-limiting, challenging an assistant that speaks with the same certainty will not happen either, whatever the policy says.
Four things make the difference, and none of them are a module:
- A named right to pause. A written rule that any employee may hold any request pending verification, with no requirement to justify the suspicion first.
- Escalation that costs nothing. A reporting route where a false alarm produces a thank you rather than a follow-up conversation. People calibrate to the second occurrence, not the policy.
- Leaders challenged in public. Someone senior visibly overriding or correcting an AI output in a meeting does more than a campaign, because it demonstrates the behavior is survivable.
- Verification built into the workflow. A required second source on defined decisions, sitting inside the process rather than depending on individual conscientiousness.
The underlying work is the same as any other security culture program, with one addition. Employees need explicit permission to be wrong about an AI, because the social cost of doubting a machine in front of colleagues is higher than most security teams assume.
What Should You Measure Instead of Click Rate?
Measure the behaviors this attack class actually requires, and demote click rate from headline number to one line among several. Click rate can stay as one line among several, but it should stop standing in for human risk on its own.
| Behavior to measure | Metric | How to baseline it | Cadence |
| Verification of AI output | Share of defined high-stakes outputs independently checked before action | Sample audit of decisions, not self-reporting | Monthly |
| Reporting anomalous AI behavior | Count of employee reports plus median time from observation to report | Reporting channel logs | Monthly |
| Untrusted content handling | Instances of external documents fed to tools holding sensitive access | Data loss prevention or platform logs | Monthly |
| Sanctioned tool usage | Share of AI work happening in approved tools, with voluntary disclosure of the rest | Anonymous survey plus discovery data | Quarterly |
| Challenge rate | Escalations raised that turn out to be benign | Same channel as incident reporting | Quarterly |
| Multi-vector susceptibility | Engagement rate across email, voice, SMS, and document-based scenarios | Simulation, scored per employee | Quarterly |
Two of these do the heavy lifting. Challenge rate is the only metric on the list that goes up when the program is working, which makes it uncomfortable to report and useful to track, because a workforce that never escalates anything is not a workforce with nothing to escalate. And reporting speed is the only one that changes an outcome mid-incident, which is why incident reporting culture deserves separate measurement from susceptibility. The broader case for behavior-based rather than completion-based security training metrics applies without modification.
Introducing these without losing board continuity takes about two quarters. Run the new metrics alongside click rate rather than in place of it, so the committee that has watched one number fall for three years can see the new lines move before the old one is demoted. Baseline each behavior before any training lands against it, since a metric introduced after an intervention cannot attribute the change it reports.
Three of the six are cheap to start. Verification rate needs a sample audit of decisions already being made, not new tooling. Anomalous AI reports need a reporting channel that accepts something other than a forwarded email. Challenge rate is a count of escalations you are probably already receiving and not recording as a positive.
The other three depend on visibility you may not have yet. Untrusted content handling needs data loss prevention or platform logs. Sanctioned tool usage needs discovery data plus an amnesty for voluntary disclosure. Multi-vector susceptibility needs a simulation platform that can run scenarios outside email. Start with the cheap three, report them for a quarter, and use the gaps they expose to argue for the rest.
A board-ready view fits on one slide: susceptibility by vector, verification rate on the decisions that matter, median time from observation to report, and the direction each moved against last quarter. That is four numbers a non-technical director can interrogate, which is more than a single percentage has ever allowed.
How to Run a Simulation for an Attack With No Link
Build the simulation around the decision rather than the click, because an attack with no link gives the employee nothing to click on. The scoring question changes from whether someone clicked to whether someone verified, escalated, or proceeded.
Three scenario types cover most of the exposure:
- The seeded document. Circulate a file that carries a visible, benign-looking instruction addressed to an assistant, for example, a line asking the tool to forward a summary to an external address. Score whether the recipient notices it before feeding the file to a tool, and whether they report it.
- The confident wrong answer. Present an AI-styled output containing an instruction that conflicts with policy, such as a vendor bank detail change presented as already validated. Score whether the person verifies against an independent source before acting.
- The helpful data request. Ask a team to supply a reference dataset for an internal AI project, routed from a plausible but unverified internal-looking source. Score whether anyone confirms the requester and the classification of the data.
Run these with the same rules that govern any credible simulation program: baseline before training, score per employee rather than per organization, and keep results non-punitive so that reporting stays honest. The delivery mechanics are no different from the best practices for employee training already in use for phishing, and multi-vector platforms such as Threatcop’s awareness training are built to run scenarios that do not depend on an inbox.
One caution. Do not score these on a pass rate and report the aggregate as an improvement. The value is in the distribution: which roles proceeded without verifying, and which teams escalated.
Where People Security Management Fits Against AI Attacks
People Security Management treats employee behavior as a security domain with baselines, owners, and metrics rather than as a training obligation, which is the frame this attack class requires. Threatcop built the approach with input from 33 security leaders across Indian industry and published it as a guide of CISO interviews, so it reads as a program structure rather than a feature list.
The cycle runs in three stages relevant here. Assess establishes a behavioral baseline through multi-vector simulation, which is where TSAT operates and where scenarios that never touch email get scored. Aware builds recognition of the specific behaviors the baseline exposed, delivered through TLMS with role-based and multi-language content. Empower turns the workforce into a detection layer through TPIR.
The reporting piece matters most against AI social engineering. An employee who notices an assistant behaving oddly has nothing to report under a phishing button, because there is no phishing email. TPIR’s WhatsApp and SMS reporting channels give people a route to raise something suspicious that never arrived in an inbox, which is increasingly how these attacks present. AI-driven simulation templates keep the scenarios aligned with lures currently in use, including synthetic voice and agent impersonation.
What to Change Before Your Next Awareness Report
The awareness metric most organizations report improved every year while the attack surface it tracks became a smaller share of the problem. That is not a failure of the metric’s design. It is a failure to notice that the delivery mechanism moved from a link a person clicks to a document a tool reads.
Three changes fit inside a single quarter. Add verification rate and anomalous AI reports to the report alongside click rate. Run one seeded-document scenario against the roles that handle external files, and score it on escalation. Then check whether your reporting channel accepts a report that is not a phishing email, because if it does not, your workforce currently has no way to tell you what they saw. Start with a multi-vector baseline and build the program from what it shows.
Frequently Asked Questions
Can employees prevent prompt injection?
Not directly. Instructions and data share one channel inside a model's context window, so no user behavior removes the risk. What employees can control is which untrusted content they feed to tools that hold sensitive access, and whether they report an assistant taking actions the task did not call for.
How many employees verify AI output before using it?
A global study by KPMG and the University of Melbourne found 66% of employees trust large language model output without checking it, with more than half reporting work mistakes caused by over-reliance. Resume Now's survey of employed US adults found 35% rarely or only occasionally review AI output before use.
What should replace click rate as the main awareness metric?
Use a small set of behaviors instead of a single number: verification rate on defined high-stakes decisions, reports of anomalous AI behavior with median time to report, challenge rate, and susceptibility across email, voice, SMS, and document-based scenarios. Click rate stays as one line rather than the headline.
Pavan Kushwaha is the Founder & CEO of Kratikal and Threatcop and a Certified Information Systems Auditor (CISA). His cybersecurity journey began in 2013 after a firsthand encounter with a sophisticated phishing attack, inspiring him to build Kratikal with his NIT Allahabad peers. Today, he leads globally recognized solutions that reduce human risk using behavioral science, automated risk detection, and agentic AI security. He specializes in penetration testing and building secure security architectures for modern enterprises. An information security researcher and author of multiple cybersecurity books, he has trained 15,000+ professionals across 130+ countries. He has filed patents (pending) for innovations in real-time email trust and people security, turning security awareness into measurable, repeatable risk reduction.
