Artificial intelligence is transforming businesses, but it’s also giving cybercriminals powerful new ways to exploit trust. In 2024, an employee at global engineering firm Arup joined what appeared to be a routine video call with senior executives. The faces looked familiar, the voices sounded authentic, and the financial request seemed legitimate.
Only later did the employee discover that every executive on the call was an AI-generated deepfake. The company lost $25 million. The incident is a stark reminder of how convincing deepfake scams have become.
Unlike traditional phishing attacks that rely on suspicious emails or malicious links, these attacks manipulate something far more difficult to detect: human trust. Powered by deepfake AI voice technology, cybercriminals can convincingly impersonate executives, colleagues, or even family members. This is how deepfake vishing is one of the fastest-growing and most dangerous forms of social engineering today.
AI lets cybercriminals create deepfakes: realistic fake audio, video, or images of a real person. When an attacker uses an AI-generated voice in combination with social engineering, the attack is a form of voice phishing, commonly known as deepfake vishing.
The approach for most attacks is predictable:
1. A sample of the target’s audio is collected.
2. The offender develops a convincing audio duplicate.
3. The offender uses social engineering techniques to contact the targeted victim.
4. The offender uses urgency or a sense of authority to convince the victim to provide confidential information or authorize a financial transaction.
Unlike “traditional” phishing attacks, these types of attacks exploit human psychology rather than using fake emails.
Discover how Threatcop protects your workforce from modern cyber threats.
In January 2024, an employee at engineering firm Arup received a request to authorize a large fund transfer during what appeared to be a routine video meeting with senior executives. The meeting featured AI-generated versions of the company’s chief financial officer and colleagues.
The employee believed the meeting was genuine and approved the transaction. The incident proved that cybercriminals no longer need to breach systems to commit fraud. They simply need to deceive the people using them.
It also exposed a critical security gap: while technical controls can detect malicious files and suspicious emails, they cannot prevent employees from trusting convincing AI-generated voices and faces. A fast way to report suspicious requests before funds move can close that gap.
Understanding voice phishing examples will help organizations recognize the warning signs of voice phishing before it’s too late for assistance.
In all three examples, trust, urgency, and familiarity are used as means of social engineering rather than technical vulnerabilities.
Attacks using deepfakes can trick workers because they manipulate people rather than systems. Most people will have a gut-level tendency to trust a person on the phone with a familiar-sounding voice, even when the person appears authentic, and the situation seems urgent.
While traditional cybersecurity tools like email gateways, firewalls, and endpoint protection can block malicious emails and files, they cannot detect whether an employee will trust an AI-generated voice.
As AI technology continues to advance, it will become harder for workers to distinguish between authentic and artificial voices. Therefore, employee education will be one of the most significant components in protecting an organization from the increasingly prevalent threat of deepfake attacks.
Businesses need to take a layered approach to mitigate the risk of deepfake fraud. Some examples are:
While technology is part of the solution to protect against fraudulent transactions, resilient cybersecurity also relies on having trained employees who know when to stop and confirm requests.
Threatcop’s Human Risk Management platform assists organizations in strengthening their human defenses. The platform enables companies to identify areas of employee vulnerability, provide adaptive security awareness training, and conduct realistic phishing simulations to give teams practice responding to the newest forms of AI-based social engineering scams.
Deepfake scams are evolving rapidly, making continuous employee awareness essential. Organizations must move beyond one-time training and adopt an ongoing, adaptive training program across the workforce.
Security teams can support this shift by measuring human risk, strengthening incident reporting, and building a culture of cybersecurity awareness. Behavioral analytics and simulation-based training reduce the risk that employees will fall victim to advanced deepfake AI voice attacks.
Deepfake scams have ushered in a new era of cybercrime, one in which attackers no longer need to target a company; instead, they can exploit the human tendency to trust others through manipulation.
As AI voice technology continues to improve and become more realistic, organizations need to go beyond conventional security measures and invest in employee education.
Strong verification processes, ongoing security training, and human risk management platforms will keep organizations ahead of the latest threats.
To be successful in the area of deepfake vishing, technology alone will not suffice – individuals who are educated and understand their role within the organization will protect assets better than any single technology can.
Cybercriminals use deepfake AI voice technology to generate fraudulent audio from internet sources, including social media, videos, news, and interviews. They then use an AI process to create an exact clone of the victim's voice for outgoing phone calls, creating a sense of urgency to get a speedy response from the victim.
Organizations may consider using multi-factor verification for high-value transactions, creating callback protocols for processing transaction requests, training staff to identify and combat social engineering tactics, and conducting periodic phishing simulations. By adopting a Human Risk Management approach, such as Threatcop's, organizations will increase employee awareness and become more resilient to evolving AI-enhanced attacks.
Organizations assume their email domains are highly protected until a spoofed message proves otherwise. Over the last few years,...
KnowBe4 is a major player in security awareness training, but it’s not the only platform to consider. Organizations are...
Email scams are becoming more of a serious problem. These attacks are more specific and difficult to detect, and...