Large-scale experiment finds little benefit from standard anti-phishing programs
Cybersecurity training programs, as commonly implemented in large organizations, do little to prevent employees from falling for phishing scams, according to a comprehensive study by researchers at UC San Diego.
The findings, based on an eight-month randomized controlled trial involving more than 19,500 UC San Diego Health employees, challenge the effectiveness of both annual cybersecurity awareness training and embedded anti-phishing exercises.
“Organizations should not expect training, as commonly deployed today, to substantially protect against phishing attacks—the magnitude of protection afforded is simply too small and employees remain susceptible even after repeated training,” the study concluded.
No clear link between training and phishing resilience
The research team sent 10 different simulated phishing campaigns to employees and found “no significant relationship between whether users had recently completed an annual, mandated cybersecurity training and the likelihood of falling for phishing emails.”
The study also evaluated embedded phishing training, where employees who click on a simulated phishing link receive immediate anti-phishing information. Here too, the difference in failure rates between trained and untrained users was “extremely low.”
“Taken together, our results suggest that anti-phishing training programs, in their current and commonly deployed forms, are unlikely to offer significant practical value in reducing phishing risks,” the researchers concluded.
Employees are more likely to become victims of HR-related phishing emails, an earlier survey found.
Phishing risk grows over time
One reason for the limited effectiveness is lack of engagement.
“The majority of people do not engage with the embedded training materials,” said co-author Grant Ho, now a faculty member at the University of Chicago. The study found that 75% of users spent a minute or less on the training, and one-third closed the page immediately.
The study also revealed that susceptibility to phishing increases as time goes on. In the first month, 10% of employees clicked a phishing link, but by the eighth month, more than half had done so at least once.
The effectiveness of phishing emails varied widely: only 1.82% clicked a link to update their Outlook password, while 30.8% clicked on a supposed update to the organization’s vacation policy.
Interactive training offers modest gains
Mandatory annual cybersecurity training, a staple in many organizations, showed no correlation with reduced phishing failures.
“Employees who recently completed such training, which has a significant focus on social engineering and phishing defenses, have similar phishing failure rates compared to other employees who completed their awareness training many months ago,” the study reported.
The researchers tested several types of training, including static and interactive formats, as well as content tailored to the specific phishing lure. Only interactive training, when completed, was associated with a lower likelihood of falling for future phishing attempts—a 19% reduction in failure rates.
However, very few users completed these interactive sessions.
“For interactive training, users who fully complete a training session have a lower likelihood of failing future phishing exercises than those who do not. This same relationship does not exist for users who receive static training,” the study found.
Incentivized training to combat phishing
Any attempt to improve training should find a way to achieve the uptake of mandatory training, without losing the “teachable moment” experience of embedded training, say the researchers.
“For example, future work could consider whether adding stronger incentives, either rewards or punitive measures, could improve users’ engagement with the training material.”
However, they warn that certain incentives (such as requiring users to complete training) do not always lead to higher intellectual engagement, “but may instead incentivize subtle compliance strategies where users simply complete the requirement without actively engaging with the material.”
Recommendation: focus on technical defenses
Given the results, the researchers recommend organizations shift their focus to technical countermeasures: two-factor authentication for hardware and applications, as well as password managers that only work on correct domains.
“For an organization with finite resources, it seems likely that focusing on technical countermeasures may offer a better return on investment. In particular, hardware multi-factor authentication (MFA) and/or relying on password managers to fill in credentials only on the correct domain offers strong protections against certain classes of phishing attacks (albeit with a number of operational and usability challenges that deserve further attention.”
With the release of Canada’s updated National Cyber Security Strategy in January, experts have an urgent message for employers: start planning to hire cybersecurity professionals now.
