Human brain power is no match for hackers emboldened with artificial intelligence-powered digital smash-and-grab attacks using email deceptions. Consequently, cybersecurity defenses must be guided by AI solutions that know hackers’ strategies better than they do.
This approach of fighting AI with better AI surfaced as an ideal strategy in research conducted in March by cyber firm Darktrace to sniff out insights into human behavior around email. The survey confirmed the need for new cyber tools to counter AI-driven hacker threats targeting businesses.
The study sought a better understanding of how employees globally react to potential security threats. It also charted their growing knowledge of the need for better email security.
Darktrace’s global survey of 6,711 employees across the U.S., U.K., France, Germany, Australia, and the Netherlands found that respondents experienced a 135% increase in “novel social engineering attacks” across thousands of active Darktrace email customers from January to February 2023. The results corresponded with the widespread adoption of ChatGPT.
These novel social engineering attacks use sophisticated linguistic techniques, including increased text volume, punctuation, and sentence length with no links or attachments. The trend suggests that generative AI, such as ChatGPT, is providing an avenue for threat actors to craft sophisticated and targeted attacks at speed and scale, according to researchers.
One of the three most significant takeaways from the research is that most employees are concerned about the threat of AI-generated emails, according to Max Heinemeyer, chief product officer for Darktrace.
“This is not surprising, since these emails are often indistinguishable from legitimate communications and some of the signs that employees typically look for to spot a ‘fake’ include signals like poor spelling and grammar, which chatbots are proving highly efficient at circumventing,” he told TechNewsWorld.
Darktrace questioned businesses in the retail, catering, and leisure sectors about their level of concern, if any, about hackers using generative AI to send scam emails that are difficult to differentiate from real correspondence. Eighty-two percent of respondents expressed concern.
More than half of those surveyed said they are aware of the factors that lead employees to mistake an email for a phishing scam. The top three are requests to open attachments or click links (68%), strange senders or unexpected material (61%), and improper grammar and writing (61%).
That is significant and troubling, as 45% of Americans surveyed noted that they had fallen prey to a fraudulent email, according to Heinemeyer.
“It is unsurprising that employees are concerned about their ability to verify the legitimacy of email communications in a world where AI chatbots are increasingly able to mimic real-world conversations and generate emails that lack all of the common signs of a phishing attack, such as malicious links or attachments,” he said.
Other key results of the survey include the following:
- 70% of global employees have noticed an increase in the frequency of scam emails and texts in the last six months
- 87% of global employees are concerned about the amount of personal information available about them online that could be used in phishing and other email scams
- 35% of respondents have tried ChatGPT or other gen AI chatbots
Human Error Guardrails
Widespread accessibility to generative AI tools like ChatGPT and the increasing sophistication of nation-state actors means that email scams are more convincing than ever, noted Heinemeyer.
Innocent human error and insider threats remain an issue. Misdirecting an email is a risk for every employee and every organization. Nearly two in five people have sent an important email to the wrong recipient with a similar-looking alias by mistake or due to autocomplete. This error rises to over half (51%) in the financial services industry and 41% in the legal sector.
Regardless of fault, such human errors add another layer of security risk that is not malicious. A self-learning system can spot this error before the sensitive information is incorrectly shared.
In response, Darktrace unveiled a significant update to its globally deployed email solution. It helps to bolster email security tools as organizations continue to rely on email as their primary collaboration and communication tool.
“Email security tools that rely on knowledge of past threats are failing to future-proof organizations and their people against evolving email threats,” he said.
Darktrace’s latest email capability includes behavioral detections for misdirected emails that prevent intellectual property or confidential information from being sent to the wrong recipient, according to Heinemeyer.
AI Cybersecurity Initiative
AI defences can assess what belongs in a given person’s mailbox by recognising what is usual. With 79% of respondents indicating that their company’s spam/security filters mistakenly prevent critical valid emails from reaching their inbox, email security systems make this mistake far too frequently.
AI can evaluate for each email whether it is legitimate and should be ignored or suspicious and should be dealt with, given a thorough understanding of the organisation and how its members interact with their inboxes.
Tools that are based on an understanding of prior attacks won’t be able to counter AI-generated attacks, according to Heinemeyer.
Attack analysis shows a notable linguistic deviation — semantically and syntactically — compared to other phishing emails. That leaves little doubt that traditional email security tools, which work from a knowledge of historical threats, will fall short of picking up the subtle indicators of these attacks, he explained.
Bolstering this, Darktrace’s research revealed that email security solutions, including native, cloud, and static AI tools, take an average of 13 days following the launch of an attack on a victim until the breach is detected.
“That leaves defenders vulnerable for almost two weeks if they rely solely on these tools. AI defenses that understand the business will be crucial for spotting these attacks,” he said.
AI-Human Partnerships Needed
Heinemeyer believes the future of email security lies in a partnership between AI and humans. In this arrangement, the algorithms are responsible for determining whether the communication is malicious or benign, thereby taking the burden of responsibility away from the human.
“Training on good email security practices is important, but it will not be enough to stop AI-generate threats that look exactly like benign communications,” he warned.
One of the vital revolutions AI enables in the email space is a deep understanding of “you.” Instead of trying to predict attacks, an understanding of your employees’ behaviors must be determined based on their email inbox, their relationships, tone, sentiments, and hundreds of other data points, he reasoned.
“By leveraging AI to combat email security threats, we not only reduce risk but revitalize organizational trust and contribute to business outcomes. In this scenario, humans are freed up to work on a higher level, more strategic practices,” he said.
Not a Completely Unsolvable Cybersecurity Problem
For a decade, the defensive side has studied the threat posed by aggressive AI. Attackers will unavoidably employ AI to advance their tactics and increase ROI, according to Heinemeyer.
“However, from a defence standpoint, this is not anything we would consider intractable. Ironically, generative AI might make social engineering more difficult, but AI that understands you might be the antidote, he suggested.
In order to regularly assess the effectiveness of its defences in front of this inevitable evolution in the attacker landscape, Darktrace has tested offensive AI prototypes against the company’s technology. The organisation is sure that AI equipped with a thorough grasp of the industry will be the most effective strategy to counter these challenges as they develop.