AI-powered social engineering attacks are seeing a huge rise - and that's a major problem

Chatbot
(Image credit: Image Credit: Shutterstock)

Hackers have fully adopted generative AI tools into their latest attacks, with unfortunately successful results, new research has claimed.

Data from cybersecurity experts Darktrace demonstrates that phishing surged with the general adoption of Chat-GPT, and that these emails have gotten a lot better and sophisticated, as well.

First, between January and February 2023, the number of phishing email attacks rose by 135%, aligning perfectly with the rising popularity of the chatbot. Darktrace customers alone have received almost three million phishing emails in December 2023 alone, representing a 14% jump compared to September the same year.

Significant impact

Then, the number of polished phishing emails - those with sophisticated language and improved punctuation - increased by 35% between September and December 2023. Darktrace bases this data on data coming in from its customers. 

In the pre-Chat-GPT era, cybersecurity researchers suggested users proofread emails they deem suspicious. Many hackers are not from English-speaking communities, don’t have high English language skills, or simply can’t be bothered to edit their emails. Hence, their emails often carry spelling and grammar errors, something companies they’re trying to impersonate can’t afford.

However, since the emergence of Chat-GPT and similar generative AI tools, hackers got a free, advanced content writer and spellchecker, resulting in more believable emails. 

As a result, almost all (89%) of IT security experts believe AI-augmented cyber threats will have a significant impact on their organization within the next two years, while 60% still remain unprepared for this inevitability. For the 1,700+ IT pros surveyed for the report, their two greatest concerns are increased volume and sophistication of malware attacks delivered via phishing, and employees leaking sensitive data by using generative AI tools. 

To tackle the growing threat of AI-powered attacks, IT teams are advised to integrate AI in their defenses, too.

“AI can help organizations to address novel threats across their entire technology footprint,” commented Darktrace’s CEO, Poppy Gustafsson.

More from TechRadar Pro

TOPICS

Sead is a seasoned freelance journalist based in Sarajevo, Bosnia and Herzegovina. He writes about IT (cloud, IoT, 5G, VPN) and cybersecurity (ransomware, data breaches, laws and regulations). In his career, spanning more than a decade, he’s written for numerous media outlets, including Al Jazeera Balkans. He’s also held several modules on content writing for Represent Communications.

Read more
Ai tech, businessman show virtual graphic Global Internet connect Chatgpt Chat with AI, Artificial Intelligence.
AI agents can be hijacked to write and send phishing attacks
Phishing
Corporate executives are being increasingly targeted by AI phishing scams
Half man, half AI.
Generative AI has a long way to go as siloed data and abuse of its capacity remain a downside – but it does change the game for security teams
A fish hook is lying across a computer keyboard, representing a phishing attack on a computer system
Everything you need to know about phishing
Cartoon Phishing
Hackers use GenAI to attack more frequently and effectively
AI tools.
Not even fairy tales are safe - researchers weaponise bedtime stories to jailbreak AI chatbots and create malware
Latest in Security
A stylized depiction of a padlocked WiFi symbol sitting in the centre of an interlocking vault.
Broadcom warns of worrying security flaws affecting VMware tools
URL phishing
HaveIBeenPwned owner suffers phishing attack that stole his Mailchimp mailing list
Ransomware
Cl0p resurgence drives ransomware attacks to new highs in 2025
Google Chrome
Google Chrome security flaw could have let hackers spy on all your online habits
cybersecurity
Chinese government hackers allegedly spent years undetected in foreign phone networks
Data leak
A major Keenetic router data leak could put a million households at risk
Latest in News
Xbox Series X and Xbox wireless controller set to a green background
Xbox Insiders are currently testing a new Game Hub feature that looks useful, but I've got mixed feelings about it
A stylized depiction of a padlocked WiFi symbol sitting in the centre of an interlocking vault.
Broadcom warns of worrying security flaws affecting VMware tools
Microsoft Surface Laptop and Surface Pro devices on a table.
Hate Windows 11’s search? Microsoft is fixing it with AI, and that almost makes me want to buy a Copilot+ PC
Oura Ring 4
Activity tracking on Oura Ring is about to get a whole lot better, but I've got bad news about your step count
Google Pixel Buds Pro 2
Cleaned your Pixel Buds Pro 2 recently? If not, you might be getting worse sound
Google Maps on a phone being held in someone's hand
Google Maps is getting two key upgrades, for easier route planning and quicker access to Gemini AI