Microsoft challenges you to hack its LLM email service

AI PC laptop
(Image credit: Shutterstock / metamorworks)

  • Microsoft is offering $10k prize for hackers who can exploit vulnerabilities in its LLM
  • The challenge will focus on prompt injection defenses
  • Software developers and hackers often work together to discover and fix flaws

Are you an experienced hacker looking to make a little extra money this Christmas? Well you might be in luck, as Microsoft is sponsoring a competition, alongside the Institute of Science, and Technology Australia, and ETH Zurich, in which contestants will try to break a simulated Large Language Model (LLM) integrated email client.

Winning teams for the LLMail-Inject challenge will be awarded a share of the $10,000 prize pool.

Participants will need to sign into the challenge using a GitHub account, and create a team. The teams will then be asked to evade prompt injection defenses in a simulated LLM-integrated email client. The LLmail service includes an assistant which can answer questions and perform actions on behalf of the user, and crucially includes defenses against indirect prompt injection tasks.

A mutually beneficial relationship

By bypassing the injection defenses, the hackers will be looking to prompt the LLM to do or reveal things it is not trained to. Through this, Microsoft is aiming to identify weaknesses in its current prompt injection defenses, and encourage the development of robust security measures.

The relationship between security researchers and software developers is often used this way, with Google often offering a ‘bug bounty’ for anyone who discovers and is able to exploit vulnerabilities in its Google Cloud Platform.

Similarly, Microsoft recently announced it was hosting its own Black Hat-esque hacking event, in which competitors would look for vulnerabilities in Microsoft AI, Azure, Identity, Dynamics 365, and M365.

Taking a proactive approach to addressing potential vulnerabilities allows software companies to mitigate the risks before they can be exploited by threat actors in real world scenarios. Slack’s AI assistant was on the receiving end of malicious prompt injections, which was luckily discovered by security researchers - but could have led to real security concerns.

Via The Register

You might also like

TOPICS
Ellen Jennings-Trace
Staff Writer

Ellen has been writing for almost four years, with a focus on post-COVID policy whilst studying for BA Politics and International Relations at the University of Cardiff, followed by an MA in Political Communication. Before joining TechRadar Pro as a Junior Writer, she worked for Future Publishing’s MVC content team, working with merchants and retailers to upload content.

Read more
A woman at a table using a Windows laptop, opposite sits a man, neither show their face
Microsoft will now pay you even more to find security bugs in Copilot
ChatGPT on smartphone and desktop.
Microsoft claims its servers were illegally accessed to make unsafe AI content
Microsoft Team Red
Microsoft's own baddie team 'attacked' more than 100 generative AI products: Here's what they learnt
AI tools.
Not even fairy tales are safe - researchers weaponise bedtime stories to jailbreak AI chatbots and create malware
Microsoft
"Another pair of eyes" - Microsoft launches all-new Security Copilot Agents to give security teams the upper hand
Concept art representing cybersecurity principles
Cybercriminals cashing in on holiday sales rush
Latest in Security
cybersecurity
Chinese government hackers allegedly spent years undetected in foreign phone networks
Data leak
A major Keenetic router data leak could put a million households at risk
Code Skull
Interpol operation arrests 300 suspects linked to African cybercrime rings
Insecure network with several red platforms connected through glowing data lines and a black hat hacker symbol
Multiple routers hit by new critical severity remote command injection vulnerability, with no fix in sight
Code Skull
This dangerous new ransomware is hitting Windows, ARM, ESXi systems
An abstract image of a lock against a digital background, denoting cybersecurity.
Critical security flaw in Next.js could spell big trouble for JavaScript users
Latest in News
OpenAI logo
OpenAI just launched a free ChatGPT bible that will help you master the AI chatbot and Sora
Monster Hunter Wilds
Monster Hunter Wilds Title Update 1 launches in early April, adding new monsters and some of the best-looking armor sets I need to add to my collection
Zotac Gaming RTX 5090 Graphics Card
Nvidia Blackwell stock woes are compounded by price hikes as more RTX 5090 GPUs soar in pricing, and I’m sick and tired of it all at this point
A collage of Elizabeth Olsen's Scarlet Witch and Tatiana Maslany's She-Hulk
Marvel fans are already tired of Doomsday and Secret Wars cast gossip as two more superheroes get linked with roles in the next two Avengers movies
Four operators survey Verdansk. One holds a sniper rifle, one binoculars, another holds is landing with their parachute, while the last wears a skull mask
New Call of Duty: Warzone trailer shows a beautiful rebuilt Verdansk, but some fans want more: 'it won't be the same unfortunately'
An Apple Music pink/pixellated poster advertising DJ with Apple Music
DJ with Apple Music lands, allowing subscribers to build and mix DJ sets directly from its +100 million-song catalog