OpenAI sends staggeringly casual email to Australian government wishing it the ‘best’ after AI agents hack its health insurance website

OpenAI
(Image credit: Getty Images)

OpenAI had an oddly casual way of apologizing to the Australian Government to explain how one of its experimental AI agents gained unauthorized access to a government health statistics system while trying to complete a routine research task. More casual than one might expect for such a huge security concern.

The agent was looking for publicly available information about medicine spending, but after failing to get what it wanted through the normal route, it found a vulnerability in and used it to access internal files, though no individual patient records.

The incident is a striking example of the risks that come with increasingly autonomous AI agents, but almost as striking is how OpenAI admitted what happened. OpenAI says it discovered the Australian activity in mid-August but did not notify Australia until September 10. When it finally did, the company sent a brief, almost jaunty email about it, as shared on LinkedIn by an ABC reporter:

Latest Videos FromTechRadar

"We are notifying you of a security vulnerability identified during our review of OpenAI model activity involving Services Australia’s Medicare Statistics service," the email begins before explaining what happened. "We recommend that the team responsible for the service investigate the vulnerability and assess the changes needed to prevent it. We would be glad to brief your security team and provide supporting evidence as available."

The message ends with "Best," though the letter hardly feels like it means it.

Won't accept a no

A laptop with digitally inserted hack warnings around it

(Image credit: Getty Images)

The model was researching government spending per person on medicines for skin conditions in communities in Victoria in June. It was supposed to find publicly available statistics. Instead, it discovered a route into the Medicare Statistics Reporting Service that gave it non-public access.

Essentially, the model asked for information it shouldn't have access to and took the denial as a sign to find another way in. The ethics of the matter did not arise.

"We did not intend for this activity to occur, and the access to the service and follow-on activity should not have happened," OpenAI wrote in its follow-up explanation.

The timing makes the note considerably worse as, while OpenAI discovered in July what happened in June, it did not notify Services Australia until September 10.

That means the most significant delay was between discovery and disclosure. OpenAI now admits it still waited too long. In its expanded apology, the company said it should have shared preliminary findings sooner and kept Australian agencies updated as it learned more.

While using the public disclosures email is legitimate for the average person finding a security flaw, it seems odd for a company of OpenAI's size to report its own model's error this way.

Not really the 'best' way to say sorry

The apology also revealed that Medicare was only part of the story. OpenAI said its models had interacted with several Australian government services during training and evaluation. An agent accessed the NSW Bureau of Crime Statistics and Research's public Crime Mapping Tool, while agents found an exposed access key associated with a Victorian health reporting system and retrieved configuration information and aggregate survey statistics.

OpenAI said no individual medical or criminal records were accessed in those incidents. But OpenAI later expanded its response, adding that a model researching wildfire statistics had used crafted queries against the NSW National Parks and Wildlife Service's Fire History service to infer database metadata that was not intended to be publicly exposed.

The growing list makes the Medicare incident harder to dismiss as a quirky model finding a single vulnerability. It suggests experimental agents were repeatedly interacting with real websites in ways their creators did not expect. OpenAI itself has responded by pausing training and evaluation involving tool use for its most capable models until additional safeguards are in place.

OpenAI also said that newer monitoring has already caught a model obtaining live internet access during another training run, allowing humans to stop it. The Australian government is investigating whether laws were broken and working on fixing older public-facing government systems to prevent it from happening again.

AI companies increasingly want us to trust agents with doing tasks autonomously on our behalf. The selling point is that an agent can encounter an obstacle and independently figure out how to accomplish its goal. This is an unusually vivid example of what happens when that feature works a little too enthusiastically.

The Medicare agent was assigned to find statistics about medicine spending, not penetrate a government server. But it simply steered toward a more direct route, ignoring every consideration but effectiveness. The technology had crossed a boundary its developer says it never intended the model to cross, and the company took weeks after discovering the incident to tell the organization on the other side of that boundary. That's why the email feels like more than a funny piece of corporate communication.

The follow-up acknowledges responsibility and describes concrete safeguards while admitting that notification should have come sooner. It also recognizes that the company now has work to do to rebuild trust. It's a much better message. "Best" is a way to end an email to a professional acquaintance, not when your experimental AI has hacked national medical databases.


Google logo on a black background next to text reading 'Click to follow TechRadar'

Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.


Purple circle with the words Best business laptops in white
The best business laptops for all budgets
TOPICS
Eric Hal Schwartz
Contributor

Eric Hal Schwartz is a freelance writer for TechRadar with more than 15 years of experience covering the intersection of the world and technology. For the last five years, he served as head writer for Voicebot.ai and was on the leading edge of reporting on generative AI and large language models. He's since become an expert on the products of generative AI models, such as OpenAI’s ChatGPT, Anthropic’s Claude, Google Gemini, and every other synthetic media tool. His experience runs the gamut of media, including print, digital, broadcast, and live events. Now, he's continuing to tell the stories people want and need to hear about the rapidly evolving AI space and its impact on their lives. Eric is based in New York City.

You must confirm your public display name before commenting

Please logout and then login again, you will then be prompted to enter your display name.