Self-doubting robots could integrate more easily into society

There are a lot of robots in the world. Mostly they live in factories, but they're increasingly making their way into everyday life - and there are some big decisions on the horizon about how we'll safeguard certain parts of our way of life as they do so.

Now, a team of roboticists at the University of California, Berkeley, has developed a simulation that indicates that adding some self-doubt to robots could help them better integrate into society. The big question is: how much?

"It is clear that one of the primary tools we can use to mitigate the potential risk from a misbehaving AI system is the ability to turn the system off", the researchers wrote in a paper published on the pre-print arXiv server.

"As the capabilities of AI systems improve, it is important to ensure that such systems do not adopt subgoals that prevent a human from switching them off. Our goal is to study the incentives an agent has to allow itself to be switched off." 

The simulation they created depicts an interaction between a robot and a human with an adjustable level of self-confidence for the robot, which has a built in off-switch.

The wishes of the human

In one simulation, the robot was asked to perform a task, and a human was then given the option of allowing it to continue or hitting the off switch. But the robot also had the ability to override its own off-switch and therefore the wishes of the human.

As you can probably expect, robots that had a lot of self-confidence turned themselves right back on. When they were given just a little confidence, however, the robot stayed off - even if it judged that it was doing a good job.

"Our analysis suggests that agents with uncertainty about their utility function have incentives to accept or seek out human oversight. Thus, systems with uncertainty about their utility function are a promising area for research on the design of safe AI systems," the researchers wrote.

They added: "This is far from the end of the story. In future work, we plan to explore incentives to defer to the human in a sequential setting and explore the impacts of model misspecification."

Duncan Geere
Duncan Geere is TechRadar's science writer. Every day he finds the most interesting science news and explains why you should care. You can read more of his stories here, and you can find him on Twitter under the handle @duncangeere.
Latest in Tech
Josie and Matt laughing in front of the Google Pixel 9a
TechRadar Podcast: Is the Pixel 9a ugly? Has Apple ruined the smartwatch market? And is Samsung's One UI in trouble?
A Lego Pikachu tail next to a Pebble OS watch and a screenshot of Assassin's Creed Shadow
ICYMI: the week's 7 biggest tech stories from LG's excellent new OLED TV to our Assassin's Creed Shadow review
A triptych image of the Meridian Ellipse, LG C5 and Xiaomi 15.
5 amazing tech reviews of the week: LG's latest OLED TV is the best you can buy and Xiaomi's seriously powerful new phone
Beats Studio Pro Wireless Noise Cancelling Headphones in Black and Gold on yellow background with big savings text
The best Beats headphones you can buy drop to $169.99 at Best Buy's Tech Fest sale
Ray-Ban smart glasses with the Cpperni logo, an LED array, and a MacBook Air with M4 next to ecah other.
ICYMI: the week's 7 biggest tech stories from Twitter's massive outage to iRobot's impressive new Roombas
A triptych image featuring the Sennheiser HD 505, Apple iPad Air 11-inch (2025), and Apple MacBook Air 15-inch (M4).
5 unmissable tech reviews of the week: why the MacBook Air (M4) should be your next laptop and the best sounding OLED TV ever
Latest in News
Nintendo Music teaser art
Nintendo Music expands its library with songs from Kirby and the Forgotten Land and Tetris
An image of Pro-Ject's Flatten it closed and opened
Pro-Ject’s new vinyl flattener will fix any warped LPs you inadvertently buy on Record Store Day
The iPhone 16 Pro on a grey background
iPhone 17 Pro tipped to get 8K video recording – but I want these 3 video features instead
EA Sports F1 25 promotional image featuring drivers Oscar Piastri, Carlos Sainz and Oliver Bearman.
F1 25 has been officially announced, with this year's entry marking a return for Braking Point and a 'significant overhaul' for My Team mode
Garmin clippd integration
Garmin's golf watches just got a big software integration upgrade to help you improve your game
Robert Downey Jr reveals himself as Doctor Doom to a delighted crowd at San Diego Comic-Con 2024
Marvel is currently revealing the full cast for Avengers: Doomsday, and I think it's going to be a long-winded announcement