Scientists make cameras work more like human eyes and this could be good news for future smartphones

Apple iPhone 15 Pro
(Image credit: Future / Lance Ulanoff)

A little-known quirk of the human eye might pave the way for better self-driving car cameras and even more effective smartphone photography.

As you're reading this, your eyes are in most cases slowly scanning from left to right but even when not reading or looking at a fixed object your eyes are constantly on the move and this, it turns out, is the key to the quality of human vision and how robots, self-driving cars, and maybe even smartphones could see more clearly.

A team of University of Maryland researchers created a camera that mimics human eye movements. Called the Artificial Microsaccade-Enhanced Event Camera (AMI-EV), it uses a rotating round wedge prism (round, but one face of the prism is sharply angled) rotating in front of an event camera, in this case, an Intel RealSense D435 camera, to move the images around.

Even though the movements are small, they're meant to mimic the saccades of the human eye. Saccades describe three different levels of movement the eye makes – Rapid, small tremors, slower eye drift, and microsaccades, which happen multiple times per second and are small enough to be imperceptible to the human eye. 

This last movement may help us see more clearly, especially moving objects where our eye shifts to put the image against the best part of our retina, replacing blurs with shape and color. 

With the understanding of how these micro-movements help human perception, the team equipped its camera with a rotating prism.

According to the paper's abstract, "Inspired by microsaccades, we designed an event-based perception system capable of simultaneously maintaining low reaction time and stable texture. In this design, a rotating wedge prism was mounted in front of the aperture of an event camera to redirect light and trigger events."

Researchers paired the hardware solution with software that could compensate for the movement and combine captured images for a stable and clear image.

According to a report in Science Daily, the experiments were so successful that AMI EV-equipped cameras detected everything from quickly moving objects to the human pulse. That's some precise vision.

AMI-EV camera paper

This excerpt from the Science robotics research paper (supplemental materials) shows how the application of the AMI-EV increases image clarity. (Image credit: Science Robotics AAAS)

Making robotic eyes see more like humans offers the potential of not only robots that can share our vision skills but, for instance, self-driving cars that could finally discern between people and other objects. There is already evidence that self-driving cars struggle to identify some humans. A self-driving Tesla equipped with an AMI-EV camera might be able to tell the difference between a bag blowing by and a child running into the street. 

Equipped with AMI EV cameras, mixed-reality headsets, which use cameras to combine real and virtual worlds, might do a better job combining them for a more realistic experience.

"...it has many applications that much of the general public already interacts with, like autonomous driving systems or even smartphone cameras. We believe that our novel camera system is paving the way for more advanced and capable systems to come.," Yiannis Aloimonos, a professor of computer science at UMD and the study's co-author, told Science Daily.

These are early days, and the hardware looks more like something you'd put in an engine than the ultra-tiny and thin camera you might need for the best smartphone

Still, the realization that something we can't see happening is responsible for what we can see and how that small but critical vision capability can be replicated in robotic cameras is a significant step on the path to a future where robots match human visual perception.

You might also like

Lance Ulanoff
Editor At Large

A 38-year industry veteran and award-winning journalist, Lance has covered technology since PCs were the size of suitcases and “on line” meant “waiting.” He’s a former Lifewire Editor-in-Chief, Mashable Editor-in-Chief, and, before that, Editor in Chief of PCMag.com and Senior Vice President of Content for Ziff Davis, Inc. He also wrote a popular, weekly tech column for Medium called The Upgrade.

Lance Ulanoff makes frequent appearances on national, international, and local news programs including Live with Kelly and Mark, the Today Show, Good Morning America, CNBC, CNN, and the BBC. 

Read more
Visual Intelligence on an iPhone 16
Don't judge Apple Intelligence by today's summaries, smartphone AI is going to be amazing... eventually
Photo triptych from left to right: the Canon EOS R5 Mark II in the hand, closeup of the Pentax 17 lens, the Fujifilm X100VI in Tokyo street held up to photographer's eye
My 7 favorite camera moments in 2024 – from innovative in-camera AI editing to film making a comeback
Two iPhone 16 Pro phones on a grey background showing its camera and settings
I’m a photographer – 5 ways to take great photos on your iPhone 16 in 2025
Apple AirPods Pro 2 in ear
Samsung and Meta are looking into earbuds with cameras, following Apple’s AirPods’ lead
Axel testing the Honor Magic 7 Pro in Slovenia
I went to the Alps to test Honor’s AI Super Zoom camera and the results are both fascinating and frightening
BleeQup ranger smart glasses
BleeqUp's new 4-in-1 cycling glasses come with an AI-powered camera, one-tap video editing and a walkie-talkie feature
Latest in Cameras
Image showing detail of the Leica D-Lux 8
Still can't get a Fujifilm X100VI? This premium Leica compact costs less, and it's in stock
Hasselblad X2D 100C camera in user's hand, their blue jacket in background
My dream Hasselblad camera is getting a sequel soon, according to new leaks – here are 5 upgrades I’m hoping for
Nikon Z6 III camera in the hand with no lens attached and full-frame sensor on display
What's the best camera sensor format? You told us your clear favorite – here's why it's not that simple
Lunar eclipse
I'm a pro photographer – here's how I'm shooting the total lunar eclipse blood moon with my camera and phone
Sigma BF silver camera in the hand at The Photography Show, UK
I tried the Sigma BF camera everyone is talking about – it's truly stunning, but has one fundamental flaw
A young man takes a portrait photo with a DSLR.
Sell Smart: Quick, easy, and secure camera gear solutions with MPB
Latest in News
Google DeepMind panel discussion
“More sovereignty and protection” - Google goes all-in on UK AI with data residency, upskilling projects, and startup investments
An image of the Nintendo Switch 2
Nintendo Switch 2 could have AI upscaling similar to PS5 Pro’s PSSR according to patent, and it could be a gamechanger for graphics on the upcoming console
PowerColor Red Devil AMD RX 9070 XT graphics card shown side-on
Your next GPU could be from AMD, not Nvidia, if Team Red’s success with PC gamers continues
Quordle on a smartphone held in a hand
Quordle hints and answers for Tuesday, March 18 (game #1149)
NYT Strands homescreen on a mobile phone screen, on a light blue background
NYT Strands hints and answers for Tuesday, March 18 (game #380)
NYT Connections homescreen on a phone, on a purple background
NYT Connections hints and answers for Tuesday, March 18 (game #646)