Most of us have wished, often at a party when we need to introduce someone whose name we've forgotten, that our smartphones or wearable devices could act as a passive memory system.
But the difficulty in building such a system is two-fold. Firstly, the device needs the battery power to be constantly listening or recording. Secondly, it needs to know what's important to remember and what it can discard.
Now, researchers at Rice University have had a go at solving both problems in one. They've built a piece of software, called RedEye, which is designed to see everything but only remember what it should.
"The concept is to allow our computers to assist us by showing them what we see throughout the day," said group leader Lin Zhong, who co-authored a new study on the subject.
"It would be like having a personal assistant who can remember someone you met, where you met them, what they told you and other specific information like prices, dates and times."
Remember, Remember
The first step was making the processes efficient enough for continuous operation. They accomplished that using software that reduces the power consumption of off-the-shelf image sensors tenfold.
"Real-world signals are analog, and converting them to digital signals is expensive in terms of energy," said Robert LiKamWa, who worked on the project. "There's a physical limit to how much energy savings you can achieve for that conversion. We decided a better option might be to analyze the signals while they were still analog."
Then to figure out what was worth remembering, they used a combination of recent research into machine learning, system architecture and circuit design.
The result is a neural network inspired by the organisation of the brain's visual cortex - the bit that processes the information we see.
Design and Testing
"The upshot is that we can recognize objects — like cats, dogs, keys, phones, computers, faces, etc. — without actually looking at the image itself," he said. "We're just looking at the analog output from the vision sensor. We have an understanding of what's there without having an actual image," said LiKamWa.
He added: "We can define a set of rules where the system will automatically discard the raw image after it has finished processing. That image would never be recoverable. So, if there are times, places or specific objects a user doesn't want to record — and doesn't want the system to remember — we should design mechanisms to ensure that photos of those things are never created in the first place."
Right now, the system is still in the design and testing stage, with a circuit layout being worked on. It needs improvements when recording data in low-light environments and other settings with a low signal-to-noise ratio.
But if those problems can be solved, then expect a future generation of wearables to be ever more aware of the world around it.
Perhaps even more than we are.
- Duncan Geere is TechRadar's science writer. Every day he finds the most interesting science news and explains why you should care. You can read more of his stories here, and you can find him on Twitter under the handle @duncangeere.
