For the first time, a computer program has been developed that is more effective than a human at telling how someone else is feeling, from their voice alone.
It's the product of a collaboration between psychologists and computer scientists at the University of Rochester in New York. While the program isn't the first to attempt the task, it's the first to score significantly higher than humans.
The software uses a selection of properties of speech – vocal tract shape, frequency, brightness, flatness, roughness and energy – to determine the emotion behind it.
It works on the fly, without needing to record the speech, and its only output is the emotions that it identifies.
Parents and children
In tests on 700 audio samples the program got the emotion right 72% of the time, compared with a group of 138 people recruited from Amazon's Mechanical Turk platform, who averaged just 60% accuracy on the same clips.
When the program was allowed to skip clips that it wasn't confident about, its accuracy rose to 85%.
The team plans to use the software to record interactions between parents and children, and work out how those emotions impact a child's development.
Na Yang, one of the computer scientists behind the research, summarised how it works in her PhD thesis, available on the University of Rochester's website.
Image credit: StartupStockPhotos // CC0
