There are still some problems to solve concerning displaying the right information for where you are, such as when you're outside a large building and you want to see details of what's inside (like the different shops and services in a railway station) or you're inside the office and you want to see what's outside the building.
That's the occlusion problem: "There's something blocking what I'm looking at – should we efface things that are not directly visible? It works in the other direction too – if you're outside, you might not want to see the points of interest that are behind the building," Kraus observes. Or if you're in a large venue, you might want to be guided to the exit without seeing the details of businesses outside. And what about handling different levels: how do you show if an exit is underground or the coffee shop you're looking for is on the second floor?
The LiveSight team is working on the best way to show all of these elements so they're meaningful. "It has to be so when the user pulls their phone out the information being given is really clear and accurate and in context – and it shows any deficiencies that may be there. The limitations [we face] are defined by the resolution of the location information that's available from hardware and how good we are at providing directions that are really constructive, and extremely robust."
Are we there yet?
LiveSight is the third-generation of Nokia's augmented reality technology. Kraus thinks it will take off this time, partly because we have fast enough processors, big enough screens and good enough location tools in our phones, along with voice recognition that works reliably. It's also due to the fact that as smartphones have become more ubiquitous, we've started to demand ways of getting information that make sense on a phone screen – and AR is ideal for that.
"I think we're getting very close because people are interested in a simpler and more humanised experience. I don't want to stand out on the street and type in 'pizza'; I want to hold my phone up and say 'show me pizza joints'. There's a desire for simplification and speed. People want it to be simple and easy to get to. It's about searching, exploring, navigating – and making that entire workflow visual and effortless and preferably one-handed. There are lots of technologies starting to appear in phones and operating systems, like the increasing amount of voice recognition, that's starting to tie in with what we can do with LiveSight."
But he's also excited about what we get with the next generation of mobile devices. We asked him when will augmented reality become commonplace, and Kraus replied: "I feel it will happen as phones get thinner and lighter, as they become transparent screens [on the world], things we hold up and look through – the glasses on your face or a tablet you hold up and unroll like a pirate map."
