Why augmented reality is the future of smart toys

People flocked to Toy Fair 2019 earlier this month to find the latest toy reveals from major brands like LEGO and Mattel for the coming year. But toy inventors, distributors and sellers also attend to pick each other’s brains on which toys will fly off shelves, not just this year, but in five to ten years, too. 

This year, the industry scuttlebutt was fairly straightforward: by 2023, consumers will spend tens of billions of dollars on augmented reality (AR) toys. 

With Apple and Google going all-in on their ARKit and ARCore platforms, and apps like Pokemon Go enjoying huge popularity, toymakers see augmented reality as the perfect way to bridge physical toys and app-loving, screen-addicted children. 

But successfully combining physical playsets with AR is no easy feat, the Toy Fair experts warned. You need both physical toy designers and AR programmers to successfully work together; a physical play experience that can stand on its own without the app; an AR interface that doesn’t have the child staring for too long at the screen and ignoring the physical toy; and a dozen other rules for making sure the augmented side of a toy doesn’t feel gimmicky or tacked on. 

We scoured the Toy Fair for all of the smart tech toys we could find, and unsurprisingly, many of them used AR. But does the AR actually make these toys more fun, or is it just a gimmick to increase the toys' price tags? Here are our thoughts on the highest-profile AR toys due to come out later this year. 

LEGO Hidden Side

LEGO Hidden Side, an 8-set series of spooky locations and possessed vehicles, is the best example of an AR toy that technically doesn't need AR for a kid to enjoy playing with it.

As with any LEGO set, kids can engage simply by building the haunted town of Newbury, then playing with their minifigs inside of them, without ever opening a phone app. 

Plus, Hidden Side's sets have a fun mechanic. Some of the blocks rotate between their normal appearance and their haunted appearance. These were designed as something for kids to interact with that would trigger an augmented reality reaction, but they easily work as a purely physical way to play. 

Once kids open the AR app, they begin a more guided experience as a ghost-busting sleuth, investigating the area for clues before driving the hidden ghosts out into the open and defeating them during a rail-shooter minigame. 

Finding the ghosts is relatively simplistic: kids will hover their phone over bright lights of "gloom"; collect enough gloom currency (which takes a few seconds), and a haunted object will be highlighted. Interact with that object in the real world, and the ghost will appear to be captured.  

There are 100 ghosts to collect, and certain ghosts are much more likely to appear in certain sets than others. Supposedly, different ghosts will have different personalities and actions while being hunted, but the core gameplay of ghost hunting should be the same across each set. 

The optimistic view of the AR app is that it extends the physical play of Hidden Side by helping kids to imagine the haunted world surrounding their sets—a world they can conceptualize and play in once the app is closed. Plus, the LEGO rep said they would be able to add more missions to the app as time went on, further adding to Hidden Side's replay value. 

The more cynical view is that this app exists for collectible-loving kids to pressure their parents into buying all 8 sets so they can collect all 100 ghosts. 

Video games and mobile games compel kids to invest more and more time into them in exchange for rewards—achievements, loot boxes, collectibles, etc. By adding an AR online element, LEGO hopes to lure its kids into a similar recurring play pattern. And we find it highly likely that other companies will try to replicate this model, and potentially sell subscriptions or season passes to toys. 

Pictionary Air

At the other end of the spectrum from the highly independent Hidden Side LEGO sets, Pictionary Air relies entirely on augmented reality tech for its playing experience: You draw in the air using a light-tracked pen, in front of someone holding a phone or tablet camera. The camera tracks your drawing and projects it onto the screen for everyone else to see, and they guess what you're drawing. 

It's a simple system, and it mostly worked very well when we tested it out at the Toy Fair. The pen uses light tracking instead of more advanced sensors to make it more affordable for consumers, but in our experience it tracked our drawings accurately. Though it's difficult to tell, since we're drawing in mid-air: without a real canvas you'll find it difficult to remember where you've already drawn, leading to some pretty inscrutable sketches. 

Also, since you have to face the person with the table so the camera can track the pen, your drawing is a mirror image of what you want the person to see. I tried to draw the United States for a clue, and California and Florida ended up flipped, making it pretty hard for the other person to recognize.

Pictionary isn't just a promising AR toy because it's fun to use, or because it fulfills the "A" requirement for a STEAM toy (Arts). It's also promising because it doesn't isolate the person using it, like many AR apps. 

During a panel on STEM/STEAM toys, littleBits founder Ayah Bdeir said that, while research is still ongoing on the effects of prolonged screen time on kids' brains,  "social interaction is a key component of helping the development of kids. And so by inference you can say that if face-to-face interaction is so important, and if the screen is getting in the way of that, then we need to make room for what’s working ... Use the screen in order to get you off the screen. Screens can sometimes facilitate interaction with other kids."

In other words, giving kids toys that revolve around screens isn't bad per se, but it's better if those screens involve work or play with other people. In this case, drawing for and responding to other people. By contrast, Hidden Side's AR app clearly restricts the play to one person, where normally multiple children could build and play together. 

PaiBotz

A lot of parents are looking to cut down on kids' screen time; an entire Toy Fair panel titled "Screenless Is In" was devoted to covering tech toys that don't use screens. So it's safe to say that toys with corresponding AR apps aren't going to be popular with parents unless they combine recreation with learning. 

PaiBotz is one such toy that strikes that balance. It teaches young children about Block-based coding languages and techniques like sequencing and looping. But not just through the AR app shown above, but also through the 150 physical building blocks your kid will use to build and rebuild the robot. 

Intriguingly, during the Pai Robotics demo, I saw that someone from Google had come to check out the little bot as well. He and one of the Pai representatives were having a serious discussion about PaiBotz's calibration issues. 

Google, Apple and Microsoft have engaged in a years-long war to determine AR dominance in the tech marketplace. Google, for instance, is reportedly developing an AR headset designed to be competitive with Hololens 2. We aren't surprised by the idea that Google would look for promising AR toymakers and developers to partner with, and that AR toymakers would want to grab some of that $100+ billion competitive marketshare. 

What the rise of AR means for kids

Technology is already rapidly changing how young children see the world. During a panel, toy business owner Kate Stone discussed how two-year-olds have been seen pinching and zooming magazine pages, not understanding why they were "broken" and not reacting to their touch. 

Before long, if augmented reality apps and toys continue to proliferate into our daily lives, children will consider seeing holograms into the real world to be the norm. 

The real question is what form all of these AR apps will take. Phil Sage, an inventor at Hasbro who was on the same panel, said that these apps will become more and more multi-modal, incorporating other senses and features beyond AR like voice assistant support. 

Eventually, your child may be able to summon holographic objects into their toy realm simply with their voice. Or, if glasses like Apple AR take off, kids could use AR apps without having to hold a phone in one hand, encouraging freer playtime in an augmented environment. 

• Here are the best AR apps available now