How Facebook plans to make VR hangouts more real, less virtual

Facebook

Facebook's purchase of Oculus in 2014 raised some eyebrows, but its presentation on how it plans to make VR a viable platform for social communication proved that it's making that money work.

At the firm's F8 developer conference, the company touched on everything from taking selfies in VR to how Oculus Rift's Touch controllers will help to make social interactions in a virtual environment a little more personable.

But perhaps the biggest takeaway from the keynote was just how heavily Facebook and Oculus are pushing VR to be more than just a technology that's fun to use alone, and something that can be experienced with other people – no matter where they are in the world.

Facebook

Creating an authentic presence in VR is a problem that Oculus' research division is working to solve. Sure, you can put on an Oculus Rift or HTC Vive right now, and feel very much a part of the virtual world that you see.

But what happens when you introduce another living, breathing human into the world?

Oculus tried that out with Toybox, a multiplayer tech demo of sorts that lets you and another person toss objects around and trade punches in a physics-based playground. In it, you can see rough outlines of faces and hands.

Oculus Toybox

There's definitely a human on the other side of that connection, but there may as well not be based on how it's rendered.

We saw more of Facebook's progress to make VR a more interactive, social experience at F8 2016. It was a demo that branched further out from the Toybox playground and allowed two users to visit each other digitally.

Despite being in two different physical locations, they were able to draw clothes onto each other with Oculus Touch controllers, and explore a 360-degree view of London together. It's not too different in concept to holoportation, a feature introduced recently for the Microsoft HoloLens that allows people to join each other in an augmented reality environment.

Both of these advancements show that we're getting there, but it would be putting it lightly to say that Facebook wants to take things even further.

You thought today's VR was futuristic?

There's no substitute for being in the same room as the person you're talking to. But, not only can you touch, you can also pick up on physical and behavioral subtleties in this VR environment that you'd probably miss through a Google Hangouts video chat.

During the F8 keynote, Oculus quoted anthropologist Edward Sapir by stating that these small, but meaningful gestures are "...an elaborate code that is written nowhere, known to no one, and understood by all."

And, advanced as it is in its early stages, VR cannot properly crack that code by putting a believable human avatar in a virtual world – yet.

But by properly capturing, displaying and predicting your every movement, the social network company thinks it can make chatting virtually feel as if you're in the same room – or as close as possible with a head-mounted display strapped to your face. Of course, it's easier said than done.

Oculus Rift

Much more expressive than a blank-faced avatar

First, capture the human face as accurately as possible. To do that, Facebook shared some early results from installing a high-resolution camera onto a Rift to capture subtle mouth movements. It needs work, but it's a heck of an improvement over flat 3D models.

Then, there's the challenge of transferring this data over a stable connection. It's an essential ingredient in creating a believable virtual one-on-one with someone. But, as the methodology of capturing every last movement gets more advanced, the amount of bandwidth required to pipe them through to the other side will likely skyrocket.

Panoptic studio

There's clearly some science going on up in here

Facebook is so interested in capturing these minute, uniquely human movements that it built another off-the-wall experiment to get even better at it. The Panoptic Studio is an igloo-looking dome with cameras capturing its subjects inside from every angle. It's highly sensitive at tracking movement, even when it isn't coming from a person.

The idea here is to intensely study the way we tend to act in various situations, social or otherwise. Ideally, the predictive learnings gathered from the studio will work to both make for more convincing social engagement in future VR apps, as well as lighten the load on the hardware-intensive capturing side of the process.

Panoptic studio

Believe it or not, this is the recorded movement of a volleyball match

Ever since its acquisition of Oculus, the question on everyone's mind has been: "So, when will we see a fully virtual version of Facebook?" The answer, just like the arrival of the Oculus Rift was, has been a long time coming, but is still a long way off.

We saw an early glimpse into the future of Facebook in VR and, thankfully, it won't just be a bunch of menus or Candy Crush invitations.

Which virtual reality headset is right for you? See for yourself:

TOPICS
Cameron Faulkner

Cameron is a writer at The Verge, focused on reviews, deals coverage, and news. He wrote for magazines and websites such as The Verge, TechRadar, Practical Photoshop, Polygon, Eater and Al Bawaba.

Latest in Virtual Reality & Augmented Reality
Ray-Ban Meta Smart Glasses
Samsung's rumored smart specs may be launching before the end of 2025
Snap Spectacles 5
Latest Snap Spectacles update teases an exciting AR future that I can't wait for
Project Moohan prototype at Samsung Galaxy Unpacked, an XR goggles headset on display in a show area
Samsung's Android XR headset could avoid the Apple Vision Pro's biggest mistake, according to this leak
Vision Pro Metallica
Apple Vision Pro goes off to never never land with Metallica concert footage
The Ray-Ban Meta Coperni smart glasses
The new Ray-Ban Meta smart glasses design is an expensive disappointment
The Meta Quest Pro on its charging pad on a desk, in front of a window with the curtain closed
Samsung, Apple and Meta want to use OLED in their next VR headsets – but only Meta has a plan to make it cheap
Latest in News
Ray-Ban Meta Smart Glasses
Samsung's rumored smart specs may be launching before the end of 2025
Apple iPhone 16 Review
The latest iPhone 18 leak hints at a major chipset upgrade for all four models
Quordle on a smartphone held in a hand
Quordle hints and answers for Monday, March 24 (game #1155)
NYT Strands homescreen on a mobile phone screen, on a light blue background
NYT Strands hints and answers for Monday, March 24 (game #386)
NYT Connections homescreen on a phone, on a purple background
NYT Connections hints and answers for Monday, March 24 (game #652)
Quordle on a smartphone held in a hand
Quordle hints and answers for Sunday, March 23 (game #1154)