Haptic feedback helps you to know when you’ve tapped a key on your phone with a reassuring little pulse. It gives you the feeling of firing a round in Call of Duty with a short, sharp jolt… and now it lets you experience the weight and pressure of lifting and cutting stomach tissue.
We’re huge fans of applications of haptic feedback that add a tactile sensory dimension to virtual reality experiences, so when we heard that there was a surgical VR experience at the Global Education and Skills Forum (GESF) that we were attending in Dubai, it was with a sense of palpable excitement that we headed to the Fundamental VR concession.
With its surgical training tool Fundamental Surgery, Fundamental VR has managed to create the most visceral haptic feedback we’ve ever experienced. Once you’re strapped into a VR headset you're handed control wands that correlate to mechanical arms that appear in front of you on-screen.
For the experience that we tried the wands controlled a set of pincers and a cauterizing tool, but they can be used to manipulate a scalpel, a bone chisel, a syringe – pretty much any tool that a surgeon will need to use during an operation.
The experience is device-agnostic (we used a Windows Mixed Reality Headset) but Fundamental had an Oculus Rift and HTC Vive too, and was happy to swap out to whichever device we wanted to try.
The experience itself is a very short, simple simulation. You’re put in the shoes of a surgeon, standing over a sheet-covered virtual patient, with a screen above an exposed fleshy belly with tools sticking out of it.
The screen shows the feed from the camera inserted into the torso, which is a view of the inside of a human body, just on the outside of the stomach.
Prepare yourself
You're tasked with grabbing and lifting the stomach lining, then cutting away a thin layer using the cauterizing tool by sliding it into a small hole and using heat to separate the section of the stomach lining.
It took us a couple of seconds to acquire the depth perception needed to grab the lining with our pincers, but when we did, the strangest thing happened… we felt the weight of the lining. It felt real. It didn’t feel like it was real, it felt real.
We’d never experienced anything like it, and we were halfway between awed and horrified. We had to lift the lining up to a point where it locked into position, and because we don’t have the deft touch of a surgeon we bungled it. The physical ‘pop’ of the lining slipping from our pincers genuinely made us gag.
Eventually we got the lining in the right place, and a hole opened up for us to slip the cauterizing tool into. Yep. That happened. Fighting back some pretty intense nausea we cut the lining using the heat of the cauterizing tool, and the session (thankfully) ended.
Talking with the maker
After taking a moment to compose ourselves, we had a chat with Fundamental VR’s co-founder and CEO Richard Vincent, and asked how it made such effective haptics, and what it plans to do with the technology.
“Surgery is about touch, and it’s about sight; with only one of them, it’s a game,” Vincent said. “Building the haptic intelligence that allows us to map all the tools and tissues, and all the possible collisions between them, took us about a year, and we’re now at a point where we can reliably deploy pretty much any tissue experience.”
And getting to that point took a lot of trial and error. “That’s a process of getting surgeons in,” Vincent continued. ”We’ve got a calibration tool that allows us to go ‘This is the point where you were inserting this into that location, did it feel right?’ and then we’ve got about seven different variables to adjust like how squishy it is, how much you have to push on it before it finally gives way or penetrates.”
Helpfully there was one real-life doctor on hand – Dimitri Amiras, a consultant radiologist from the UK-based Imperial College NHS Trust. We took the opportunity to ask him whether the ‘gamification’ of surgery would reduce the emotional risk associated with a real-life operation.
“The gamification will take the surgeon a step back from what he’s actually doing, and actually allow them to focus on the operation itself,“ Amiras told us. “If the surgeon’s thinking about the person’s kids waiting at home there’s no way they can do the operation. You can have empathy afterwards, but at that time you need to focus on the procedure.”
For an experience of a real-life brain surgery in 360-degree video, check out this video made by Fundamental:
While the simulation experience we tried felt very real, it still looked very game-like, and we wondered how many of Vincent’s team came from the gaming industry.
“The team’s made up of a lot of people from the gaming background because that’s where the skills in VR performance have really been refined over the last two years,” he told us. ”The best place to find Unity or Unreal programmers with really good experience is gaming.”
At the moment Fundamental Surgery is being used to train would-be surgeons, but we can see obvious applications in consumer VR, and we’re not the only ones – Vincent said he's been approached by figures from a number of different industries for his algorithm. For the time being, however, he’s keeping the focus purely surgical.
- Want to read some VR news next? Check out: Oculus Go standalone VR headset could launch in early May
