Polycam pulls in the real world and makes sense of it in 3D – and I am hooked

(Image credit: Polycam)

For as much time as we spend on our computers and phones, we can forget that much of the digital world lacks dimension. It’s a strange dichotomy, existing in a 3D world but living and working in 2D spaces.

Combining the real 3D dimensional world with the digital 2D one isn’t easy. Mixed reality tries to do it, but you have to wear a clunky headset. It’s not that we must drag fully-dimensioned objects and environments kicking and screaming into our computers and phones, but sometimes we need them there, or at least we need the 3D information that can make them useful in a digital world.

That is the space where Polycam lives. The five-year-old 3D scanning company – they claim to be one of the biggest in the world – makes a business of helping people scan objects ranging in size from a remote control to a car and spaces ranging from rooms to complete houses and factory floors and putting 3D replicas of them in their app (and vast online library) and a growing variety of projects.

My experience with handheld 3D scanning tools goes back at least a decade to when I was using the Structure Sensor, an iPad accessory that I bolted onto the tablet and used to scan small objects and people’s heads. With the advent of built-in LiDAR camera on iPhone Pro-level phones, there’s no longer a need for extra expensive hardware, just software smart enough to let you point it at spaces, objects, rooms, and houses, collect all that mesh data, and then turn it into useable 3D renders.

A big 3D vision

Polycam, which is prepping some major updates to its smartphone, tablet, and desktop-based software is working on making 3D scanning as obvious and simple as taking digital photos. It calling the suite of changes Polycam Vision 25 release and it includes Space Mode, which will use LiDAR to automatically generate 2D and 3D models, and even floorplans.

Company executives showed me a full home scan they created using the platform’s new Scene Editor, which used, in part, a drone to capture the external scans and then an interior scan that, with some work on their part, stitched together multiple space scans to make the complete home render. It also included a detailed 2D scan that looked like an architectural drawing.

Polycam’s been around long enough that it’s amassed a massive library of 3D objects and is using that information to inform its small language model (SLM) and help it identify objects and, in the near future, use the data to help Polycam build full 3D rendered scans out of a single image.

In the meantime, though its iPhone app, which I tried out, is a powerful tool for quickly capturing 3D image scans and a lot of the underlying spatial data that can help anyone from consumers seeking to do some redecorating to enterprises trying to understand how much paint a factory floor will take get the instant answers they need from one scan.

The app, which is switching up camera mode presets to be more consumer-friendly (Photo Mode becomes Object Mode, for instance), is quite easy to use. I opened it and without any guidance used the Room Mode to scan a hallway in my office. To scan, I walked the hall and pointed the phone at the walls and floors as the app and phone pulsed gently in my hand.

What was interesting was to see how the system was clearly gathering multiple levels of information. There was 360-degree evidence of the space and then highly accurate measurements that showed the wall height and floor plan.

Later I used the app to quickly capture a 3D image of a bike chained to a Manhattan lamppost. For this scan, I chose Gaussian Splat, a really unusual 3D capture mode that had to send the data up to Polycam for processing. The result, which was rendered on my phone when I reopened the app, looks like a mix between a Dali painting and a complex 3D model built of Pickup Sticks.

I also used the app to scan my house's living room, which I plan on redoing soon, and was impressed with how it automatically detected the furniture and with a tap of one layer could instantly remove all of it to show me a clean 3D schematic of the space.

Company execs tell me that the system’s new Space Mode will capture multiple layers of information with a single scan and even spit out a 10-to-20 page PDF, called an AI Spatial Report, with all the critical data about a room, space, or, eventually, entire home.

While that PDF full of space data is actually generated via Polycam’s SML, the software does not yet include any prompt-based activity. It knows a lot about the space and objects but you can’t type in a question asking about floor space and how much wood it will take to replace the flooring – at least not yet.

Polycam gave me a free account to try out the app. It’s worth noting though that this is not a free app. A Pro-level account will cost you $150 a year. So it might not be the best consumer solution, but realtors, architects, and interior designers might appreciate it. Businesses can get the whole system for $300 a year, and enterprises pay $600 a seat. The company’s Vision 25 release also includes a fair number of enterprise-grade updates including advance floor plans, saved measurements, cloud space processing, and pro-grade 3D file exports.

Polycam cofounder Eliott Spelman told me that the ideal system, and perhaps one they’re building right now, has the power of a 3D game engine with the ease of use of a current smartphone.

A 38-year industry veteran and award-winning journalist, Lance has covered technology since PCs were the size of suitcases and “on line” meant “waiting.” He’s a former Lifewire Editor-in-Chief, Mashable Editor-in-Chief, and, before that, Editor in Chief of PCMag.com and Senior Vice President of Content for Ziff Davis, Inc. He also wrote a popular, weekly tech column for Medium called The Upgrade.

Lance Ulanoff makes frequent appearances on national, international, and local news programs including Live with Kelly and Mark, the Today Show, Good Morning America, CNBC, CNN, and the BBC.