How photogrammetry brought Resident Evil's world of undeath to life

Photogrammetry
Image Credit: Capcom

A symphony of staccato shutter snaps filled the air. Xenon gas strobed from dozens of camera flashes. Capcom was visiting a meat processing plant with a trove of camera equipment.

Iron-stomached photographers shot photo after photo of wild game meat along with ivory bones and flowing blood. Afterwards, they ordered tofu and pizza, but not to cleanse their palates. Those, too, were photographic fodder. 

The meat, bones, blood, and viscera translated into zombified flesh in their recent remake of Resident Evil 2. The tofu photographs helped them create the character Tofu from the game's eponymous Tofu Survivor mode. As for the pizza, that was actually one of the first images audiences saw in the game's E3 2018 trailer as a rat skittered across it.

The ability to scan objects and then separately collect texture data has existed for some time, but photogrammetry captures both at once. Streamlining that process allows us to create accurate, properly textured 3D models faster than ever before.

Members of the RE2 dev team

The process that Capcom employed to transduce multitudes of photos into what players saw reflected in their world of undeath is called photogrammetry, and Capcom has been tapping into the art and technique of this process since 2017's Resident Evil 7.

"The photos we take are able to capture fine-grained details such as an object's texture, which in turn allows us to create realistic, high-quality 3D models,” members of the Resident Evil 2 development team told us in an interview. 

“The ability to scan objects and then separately collect texture data has existed for quite some time, but photogrammetry captures both at once. Streamlining that process allows us to create accurate, properly textured 3D models faster than ever before."

Streamlining processes like rendering is key when you want to make big budget games: it reduces cost, saves time and ultimately, creates a better product - in this case, one of the highest-rated games of 2019.

Photogrammetry 101 

The field of photogrammetry uses photographs to take precise and advanced measurements and has applications that range from cartography to archaeology.

The game industry has, over the last several years, adopted the science of photogrammetry to create highly detailed 3D models like the those found in Resident Evil 2's art deco museum-turned-police station and the sanitized white laboratory stained in stark relief with arterial spurts of blood.

This process works by first taking an array of overlapping photos of a subject from all angles to build a comprehensive set of measurements of the subject in 3D space. The photos are then stitched together using specialized software which takes into accounts for scale, angle, and perspective. In that way, a series of 2D images become a seamless 3D reconstruction.

Overlapping aerial photographs can be used to create topographical maps, whereas 3D modelers more often get up close and personal with their subjects in employing this technique.

Photogrammetry is also remarkably scaleable. A cursory glance around the internet reveals tutorials on how to do rudimentary photogrammetry with a smartphone camera. Of course, the technology employed by AAA-development studios is a bit more advanced than that.

A match made in heaven (or... hell) 

This versatile approach to creating high-fidelity 3D models has allowed Capcom to push the boundaries into literally photorealistic territory. Not only can it be applied to architecture and organic materials, but facial capture and even details as minute as creases and wrinkles in clothing as well.

"For Resident Evil 2, we hired live models who fit the descriptions of our characters, fitted them with the appropriate costumes, and then used photogrammetry to capture the data from which we built the 3D models. The process was pretty similar to filming a live-action movie. Many of the objects in the game were created using the same process," the developers told us. 

"Something we had to contend with in this game was the difference in size between the stunt actors and the facial model actors. We had to find a way to link the two of them by performing some initial facial scans to use as a target. We also use photogrammetry to capture the contortions that appear in the actors' outfits as they move around and in their faces as they emote."

To accomplish this, Capcom's 3D-scanning studio houses 141 commercially available digital single-lens reflex cameras across two booths. One booth is dedicated to creating lavishly-detailed full-body scans composited from photos taken across 103 camera, while the other 38 reside in a separate booth dedicated to facial scans. 

According to Capcom, the number of photos needed per object or person is directly proportional to the desired quality. They took, on average, about 100 photos for each model in the game for things like ...

Is the future of gaming picture-perfect? 

According to the Resident Evil 2 development team, there are a few drawbacks to employing photogrammetry - some objects can be cost-prohibitive to either procure or manufacture, while other objects that are constantly in motion (like trees and foliage that could sway in the wind) can introduce noise into the 3D data when converted if they move during the scanning process. 

That’s also not even mentioning the setup cost, which is rather sizable when you factor in a scanning studio and acquiring the equipment therein.

"However, assuming the object we want is easily deliverable, the quality of the final product compared to what's possible with traditional methods is more than worth the extra cost," according to Capcom. "We take the investment and operating costs of the photogrammetry studio into consideration as well, but we think this is an incredibly effective way to create assets, particularly in terms of the quality most contemporary games demand."

That is why photogrammetry has continued to leave a footprint on the game development industry, and why Capcom has said they will continue to utilize it for their future projects. 

Moreover, Capcom tells us, they are looking towards a future where it isn't something noteworthybut another commonplace tool in the belts of creators.

"Photogrammetry will eventually cease to be special, and in some ways it's already reaching that point. What's important is to have a kind of auteur mindset that is always aiming for higher quality and bringing even more value to the title. Photogrammetry is, in the end, just a part of the foundation used to create games and images."

Latest in Consoles & PC
Nintendo Switch 2 Joy-Con up-close from app store
Nintendo's new app gave us another look at the Switch 2, and there's something different with the Joy-Con
Nintendo Music teaser art
Nintendo Music expands its library with songs from Kirby and the Forgotten Land and Tetris
Image of Naoe in AC Shadows
Assassin's Creed Shadows is hands-down one of the most beautiful PC ports I've ever seen
Image of AC Shadows cover art & Steam Deck
It's not perfect, but Assassin's Creed Shadows' performance is impressive - it runs smoothly on the Steam Deck and Asus ROG Ally
Steam Deck OLED in limited edition white color
With a single update SteamOS could turbocharge handheld PCs – here's how
Samus Aran leaping through space
Metroid Prime 4 tipped to be at the heart of April's Nintendo Switch 2 deep-dive
Latest in News
Nintendo Switch 2 Joy-Con up-close from app store
Nintendo's new app gave us another look at the Switch 2, and there's something different with the Joy-Con
cheap Nintendo Switch game deals sales
Nintendo didn't anticipate that Mario Kart 8 Deluxe was 'going to be the juggernaut' for the Nintendo Switch when it was ported to the console, according to former employees
Three angles of the Apple MacBook Air 15-inch M4 laptop above a desk
Apple MacBook Air 15-inch (M4) review roundup – should you buy Apple's new lightweight laptop?
Witchbrook
Witchbrook, the life-sim I've been waiting years for, finally has a release window and it's sooner than you think
Amazon Echo Smart Speaker
Amazon is experimenting with renaming Echo speakers to Alexa speakers, and it's about time
Shigeru Miyamoto presents Nintendo Today app
Nintendo Today smartphone app is out now on iOS and Android devices – and here's what it does