Traditionally, data centers operated by cloud providers like AWS, Azure and Google Cloud have relied on air conditioning and liquid cooling to keep servers and other equipment from overheating.
However, these cooling methods are inefficient, expensive and resource-intensive. And this is especially true in the context of facilities located in tropical climates (e.g. Hong Kong or Singapore), which are locked in a constant battle with the environment.
Over the last few years, attention has turned to identifying methods that can help cool data centers more effectively, and in a more sustainable manner. Although a few different techniques show promise, one in particular stands out: two-phase liquid immersion.
Under this system, server racks are completely submerged in tanks of non-conductive fluid. As the hardware starts to generate heat under load, the fluid begins to boil and rise to the top of the tank. In the second phase of the process, a condensing coil returns the gas to liquid, which drops back into the chamber below - and so the cycle continues.
Although liquid immersion cooling was first deployed way back in 1985 with the Cray-2 supercomputer, which cooled its hardware with waterfalls of a fluid called Fluorinert, the technique has since matured and is starting to gain traction.
Leading the charge in this space is a company called LiquidStack, which offers immersion tanks for a variety of rack sizes. Although its technology is not yet used widely, it’s beginning to attract the interest of some of the world’s largest hyperscalers.
“Change is always difficult in the data center industry; the sector is risk-averse and slow to move,” said Joe Capes, LiquidStack CEO, in conversation with TechRadar Pro. “But I predict we’ll see the first large-scale cloud facility using only two-phase immersion within the next twelve months.”
A new identity
Founded in 2012, LiquidStack began life under a different identity entirely: Allied Control Ltd. At the time, the ambition was to build the first ever Bitcoin mining data center in Hong Kong, back when the cryptocurrency was trading at just $13 per unit.
Having dismissed all available cooling methods on the grounds of cost, Allied Control was forced to develop a technique of its own in collaboration with a company called 3M, which now manufactures the fluid used in LiquidStack immersion tanks.
In other words, two-phase liquid immersion began as a means to an end for the company, a utility in the quest to achieve a separate objective. But over the last few years, the novel cooling technique has become the primary focus.
Although Allied Control still believes the crypto sector represents a major addressable market, the company took the decision to pivot last year to focus on high performance computing (HPC), the data center and the edge. As part of this process, the firm rebranded as LiquidStack.
Asked about the rationale behind the change of name and direction, Capes explained the team had identified emerging opportunities on which it was well-positioned to capitalize.
“We wanted to do two things: take advantage of the clear traction liquid immersion had been gaining in the data center industry since circa 2017, and also unpeg our business from the price of Bitcoin,” he said.
“I think that was a really wise choice, because it has allowed us to diversify our market segmentation to high-growth areas, while also taking the opportunity to apply ourselves in certain crypto scenarios, where the fit is good.”
The strategy appears to be paying dividends; the company is fast gaining momentum and recently added Microsoft to its list of customers. The next challenge LiquidStack faces is taking two-phase liquid immersion to market at scale.
The perfect fluid
The single most important ingredient in the immersion cooling equation is dielectric fluid. These compounds have a very specific set of attributes, which means they are able to cool computer hardware extremely effectively, but without causing any form of damage.
LiquidStack currently uses a fluid called Novec 7000 for its immersion tanks, which is manufactured by 3M. The fluid is non-toxic, non-conductive and has a low boiling point of just 34 degrees celsius (93 degrees fahrenheit), which is perfect for two-phase immersion cooling scenarios.
Unlike with oil-based immersion, fluids like Novec 7000 are also non-flammable, so there is less risk a data center might go up in flames, with disastrous consequences for the customers of web hosting, bare metal hosting, server hosting and colocation providers.
“These fluids were originally designed to clean computer components on circuit boards and inside servers,” Capes explained. “And they are still used widely as a mild solvent in the semiconductor industry.”
“When you pull a server out of this fluid, it’s bone dry within 30 seconds. That’s a massive advantage from a serviceability perspective.”
However, the same attributes that make these fluids excellent solvents also pose challenges in a data center context; the high rate of evaporation means it’s easy to lose large quantities of fluid, and quickly.
For this reason, LiquidStack tanks are hermetically sealed in operation (Capes says neither dust nor air can penetrate), and designed such that only minimal fluid is lost when hardware is swapped in and out.
The next generation of dielectric fluids is the most exciting, however. According to Capes, companies like 3M, Chemours and others are on the cusp of developing fluids with next to no global warming potential (GWP).
By swapping ozone-depleting data center refrigerants (like R410A and R407c) for zero-GWP dielectric fluids, and replacing water- and energy-intensive cooling techniques with immersion, LiquidStack hopes to reduce the environmental impact of the data center industry by a massive margin.
“This year alone, data centers are going to consume 300 billion liters of water to reject outside heat. The ability to bring that number down to zero is transformational,” said Capes.
“My ambition is to reach a day where there is no water evaporated for heat rejection. That alone could provide a tremendous benefit to the planet.”
Performance boost
The opportunity to make data center facilities more sustainable is by no means the only benefit of liquid immersion, however; new research from Microsoft shows the technique has the potential to unlock massive performance gains too.
“Computing capacity has always been on the upward climb due to the constant technological improvements in the semiconductor manufacturing and packaging industry. This growth in computing capability is usually accompanied by a steep rise in heat flux density,” explain the researchers.
“State-of-the-art air cooling technologies throttle after a certain heat flux level and would require bigger heat sinks driving enormous airflow through it, which may not be desirable from a data center operation standpoint. Hence, a lookout for advanced thermal management techniques is quite imperative.”
According to data collected by Microsoft, which is experimenting with LiquidStack immersion tanks in its Azure facilities, some server chipsets can be overclocked by more than 20% thanks to the superior thermal management on offer.
The opportunity to complete demanding AI workloads markedly faster with immersion cooling could have major consequences in almost all industries, from finance and pharma to manufacturing and more. It’s possible we might have ended up with Covid vaccines even sooner, for example.
As Capes explains, immersion opens up opportunities in the context of rack design and facility layout too. Because there is no longer a need for bulky heat sinks, a larger number of servers can be squeezed into a far smaller area.
In theory, this means the compute density of data center facilities could be increased by a significant margin, without the need to upgrade server processors or expand the physical footprint.
There are also major ramifications at the edge, where space comes at a premium but more and more computing is expected to take place in the coming years. The ability to squeeze greater compute power into a more compact space could enable a range of new edge computing use cases in suburban and even urban environments, cutting latency to almost zero.
Capes even says it’s possible liquid immersion cooling will eventually trickle down to the consumer sector in some capacity, perhaps paving the way for the next generation of monstrous gaming rigs.
“Fundamentally, we’re in a really exciting time. I think change is hard, but it’s very real and we’re experiencing great traction in the market,” added Capes.
“We’re seeing a full-blown renaissance of the semiconductor and hardware industry, which will only help liquid immersion accelerate.”
Update (December 28): This article has been updated to reflect that liquid immersion was first used in the Cray-2 supercomputer.
- Also check out our list of the best cloud storage services
