Intel has lifted the lid on a second generation of Gaudi accelerators that could reduce the time it takes to train large-scale AI models significantly.
Announced at Intel Vision 2022 in Dallas, the Gaudi 2 processors are built on a 7nm process, feature 24 integrated 100GbE RoCE ports and boast the largest quantity of memory of any accelerator on the market (96GB HBM2e).
The new processors are a product of Israel-based Habana Labs, which was absorbed by Intel back in 2019, and are designed for servers dedicated to deep learning workloads.
Training AI models
In recent years, a number of large-scale natural language processing (NLP) and computer vision models have emerged, delivering performance far superior to previous entries in the respective disciplines.
The problem is that training these multi-billion parameter models is incredibly compute intensive, and therefore expensive and time-consuming, a limiting factor in the development of the technology.
However, with the new Gaudi 2 accelerators, both the cost and time it takes to develop sophisticated new AI models will be cut significantly, Intel says.
According to Eltan Medina, COO at Habana, price to performance ratio is a key factor for customers, and was therefore made a priority during the development of the second-generation accelerators.
Benchmarks presented at Intel Visions suggest Gaudi 2 processors deliver roughly 2x the training throughput across popular NLP and vision workloads (BERT and Restnet-50), as compared with Nvidia’s A100 GPU.
At the same time, the new Gaudi chips are said to deliver a circa 40% cost saving across both workload types, again in comparison with A100 GPUs.
“Intel is advancing AI and the value for data center customers with Habana accelerators, which are the optimal solution for servers dedicated to deep learning,” said Medina. “We believe this category will be incredibly important.”
Gaudi 2 processors are available to customers immediately, and are also likely to underpin cloud instances from AWS further down the line, as with the previous generation.
