What you can do with a GPU
Microsoft will be using the new GPU VMs itself, especially for workloads that rely on deep neural networks and machine learning. "The internal image recognition that's done by Bing, Skype Translator; they will all be done on Azure in the future," said Karan Batta of the Azure team. That will take advantage of the strength of CNTK – this open source deep computing toolkit is Microsoft's equivalent of Google TensorFlow and it can scale to multiple GPU systems for better performance.
CNTK was used to build the CaptionBot service that provides a caption describing what's happening in an image. At the moment that runs using standard virtual machines on Azure, but it will get better performance when it's moved over to the new GPU VMs.
Running CNTK on a virtual machine with four GPUs, Batta was able to process 10,000 samples a second using a single GPU, 19,000 samples a second with two GPUs and 35,000 samples a second with four GPUs, showing that the toolkit can keep taking advantage of more hardware.
And the big data you want to use for machine learning is often stored in the cloud already. "We're trying to close the loop on data. You want to work with data where it is and GPUs help with that," Batta explained.
All manner of applications
But Batta also predicts that GPU computing will be useful for a lot of high-performance computing tasks in finance and manufacturing as well as in traditional graphics areas like media and rendering. "You could make a Netflix-like offering using public cloud. We're able to do rendering with V-Ray, Arnold, all these ray tracers, in the cloud; you can spin up the number of GPUs you need to render your frames and then spin them back down again," he said.
Batta added: "Taking the GPU and directly passing it through to the guest allows us to give you good performance, because we're essentially giving all the GPU performance to the guest. That allows close to bare metal performance that these workloads can really take advantage of."
With all that power, you will need to allow some time for N-Series VMs to start up – it will take five to ten minutes for them to spin up and load the operating system and your applications.
