» Current | 2020 | 2019 | 2018 | Subscribe

Linley Newsletter

Alibaba Takes AI Performance Lead

November 26, 2019

Author: Linley Gwennap

Alibaba’s first deep-learning chip is a doozy, achieving 10x the performance of Nvidia’s fastest GPU on the popular ResNet-50 model. The HanGuang 800 features four big cores, each with a set of MAC arrays optimized for inference operations. The Chinese web giant has already developed accelerator cards and systems using the new chip, and it has tested them in its data centers. It will also allow Alibaba Cloud customers to access HanGuang-enabled systems. Mass production is scheduled for December.

Alibaba subsidiary T-Head (also known as PingTouGe Semiconductor), in collaboration with the parent company’s Machine Intelligence Technology Lab, designed the AI accelerator. The chip group previously developed a 16-core RISC-V server processor. Alibaba designed its own deep-learning chip to support its surging deployment of neural networks. The HanGuang 800 focuses on inference, which is growing faster than training.

The company has released limited information on the HanGuang (含光) chip. It comprises four large accelerator cores that handle all of the computation. A small CPU (likely RISC-V) accepts commands from the host processor through a PCIe Gen 4 interface (presumably x16) and controls the accelerator cores. A high-bandwidth crossbar connects the cores, enabling several of them to coordinate when operating on a single large matrix. The chip doesn’t connect to DRAM or other external memory; any data that doesn’t fit on chip must come from host memory through the PCIe link.

Alibaba’s headline number is a ResNet-50 score of 78,563 images per second (IPS). The company also quotes 500 IPS per watt, but this power efficiency comes when the chip runs in a lower-voltage mode, achieving 53,983 IPS at 108W. Alibaba expects to run most workloads in that mode, since the overall performance is still much better than that of any other deep-learning accelerator.

Subscribers can view the full article in the Microprocessor Report.

Subscribe to the Microprocessor Report and always get the full story!

Purchase the full article

Free Newsletter

Linley Newsletter
Analysis of new developments in microprocessors and other semiconductor products
Subscribe to our Newsletter »

Events

Linley Fall Processor Conference 2020
October 20-22 and 27-29, 2020 (All Times Pacific)
Register Now!
More Events »