» Current | 2020 | 2019 | 2018 | Subscribe

Linley Newsletter

Nvidia A100 Tops in AI Performance

June 9, 2020

Author: Mike Demler

Nvidia’s A100 GPU delivers huge performance increases to data centers running AI workloads, data analytics, and high-performance computing (HPC). It’s the first chip to implement the company’s next-generation Ampere architecture, succeeding the Volta V100. The new GPU achieves its impressive performance gains by implementing third-generation tensor cores that offer enhanced integer and floating-point capabilities, along with a new sparse-compute feature. It also boosts the scale-up performance in systems that distribute neural-network training across multiple GPUs. The third-generation NVLink doubles the V100’s bandwidth, and the new PCIe Gen4 interface delivers twice Gen3’s throughput. The chip connects to 40GB of HBM2 DRAM, increasing capacity and bandwidth.

Ampere’s tensor cores combine the Turing and Volta capabilities with three additional data formats: Bfloat16 (BF16), FP64, and a new 32-bit floating-point format the company calls tensor float (TF32). TF32 combines FP32’s range with FP16’s precision, delivering 10x Volta’s peak performance on FP32 calculations. The new tensor core enables the A100 to execute 10x more INT8 operations per second than Volta and more than 2x more than the Turing-powered Quadro RTX 6000. By exploiting the sparsity common to most neural networks, it increases peak performance by another factor of two for most data types.

Although the A100’s peak power jumps by 33% compared with the V100, it nearly doubles the power efficiency for training. Nvidia optimized Turing for inference, but the RTX 6000 delivers 20% better floating-point efficiency than Volta as well as 5x the INT8 efficiency. The A100 further improves inference thanks to its 56% better INT8 power efficiency relative to the RTX 6000. Ampere’s sparsity capabilities will further boost inference efficiency. 

Subscribers can view the full article in the Microprocessor Report.

Subscribe to the Microprocessor Report and always get the full story!

Purchase the full article

Free Newsletter

Linley Newsletter
Analysis of new developments in microprocessors and other semiconductor products
Subscribe to our Newsletter »


Linley Fall Processor Conference 2020
October 20-22 and 27-29, 2020 (All Times Pacific)
Register Now!
More Events »