» Current | 2020 | 2019 | 2018 | Subscribe

Linley Newsletter

Nvidia Ampere Delivers AI Innovation

June 9, 2020

Author: Linley Gwennap

For its new Ampere A100 accelerator, Nvidia could have simply used the 7nm shrink to double the tensor-core width and then called it a day. That single change would have boosted performance above what competitors offer. But the company went much further, delivering a series of hardware and even software innovations to gain additional performance.

These innovations include a new data type called TF32 that boosts peak TOPS by 8x relative to traditional FP32 operations. It’s of particular value for training, which often employs FP32 data. For inference, Ampere adds new integer and binary data types that allow users to trade lower precision for greater throughput. A new sparsity mode eliminates 50% of the weights, effectively doubling throughput. Sparsity can reduce model accuracy, but Nvidia claims the effect is negligible.

The new design can also compress activation values in the cache, effectively increasing the cache size. A new multi-instance-GPU (MIG) feature partitions a single chip into several virtual GPUs, enabling cloud-service providers to more securely rent GPUs to multiple customers (“AI as a service”). To help programmers efficiently write code that takes advantage of Ampere’s parallelism, Nvidia worked with the ISO to define a new C++ barrier function that simplifies asynchronous algorithms.

These and other improvements raise the bar for other companies marketing deep-learning accelerators (DLAs). Some of those vendors have deployed different types of sparsity and compression, and others have implemented new data types. But none offers this full suite of innovations—particularly the focus on virtualization and software development.

Subscribers can view the full article in the Microprocessor Report.

Subscribe to the Microprocessor Report and always get the full story!

Purchase the full article

Free Newsletter

Linley Newsletter
Analysis of new developments in microprocessors and other semiconductor products
Subscribe to our Newsletter »


Linley Fall Processor Conference 2020
October 20-22 and 27-29, 2020 (All Times Pacific)
Register Now!
More Events »