» Current | 2020 | 2019 | 2018 | Subscribe

Linley Newsletter

Google Details TPUv3 Architecture

September 1, 2020

Author: Linley Gwennap

Google recently disclosed details of its TPUv2 and TPUv3 chips for the first time, more than two years after the former entered production. The disclosures describe how the company co-developed the chip architecture and software tools to produce a powerful accelerator that can handle some of the largest AI workloads in the world. The TPUs deliver up to 99% scalability in titanic 1,024-chip configurations. To simplify programming, they employ only two brawny cores per chip, but each core implements a flexible VLIW architecture that relies on 16K-unit MAC arrays for much of the computation.

Google’s first custom AI chip was the TPUv1, which entered production in 2015. That chip focused on inference tasks and supported only integer data. For the TPUv2, the company added floating-point capability so it could accelerate neural-network training as well. Whereas the original TPU was more of a fixed-function architecture, the newer one is more programmable and can handle a broader repertoire of models. Each TPUv2 connects to four stacks of High Bandwidth Memory (HBM) and provides four chip-to-chip connections for creating large systems organized as a 2D torus.

The TPUv3 implements a similar architecture and uses the same 14nm-class technology as its predecessor; as a result, it reached production in early 2019, only a year after the TPUv2. The primary architecture change is a second matrix accelerator in each core. The TPUv3 also achieves a 30% faster clock speed, yielding a 2.7x rise in peak compute rate.

Google has already completed the TPUv4 design. Its recent MLPerf Training benchmark results show an average gain of 2.4x over the TPUv3, allowing it to keep pace with Nvidia’s newest GPU, the Ampere A100. We expect the TPUv4 to become available for customer use by the end of this year.

Subscribers can view the full article in the Microprocessor Report.

Subscribe to the Microprocessor Report and always get the full story!

Purchase the full article

Free Newsletter

Linley Newsletter
Analysis of new developments in microprocessors and other semiconductor products
Subscribe to our Newsletter »

Events

Linley Fall Processor Conference 2020
October 20-22 and 27-29, 2020 (All Times Pacific)
Register Now!
More Events »