» Current | 2020 | 2019 | 2018 | Subscribe

Linley Newsletter

Tenstorrent Scales AI Performance

April 14, 2020

Author: Linley Gwennap

Tenstorrent wants to scale its AI architecture from milliwatts to kilowatts, but it’s beginning with a powerful data-center chip. Hoping to find an efficient middle ground between large monolithic architectures and arrays of tiny cores, the startup is sampling a chip with 120 cores, each capable of executing three trillion operations per second (TOPS). At a peak rate of 368 TOPS, the chip runs on just 65W. When inferencing popular neural networks, Tenstorrent expects its flexible architecture to outperform competing devices that burn as much as 300W. At the recent Linley Spring Processor Conference, it disclosed initial results for the ResNet-50 and Bert models to bear out this claim.

To increase power efficiency, the startup designed its Tensix architecture to take advantage of sparsity, meaning the cores don’t waste power on operations that produce no meaningful results. Although the architecture isn’t neuromorphic, it employs conditional execution to achieve similar savings in software. To further improve efficiency, it features “dense math units” that can perform thousands of operations based on a single instruction. Finally, Tenstorrent optimized the core-to-core interconnect to minimize packet overhead.

The basic architecture unit is the Tensix core, which is built around a large compute engine that produces most of its 3 TOPS from a single dense math unit. The core contains 1MB of SRAM to hold weight values and feed data to the compute engine. Five simple CPUs handle scalar processing and manage the conditional execution. Each tile connects to adjacent tiles via four bidirectional connections that form a double 2D torus, reducing the number of hops required to move data. A packet engine processes data from the torus, decompressing it on the fly.

Subscribers can view the full article in the Microprocessor Report.

Subscribe to the Microprocessor Report and always get the full story!

Purchase the full article

Events

Linley Fall Processor Conference 2020
Coming October 28-29, 2020
Hyatt Regency, Santa Clara, CA
More Events »

Newsletter

Linley Newsletter
Analysis of new developments in microprocessors and other semiconductor products
Subscribe to our Newsletter »