» Current | 2021 | 2020 | 2019 | Subscribe

Linley Newsletter

Groq Gets Connected

November 10, 2020

Author: Linley Gwennap

Advancing toward production, startup Groq disclosed additional details about the initial TSP chip, focusing on the high-speed-I/O subsystem that enables scalable high-performance AI inference for data-center workloads. At the recent Linley Fall Processor Conference, chief architect Dennis Abts described the TSP’s proprietary chip-to-chip links, which provide 480GB/s of total bandwidth. Taking advantage of these links, Groq has developed a 4U server containing eight TSP cards and delivering up to 5,760 trillion operations per second (TOPS) for 8-bit integer (INT8) calculations.

The company separately reported that it has raised new funding, adding to its prior $67 million stash. It withheld details of the amount and investors, but the round must be sizable: Groq plans to double its headcount to 120 employees by the end of this year and then double in size again next year. It plans to sell both PCIe cards and servers, but it withheld prices.

The TSP implements a unique microarchitecture that features a single massive core comprising hundreds of streaming function units. Each chip provides a single x16 PCIe Gen4 port for the host-processor connection plus 16 chip-to-chip links. Each link comprises four 30Gbps lanes, but the link speed slows to 28Gbps when cabled. Thus, the peak card-to-card bandwidth is 14GB/s per link in each direction. By comparison, Nvidia’s NVLink operates at 25GB/s per link.

Although the chip provides numerous coherent ports, the card exposes only three. This number is adequate to create fully connected four-card clusters; alternatively, eight cards can connect in a virtual cube. The 4U server contains two four-slot bays, enabling it to implement two four-card clusters or one eight-card cluster. The server contains two AMD “Rome” processors, which offer enough PCIe Gen4 lanes to control all the cards. It also has 200G Ethernet and InfiniBand HDR ports to connect multiple nodes.

Subscribers can view the full article in the Microprocessor Report.

Subscribe to the Microprocessor Report and always get the full story!

Purchase the full article

Free Newsletter

Linley Newsletter
Analysis of new developments in microprocessors and other semiconductor products
Subscribe to our Newsletter »


Linley Fall Processor Conference 2021
Held October 20-21, 2021
Proceedings available
Linley Spring Processor Conference 2021
April 19 - 23, 2021
Proceedings Available
More Events »