» Current | 2022 | 2021 | 2020

Linley Newsletter

It’s Getting Hot in Here

June 21, 2022

Author: Linley Gwennap

Server processors, switch chips, and AI accelerators can generate lots of heat, which is why data centers resound with fan noise. For high-end chips, 300W was once considered the upper limit, but recent announcements have smashed through that barrier, in some cases topping 900W. New process nodes bring greater transistor counts but only minor power-efficiency increases. Chip designers wanting greater performance have little choice but to use the extra transistors and let their customers worry about cooling.

AI accelerators have led this trend. Nvidia limited its older V100 to 300W but stretched to 400–500W for its A100 two years ago, and its new H100 unabashedly carries a 700W TDP. Broadcom’s 12.8Tbps Tomahawk 3 switch chip used 300W (typical), but the 25.6Tbps Tomahawk 4 jumps to about 450W. Startup Tachyum rates its 128-core server processor, which has large matrix units, as high as 950W.

Transistor progress has slowed, with 5nm delivering modest benefits and 3nm looking worse. Although foundries claim that each new node reduces power per transistor by 20–30%, these figures fail to account for rising signal resistance as metal layers become narrower. In a typical processor design, this greater resistance cancels most or all of the power-efficiency gain. Supply voltages are no longer dropping, leaving no easy way to save power.

The changes affect data centers as well. Many are built to supply around 15kW per rack, enough for dozens of standard servers. But such a rack can hold only a single DGX H100 system (with eight GPUs), which burns 10kW. A full rack of H100 systems requires 70kW. Thus, data-center operators need new wiring to supply these voracious devices.

Subscribers can view the full article in the Microprocessor Report.

Subscribe to the Microprocessor Report and always get the full story!

Free Newsletter

Linley Newsletter
Analysis of new developments in microprocessors and other semiconductor products

Events

Linley Spring Processor Conference 2022
Conference Dates: April 20-21, 2022
Hyatt Regency Hotel, Santa Clara, CA
Linley Fall Processor Conference 2021
Held October 20-21, 2021
Proceedings available
Linley Spring Processor Conference 2021
April 19 - 23, 2021
Proceedings Available
More Events »