» Current | 2022 | 2021 | 2020

Linley Newsletter

Merging Memory and Compute

April 12, 2022

Author: Bryon Moyer

In high-performance computing (HPC) and artificial intelligence (AI), data movement has surpassed computing as the main energy consumer. Moving a 64-bit word from off-chip DRAM to a CPU can consume a thousand times the energy of performing a compute operation on that word. For AI workloads, traditional von Neumann approaches can dedicate as much as 90% of energy to fetching data and delivering it to the compute engine. Reducing energy is important for both data centers and battery-powered edge equipment. Developers therefore seek creative new approaches to reducing the number of memory fetches.

Among these approaches are ideas variously called near-memory and in-memory compute, although no formal definition exists for either term. Near memory refers to moving memory close to the compute function or vice versa. In memory has many embodiments, such as adding compute resources to a memory array, adding logic or other computation circuits to a memory bit cell, and having a memory element perform the computation.

Many of these techniques started in universities or other research labs, but they are beginning to show up in commercial products. Ambient Scientific and Upmem are approaching production of near-memory chips. Samsung is pursuing a near-memory proof-of-concept, and Untether has a near-memory chip in production. Mythic is sampling a flash-based in-memory chip, and TetraMem has announced an in-memory chip that employs resistive RAM (RRAM). Practical challenges blocking commercial production include differences between DRAM and logic processes, analog-process variability, and digital/analog conversion.

Subscribers can view the full article in the Microprocessor Report.

Subscribe to the Microprocessor Report and always get the full story!

Free Newsletter

Linley Newsletter
Analysis of new developments in microprocessors and other semiconductor products

Events

Linley Spring Processor Conference 2022
Registration now open: April 20-21, 2022
Hyatt Regency Hotel, Santa Clara, CA
Linley Fall Processor Conference 2021
Held October 20-21, 2021
Proceedings available
Linley Spring Processor Conference 2021
April 19 - 23, 2021
Proceedings Available
More Events »