» Current | 2021 | 2020 | 2019 | Subscribe

Linley Newsletter

Deep AI Retrains Networks at the Edge

November 3, 2020

Author: Bob Wheeler

Startup Deep AI is enabling infrastructure-edge hardware to perform both retraining and inference. Otherwise, before running AI inference, customers must retrain their neural network using costly on-premises hardware or in the cloud. Retraining refers to adapting a pretrained model to customer data sets rather than training from scratch. Deep AI initially targets applications in markets such as retail, manufacturing, smart cities, and health care.

 The company licenses its software and RTL for use with FPGA-based accelerator cards from Xilinx. Its initial offering is for on-premises deployment in Alveo U50 cards and is available now. In 1Q21, it plans to enable FPGA-as-a-service instances based on the Alveo U250 to use its software/RTL.

Delivering acceptable training time from such modest hardware requires a secret sauce, which Deep AI provides through reduced-precision math and sparsity. Inferencing with 8-bit integer (INT8) data is already popular, but it requires quantizing models trained at higher precision, such as 32-bit floating point (FP32). In Deep AI’s system, inferencing directly employs the trained model without any intermediate conversions. The startup claims these models, including sparsity, are up to 95% smaller than the original model trained using FP32 without sparsity.

By retraining at the edge, customers needn’t send data to the cloud, improving privacy and security. Although researchers have studied INT8-based training, frameworks that achieve acceptable accuracy on a range of neural networks have yet to emerge. If Deep AI can deliver on its claims, it will set itself apart from other vendors targeting the infrastructure edge.

Subscribers can view the full article in the Microprocessor Report.

Subscribe to the Microprocessor Report and always get the full story!

Purchase the full article

Free Newsletter

Linley Newsletter
Analysis of new developments in microprocessors and other semiconductor products
Subscribe to our Newsletter »


Linley Fall Processor Conference 2021
Held October 20-21, 2021
Proceedings available
Linley Spring Processor Conference 2021
April 19 - 23, 2021
Proceedings Available
More Events »