AI accelerators continue to evolve rapidly. In the data center, new chips focus on sparsity and other techniques for improving performance per watt. Scalability is paramount, particularly for training. Architecture innovation is particularly rampant at the edge, where power limitations are stringent and software stacks are smaller. Numerous vendors, large and small, are deploying a variety of approaches including CGRA, neuromorphic, and analog computation. This presentation will describe the latest architecture trends for AI accelerators of all types. 

Training AI models is typically performed on high-end GPUs using expensive floating-point operations (FLOPs). Inference, on the other hand, typically requires no more than modest INT8 hardware. AI development is then limited to ongoing communications between the data-facing inference devices and a massive training center, with the resulting performance, cost, security and data-privacy implications. We discuss the challenges in low precision, sparsity constrained training and Deep-AI's breakthrough technology that enables a seamless integrated INT8 training-inference workflow and edge device personalization.

Obtaining the highest IPS/W on real neural network workloads requires a complete rethink of the AI architecture. This presentation explores a unique approach leveraging networking principles that achieves high utilization and lowest latency while minimizing power, external bandwidth, and die area. The architecture uses a hardware scheduler, innovative memory management to handle compute and data efficiently, and a unified pipeline with zero context switch penalty. Executing neural networks natively with only metadata from each layer greatly simplifies the software stack.

With the proliferation of edge AI applications, questions around the right balance between performance, power and accuracy arise. This presentation will describe Hailo’s architecture, which is leveraged by customers across industries to perform high-performance AI inferencing at the edge. We will show real-life benchmarks in an embedded power envelope and discuss their implications and importance. We will then continue to actual usage examples that illustrate the intrinsic properties required in edge AI processing.
 

Noise, non-idealities, and the complexity of spatiotemporal processing make it challenging to achieve latency and power targets in always-on applications at the sensor-edge. Spiking neural networks are a compact, powerful means of processing spatiotemporal data within stringent operating envelopes, leveraging an inherent notion of time to realize advanced signal processing functions. This presentation introduces Innatera’s ultra-low-power mixed-signal approach to neuromorphic computing with spiking neural networks, and delves into the challenges of accelerating them at the sensor-edge.