Edge AI Accelerators using Non-volatile Analog Optical Tensor Processing

Abstract

Our goal in this project is to design, implement, and benchmark a hybrid electro-optical analog AI accelerator for deploying compute-intensive edge AI applications such as inference for autonomous vehicles or generative AI applications at edge. While electrical analog AI processors could achieve high TOPS/W, the actual TOPS is relatively low (~100MHz clock speed) which makes them impractical for complex AI applications such as self-driving or generative AI. We address this issue by preforming analog-based electro-optical processing which allows for much faster operation speed (+10GHz). Proposed accelerator chip can be co-packaged with host CPU and HBM for deployments of less than 50W AI accelerators in edge applications. Compared with pure electrical analog-based accelerators it has 100x higher compute power at similar TOPS/W, and compared with edge digital accelerators it can achieve 10x smaller area and +10x better TOPS/W.