A Flash-Based Reconfigurable QCNN Inference Accelerator for Edge Applications

This paper presents a novel in-memory flash-based Quantized Convolutional Neural Network (QCNN) inference accelerator Integrated Circuit (IC), which achieves extremely high throughput and extremely low power, energy, and memory requirements. In this manner, our design significantly improves upon state-of-the-art inference accelerators operating at the edge. Our accelerator IC is designed as a field-programmable, reconfigurable dataflow architecture, which allows a wide variety of trained QCNN models to be programmed to our IC. Once programmed with a model, our IC performs inference withzeroaccess to off-chip memory. In our accelerator, the core computing structures are arrays of flash Field Effect Transistors (FETs) integrated in three dimensions (3D). We use 3D NOR flash stacks, thus achieving an area efficient design with the promise of further improvements as the depth of 3D NOR flash process technology advances. Our flash arrays perform dot product operations in the analog voltage domain, using an extremely low supply voltage (V DDlo, which is 100 mV in our design), resulting in a very high energy efficiency. The digital I/Os of our arrays are in the nominal supply voltage domain for our process (V DDhi, which is 0.8 V in our design), which allows them to be easily stored in registers without the need for level shifters. Neuron weights in our QCNN accelerator are stored in-memory in our flash arrays. These arrays perform dot product operations between a vector of all neuron inputs (in theV DDhidomain) and a vector of corresponding weights (stored in the flash devices as programmedVTHvalues), resulting in an output in theV DDlodomain. We show that for inference on ImageNet with a quantized AlexNet model, our IC achieves extremely low inference latency (28.7 ms at most), power consumption (5.82 mW at most), and energy consumption (4.35 µJ at most), making it especially suited for use in resource-constrained edge devices, such as portable medical devices used for disease diagnosis and ECG signal classification. We prove that our design is robust to Process, Voltage, and Temperature (PVT) variations, and report a drop in Top-1 (Top-5) ImageNet validation accuracy of at most 0.90% (1.07%) across PVT variations. We compare our QCNN accelerator against recent state-of-the-art in-memory neural network accelerators and report an energy efficiency that is at least 15.2µ higher and a parameter density that is at least 3.32µ higher than the best existing design.