RRAM: A Reconfigurable Request-Rate-Aware Multi-DNN Accelerator Based on FPGA

The latency-sensitive image recognition systems in edge data centers are facing the workloads of multiple deep neural networks (Multi-DNN) and multi-user dynamic requests. Most of the existing solutions are based on GPUs or ASICs, whose fixed hardware architecture has limited low-latency capabilities when dealing with varying request rates in random dynamic scenarios. The dynamic reconfigurability of Field-Programmable Gate Arrays (FPGAs) can enable the hardware architecture to adapt in accordance with fluctuating request rates. Therefore, this study proposes a Request-Rate-Aware Multi-DNN accelerator framework (RRAM) based on FPGA, thereby improving latency performance. The RRAM includes a reconfigurable, computation-memory-interleaved multi-core accelerator architecture, a dynamic bandwidth adaptor for stable data transfers, and a performance analytic model for random request scenarios that enables an end-to-end latency evaluation. To efficiently explore the design space, a workload-aware resource boundary search strategy is also introduced, which eliminates a majority of invalid solutions. Implemented on the U250 platform, RRAM achieves better Service-Level Agreement (SLA) satisfaction rates compared to the fixed-architecture baseline accelerators under varying request rates, and improves the Average Normalized Turnaround Time (ANTT) by 1.3x to 6.1x. Furthermore, the average power consumption of RRAM is only 22.8% and 44.9% of the GPUs of RTX3090 and A100, while the latency is improved by up to 18.2x and 4.6x.