Device Endurance-Aware Adaptive Layer Selection for DNN Training on IMC-Based 2.5-D Systems

Deep Neural Network (DNN) training is both a compute- and memory-intensive operation. There exist several techniques that make training faster and more energy-efficient on state-of-the-art von-Neumann architectures. RRAM-based In-Memory Computing (IMC) together with 2.5D technology delivers an energy-efficient solution to execute DNN training. However, due to low endurance of RRAM devices, limited number of weight updates can be performed. Therefore, in this work, we propose an adaptive layer selection approach for DNN training to improve lifetime of 2.5D system with RRAM-based IMC devices. Specifically, during training, DNN layers are adaptively selected for weight updates at each epoch based on their importance to overall training accuracy and their energy-delay product (EDP) information. Thorough experimental evaluations on proposed IMC-based 2.5D system with different DNNs show 19% improvement in energy-delay product on average, with only 0.5% loss in accuracy when compared to State-of-the-art (SOTA) technique where all layers are trained. Moreover, on an average our proposed technique improves the device lifetime by up to 15% with respect to the SOTA approach.