Enhancing Layer Attention Efficiency Through Pruning Redundant Retrievals

Growing evidence highlights the impact of layer attention mechanisms in deep neural networks, which enhance inter-layer interactions and representational ability. However, existing layer attention methods suffer from redundancy, as adjacent layers often learn highly similar attention weights. This redundancy leads to multiple layers retrieving nearly identical features, diminishing the model’s representational capacity and increasing training time. To address this issue, we propose an Efficient Layer Attention (ELA) architecture composing two key components: first, we quantify redundancy by leveraging Kullback-Leibler (KL) divergence between adjacent layers; second, we introduce an Enhanced Beta Quantile Mapping (EBQM) algorithm to accurately detect and skip redundant retrieving, ensuring model stability while boosting efficiency. Experimental results demonstrate that ELA not only enhances overall performance but also reduces training time by 30%, yielding improvements in tasks such as image classification and object detection.