A bio-inspired method for investigating and boosting the performance of deep neural networks

Understanding which internal activations of deep neural networks are most influential for decision making remains a fundamental challenge in model interpretability. In this work, we propose Salient Neural Activation (SaNA) analysis, a post-training, model-agnostic framework that decomposes convolutional activations into a compact set of salient components and complementary minor features without modifying or retraining the network. Unlike pruning or attribution methods, SaNA operates directly on learned activations using singular value decomposition to isolate class-relevant information while preserving predictive behavior. Extensive experiments on object recognition and video-based activity recognition tasks demonstrate that a small number of SaNA components is sufficient to maintain correct classification, whereas minor features alone fail to do so. These results provide new insights into internal feature organization and offer a lightweight and interpretable alternative for analyzing deep neural representations.