Steering Sparse Autoencoder Latents to Control Dynamic Head Pruning in Vision Transformers (Student Abstract)

Read full text Summarise with VIRAT
Authors: YouSung Lee, Dongsoo Har
Cited by: 0
References: 4

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Dynamic head pruning in Vision Transformers (ViTs) improves efficiency by removing redundant attention heads, but existing pruning policies are often difficult to interpret and control. In this work, we propose a novel framework by integrating Sparse Autoencoders (SAEs) with dynamic pruning, leveraging their ability to disentangle dense embeddings into interpretable and controllable sparse latents. Specifically, we train an SAE on the final-layer residual embedding of the ViT and amplify the sparse latents with different strategies to alter pruning decisions. Among them, per-class steering reveals compact, class-specific head subsets that preserve accuracy. For example, bowl improves accuracy (76%→82%) while reducing head usage (0.72→0.33) via heads h2 and h5. These results show that sparse latent features enable class-specific control of dynamic pruning, effectively bridging pruning efficiency and mechanistic interpretability in ViTs.

Fields of Study & Subjects

Advanced Neural Network Applications Domain Adaptation and Few-Shot Learning Explainable Artificial Intelligence (XAI)

Publication details

Journal / source
Proceedings of the AAAI Conference on Artificial Intelligence
Volume
40
Issue
48
Pages
41263–41265
Publication date
14 March 2026
Language
EN