S 2 Mamba: An Efficient Mamba Accelerator With Word-Importance SSM Sparsity

The Mamba model, based on state space models (SSM), excels at long-sequence modeling with linear complexity, surpassing Transformers and emerging as a strong LLM candidate. However, various low-arithmetic-intensity operations in Mamba, along with the complex computational dependencies of the SSM, result in low efficiency on general-purpose computing platforms. Therefore, this paper introduces S$\rm ^{2}$Mamba, an efficient Mamba accelerator that leverages the sparsity of SSM. First, we develop a Mamba processing core (MPC) for low-arithmetic-intensity linear operations. By combining the linear-conv layer fusion scheme, the MPC facilitates fast depthwise convolution (DWC) computation, while continuous element-wise (EW) operations are designed to achieve efficient SSM computation. Second, we propose a word importance sparsity (WIS) algorithm that takes advantage of redundancy in natural language to filter 40.20% of unimportant words, leading to an average 37.25% reduction in the SSM computation. The saved computation is accelerated dynamically by the Dynamic Series Modules. Finally, we introduce a reconfigurable SiLU/Softplus unit (RSSU) for performing low-arithmetic-intensity nonlinear operations. The accelerator is implemented using a 28 nm CMOS process with an area of 17.56 mm$\rm ^{2}$. Extensive evaluations on representative benchmark tasks show that, S$\rm ^{2}$Mamba delivers 35.48–$97.11\times $speedup and 147.72–$332.47\times $energy efficiency gains over GPUs, while outperforming state-of-the-art Mamba accelerators with 1.52–$25.67\times $speedup and 1.10–$25.25\times $energy savings.