ROSETTA: ROM-Overlaid STT-MRAM for Efficient MVM and Softmax Operations Toward Accelerating Transformer Inference

Deep learning architectures are increasingly limited by the memory wall — the “von Neumann bottleneck” that forces frequent data transfers between memory and processing units, throttling throughput and energy efficiency. To alleviate this bottleneck, researchers have explored Compute-in-Memory (CiM) architectures to accelerate the core computing primitives such as the matrix–vector multiplications (MVMs), within memory arrays. Yet, large language models based on transformers also require many non-linear softmax operations, creating a new bottleneck. From a technology perspective, non-volatile STT-MRAMs are promising for CiM thanks to their endurance, nonvolatility, and area footprint. However, conventional 1T–1R STT-MRAM CiM arrays place bit-cells in parallel, yielding lower MVM energy efficiency than 8T-SRAM CiM due to low device resistance; line parasitics and low OFF/ON ratios of STT-MRAM devices further limit row parallelism and scalability. In this context, we propose ROSETTA, a CiM array based on 3T–2R STT-MRAM bit-cells that uses series-resistance sensing and time-to-digital conversion (TDC) for low-power MVMs. An added word line also stores a ROM bit, enabling in-array tables for fast softmax, without increasing bit-cell area. We introduce a novel palindromic encoding of inputs and weights to mitigate the data-dependent nonlinearity in STT-MRAM CiM arrays based on TDC sensing. This enables 8× higher row-level parallelism than standard 1T–1R MRAM CiM and facilitates scaling to larger CiM arrays. Our macro occupies ∼ 47% less area and consumes 3.1× less MVM energy than an equivalent ROM-overlaid 8T-SRAM macro, at similar latency and accuracy. At the system level, a spatial architecture comprising our macros exhibits 11.7−14.5× and 1.44 − 1.5× higher energy efficiency than equivalent architectures comprising 1T-1R MRAM and 8T-SRAM ROM-overlaid CiM macros, respectively.