A Lossless, Reconfigurable FP8 Compute-in-Memory Accelerator With Domino Logic-Based In-Memory Multiplication and Sign-Group Aggregation for Transformers

Efficient on-device execution of transformer-based models (TBMs) requires specialized hardware acceleration. Yet, stringent memory and power constraints hinder the deployment of TBMs on edge platforms, necessitating aggressively quantized models to maintain energy efficiency. FP8 data types (E5M2/E4M3) provide a compelling 8-bit quantization alternative to FP16/BF16/FP32, enabling more efficient MAC computations. However, most existing FP CIM accelerators rely on serialized pre-alignment, incurring significant accuracy degradation and energy overhead. To overcome these limitations, we propose a novel FP8 CIM accelerator that integrates 1) bit-parallel, reconfigurable, Domino-logic-based FP8 multipliers with 3.2× improved energy-delay-product to support dual FP8 formats, 2) post-alignment, reconfigurable sign-group aggregation units to enable efficient and precision-preserving computing, and 3) a lossless mantissa mapping scheme to facilitate seamless higher/mixed-precision MAC operations. Fabricated in 40 nm CMOS, our test chip achieves macro-level energy efficiencies of 42.13 TFLOPS/W for E5M2 and 38.23 TFLOPS/W for E4M3, respectively. Moreover, it delivers a system energy efficiency of 9.78 TFLOPS/W and a system compute density of 0.047 TOPS/mm2, 4.3× and up to 1.42× higher, respectively compared to the state-of-the-art TBM-oriented CIM accelerator. These results demonstrate that the proposed CIM circuitry provides a highly energy-efficient solution for on-chip transformer acceleration while maintaining computational accuracy.