Optimization of a GEMM Implementation using Intel AMX

In high-performance computing, general matrix multiplication (xGEMM) routines form the core of Level-3 BLAS kernels, which enables efficient matrix operations. Among these, the low-precision GEMM such as BFloat16 has become indispensable in machine learning and deep learning because it reduces memory usage and power consumption. To meet this demand, recent hardware platforms are equipped with dedicated matrix computation units separate from the CPU, and research efforts have focused on maximizing their performance. Intel’s Advanced Matrix Extension (AMX) is one such hardware accelerator designed specifically for low-precision matrix operations. In this study, we implement and optimize matrix multiplication-accumulation using AMX by applying blocking and tile register level optimizations and evaluate its performance. Our results demonstrate a performance improvement in the range of 7.27–20.40% compared to that of the BFloat16 GEMM implementations provided by MKL and OpenBLAS.