MetaAttention: A Unified and Performant Attention Framework across Hardware Backends

Proceedings of the 31st ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming 2026

View at publisher
Authors: Feiyang Chen, Yu Cheng, Lei Wang, Yuqing Xia, Ziming Miao, Lingxiao Ma, Fan Yang, Jilong Xue, Zhi Yang, Mao Yang, Xingda Wei, Haibo Chen
DOI: 10.1145/3774934.3786444
Cited by: 1
References: 6

Abstract

Computing attention is the backbone of transformer-based models like large language models. However, the increasing diversity of attention algorithms presents significant challenges for unleashing hardware performance. State-of-the-art variants like FlashAttention target a specific attention algorithm or hardware platform, which fail to generalize to other algorithms and platforms.

Fields of Study & Subjects

Big Data and Digital Economy Advanced Neural Network Applications Natural Language Processing Techniques

Publication details

Journal / source
Proceedings of the 31st ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming
Publisher
ACM
Pages
635–647
Publication date
28 January 2026