FLARE: A Fine-Grained In-Memory Computing Heterogeneous Multi-Chiplet Accelerator for Large Language Models

View at publisher Summarise with VIRAT
Authors: Abhishek Tyagi, Zixuan Wang, Avadh Harkishanka, Hong Gao Wong, Bharadwaj Veeravalli, Xuanyao Fong
Cited by: 0
References: 32

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Deploying multi-billion-parameter Large Language Models (LLMs) on modern AI accelerators requires both low-latency communication and energy-efficient computation. Emerging heterogeneous in-memory computing (IMC) architectures offer a promising solution by co-locating compute and storage, thereby reducing data movement and power consumption. However, efficiently partitioning and distributing model workloads across cores remains a key open challenge in multi-chiplet, network-based accelerator systems for LLMs, as it directly impacts the workload’s performance and hardware utilization. We present a heterogeneous multi-chiplet accelerator architecture accompanied by a co-designed fine-grained hardware–software stack. This stack incorporates custom algorithms for LLM partitioning, workload mapping, and identification of hardware configurations that ensure efficient execution. In contrast to prior work, our methodology adopts a bottom-up perspective, addressing execution from the core level up to the chiplet-level coordination to enable scalability and flexible dataflows. Moreover, we assess and recommend hardware setups optimized for different classes of LLMs, offering practical guidance for selecting architectures best suited to specific models. Through cycle-accurate experimental evaluation, we demonstrate that our approach achieves up to 4× higher token throughput and 30× better energy efficiency compared to state-of-the-art H100 GPUs on the same workloads.

Fields of Study & Subjects

Parallel Computing and Optimization Techniques Advanced Memory and Neural Computing Embedded Systems Design Techniques

Publication details

Journal / source
IEEE Journal on Emerging and Selected Topics in Circuits and Systems
Volume
16
Issue
2
Pages
500–513
Publication date
25 March 2026
Language
EN