SiPAM: Silicon Photonic Accelerated Memory Pooling Architecture

Training and inference of large-scale machine learning models demand significant processing power, memory capacity, and memory bandwidth. Current compute units are equipped with a fixed amount of high-bandwidth memory (HBM) modules placed near the compute die which limits the system’s flexibility to adapt to different compute and memory demands. In this work, we present SiPAM, a silicon photonic accelerated memory pooling architecture that integrates dense wavelength-division multiplexing-based photonic I/Os directly along the compute die’s shoreline. This design creates a unified high-bandwidth communication domain that decouples memory and network scaling from packaging constraints and enables demand-driven dynamic resource allocation. We develop a quantitative optimization methodology to allocate compute and memory resources efficiently, with up to 3.5× faster iteration time and 4.5× reduction in energy consumption compared to state-of-the-art compute systems.