Sustaining the AI Revolution With Analog Compute-in-Memory [Industrial and Governmental Activities]

Artificial intelligence (AI), once a mere figment of science fiction, has now entrenched itself deeply into our lives, reshaping everything from adjusting the tone of your email to assisting scientists analyzing deep space images captured by the James Webb space telescope. However, beneath the glorious appearance of the rapid development of AI technology, increasingly serious environmental problems are gradually emerging. To train and operate AI systems, power demand from data centers has skyrocketed, having tripled in just five years and being expected to triple again by 2030 [1]. This staggering increase starkly contrasts with less than 3% of annual global electricity production growth witnessed over the past 40 years, far outpaced by AI's ballooning energy needs [2], as shown in Figure 1 left. If we continue down this path relying on traditional processors, the growth of large language models (LLMs) may soon reach an unsustainable ceiling. This urgency is amplified by the reality that early efficiency gains in AI hardware are plateauing. Data centers nowadays principally utilize GPUs for both training and inferences. While GPU manufacturers often highlight improvements in efficiency—claiming a 40,000x improvement over the past decade—these numbers obscure several underlying concerns [3]. Upon scrutinizing the sources of the improvement, it becomes evident that sustaining such a pace of progress will be fraught with challenges. The main drivers of efficiency enhancement stem from circuit/architecture innovations, semiconductor process advancements, and data format optimizations.