When AI Meets Cache: Intelligent Memory Hierarchy Management in Modern CPUs

Read full text Summarise with VIRAT
Authors: Yihan Wang, Suchuan Xing
Cited by: 0
References: 56

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

The persistent and widening gap between processor execution speed and memory access latency—commonly termed the "memory wall"—has made efficient cache management one of the most consequential challenges in modern computer architecture. Traditional heuristic-based strategies such as least recently used (LRU), re-reference interval prediction (RRIP), and signature-based hit prediction (SHiP) have long served as foundations of cache control, yet they fundamentally fail to capture complex, application-specific, or workload-diverse memory access patterns. The rapid maturation of artificial intelligence (AI) and machine learning (ML) techniques now offers transformative opportunities to reimagine every tier of the memory hierarchy—from L1 instruction caches to last-level cache (LLC) partitioning and speculative hardware prefetching. This review surveys the state of the art at the intersection of AI and cache management, examining techniques including deep reinforcement learning (RL), long short-term memory (LSTM) networks, transformer-based sequence models, and graph neural networks (GNN) as applied to the three principal problem domains of cache replacement, hardware prefetching, and adaptive resource partitioning. We critically analyze trade-offs between prediction accuracy, inference latency, hardware overhead, and practical deployability, synthesizing empirical findings from recent simulation studies and silicon implementations. Key open challenges are identified, including online learning constraints, workload generalization, and the integration of AI inference within microarchitectural timing budgets.

Fields of Study & Subjects

Parallel Computing and Optimization Techniques Network Packet Processing and Optimization Caching and Content Delivery

Publication details

Journal / source
Jisuanji shenghuojia.
Volume
14
Issue
1
Pages
19–26
Publication date
25 March 2026
Language
EN