A Low-Cost Multi-Objective Cache Prefetcher for Complex and Irregular Memory Access Patterns

Hardware prefetching in processors, as an effective solution to hide the cache miss latencies, needs to be redesigned according to emerging applications behavior. Conventional prefetchers try to predict predefined patterns but are unable to prefetch a mixture of even simple detectable patterns, e.g., access pattern of convolution operation in neural networks, or randomness in workload behavior. In this paper, we expose and define new memory access patterns and propose a prefetcher capable of detecting them in addition to its capability to detect all patterns detectable by existing prefetchers in L2 caches. The proposed scheme, namedLearning-based Adaptive Multi-objective Prefetcher(LAMP), not only supportsintra-pagepatterns but also detectsinter-pagepatterns as its unique feature. It is also able to extrapolate a pattern from one page to the other pages by detecting a relation between them, offeringcross-pageprefetching. The other feature is to detectrandom accessesto a limited memory area as well asirregular traversalof a memory address path. LAMP learns the patterns in less than or equal to three block accesses offering the minimum learning time not to miss prefetching opportunities. It adaptively adjusts the prefetching parameters and decisions to maximize efficiency via a tiny tunable learning-based engine. The tradeoff between accuracy and coverage can be tuned with the LAMP mechanism, which can (automatically) adjust the prefetch aggressiveness and depth. The evaluation demonstrates that the proposed approach surpasses existing state-of-the-art methods by achieving superior coverage and accuracy. Relative to the baseline, it improves performance by 17.3%, 27.0%, and 7.6% on the SPEC CPU2017, GAP, and CloudSuite benchmark suites, respectively. In addition, the scheme reduces cache energy consumption by lowering the proportion of ineffective prefetching requests from over 43% in prior methods to below 26%, while incurring substantially less hardware overhead.