AoI-Aware Collaborative Data Caching in IIoT: A Multi-Agent Reinforcement Learning-Based Approach

To cope with the challenges of the rapid growth of Industrial Internet of Things devices, deploying cache servers (CS) at the edge of the network becomes a promising solution. The cache node (CN) can reduce request latency and the traffic load on backhaul link by pre-caching relevant data generated by sensors. However, the storage resources of CSs are limited, and select the appropriate data item to cache is a critical issue for improving efficiency. In addition, we introduce the Age of Information (AoI) to describe the freshness of cached data. However, increasing the freshness of data through frequent caching updates could result in increased energy consumption of sensor. In this paper, we study the caching problem of multiple CNs, and further model the problem as a multi-agent Markov Decision Process to minimize long-term request latency, AoI, and energy consumption. Furthermore, to solve this problem, we propose a variant of the Soft Actor-Critic-based multi-agent caching algorithm. Finally, experiments show that the proposed algorithm can converge to a better caching strategy and has better performance in multiple objectives than existing algorithms.