Efficient Information Updates in Compute-First Networking via Reinforcement Learning With Joint AoI and VoI

Timely and efficient dissemination of service information is critical in compute-first networking systems, where user requests arrive dynamically and computing resources are constrained. In such systems, the access point (AP) plays a key role in forwarding user requests to a server based on its latest received service information. This paper considers a single-source, single-destination system and introduces a PPO–based reinforcement learning framework for efficient information updating, guided by a newly designed reward metric called Age-and-Value-Aware (AVA). Unlike traditional freshness-based metrics, AVA explicitly incorporates variations in server-side service capacity and AP’s forwarding decisions, allowing more context-aware update evaluation. Under this reward structure, the PPO agent autonomously learns when to trigger service information updates, achieving a dynamic balance between communication cost and decision accuracy. Extensive simulations under diverse user request patterns and varying service capacities demonstrate that AVA reduces the update frequency by over 90% on average compared to baselines, with reductions reaching 98% in certain configurations. This reduction is achieved without compromising the quality of decision making.