Adaptive Beam Steering in WPCNs under Slotted ALOHA via Deep Q-Learning

In this article, we propose a Deep Q-Learning-based approach to improve the throughput of a Wireless Powered Communication Network (WPCN) by controlling the transmissions of energy harvesting devices through adaptive beam steering. We examine the dynamics of the slotted ALOHA protocol in conjunction with the charging process of energy harvesting devices to develop a framework for maximising throughput through beam steering, despite partial observability constraints, such as unknown device locations and current charge levels. We design a Deep Q-Network that learns the optimal beam steering policy only from the past observations of transmission attempts by the energy harvesting devices. Extensive simulations illustrate that our proposed approach offers as much as 26% improvement in throughput over deterministic round robin beam steering and 9% over random beam steering strategies for a dense network size of 50 devices. The performance is found to improve with increasing length of past observations.