Actively Adaptive Bandit Algorithms for Beam Tracking in mmWave Networks

Beam acquisition and maintenance in millimeter-wave (mmWave) cellular networks present significant challenges. User mobility and environmental blockages necessitate beams to be regularly updated to ensure a certain quality of service, making beam tracking a crucial aspect of mmWave beamforming. To this end, we propose a multi-armed bandit based algorithm for beam tracking in mmWave networks for a single user scenario. In this algorithm, we propose a sliding window based technique to detect changes in the mmWave channel and update the beams. Furthermore, we consider the beam tracking problem when multiple users are associated with a single base station. In this scenario, the presence of inter-beam interference leads to additional challenges. We propose the use of matroid bandits, a combinatorial optimization technique, with an objective of maximizing the sum of spectral efficiencies of users. Towards this, we propose another multi-armed bandit based algorithm for the simultaneous selection of beams for multiple users. We evaluate the proposed algorithms on mmWave channel samples generated using the simulator NYUSIM, and compare the results with state-of-the-art bandit-based tracking algorithms.