ACK-UCB: An Asynchronous Contextual Kernel-Based Bandit Approach for User Association in mmWave Vehicular Networks

Timely channel conditions are essential for vehicles to determine which base station (BS) to connect to, but acquiring them in mmWave vehicular networks is costly. Without additional channel estimations, the proposed asynchronous contextual kernelized upper confidence bound (ACK-UCB) algorithm estimates the current instantaneous transmission rates based on the historical transmission rates and contexts, such as the vehicle’s historical locations, velocities, and numbers of concurrent transmissions at the BS. ACK-UCB captures the nonlinear relationship between context and transmission rate, mapping the context into a reproducing kernel Hilbert space (RKHS), where a linear relationship becomes observable. To enhance estimation accuracy, a novel kernel function incorporating mmWave signal propagation characteristics is introduced in RKHS, allowing for a more precise evaluation of context similarity in relation to transmission rates. Furthermore, ACK-UCB encourages vehicles to share only reward distribution features after sufficient explorations, accelerating the learning process while keeping communication costs manageable. Numerical results show that ACK-UCB achieves 99.5%–100.5% network throughput and reduces 89%–91% communication cost of a benchmark algorithm that directly shares all local historical contexts and transmission rates, demonstrating the sharing efficiency of the ACK-UCB algorithm.