Enhanced Deep Q-Learning through Adaptive Experience Clustering for Vehicular Task Offloading

With the increasing adoption of AI in connected and autonomous vehicles (CAVs), deep reinforcement learning (DRL) has emerged as a core approach for decision-making in dynamic vehicular edge computing (VEC) environments. These settings require rapid adaptation to changing mobility patterns, network conditions, and resource availability. However, conventional experience replay strategies such as uniform sampling and prioritized experience replay (PER) struggle to support efficient learning under such nonstationary conditions. To address this, we propose enhanced Deep Q-Learning through experience clustering mechanism, referred to as ARCS (Adaptive Reward Clustering and Sampling) which is a reward-guided replay mechanism that periodically clusters experiences based on reward distributions and prioritizes sampling from transitions near cluster centroids. This design filters out outdated or low-utility experiences, promoting stable, sample-efficient learning for Q value estimation in DQN network. Extensive simulations show that ARCS accelerates convergence while maintaining high task performance and energy efficiency. Compared to Uniform and PER, ARCS requires far fewer transitions, achieves approximately 5% higher task completion, 4% lower energy consumption, and maintains performance with a model architecture 16% smaller in size. These results underscore ARCS's scalability and effectiveness for resource-constrained, high-variability edge learning scenarios.