Joint Throughput And Energy Optimization In GSM Networks Using Watts–Strogatz Graph Modeling And Reinforcement Learning

In this paper, the problem of simultaneous optimization of throughput and energy consumption in GSM networks is studied as a multi-objective problem. For realistic modeling of the network structure, the cell topology is represented as a “small world” graph using the Watts–Strogatz stochastic graph model, and by exploring the graph, important structural and traffic features of each cell, including degree, centrality, traffic load, and QoS indices, are extracted. These features, together with the instantaneous channel information and the power status of the base stations, form the state space of a deep reinforcement learning agent. Using a neural network-based Q-value function approximation, the DRL agent learns control actions such as adjusting the transmit power, turning cells on/off, and routing traffic between neighboring cells in such a way that a reward function including normalized throughput, normalized energy consumption, and QoS violation penalty is maximized. Simulation results in the scenario of one macro cell and 12 micro cells with 200 to 2000 users show that the proposed method has a significant advantage over the three reference methods (basic, heuristic and classical Q-learning); so that the average cumulative reward of the DRL algorithm reached 3.41, while for Q-learning and the heuristic method it was 2.28 and 0.7, respectively, and for the basic method it was reported to be negative 3.55. Also, the network energy efficiency, in terms of the ratio of total throughput to base station power consumption, is maintained at a stable level of close to 2.5 for the proposed method, while this value is about 2.05 for Q-learning and 1.56 and 1.21 for the two non-intelligent methods, respectively. These results indicate that the combination of small-world graph modeling and deep reinforcement learning can provide an efficient framework for intelligent energy and capacity management in GSM networks.