Hierarchical reinforcement learning framework for adaptive network slice resource orchestration in heterogeneous base station systems

The proliferation of IoT devices drives mobile networks toward heterogeneous architectures combining macro and small base stations, where network slicing enables flexible resource block allocation to optimize multiservice quality under dynamic user access scenarios. This paper presents a comprehensive framework integrating greedy marginal utility algorithms, hierarchical Markov decision processes, and game-theoretic mechanisms for network slice resource orchestration. For static allocation, the greedy algorithm achieves optimal distribution with eMBB receiving 22 blocks, URLLC 18 blocks, and mMTC 10 blocks, validated through grid search. Under dynamic scenarios, a two-layer hierarchical MDP combined with Q-learning realizes adaptive scheduling across 10 decision windows with 100ms time steps, maintaining stable resource ratios and achieving overall QoS of 9.18. For interference management, cooperative game theory coordinates power control among small base stations, stabilizing average power at 15-25 dBm while significantly reducing interference. In heterogeneous macro-small base station scenarios, non-cooperative game-based user association enables joint optimization where macro-stations maintain 28-30 dBm and small stations 17-22 dBm. Multi-objective optimization balancing QoS and energy consumption reduces energy by 23.1% compared to single-objective approaches while maintaining service quality, demonstrating effective performance-efficiency tradeoffs.