Decentralized Federated Multi-Agent Reinforcement Learning for RAN Controller Orchestration in 6G

6G mobile networks are increasingly using Artificial Intelligence to manage highly dynamic environments characterized by time-varying traffic demands, user mobility, and heterogeneous resources. The dynamic behavior of User Equipment makes timely and accurate control decisions challenging, while distributed data exchange introduces communication overhead and privacy concerns. These challenges call for scalable and communication-efficient learning mechanisms for Radio Access Network (RAN) orchestration. In this paper, we propose DERRIC-FRL, a decentralized Federated Reinforcement Learning framework to orchestrate RAN intelligent controllers. DERRIC-FRL jointly optimizes controller placement and user power allocation through a selective two-level aggregation mechanism, reducing data exchange to only selected orchestrators and controllers while preserving user privacy and improving overall network performance. Specifically, our method significantly reduces total training communication costs by 34% across inter-domain connections, and up to 77% across intra-domain connections, compared to the FedAvg approach. Furthermore, DERRIC-FRL improves user throughput by up to 53% and 61% compared to the DERRIC and FedAvg baselines across a broad range of simulated scenarios.