A Constrained Decision Transformer Framework for Fast Adaptive Optimization in RIS-Assisted Wireless Communications

Reconfigurable intelligent surface (RIS) has been recognized as a revolutionary technique for reconfiguring wireless environments with low cost and high array gain. However, achieving a high passive beamforming gain requires a large number of activated reflecting elements, incurring non-negligible power consumption. To balance the fundamental tradeoff between transmission rate and energy consumption, we jointly optimize the base station (BS) transmit power, RIS beamforming, and RIS element switching control. The objective is to maximize the average achievable rate while satisfying constraints on the transmit power of the BS, the average power of the RIS, and the unit modulus of the RIS phase shifts. We formulate this problem as a constrained Markov decision process (CMDP) and propose the proximal policy optimization-Lagrangian (PPO-Lag) algorithm. Furthermore, to improve the adaptability of decision-making models across various scenarios, we develop a constrained Decision Transformer (CDT) framework that integrates offline pre-training with online fine-tuning. Specifically, we first propose a PPO-Lag algorithm to solve the resource allocation for RIS systems. Then we collect samples based on PPO-Lag in varying scenarios, i.e., different power constraints and numbers of RIS elements. Afterwards, CDT is pre-trained based on the collected offline samples and fine-tuned online utilizing a limited number of samples under the new scenario to enhance its generalization ability. Simulation results show that the proposed CDT framework can improve the achievable rate by 4.4% and converges four times faster compared to the benchmark algorithm.