Synthesis of Reconfigurable Intelligent Surface Patterns With Deep Reinforcement Learning

We propose a deep reinforcement learning (RL) method for three-dimensional (3-D) field pattern synthesis and first demonstrate it on a 1-bit reconfigurable intelligent surface (RIS). Desired patterns are specified as lower/upper 3-D masks on the normalized scattered field. A U-Net-based neural network maps these masks to RIS configurations. This mask-to-configuration (M2C) network includes several tiny U-Nets stacked together. For a lower/upper mask pair, these “stacked U-Nets” iteratively update a grid of logits, i.e., unbounded real values whose sigmoids give probabilities for each RIS cell to work in a mode of operation (either “0” or “1”). A subsequent “multi-head U-Net” produces multiple alternative logits (“heads”), from which several candidate configurations are drawn by probabilistic sampling. The RL loop is: 1) the policy (i.e., the M2C network) proposes candidate configurations; 2) an efficient, mutual coupling-aware model of RIS scattered fields (empowered by a convolutional neural network) evaluates each one with a mask-violation loss; 3) the best candidate’s reward (negative loss) drives a policy gradient update. No pre-collected dataset is required to train the M2C network. After training, the M2C network produces satisfactory configurations for unseen masks in real time; a brief local search can further refine outputs with small mask violations. A case study that demonstrates the extension of the proposed method to a multi-bit RIS is also presented. Our method removes the need for heavy dataset preparation of existing supervised machine learning methods and avoids the repeated runs and long runtime of search algorithms (such as genetic algorithms), while achieving high-quality 3-D pattern synthesis.