Generalized Multi-Granularity Design of mmWave and THz On-Chip Antennas Based on Enhanced Reinforcement Learning

This paper presents a generalized multi-granularity design of Millimeter-wave (mmWave) and terahertz (THz) on-chip antennas based on improved reinforcement learning (RL). The proposed framework takes the advantages of the modeling strategy, enhanced RL, and multi-granularity method, enabling high-degree-of-freedom on-chip antenna design with low-dimensional modeling parameters. Firstly, a parametric modeling strategy is introduced to achieve a pixelation-like design effect but circumventing high data quality and quantity requirements typically associated with conventional pixel-based methods. Secondly, the enhanced RL uses a Maximum Entropy Actor-Critic (MEAC) algorithm to improve convergence speed, robustness, and efficiency. Thirdly, field theory analysis is incorporated to decompose the optimization process into coarse and fine granularities, effectively minimizing redundant modeling in non-essential regions. This method is applied to D-band on-chip antenna design for the commercial1 65nm CMOS process and W-band for the commercial2 55nm CMOS process. The experimental results show that the D-band on-chip antenna achieves a relative bandwidth of 50.9% (100-177 GHz), a peak gain of 2.4 dBi, and a radiation efficiency of 62%, respectively, while the W-band achieves a relative bandwidth of 30.6% (69-95GHz), peak gain of 6.4dBi, and radiation efficiency of 72%. Compared with existing studies, the multi-granularity with enhanced reinforcement learning significantly improves bandwidth, gain, and radiation efficiency, with strong generalizability across different frequency bands and fabrication processes.