Covert Transmission for H 2 AD MIMO-Based ISAC Systems With Deep Reinforcement Learning

A covert ISAC transmission scheme based on an innovative heterogeneous sub-connected hybrid analog and digital (H2AD) multiple-input multiple-output (MIMO) transceiver is investigated in this paper. Specifically, H2AD ISAC system possesses the capability to detect the point-like target while covertly transmitting confidential information to a singleantenna legitimate user, and enabling secure transmission without detection by the warden. The objective is to maximize the covert transmission rate for legitimate users while adhering to the Cramér-Rao bound (CRB) threshold. However, due to the coupling of multiple variables under the H2AD transceiver framework, the optimization problem becomes non-convex. To tackle the challenging, an alternating optimization algorithm based on Dinkelbach’s transformation and semidefinite relaxation (DTSDR) is proposed to design the analog and digital beamforming along with the sensing signal. Then, by utilizing historical system states and optimizing for long-term returns, an improved distributional soft Actor-Critic with three refinements (DSACv2) algorithm framework based on deep reinforcement learning (DRL) is proposed. Simulation results demonstrate that incorporating the novel H2AD MIMO antenna array into ISAC system design enhances the covert performance while ensuring target sensing performance.