UDRSim: an underwater dynamic real-time multimodal imaging simulator

Underwater robots typically utilize forward-looking sonar (FLS) and visible light sensors for task execution. Perceptual image data underpin obstacle avoidance, detection, planning, and decision-making, where real-time underwater multimodal data are particularly critical to advance intelligent decision-making in underwater robotics. However, marine environmental complexity and high experimental costs lead to scarce underwater data, exemplified by the small-sample problem. Current methods lack real-time perception capabilities in dynamic underwater environments. This paper proposes UDRSim, a multimodal image simulator leveraging the Unreal Engine (UE) platform and raytracing methods, which can solve moving target imaging simulation underwater. We develop a comprehensive FLS reflection intensity model that incorporates acoustic spreading loss, absorption loss, reflection angles, and target wave impedance. This approach enables efficient, physics-aware sonar data generation. We enhance UE’s native visible light sensor to simultaneously produce underwater optical images with FLS data. Our evaluation of ray counts and tracing methods determines that Single Line Trace by Channel is optimal for sonar simulation. At 7,200 rays, UDRSim achieves 5.5 FPS on an NVIDIA RTX 2000 Ada GPU, balancing efficiency with quality. Real-time data generation supports automated parametric annotation and storage. Within one hour, UDRSim generates more than 1,800 sets of annotated multimodal image data for identical underwater targets from diverse perspectives.