Integrating Proximal Policy Optimization with Physically Realistic Simulation for Robust Autonomous Underwater Vehicle Control

Read full text Summarise with VIRAT
Authors: MAHMUT MOL, Ahmet Karaarslan
Cited by: 0
References: 27

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This study presents the design and implementation of a reinforcement learning (RL)-based framework for the control of an autonomous underwater vehicle (AUV) directly within Unreal Engine (UE). A high-fidelity aquatic environment was created using UE’s native Water System to simulate hydrodynamic forces and buoyancy. Unlike studies assuming continuous control, this research addresses the challenge of stabilizing an AUV subject to severe discrete 'bang-bang' hardware constraints. A parallelized Proximal Policy Optimization (PPO) algorithm was employed to synthesize adaptive control policies. Comparative analysis against tuned Proportional-Integral-Derivative (PID) baselines demonstrates that the RL agent outperforms classical methods in three key metrics: (1) in longitudinal navigation, the agent learned an emergent "pulsing" strategy—mimicking Pulse-Width Modulation (PWM)—to overcome these discrete actuation constraints, reducing steady-state error compared to the Proportional-Derivative (PD) baseline; (2) in vertical depth control, the agent autonomously learned gravity compensation, settling faster than integral-based controllers while avoiding buoyancy-induced stalling; and (3) in heading control, the agent demonstrated superior dynamic handling, completing stabilization maneuvers faster than the baseline. The key architectural innovation lies in the direct integration of UE’s Learning Agents plugin, eliminating the need for external middleware. This native integration enables real-time synchronization between simulation physics and learning processes, establishing a high-fidelity platform for developing intelligent underwater control systems.

Fields of Study & Subjects

Reinforcement Learning in Robotics Underwater Vehicles and Communication Systems Maritime Navigation and Safety

Publication details

Journal / source
Gazi University Journal of Science Part A Engineering and Innovation
Volume
13
Issue
1
Pages
165–199
Publication date
31 March 2026
Language
EN