Optimal Policy Determination for Autonomous Underwater Robots Using MDP and POMDP Frameworks

Read full text Summarise with VIRAT
Authors: Janakkumar Patel
Cited by: 0
References: 8

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Autonomous underwater robots operating in dynamic and uncertain ocean environments must navigate safely and efficiently toward predefined objectives while avoiding obstacles. This paper investigates the use of Markov Decision Process (MDP) and Partially Observable Markov Decision Process (POMDP) frameworks as foundational tools for optimal policy determination in such cyber-physical systems. A two-dimensional 5x5 grid world models the seafloor environment, incorporating stochastic transitions that reflect real-world disturbances such as tides, currents, and sensor noise. Value iteration is applied to derive the MDP optimal baseline policy, which serves as an upper-bound benchmark for the POMDP solutions. Three offline POMDP solvers — QMDP, Fast Informed Bound (FIB), and Successive Approximations of the Reachable Space under Optimal Policies (SARSOP) — are evaluated under four observation model accuracy levels ranging from 70 to 100 percent. Monte Carlo simulations involving 15,000 independent trials are used to assess each solver. Results confirm that expected rewards increase with improved observability across all three methods. SARSOP consistently provides the tightest balance between computational tractability and solution quality. The analysis reveals that the greedy action selector, rather than the solution methods themselves, is the primary driver of performance variability across observation models. These findings contribute to the growing body of work on decision-making under uncertainty for autonomous marine cyber-physical systems and suggest several directions for practical enhancement.

Fields of Study & Subjects

Underwater Vehicles and Communication Systems Reinforcement Learning in Robotics Robotic Path Planning Algorithms

Publication details

Journal / source
SSRN Electronic Journal
Publication date
1 January 2026
Language
EN
Document type
Preprint