NMPC-Augmented Visual Navigation and Safe Learning Control for Large-Scale Mobile Robots

A large-scale mobile robot (LSMR) is a high-order multibody system that often operates on unconsolidated terrain, which reduces traction. This paper presents an integrated navigation and control framework for an LSMR that ensures stability and safety-defined performance on slip-prone terrain by combining high-performance methods. The proposed architecture comprises four main modules: (1) a visual pose-estimation module that fuses onboard sensors and stereo cameras to provide an accurate, low-latency robot pose, (2) a high-level nonlinear model predictive control (NMPC) that updates the wheel motion commands to correct robot drift from the robot reference pose, (3) a low-level deep neural network control policy that approximates the complex behavior of the wheel-driven actuation mechanism in LSMRs, augmented with robust adaptive control to handle out-of-distribution disturbances, ensuring that the wheels accurately track the updated commands issued by high-level control module, and (4) a logarithmic safety module to monitor the entire robot stack and guarantee safe operation. The proposed actuator-level control framework guarantees uniform exponential stability for tracking the NMPC-generated commands, while the safety module ensures overall system safety during operation. Comparative experiments were conducted on a 6,000 kg LSMR actuated by two electro-hydrostatic drives.