Autonomous Exploration in Unknown Environments With Mobile IoT Device: An Intelligent Reward Strategy Cloning Approach

Autonomous exploration in unknown environments is a fundamental capability for intelligent mobile IoT systems, especially in scenarios where prior environmental information is unavailable. In such settings, mobile IoT devices are required to achieve safe and efficient full-area coverage based solely on onboard sensing and limited computational resources. However, the unpredictable and continuously changing nature of unknown environments poses significant challenges to adaptive and collision-free exploration, particularly for resource-constrained mobile IoT devices. To address these challenges, we propose an autonomous full-area exploration algorithm for mobile IoT devices based on reward strategy cloning. Specifically, a state representation method using color mapping is designed to improve the information intensity of input state in full-area coverage exploration missions. Simultaneously, to address the issue of sparse rewards in full-area exploration missions, we construct an intensive reward shaping function that integrates exploration rewards, collision penalties, and incentives for exploring frontier trends. Furthermore, a lightweight exploration model that maps state to action reward is designed for mobile IoT devices with limited computing power and storage resources. Moreover, we propose a reward-sensitive dynamic ϵ-greedy strategy that adaptively balances exploration and exploitation based on real-time performance trends. Finally, empirical results demonstrate the robustness of the proposed algorithm in exploring various complexities and dynamic environments. In particular, the computational complexity of the proposed exploration model is significantly reduced compared to other models.