Optimization of Interaction Experience in a Smart Home Control System Based on Pose Estimation: Proposing a New Method

The control of household devices relies heavily on physical interaction, with its high frame rate, lower latency, and real-time response to user operations. With the increasing number of artificial-intelligence (AI) models being used in smart home control systems (SCSs), there are two problems that limit the application of SCS: 1) AI models usually need to be processed in the cloud, which leads to high communication latency with smart home devices, and 2) the running of AI models on edge devices is usually limited by insufficient computational power. To address the problems, a pose estimation-based SCS (PE-SCS) is investigated in this study. First, a logic-based action judgment method with visual feedback is proposed to comprehensively judge the PE of consecutive frames and the response of the visual interface so that the action signal of SCS is under control. Then, an optimization method calledOM-HCIoPEis proposed to reduce the dependence on model output through parallel prediction in order to improve the reliability and the performance of the system. A series of experimental evaluations shows that the proposed method achieves a higher frame rate, lower latency, and real-time response to user operations, which provides a user-friendly experience for a wide range of audiences.