Building an Intelligent Home Perception System Based on Multi-Modal Information Interaction

Read full text Summarise with VIRAT
Authors: Guo Zhanmiao, Qian Zhongli
Cited by: 0
References: 21

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

In response to the problems of single interaction modality and weak perception ability in traditional smart homes, this paper proposes a multi-modal information perception Artificial Intelligence(AI) model invocation framework. It schedules visual, voice, and sensor data through natural language prompts, and combines the zero-shot visual recognition method of the cloud-based visual-language hybrid large model workflow to achieve cross-scene generalization ability without labeled training. This framework can innovatively solve the problems of heterogeneous data fusion and insufficient computing power of edge devices. Experimental results show that the multi-modal smart home perception system designed in this paper achieves an accuracy rate of over 90% in environmental perception and a precision rate as high as 92% in user intention recognition, which can provide new ideas and practical foundations for the multi-modal perception of future smart home technology.

Fields of Study & Subjects

IoT-based Smart Home Systems Multimodal Machine Learning Applications Context-Aware Activity Recognition Systems

Publication details

Journal / source
ICST Transactions on Scalable Information Systems
Volume
12
Issue
8
Publication date
31 March 2026
Language
EN