Fusing Channel State Information and Computer Vision in an Agentic AI Workflow for Accurate Human Activity Recognition and Crisis Prevention

This paper presents a novel multimodal framework for Human Activity Recognition (HAR) in homes that combines Channel State Information (CSI) from WiFi access points with visual data from camera images. This approach introduces a dual-stream classification pipeline that leverages a Transformer-based deep learning model for processing CSI features and a computer vision model based on a fine-tuned Convolutional Neural Network (CNN) for image-based activity classification. To enable autonomous decision-making, an agentic AI module based on the Falcon Large Language Model (LLM) has been incorporated, which fuses predictions from both modalities through a reasoning-driven mechanism. This integrated system is capable of recognizing activities such as walking, sitting, standing, and falling, and is particularly effective in scenarios requiring real-time monitoring and emergency detection. Agentic AI-based fusion is particularly important because false negatives can be risky. Experimental results demonstrate that the fusion of RF and visual cues significantly improves recognition accuracy compared to unimodal approaches, offering a robust solution for intelligent, non-intrusive HAR systems.