Prevent fire risks with indoor surveillance videos of buildings: Develop a 3D convolutional neural network-based real-time fire and smoke detection model via red-green-blue and near-infrared feature fusion

Current indoor fire detection systems often have limitations, including insufficient intelligence and slow response times. Recent studies have started to explore AI vision-based methods for real-time fire and smoke detection to replace traditional fire sensors. However, indoor environments present many interference factors, and few methods can effectively demonstrate superior fire and smoke detection performance under multiple simulated interference conditions. To meet the requirements of real-time, high accuracy and anti-interference, a 3D convolutional neural network (3D-CNN)-based real-time fire and smoke detection model has been developed via red-green-blue (RGB) and near-infrared (NIR) feature fusion. RGB and NIR images with complementary information were used as dual-stream inputs to the model. The feature extraction module was enhanced by 3D attention mechanisms to simultaneously capture dynamic features between frames and static features within frames of fire and smoke. A non-linear feature gated fusion module has been developed to solve the challenges of missing multi-modal features and insufficient fusion. To validate the engineering effectiveness of the model, a large-scale multi-modal dataset was constructed in real indoor building scenarios, and a real-time indoor fire and smoke detection system was also developed and deployed at the experimental site.