Automated Window and Blind States Detection Using Building Façade Images and Faster R-CNN

Images of building façades, captured using commonly available devices such as smartphones and digital cameras, can be utilized to analyze the states of windows and blinds in non-residential buildings. These states are determined by identifying the open and closed sections of windows and blinds. However, manual classification of such images is labor-intensive and time-consuming. Existing studies using traditional and deep learning methods have made significant efforts to automate this process, but the accuracy in the state detection remains low due to inconsistent image characteristics of building facade, such as non-uniform lighting conditions and differences in window scale. To address these limitations, this study proposes a deep-learning-based approach using Faster R-CNN (Region-based Convolutional Neural Network). A specialized Faster RCNN architecture was developed to handle common window types (hinged and sliding) and common blind types (roller and Venetian). The performance of the proposed approach was evaluated using façade image datasets collected under diverse conditions from London, UK, and Seoul, South Korea. For hinged windows (open/closed), the model achieved an overall accuracy of 96.81% on the UK dataset and 95.52% on the South Korean dataset. For sliding windows (the opening area per window), accuracies were 93.76% for the UK dataset and 93.38% for the South Korean dataset. For blinds, the model accurately detected occlusion areas per window, achieving 93.75% (UK) and 93.37% (South Korea) for roller blinds, and 94.32% (UK) and 94.23% (South Korea) for Venetian blinds. Additionally, the classification of Venetian blind slat angles (open/closed) achieved 100% accuracy in both datasets. The findings demonstrate that the proposed Faster R-CNN-based model reliably detects window and blind states across various conditions of building façade images, with high accuracy and strong generalization between the two datasets. Based on these results, this research explores the potential applications of the proposed method in real-world scenarios.