Enhancing scene understanding using RGB‐D visuals and deep learning segmentation models

Abstract Machine intelligence has advanced rapidly, and researchers strive to endow robots with cognitive and perceptual abilities similar to those of humans. Although machines collect and analyze sensor data, a large gap remains in interpreting complex scenarios in the real world. Hence, we propose a model for scene comprehension using depth images only in indoor and outdoor settings. This method closely resembles human perception and comprehension, allowing robots and machines to analyze scenes in real time. It has high potential for applications where precise scene detection from depth data is crucial, such as robotics, autonomous navigation, and augmented reality. The proposed system performs reliable multi‐object segmentation on depth images using the SegNet deep learning model. To enhance scene recognition accuracy, entropy‐based features are extracted from segmented regions and fed into a convolutional neural network classifier. Our method, which combines deep learning‐based segmentation, convolutional neural network classification, and distinctive feature engineering, substantially improves depth‐based complex multi‐object scenario interpretation, surpassing the state of the art.