Feature Enhancement Method for RGB-D Image Through Convolution over Plane Residuals and Plane Parameters

We propose a feature enhancement method for depth images that applies convolutions over residuals and parameters obtained from dominant plane. The proposed method can obtain the initial features of depth images that are less sensitive to surface orientation and more representative of intrinsic geometric properties. Specifically, the features are obtained through the plane-based convolution that performs operation on residuals with respect to the dominant plane within a local patch of the depth image. For each patch, a dominant plane is fitted to the corresponding depth pixel values using a least-squares method. Then, convolutional operations are performed on plane residuals computed between the original depth values and the corresponding depth values on the dominant plane. In addition, standard convolution is applied to the dominant plane parameters to capture local variations and spatial consistency of surface orientation. A plane-based convolution module incorporating these convolutions is attached to the initial layer of the existing feature extractor in parallel to supplementarily obtain surface geometric features. Experiment results demonstrate that the proposed method consistently achieves performance gains on both segmentation and classification tasks.