Depth map curvatures as pose-invariant features for RGB-D object recognition

Computer vision tasks, such as object recognition, using deep learning find their place in a variety of contexts including agriculture. Regarding data, the coupling of RGB and depth modalities has already proven to be beneficial for object recognition over the use of RGB-only images. However, the lack of neural network architectures and large-size datasets dedicated to the depth modality forces us to use backbones pre-trained on RGB data using large datasets such as ImageNet. While works proposed by Eitel et al. and Aakerberg et al. rely on colorizing the depth values to match an RGB format, they do not take full advantage of the geometric properties carried by the depth modality. We demonstrated principal curvatures when used to color-encode the depth values retain more information related to the object's shape. The proposition was evaluated on two datasets: Washington RGB-D and a homemade synthetic dataset. With the introduction of superclasses based on the geometric shape of objects (sphere, cylinder, cube, …) from the Washington RGB-D dataset our model performed higher than the previous work, eg. 3.1% precision increase for the sphere superclass. Results obtained using our synthetic dataset have demonstrated the better generalization capability of our curvature-based approach. While presenting some limitations, this work opens the path for further developments.