Exploiting multiple representations: 3D face biometrics fusion with application to surveillance

3D face reconstruction (3DFR) aims to recover a 3D face model from 2D images, providing richer representations that can enhance downstream face analytic tasks. Many 3DFR methods have been developed based on specific assumptions tailored to the limits and characteristics of the different application scenarios. However, their fusion remains largely unexplored despite the potential of ensembling to boost face recognition performance by leveraging the unique advantages of each model. In this study, we investigate how multiple state-of-the-art 3DFR algorithms can be used to generate a better representation of subjects, with the final goal of improving the performance of face recognition systems in challenging uncontrolled scenarios. We also explore how different parametric and non-parametric score-level fusion methods can exploit the unique strengths of multiple 3DFR algorithms to enhance biometric recognition robustness. For this purpose, we propose a comprehensive analysis of several face recognition systems across diverse conditions, such as varying distances and camera setups, intra-dataset and cross-dataset. The results demonstrate that the distinct information provided by synthetic views from different 3DFR algorithms can alleviate the problem of generalizing over multiple application scenarios. In addition, the present study highlights the potential of advanced fusion strategies to enhance the reliability of 3DFR-based face recognition systems, providing the research community with key insights to exploit them in real-world applications effectively. Although the experiments are carried out in a specific face verification setup, our proposed fusion-based 3DFR methods may be applied to other tasks around face biometrics that are not strictly related to identity recognition. • Combining multiple 3D face reconstructed models may boost biometric recognition. • This article explores the use of score-level fusion to exploit their complementarity. • It tests deep learning recognition systems across distances, cameras, and datasets. • Fusion improves accuracy and stability in challenging conditions like surveillance. • We provide guidelines to leverage the proposed fusion strategy in biometric systems.