Chest X-ray deep learning comparative diagnosis using visual and semantic similarity with variational autoencoders

Medical image retrieval enhances the diagnostic process by providing physicians with relevant visual references, thereby supporting clinical decision-making, particularly in complex cases. In this study, we introduce a novel custom-built system that improves retrieval models in medical tasks, focusing exclusively on image information. The proposed network simultaneously focuses on the anatomical and pathological features of each case, obtaining a precise and easily interpretable recommendation. Results obtained using the PadChest dataset show that the proposed architecture outperforms previous architectures improving diagnosis by 0.68–12,93 percentage points using the precision@k and visual similarity between 0.0169–0.0648 points of the Sliced Wasserstein distance. From a clinical perspective, the results suggest that the recommended cases present relevant information, both semantically and visually similar and could be used by professionals as an assistant tool.