OT-Fusion: cortex-conditioned fusion of matched PET and MRI images using optimal transport

Abstract Positron Emission Tomography (PET) and Magnetic Resonance Imaging (MRI) are crucial in diagnosing various medical conditions related to brain. FDG-PET provides essential functional information by capturing synaptic transmission activity, while MRI offers anatomical details. The integration of these modalities has become increasingly valuable, especially with the advent of hybrid PET/MRI systems that enable acquisition of both images in a single session. However, a notable challenge arises due to the inherent differences in spatial resolution between PET and MRI. PET images often have lower spatial resolution, leading to metabolic signals from active regions, such as the cerebral cortex and basal ganglia, appearing diffused and extended into adjacent areas like the white matter. This diffusion can complicate the accurate localization of metabolic activity, posing difficulties in clinical assessments, particularly in neurological pathologies such as epilepsy where precise mapping is crucial. To address this issue, we propose a novel approach that effectively maps the metabolic activity observed in PET onto the cortical structures delineated in MRI. We formulate this mapping as an optimal transport problem that corresponds to finding the optimum way to transform one probability distribution into another. Our method aims to enhance the structural quality of PET images, making them more detailed and informative for clinical decisions related to neurological disorders. The experiments using paired PET and MRI images indicate that our method performs better than the state-of-the-art image fusion methods on quantitative metrics and improves image quality based on visual qualitative evaluation by an expert physician.