Using multimodal PET+MR data as conditional generative adversarial network inputs improves pseudo-CT and attenuation correction estimates for brain PET/MR

To obtain qualitatively and quantitatively accurate positron emission tomography (PET) images, the recorded PET emission data must be corrected for photon attenuation. Attenuation correction (AC) factors are typically estimated from X-ray computed tomography (CT) data acquired during an integrated PET/CT study. Estimating these factors from magnetic resonance (MR) data in an integrated PET/MR scanner is challenging, as MR images don't provide direct information about annihilation photon attenuation. Conditional generative adversarial networks (cGANs) have shown promising results for both emission-based and MR-based AC. This study explored whether combining these approaches could further improve brain PET AC accuracy. Thirty-five patients who received same-day whole-body PET/MR and PET/CT scans participated in this study. The non-attenuation-corrected and non-scatter-corrected (NASC) PET, MR, and CT reconstructed head regions were cropped and automatically co-registered. Four networks were trained to translate NASC PET and MR images into pseudo-CTs. Three used single-modality input, and the fourth used multi-modality. The multi-modality cGAN produced significantly better pseudo-CTs vs. the single-modality cGANs, with an average structural similarity index (SSIM) and dice similarity coefficients for bone, soft-tissue, and air of 0.865±0.001, 0.715±0.002, 0.915±0.001, and 0.567±0.004, respectively, vs. 0.841±0.001, 0.660±0.003, 0.894±0.001, and 0.524±0.005, for the single-modality cGANs with the best results. When comparing the AC PET reconstructed images, all cGANs outperformed the clinical atlas-based method used in commercially available PET/MR systems, and, as expected, the multi-modal cGAN achieved the highest quality results with average SSIM, and peak signal-to-noise ratio of 0.9987±0.0001, and 50.0±0.4, respectively, vs. 0.9913±0.0024, and 44.3±0.3 for the atlas method.