Foundation Deep Learning Models For Precision Medicine Using Multimodal Big Data

Deep learning models pretrained on multimodal Big Data may be used to address many problems in Precision Medicine. Choi et al. (2022) provided background on the core model architectures and training paradigms; on the prevalent types of multimodal medical data; and on the preliminary works aimed at combining model frameworks and training paradigm across existing modalities. Multimodal Foundation Models are introduced on the basis of learned medical representation and multimodal representation transfer and design. The explored areas are centered on data ecosystems that can support the future development of Deep Learning within Precision Medicine using multimodal Big Data. As subtitled, the section on Data Ecosystems and Governance focuses on data acquisition, curation and quality assurance, privacy, security and ethical considerations. Applications involve the integration of genomics and transcriptomics; the use of medical image and radiomic data; and the building–testing of benchmark datasets to alleviate data bias. The addressed methodological challenges discuss data bias, fairness and generalizability; interpretability and clinician trust; benchmarking protocols; reproducibility; and open science practices.