Efficient Machine Learning for Scientific and Medical Applications

The rapid advancement of deep learning has been largely driven by the continuous growth of datasets and model parameters. While effective in domains such as vision and language, this approach is increasingly unsustainable in machine learning for scientific domains. In scientific and medical domains, scaling is limited by the high computational cost of specialized architectures and the scarcity or privacy constraints of available data. This dissertation aims to address these challenges by exploring two complementary paradigms for sustainable AI in science: model efficiency and data efficiency. First, we address model efficiency by targeting the computational overhead of large-scale foundation models in molecular chemistry. We investigate the architectural redundancy of the Joint Multi-domain Pre-training framework, revealing that deep interaction blocks contribute diminishing returns to predictive accuracy. Second, we address data efficiency by maximizing the utility of available information through strategic dataset selection, cross-modal transfer, and collaborative learning. In atomic property prediction, we challenge the assumption that larger pretraining datasets yield better performance, demonstrating instead that prioritizing dataset alignment enables computationally efficient training on small, high-utility subsets. We further address the scarcity of high-fidelity geometric 3D data by transferring knowledge from abundant 2D molecular graphs to improve 3D conformer generation, reducing the reliance on expensive 3D pretraining. Finally, we extend these principles to the medical domain to address distributed and evolving data. We introduce a holistic framework for federated continual learning that enables efficient adaptation to distribution shifts without the computational overhead of full retraining.