Foundation Models for Scientific Discovery: A Survey of Modalities, Architectures, and Pre-training Paradigms

The integration of artificial intelligence with scientific discovery has entered a transformative era with the emergence of foundation models. These large-scale pre-trained models, characterized by their ability to transfer knowledge across diverse tasks and domains, are revolutionizing how we approach scientific problems in physics, chemistry, biology, and materials science. Moving beyond application-centric overviews, this survey offers a methodologically-driven analysis of foundation models in scientific discovery. We propose a systematic taxonomy organized along three orthogonal dimensions: data modality, model architecture, and pre-training paradigm. Through detailed analysis of representative models, we examine how these three dimensions interact to determine model capabilities, and identify effective combinations for different scientific tasks. We further discuss critical challenges including data scarcity, physical constraint integration, interpretability, and computational accessibility, with concrete technical recommendations. Finally, we outline actionable future directions toward more general, physics-informed, and interpretable foundation models for science. To facilitate ongoing research, we release an up-todate repository of resources at https://github.com/ Pinlong-Zhao/Scientific-Foundation-Models.