AI in Chemistry: A Review of SMILES-Based Transformer Models for QSAR Applications

Quantitative Structure Activity Relationship (QSAR) is a long-standing research area that seeks to predict physicochemical and biological properties of compounds based on existing data. Since the 1960s, it has drawn attention as means of inferring molecular properties without direct experimentation. The advent of Transformer architecture in 2017 marked a paradigm shift. This architecture has proven highly effective for interpreting SMILES (Simplified Molecular Input Line Entry System) strings, enabling inference of molecular properties from atomic and functional group contexts. Building upon this progress, since 2019 numerous Transformer models utilizing SMILES-based data have emerged, these models generally feature Seq2Seq or BERT-style, yet the strategies related to pre-training with SMILES as input or the downstream tasks for extracting target properties are highly diverse. This review synthesizes recent progress in SMILES-based Transformer models in QSAR and discusses how transfer learning has reshaped molecular representation learning in cheminformatics.