Benchmarking Generative AI Protein Models Reveals Differences Between Structural and Sequence-based Approaches

Recent advances in artificial intelligence have led to the development of generative models for de novo protein design. In this study, we compared 13 state-of-the-art generative protein models, assessing their ability to produce feasible, diverse, and novel protein monomers. Structural diffusion models generally create designs with higher confidence in predicted structures and more biologically plausible energy distributions, but exhibit limited diversity and strong sequence biases. Conversely, protein language models generate more diverse and novel designs but with lower structural confidence. We also evaluated the ability of these models to generate unique proteins, conditionally based on the tobacco etch virus (TEV) protease. Generative models are successful in producing functional enzymes, albeit with diminished activity compared to the wild-type TEV. Our systematic benchmarking provides a foundation for evaluating and selecting generative protein models, while highlighting the complementary strengths of different generative paradigms. This framework will facilitate informed application of these tools for biomedical engineering and design.