PhononBench:A Large-Scale Phonon-Based Benchmark for Dynamical Stability in Crystal Generation

Recent advances in generative artificial intelligence have enabled crystal-design methods based on graph neural networks, diffusion models, and large language models. Existing evaluations commonly follow the stability–uniqueness–novelty (S.U.N.) framework, but typically assess stability using thermodynamic criteria, which do not fully capture dynamical stability. Although dynamical stability influences material synthesis and persistence, experimental realization also depends on anharmonicity, finite temperature, defects, disorder, and synthesis conditions. Its high computational cost has hindered systematic large-scale evaluation of generated crystals. Here, we introduce PhononBench, the first large-scale benchmark of dynamical stability in AI-generated crystals. Using the MatterSim interatomic potential, which provides density-functional-theory-level accuracy for phonon predictions across more than 10,000 materials, PhononBench enables efficient phonon calculations for 133,838 structures generated by seven leading crystal-generation models. Unless otherwise specified, all stability metrics use a phonon-frequency threshold of −0.1 THz. The results reveal a major limitation of current models: the average dynamical-stability rate is only 32.15%, while the best-performing model, MatterGen, reaches 45.05%. Further analyses show that, for MatterGen band-gap-conditioned generation, the stability rate remains 41.0% even under the optimal 0.5 eV condition. In space-group-controlled generation, higher-symmetry crystals are generally more stable, with cubic systems reaching 53.4%; nevertheless, the overall average is 44.7%. We identify 32,995 structures whose minimum phonon frequencies remain above the adopted threshold along sampled high-symmetry paths; these labels do not establish stability throughout the full Brillouin-zone volume. Multi-threshold sensitivity analyses and investigations of structural and physical factors further clarify the origins of dynamical instability. By establishing the first large-scale benchmark for dynamical stability, PhononBench systematically exposes limitations of current crystal-generation models and provides practical criteria for their evaluation and improvement. Together, these results create a scalable basis for comparing generative approaches, diagnosing failure modes, and guiding future model development toward crystals with improved physical plausibility and greater relevance to experimental materials discovery