PervoTransformer: Transformer-Based PCE Predictionand Device Design with MACE MLIPs for CsSnX3 Perovskites

Perovskite solar cell performance depends on the joint configuration of materials, interfaces, and layer-specific physical parameters, forming a structured design space that is naturally sequential but rarely modeled as such. This work introduces PervoTransformer, a transformer-based framework that represents complete device architectures as sequences and learns device-level behavior directly from this representation. A RoBERTa model is finetuned on SCAPS-generated device sequences spanning six perovskite families, including MAPbI3, FAPbI3, CsPbI3, and their Sn-based analogs. The model predicts multiple performance metrics simultaneously, including power conversion efficiency (PCE), short-circuit current density Jsc, open-circuit voltage Voc, and fill factor (FF), demonstrating that transformer encoders can capture coupled, non-linear dependencies across the full device stack. To the best of our knowledge, this is the first application of transformer models to perovskite solar cell device modeling using a sequence-based formulation. To explore device configurations beyond the training distribution, a GPT-2 model is finetuned on the same device sequence formalism and used to generate candidate architectures with novel combinations of compositional and physical parameters. Generated sequences are screened for validity, uniqueness, and physical consistency using a retrieval augmented generation (RAG) based constraint system. The validated devices are then passed back through the finetuned RoBERTa model to predict performance metrics, enabling selection of high performing candidates. Selected absorber configurations are further evaluated through device-level simulations, forming an end-to-end loop for device prediction, generation, and screening. Additionally machine-learned interatomic potentials based on the MACE framework are finetuned for CsSnX(X = Cl,Br, I)3 using available DFT data, enabling efficient exploration of structural and energetic trends in lead-free Sn perovskites and supporting absorber materials design. Together with the transformer based predictive, generative, and screening workflow, this establishes a multi-scale strategy in which sequence-aware transformers optimize device architectures while MLIPs guide atomic-scale materials exploration for perovskite solar cells.