Integrating Large Multi-Modal Models for Automated Powdery Mildew Phenotyping in Grapevines

Powdery mildew is a major fungal disease of grapevines, yet its field quantification remains constrained by subjective visual assessments, inconsistent scoring protocols, and the high cost of annotated datasets. Large multi-modal models that integrate vision and language offer new opportunities for scalable phenotyping, but applying them to agricultural imagery is challenging due to subtle symptoms and domain shifts across environments. This study presents an automated phenotyping pipeline that combines active illumination imaging with a unified Segment Anything Model-Contrastive Language-Image Pretraining (SAM-CLIP) framework for powdery mildew segmentation and vine-level severity quantification.The PM-SAM-CLIP model achieved 58.43% image-level mean Intersection over Union (mIoUI), substantially outperforming baselines. For canopy segmentation, Canopy-SAM-CLIP reached 95.48% mIoUI, providing a stable foundation for severity normalization. Applied to independent field datasets without fine-tuning, the pipeline demonstrated strong zero-shot generalization, with over 98% of vines scored within one category of expert ratings. Multi-date analyses further produced spatial and temporal infection maps that captured vineyard-scale heterogeneity and disease progression.To assess biological relevance, a downstream quantitative trait locus (QTL) analysis was performed using image-derived severity metrics. This analysis identified a major resistance locus on chromosome~13, consistent with QTL mapped from human scouting scores and aligned with previously reported loci, supporting the validity of the automated phenotypes.Overall, this work demonstrates that large multi-modal models adapted through vision-language integration can generate accurate, reproducible, and scalable phenotypes for plant disease assessment. The proposed pipeline reduces human subjectivity and provides high-quality quantitative traits that support downstream genotype-phenotype analyses in viticulture.