Explainable Multimodal Plant Disease Classification with Prompt-Guided Transformers

Plant pathogens are serious challenges to food security in the world, and thus automated diagnosis systems are required to ensure timely response. Although the current deep learning models can classify with high accuracy, they are black boxes that are not interpretable and based on visual features only, which makes them not as reliable when applied in real-life agricultural scenarios. The existing methods do not incorporate semantic data with visual cues, and do not offer any quantified values of model explainability. The current piece suggests PCViT-DRS, a Prompt-Guided Cross-Modal Vision Transformer with Disease Region Supervision, which incorporates textual symptom description in the form of handcrafted descriptions on the visual embeddings using cross-attention, and the lesion-centered attention through pseudo-ground-truth masks. Stratified splits, data augmentation, and weighted cross-entropy loss were applied to the PlantVillage dataset (54,306 images, 38 classes) to train and evaluate the model to address imbalanced classes. PCViT-DRS was able to reach 98.87% with standard deviation of 0.11% and it is a 2.75% absolute accuracy improvement (68% reduction of error) over the ViT base and significantly better than EfficientNetV2B3 + ViT ( 85.06% ) with statistical significance ( p < 0.01 ). Measures of quantitative interpretability showed significant positive changes: the IoU improved by 25.6% points to 67.1%, and the accuracy of the Pointing Game by 68.2% to 88.6%, which confirmed faithful disease localization. The multimodal scheme offers an understandable and strong basis of credible diagnosis of plant diseases in precision agriculture practices.