Few-shot vision-language models grounded with hierarchical retrieval for wind turbine blade inspection

Read full text
Authors: Qianyu Zhou, Yang Zhang, Zhiling Chen, Fardin Jalil Piran, Farhad Imani, Jiong Tang
Cited by: 1
References: 25

Abstract

Wind turbine blades are a load-bearing element whose undetected damage propagates rapidly. Reliable, automated inspection is therefore a high-leverage problem. Existing supervised deep-learning approaches, however, are constrained by the scarcity of pixel-annotated damage data and produce models that cannot adapt to new defect types without retraining. Vision-language models (VLMs) offer a path around the data bottleneck through their generalist visual reasoning, but two obstacles remain: they lack domain-specific knowledge, and they cannot produce the pixel-level spatial outputs that maintenance planning requires. This work introduces a unified few-shot framework that resolves both obstacles within a single inference pass. We establish that VLM cross-modal attention, generated as a byproduct of caption synthesis, encodes spatially structured task-relevant information, and we introduce a knowledge-guided IoU-based mechanism that converts this passive signal into pixel-level segmentation masks without any segmentation training. Domain expertise is injected through a three-tier hierarchical knowledge base, defect taxonomy, visual exemplars, and spatial patterns, that mirrors how expert inspectors progressively refine their assessment. On the public DTU dataset, the framework achieves 98.3% classification accuracy and 0.862 Dice coefficient using only 15 training samples per class, outperforming the strongest supervised baseline by 13.3%; cross-domain validation on the NEU surface defect benchmark confirms the framework transfers to a fundamentally different industrial inspection setting. The significance is twofold: methodologically, we show that attention as a side product of generation can be repurposed as an active grounding signal; practically, we demonstrate that few-shot VLM-based inspection is viable for industrial deployment where labeled data is scarce.

Fields of Study & Subjects

Multimodal Machine Learning Applications Advanced Neural Network Applications Infrastructure Maintenance and Monitoring

Publication details

Journal / source
Results in Engineering
Publisher
Elsevier BV
Volume
30
Pages
111187–111187
Publication date
23 May 2026
Language
EN