Dependency Then Compression: Global Dependency Network With Three-Stage Knowledge Transfer for Visible-Infrared Transmission Line Detection

Although deep learning techniques have largely solved the transmission line detection (TLD) problem, three major challenges remain. First, most methods ignore the effects of visible and infrared images on the predicted maps. Second, they fail to generalize adequately to future unseen data. Third, methods that limit the number of model parameters often reduce the model complexity and degrade the performance. To resolve the aforementioned issues, we designed global dependency networks (GDNet-S∗ and GDNet-S) utilizing three-stage knowledge distillation (KD) for TLD. First, to extract useful information from visible and infrared images, we developed two generators that extract essential information from the main and auxiliary images, respectively. Second, we designed a Bayesian module to enhance the ability of the model to generalize to new data. Finally, we designed three KD methods for encoding the basic, relational, and output features of the student and teacher networks to improve the performance without changing the GDNet-S parameters. Comprehensive experiments on a TLD dataset demonstrated that both GDNet-T and GDNet-S∗ achieved outstanding performance. Furthermore, to validate the generalizability of GDNet-S∗, we tested it on three public RGB-D datasets. GDNet-S∗ achieved moderately competitive results on these datasets. The experimental results and network code can be accessed at github.com/BeibeiIsFreshman/GDNet.