Density-Aware Video Desnowing With Robust Alignment on a Large-Scale Dataset

Existing desnowing methods have primarily focused on single snowy images, relying on hand-crafted or learned convolutional features. However, these methods often face challenges when it comes to video desnowing tasks due to their neglect of video temporal information. Moreover, existing image datasets contain limited snow densities, and thus existing desnowing CNNs trained on these datasets are not effective in handling inputs with different snow densities. To alleviate this issue, we generate the first high-quality snowy video dataset with 1250 videos and diverse snow densities. We believe that this is the first learning-oriented dataset for video desnowing. Moreover, we devise a density-aware video desnowing network (DvD-Net) by learning inter-video and intra-video snow density information. We utilize the window-based mechanism to align adjacent video frames to reduce the influence of inaccurate optical flow estimation caused by snow particles. Then, we predict additional snow density information and devise a density-aware transformer block to embed it into a transformer to learn global intra-video information for video desnowing. Meanwhile, we devise a density contrastive loss to enlarge the snow density similarities within frames of the same videos and reduce the density similarities among frames from different videos. Experimental results on our generated snowy video datasets and real-world videos show that our network clearly outperforms state-of-the-art methods.