A Heterogeneous CNN With Transformers for Image Denoising

Deep learning models often emphasize structural information over long-range dependencies when producing cleaner images. To enhance the robustness of the resulting denoisers, this work introduces a heterogeneous architecture, termed HTDNet, which synergistically integrates a convolutional neural network (CNN) with Transformers. CNNs are proficient at capturing intricate local structural features, thereby enabling a more effective representation of the noise profile, whereas Transformers are adept at modeling long-range dependencies and global context. These features are complementary and informative, thereby facilitating the development of a more robust denoising model. HTDNet is composed of two shallow feature extraction blocks (SFE blocks), lightweight Transformer blocks (LT blocks), an augmented mechanism (AM), and a heterogeneous architecture refinement block (HAR block). The two SFE blocks extract distinct structural features to bolster the denoiser’s expressive capacity, inspired by the principle of binocular vision, which mimics how human eyes capture complementary perspectives. To improve training efficiency, the two LT blocks employ enhanced Transformer mechanisms in parallel to extract salient noise information by leveraging global inter-pixel dependencies. To enhance the reliability of the acquired noise information, the augmented mechanism leverages a symmetrical architecture to facilitate interactions between diverse structural features (i.e., linear and non-linear) derived from hierarchical representations and global context for image denoising. Finally, the HAR block is utilized to mitigate adverse effects arising from the network depth and architectural complexity. Experimental results demonstrate that HTDNet is competitive with state-of-the-art methods across various noisy-image denoising benchmarks.