Learning Degradation-Aware Diffusion Prior for Hyperspectral Reconstruction From RGB Image

Hyperspectral image (HSI) is applicable in many fields due to the ability in discriminating different materials. Collecting HSI usually requires expensive hardware and long period. Reconstructing HSI from RGB image, also called spectral super-resolution (SSR), is an affordable and feasible way for HSI acquisition. Despite the SSR results achieved by existing deep unfolding networks (DUNs), they still face challenges in: 1) recovering the fine-grained and realistic details; 2) suppressing the spectral distortion. Diffusion model has advantages in generating diverse and realistic contents, while its fidelity is limited due to the inherent randomness. In this study, to reconstruct a faithful and realistic HSI, we integrate the diffusion model in DUN, and propose a degradation-aware unrolling diffusion model for SSR (deDiff-SSR). The generative diffusion prior is jointly leveraged with the spectral degradation and deep prior learning. Specifically, we first pre-train a channel attention enhanced denoising diffusion probabilistic model (DDPM), the spectral correlation is exploited for learning the diffusion prior of HSI. To aware the degradation, by optimizing a diffusion and deep priors regularized HSI SSR model, we propose a degradation-aware diffusion sampling method, the spectral degradation is learned to refine each diffusion sampling step. Via unrolling the degradation-aware diffusion sampling steps, we build the deDiff-SSR network. It contains diffusion and deep proximal operators to represent the diffusion and deep priors, respectively. We implement the diffusion proximal operator with one sampling step of the pre-trained DDPM. Moreover, we design a state-space Transformer as the deep proximal operator, the spectral-spatial long-range relationship of HSI can be efficiently captured. The experiments on several indoor and remote sensing datasets demonstrate the effectiveness of deDiff-SSR.