SE-GS: Sensor-Enhanced Gaussian Splatting

The degradation issues of visual sensors, such as noise and motion blur, limit their applications. Traditional discrete-view approaches to addressing these problems often rely on prior neural networks and cannot leverage multi-view geometry for unsupervised solutions. Recently, 3D Gaussian Splatting (3DGS) has demonstrated remarkable performance in the field of multi-view synthesis. Although some 3D Gaussian-based methods have proposed frameworks for single tasks, they lack effective capabilities for generalizable multi-task solutions. Therefore, we propose SE-GS, a joint optimization framework that combines a sensor enhancement network with 3D Gaussian parameters. Our innovations include: We introduce a self-supervised learning module that can achieve adaptive task optimization. We propose a perception enhancement module based on frequency-domain transformations to adaptively acquire pixel masks for degraded interference. We propose a coarse-to-fine feature optimization method based on geometry-optical flow to reduce the model size, thereby improving training speed and eliminating artifacts. We propose a joint optimization loss module that includes multi-view consistency loss and structural similarity loss to achieve rendering optimization.Our method can adapt to multi-task constraints and demonstrate good generalizability. It avoids the view separation caused by decoupled training and reconstruction, which significantly reduces reconstruction artifacts caused by sensor degradation. Extensive experiments conducted on multiple datasets demonstrate that our method achieves competitive performance in rendering and real-time metrics compared to state-of-the-art methods.