ShadowNeRF: Learning Neural Radiance Field With Sight Degradation and Recovery

Inherently equipped with arbitrary resolution and multi-view consistency, the Neural Radiance Field (NeRF) as an implicit scene representation has drawn extensive attention. While traditional NeRFs excel at novel view synthesis (NVS) under ideal conditions, they overlook the potential of learning consistent geometric representations across varying sight qualities. Current methods mainly focus on optimizing synthesis under clear visibility, which limits their effectiveness in downstream scene understanding tasks where robust geometry comprehension is crucial. In this paper, we propose a NVS pre-training technique named ShadowNeRF which firstly synthesizes degraded views with shadowed regions to challenge the model in inferring complete scene geometries. We then design a self-supervised sight recovery process with a two-stage unshadowing framework, which progressively recovers neighboring areas and reveals geometric properties of invisible regions. This pre-training strategy of degradation synthesis and recovery, when combined with taskspecific fine-tuning, enhances the understanding of underlying scene structure for the model and strengthens its ability to process scenes under varying sight conditions. Through extensive experiments, we demonstrate that our pre-training and finetuning pipeline significantly improves the model performances in semantic segmentation and 3D object detection, as well as the reconstruction quality of complex scenes.