Few-shot 3D Novel View Synthesis with Depth Guidance

Novel view synthesis is a computer vision technique that uses collected views to learn the scene's underlying 3D geometry and appearance in order to recreate photorealistic representations of the scene from angles not included in the original input. Current learning-based approaches focus on Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS). However, both NeRF and 3DGS suffer from sample inefficiency, resulting in quality degradation and overfitting in few-shot settings. In this thesis, we focus on tackling few-shot prediction with NeRF and 3DGS through introducing guidance from depth. We firstly apply a depth-supervision loss that directly penalizes deviations between rendered and ground-truth depths, which works well for the NeRF model under sparse-view conditions. Considering the efficiency of 3DGS, we next apply our depth-aware optimization framework to 3DGS, a point-based volumetric representation that enables real-time rendering. However, we observe that the standard depth-guided supervisions easily introduce undesired effects to 3DGS. After analyzing the effects and revealing the cause, we introduce a robust-loss formulation that down-weights outlier depth and color errors via adaptive, patch-level weighting, thereby decreasing noise in monocular depth estimates. Our experiments systematically evaluate the proposed methods across a wide range of datasets, and the results show that integrating depth markedly improves NeRF while providing only modest gains for 3DGS.