From Images to Neural Scenes: Efficient Rendering, Robust Editing, and Human-Aware Reconstruction

Casual multi-view capture with commodity cameras is increasingly used to create 3D content for augmented and virtual reality, virtual production, and built-environment applications. Neural scene representations such as neural radiance fields make it possible to reconstruct photorealistic scenes from unstructured photographs, but three blockers limit deployment in real systems: real-time rendering on constrained devices, semantics-aware and view-consistent editing, and robust reconstruction in human-centric scenes. This thesis addresses these three blockers. First, we develop a distillation strategy that converts a volumetric radiance field into a mesh-based representation with compact, rasterization-friendly appearance parameters, enabling single-query color evaluation per ray and real-time rendering across devices. Second, we propose a text-guided diffusion editing pipeline that conditions on depth predicted by the radiance field and propagates edits across views via reprojection and hybrid inpainting, yielding geometrically and photometrically consistent 2D edits that can be fused back into a coherent edited scene. Third, we extend a learning-based reconstruction backbone with explicit human understanding, predicting dense human segmentation and body-part correspondences to improve camera and geometry estimation in human-centric scenes. Taken together, these contributions operationalize casual multi-view capture into deployable 3D assets by enabling device-portable rendering, semantics-aware, view-consistent editing, and robust, human-aware reconstruction within efficiency budgets.