Silhouette Sculptor: A Silhouette-Based Volume and 3D Mesh Refinement

We introduce an innovative method called Silhouette Sculptor, which utilizes an object’s silhouetteto extract hole-free, precise 3D meshes from volumetric data generated by Neural Radiance Fields(NeRF). Silhouette Sculptor leverages NeRF alongside a machine learning model specialized in objectsilhouette segmentation for enhanced accuracy. NeRF has recently gained significant attention forrepresenting complex scenes using a continuous volumetric representation. However, reconstructingprecise 3D meshes from NeRF volumes remains challenging due to potential defects such as surfaceholes, noise, and inaccuracies introduced during volumetric scene construction and rendering.Our approach addresses these challenges by incorporating stenciling, volume refinement, and meshrefinement stages into the mesh reconstruction pipeline. We use machine learning models trainedto accurately segment the silhouettes of various objects, particularly humans, from rendered 2Draster images of the volumetric scene. By refining defects and integrating 2D silhouette informationfrom different viewing angles with the 3D volumetric rendering process, we enhance the volumetricrendering quality, and the accuracy of the extracted 3D meshes.