DiffScene3D: sparse diffusion for high-fidelity indoor scene geometry generation

Diffusion models have demonstrated significant effectiveness in generating 2D images and single 3D objects, but they still face major challenges when generating indoor scenes with highly complex geometric structures and topological diversity. This paper proposes a multi-stage diffusion-based indoor scene generation framework that achieves high-quality 3D scene generation from scratch. The framework employs a three-stage diffusion pipeline to progressively optimize details through incremental geometric reconstruction. To address the complexity of 3D scenes, a multi-scale autoencoder is innovatively introduced to encode spatial occupancy features into the latent space, providing multi-level guidance for the generation process. A novel Symbolic Distance Field (SDF) fusion algorithm is proposed, combining iterative diffusion with locally truncated SDF (TSDF) voxel fusion to achieve full-scene reconstruction. A sparse 3D diffusion model is employed to denoise only the TSDF occupancy voxels and latent occupancy features, significantly reducing computational overhead and memory consumption by two orders of magnitude. Experimental results demonstrate that the proposed method can generate high-quality indoor scene meshes, with visual quality significantly superior to the ScanNet dataset benchmark in user survey evaluations. Its output module can serve as a post-processing unit, collaborating with multi-view stereo (MVS) technology to effectively enhance scene details and geometric completion, thereby improving the completeness and precision of 3D scenes.