ISDNet: High-Fidelity Single-View Reconstruction of Indoor Scenes via Instance Separation and Deformation

In this work, we aim to reconstruct the 3D shape of an indoor scene from a single view, which includes multiple objects and the background. This task is challenging for existing methods since those instances of indoor scenes regularly occlude each other and contain diverse topologies. To address this, we propose a novel framework, ISDNet, to adaptively separate mixed instances and perform topology-aware reconstruction. Specifically, Specifically, ISDNet consists of two cascaded subnetworks: an instance separation module (ISM) and an instance deformation module (IDM). The ISM learns to separate occluded objects through stepwise sampling, inferring clean features for each instance. On the basis of these features, IDM generates an instance-topology-aware template and deforms it with learned offsets to reconstruct detailed geometry. Quantitative and qualitative experiments on the SUNRGB-D and 3D-FRONT datasets demonstrate that ISDNet outperforms the state-of-theart methods in terms of local details and overall shapes.