Image2CADSeq: Computer-Aided Design Sequence and Knowledge Inference From Product Images

Abstract Computer-aided design (CAD) tools empower designers to design and modify 3D models through a series of CAD operations, commonly referred to as a CAD sequence. In scenarios where digital CAD files are inaccessible, reverse engineering (RE) has been used to reconstruct 3D CAD models. Recent advances have seen the rise of data-driven approaches for RE, with a primary focus on converting 3D data, such as point clouds, into 3D models in boundary representation (B-rep) format. However, obtaining 3D data poses significant challenges, and B-rep models do not reveal knowledge about the 3D modeling process of designs. To this end, our research introduces a novel data-driven approach based on representation learning to infer CAD sequences from product images, coined as Image2CADSeq. These sequences can then be translated into B-rep models using a solid modeling kernel. Unlike B-rep models, CAD sequences offer enhanced flexibility to modify individual steps of model creation, providing a deeper understanding of the construction process of CAD models. One unique contribution of this study is the development of a multilevel evaluation framework for model assessment, so the predictive performance of the Image2CADSeq model can be rigorously evaluated. The model was trained on datasets generated using a proposed data synthesis pipeline, and two different neural network architectures were explored to optimize the Image2CADSeq performance. The experimental results show that our methods are promising in data-driven reverse engineering of 3D CAD models (CAD sequences) from 2D images.