Simultaneous training of autoencoders for coherent encoding of features from different domains: An approach to unified representation of technical components from 2D and 3D modality

As companies shift towards purely 3D-based design and manufacturing, a gap emerges in the relevant databases. Companies may have accumulated a large number of technical drawings and other geometric 2D data over decades. Still, due to the differences between the domains, these can no longer be used as training data. This issue could be overcome if AI models can interchangeably handle 2D and 3D data. One can utilize both datasets as input by transferring the domains into a comparable space. For this purpose, we propose a new concept called Simultaneous Training of Autoencoders (STAE). The fundamental principle is to link the training process of two separate autoencoders so that they can abstract relevant information concerning the target of the second model and vice versa. We present this concept by processing geometric data, where each object exists in 2D and 3D. The linked models are then trained to encode the geometric information. Additionally, we enforce similarity in the latent space. Evaluations show that the models can reconstruct the geometries. To demonstrate the comparability of the encoded data, we conducted a case study on predicting processing times using a real-world dataset of technical parts. The results show that the tested regression models behave and perform similarly, regardless of the source domain from which the dataset was generated. Moreover, the quality of the predictions was increased compared to a benchmark model. The models are trained on the newly created Imgvox dataset, a derivation of the ABC-dataset , which contains both modalities for each sample.