Deep-JGAC: End-to-End Deep Joint Geometry and Attribute Compression for Dense Colored Point Clouds

Colored point cloud becomes a fundamental representation in the realm of 3D vision. Effective Point Cloud Compression (PCC) is urgently needed due to the huge amount of data. In this paper, we propose an end-to-end Deep Joint Geometry and Attribute Compression (Deep-JGAC) method for dense colored point clouds. First, we propose a flexible Deep-JGAC framework, where the geometry and attribute encoders are compatible with either learning or non-learning encoders. Second, we propose an end-to-end deep residual self-attention-based geometry encoder to improve geometry coding efficiency, where a Hybrid Residual Self-attention Module (HRSM) is proposed to enhance geometry representation by considering its geometrical importance. Third, to solve the mismatch between the point cloud geometry and attribute caused by the geometry compression distortion, we propose an optimized re-colorization module to attach attribute to the geometrically distorted point cloud for attribute coding, which lowers the computational complexity. Extensive experimental results demonstrate that, in terms of the geometry quality metric D1-PSNR, the proposed Deep-JGAC achieves average Bjøntegaard Delta Bit Rate (BDBR) of -82.96%, -44.63%, -36.46%, -41.72%, and -31.16% compared to the G-PCC (Octree), G-PCC (Trisoup), V-PCC, GRASP, and PCGCv2, respectively. For the perceptual joint quality metric MS-GraphSIM, Deep-JGAC achieves an average BDBR of - 48.72%, -57.14%, -14.67% and -13.37% against G-PCC(Octree), IT-DL-PCC, V-PCC, and DeepPCC, respectively. In addition, the costs of encoding/decoding time are reduced by 32.8%/30.8%, 80.1%/81.8%, 97.2%/35.7%, 98.4%/92.3%, and 96.4%/99.6% on average compared to G-PCC (Octree), G-PCC (Trisoup), V-PCC, IT-DL-PCC and DeepPCC. The code and pre-trained models are available at https://github.com/SYSU-Video/Deep-JGAC.