Text-KeyPoint Human Representation Based Multi-Level Semantic Guided Model for Human Mesh Recovery

Human mesh recovery (HMR) from the monocular RGB images has attracted a lot of attention from computer vision. Affected by the occlusion between joints and the fact that a 2D pose may match with multiple 3D poses, HMR remains a difficult task. Some mainstream neural network methods realize information exchange in the process of recovering key points extracted from images into meshes, enhance the perception of the vertices and improve the structural rationality of the estimation results. However, the information exchange of vertices without reasonable guidance of global semantic information lacks attention to the global rationality of the estimation results. In order to solve this problem, we propose Text-KeyPoint human representation, which augments key points with semi-structured textual information generated from manual rules to describe the human body at different levels, thus introducing global semantic information. Based on Text-KeyPoint human representation, we propose a novel semantic guided model, which fuses multi-level text features extracted from semi-structured text and multi-level geometric features pooled from key points to provide multi-level semantic guidance in the process of human recovery, thus improving the global rationality of results. Comprehensive experimental results on widely used HMR benchmarks H3.6M, 3DPW and SURREAL show that the proposed method achieves competitive performance compared with the state-of-the-art methods.