SCALE-Pose: Skeletal Correction and Language Knowledge-assisted for 3D Human Pose Estimation

View at publisher Summarise with VIRAT
Authors: Yaochen Li, Xinnan Ma, Limeng Zhao, Chenxu Zhou, Jiahui Feng, Yuehu Liu
Cited by: 0
References: 50

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Transformer-based 3D human pose estimation methods typically use 2D joint sequences as inputs, leveraging spatial and temporal transformer encoders to model the 3D human pose. However, these methods often fail to incorporate skeletal constraints to limit joint motion. The integration of prior category knowledge to enhance joint representations is also neglected. To address these challenges, a novel approach named SCALE-Pose is proposed in this article. Our method first constructs a feature extraction network based on spatiotemporal skeleton refinement, where the skeletal correction modules are designed within both the spatial and temporal skeleton encoders to enhance the backbone network’s understanding of skeletal features. Meanwhile, a weighted average joint position error loss function is applied to improve the network’s ability to represent the joints with varying difficulty levels. A new radian-based loss function for skeletal joint angles is also designed, further enhancing the model’s capability to capture subtle skeletal movements. Furthermore, a training strategy based on large language model (LLM) priors is proposed to generate category-specific prior semantic knowledge from category keywords, which is then incorporated as auxiliary information to extract motion features. The experimental results based on the Human3.6M and MPI-INF-3DHP datasets well demonstrate the effectiveness of the proposed method.

Fields of Study & Subjects

Human Pose and Action Recognition Human Motion and Animation Hand Gesture Recognition Systems

Publication details

Journal / source
ACM Transactions on Multimedia Computing Communications and Applications
Volume
22
Issue
5
Pages
1–23
Publication date
25 March 2026
Language
EN