Dstadapter: divided spatial-temporal adapter fine-tuning method for sign language recognition

Read full text Summarise with VIRAT
Authors: Qiuhong Tian, Yijie Yang, Bin Chen, Jiacheng Chen, Junxiao Ning, Lizao Zhang
Cited by: 0
References: 57

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Abstract The commonly adopted approach of full fine-tuning for video-based sign language recognition models encounters two critical limitations: high computational resource consumption and compromised generalization capabilities. To address these challenges, we propose DSTAdapter, a parameter-efficient transfer learning framework that activates frozen CLIP models for video understanding through spatial-temporal decoupled adaptation. Our methodology introduces three key technical contributions: (1) dual-branch adapter architecture with separate adapter branches dedicated to capturing spatial hand shapes and temporal gesture dynamics, (2) channel-aware feature fusion modules that dynamically optimize the interaction between adapter-enhanced features and backbone representations, and (3) a lightweight framework design enabling efficient deployment on resource-constrained devices. Requiring only 4% tunable parameters, the proposed method establishes new state-of-the-art performance across four benchmark sign language datasets. Comprehensive evaluations demonstrate significant efficiency improvements, particularly on the Bukva benchmark where DSTAdapter achieves a 30% reduction in training time and 60% decrease in GPU memory consumption compared to conventional full fine-tuning approaches. The compact architecture further facilitates practical multitask deployment scenarios. These technical advancements present a promising solution for developing real-world assistive technologies, particularly benefiting hearing-impaired communities through improved accessibility. The code of this work is available at https://github.com/BLOOM0-0/DSTAdapter.

Fields of Study & Subjects

Hand Gesture Recognition Systems Human Pose and Action Recognition Domain Adaptation and Few-Shot Learning

Publication details

Journal / source
International Journal of Machine Learning and Cybernetics
Volume
17
Issue
5
Publication date
21 March 2026
Language
EN