Toward Injury-Prone Posture Recognition in Piano Practice via Multi-sensor Fusion and Deep Learning

Read full text Summarise with VIRAT
Authors: Daniel Sheldon, Fangshi Zhou, Tianming Zhao, Zhongmei Yao
Cited by: 0
References: 17

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Extended piano practice often prioritizes note and rhythm accuracy while overlooking the physical strain caused by inefficient movement. Over time, repeated poor technique can lead to chronic musculoskeletal injury. To address this, we present a multimodal, injury-aware gesture recognition framework designed to detect harmful gestures during piano performance. We introduce a novel dataset comprising 14 representative incorrect piano gestures, collected from 10 participants using a custom wearable prototype that integrates surface electromyography, inertial measurement units, and pressure sensors. To reduce label noise, gestures are performed in isolation. Using this dataset, we evaluate three deep learning architectures that take different approaches to multimodal data fusion and multi-label classification: (1) a parallel baseline with gesture-specific classifiers, (2) a backbone-based multi-head model, and (3) a prototype-based distance model trained with a composite loss. Experimental results show that the prototype-based model achieves the highest overall performance, with a precision of 0.97 and a recall of 0.70, significantly reducing false positives in gesture recognition. This work establishes a foundational, data-driven framework that paves the way for developing real-time, biomechanics-informed feedback systems to support healthier, injury-preventive piano practice.

Fields of Study & Subjects

Context-Aware Activity Recognition Systems Hand Gesture Recognition Systems Musicians’ Health and Performance

Publication details

Journal / source
AI Computer Science and Robotics Technology
Volume
5
Publication date
26 March 2026
Language
EN