Object Recognition Using Deep Learning‐Powered Glove With Flex and MPU Sensors Fusion

Read full text Summarise with VIRAT
Authors: Joao Andre Ndombasi Diakusala, Zhigeng Pan, Ginias Lulendo Kiveni, Nenglun Chen, Habachi Bilal, Marcellin Atemkeng, Thierry Bouwmans, M. D. Sabbir Rahman, Ange Ntokusi, Sébastien Lalléchère, Blaise Ravelo
Cited by: 0
References: 29

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

The integration of multimodal sensor data is critical for developing intelligent systems capable of extracting complex, insightful information. We introduce a robust platform for high‐fidelity and real‐time hand posture replication, as well as object classification, centered on a novel tactile‐sensing glove. The glove design incorporates 10 flex sensors spanning the PIP and MCP joints, complemented by two MPU6050 IMUs strategically mounted on the thumb’s TM and the hand’s dorsal CMC bones for comprehensive motion capture. We leverage this rich sensory data to drive a digital twin—constructed in Blender and visualized in the Unity3D engine—achieving precise real‐time hand posture reproduction, necessary for visual feedback and sensor anomaly detection during sensor calibration and data collection by the operator. Furthermore, we evaluate the glove’s classification capability on a custom dataset of 15 objects. Through a grid search optimization, we trained One‐dimensional Convolutional Neural Network (CNN‐1D), Long‐Short Term Memory (LSTM), and Temporal Convolutional Network (TCN) architectures. The models achieved classification testing accuracies confidence intervals of [95.40%, 96.50%], [92.55%, 93.84%], and [93.47%, 94.76%], respectively, validating the high‐performance and utility of our multimodal sensing approach.

Fields of Study & Subjects

Hand Gesture Recognition Systems Robot Manipulation and Learning Context-Aware Activity Recognition Systems

Publication details

Journal / source
Journal of Robotics
Volume
2026
Issue
1
Pages
6642796
Publication date
1 January 2026
Language
EN