Object Recognition Using Deep Learning‐Powered Glove With Flex and MPU Sensors Fusion
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
The integration of multimodal sensor data is critical for developing intelligent systems capable of extracting complex, insightful information. We introduce a robust platform for high‐fidelity and real‐time hand posture replication, as well as object classification, centered on a novel tactile‐sensing glove. The glove design incorporates 10 flex sensors spanning the PIP and MCP joints, complemented by two MPU6050 IMUs strategically mounted on the thumb’s TM and the hand’s dorsal CMC bones for comprehensive motion capture. We leverage this rich sensory data to drive a digital twin—constructed in Blender and visualized in the Unity3D engine—achieving precise real‐time hand posture reproduction, necessary for visual feedback and sensor anomaly detection during sensor calibration and data collection by the operator. Furthermore, we evaluate the glove’s classification capability on a custom dataset of 15 objects. Through a grid search optimization, we trained One‐dimensional Convolutional Neural Network (CNN‐1D), Long‐Short Term Memory (LSTM), and Temporal Convolutional Network (TCN) architectures. The models achieved classification testing accuracies confidence intervals of [95.40%, 96.50%], [92.55%, 93.84%], and [93.47%, 94.76%], respectively, validating the high‐performance and utility of our multimodal sensing approach.
Fields of Study & Subjects
Hand Gesture Recognition Systems
Robot Manipulation and Learning
Context-Aware Activity Recognition Systems
Publication details
- Journal / source
- Journal of Robotics
- Volume
- 2026
- Issue
- 1
- Pages
- 6642796
- Publication date
- 1 January 2026
- Language
- EN