MHFS: Multimodal Haptic Feedback System for Enhanced Immersive Experience

Haptic feedback devices provide a sense of immersion that makes them play a key role in the domain of human-environment interactivities, such as in telemedicine, teleoperation, and online education. However, widely used unimodal or limited-modal approaches, such as vibrotactile, electrotactile, force-feedback, thermal, and emerging mid-air ultrasound/magnetorheological systems, are merely capable of simulating few virtual materials, exhibit a deficiency in fidelity, and necessitate substantial training expenditures. To address these limitations, we propose a multivirtual object recognition system based on multimodal haptic feedback and develop an evaluation and training scheme. The system is based on electrohaptic, vibrohaptic, and thermohaptic, acquires four haptic perception modalities, roughness, sharpness, hardness, and temperature, and simulates the haptic sensation of eight different materials. In experiments, the fidelity alignment of haptic feelings of real and virtual objects is 85.8%, and after averaged 2.8 trainings, the recognition accuracy reaches 100%. Our research not only improves the realism of haptic feedback but also significantly reduces the training cost, providing a more cost-effective and practical solution for user experience enhancement in virtual reality (VR) and meta-universe applications.