RVQ-NVC: Taming RVQ-VAE for High Fidelity Neural Vibrotactile Compression

With the rise of the metaverse, haptic technologies have gained significant attention, striving to enhance human-machine interactions by incorporating a sense of touch alongside audio and visual experiences. Vibrotactile signals, which deliver haptic information through vibrations, are commonly used in commercial haptic interaction devices. The IEEE and MPEG have begun efforts to standardize the data compression of these signals. However, achieving efficient compression without sacrificing fidelity continues to pose a significant challenge. Inspired by the state-of-the-art audio compression techniques, we develop a novel residual vector quantization (RVQ)-based neural vibrotactile codec (RVQ-NVC) that capitalizes on the similarities between vibrotactile and audio signals in both time and frequency domains, strategically adapting RVQ to suit vibrotactile data. The RVQ-NVC scheme consists of a fully convolutional encoder-decoder network combined with an RVQ module, trained in an end-to-end manner. We employ perceptual and adversarial loss functions to ensure the high-quality reconstruction of vibrotactile signals while enabling variable bitrate coding through structured dropout techniques. Experimental results reveal that RVQ-NVC excels in signal reconstruction at low compression ratios, maintains high perceptual quality at reduced bitrates, and supports low-latency real-time processing, making it particularly well-suited for haptic interaction applications.