Hand Tracking Streamer

Read full text
Authors: Zhengyang Kris Weng
Cited by: 0

Abstract

Hand Tracking Streamer is a lightweight hand telemetry utility that turns a Meta Quest headset into a precision controller for robotics teleoperation and motion capture. Built on the Meta Interaction SDK, it streams 21-landmark hand data and 6-DoF wrist pose to a PC in real time over Wi-Fi using UDP for ultra-low latency or TCP for reliable data logging. The app supports left, right, or dual-hand modes with in-headset configuration, and includes a live log console and phantom hand visualization for debugging. Data is sent as structured packets of joint positions and orientations in Unity world space, making it suitable for robot control, imitation learning, and gesture-based prototyping. Open source available on github. Read more about connection and deployment examples on: https://github.com/wengmister/hand-tracking-streamer

Fields of Study & Subjects

Hand Gesture Recognition Systems Teleoperation and Haptic Systems Interactive and Immersive Displays

Publication details

Publication date
23 January 2026
Language
EN