YOTO++: Learning Long-Horizon Closed-Loop Bimanual Manipulation From One-Shot Human Video Demonstrations
Abstract
Bimanual robotic manipulation remains a fundamental challenge due to the inherent complexity of dual-arm coordination and high-dimensional action spaces. This paper presents the extended YOTO++ (You Only Teach Once), which is a unified one-shot learning framework for teaching bimanual skills directly from third-person human video demonstrations. Our method extracts structured 3D hand motions using binocular vision and distills them into compact, keyframe-based trajectories for dual-arm execution. We develop a scalable demonstration proliferation strategy that synthetically augments one-shot demonstrations into diverse training samples, enabling effective learning of a customized bimanual diffusion policy. Extensive evaluations across a broad spectrum of long-horizon bimanual tasks, including asynchronous, synchronous, contact-rich, and non-prehensile scenarios, demonstrate strong generalization to novel skills and objects. We further introduce a visual alignment mechanism at the initial manipulation stage for closed-loop control, enabling the system to timely adapt to perturbations during execution. We validate the framework on an unseen dual-arm robotic platform to show seamless cross-embodiment transfer without additional retraining. YOTO++ achieves impressive performance in accuracy, robustness, and scalability, advancing the practical deployment of general-purpose bimanual manipulation systems.
Fields of Study & Subjects
Robot Manipulation and Learning
Human Pose and Action Recognition
Generative Adversarial Networks and Image Synthesis
Publication details
- Journal / source
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Publisher
- Institute of Electrical and Electronics Engineers (IEEE)
- Volume
- 48
- Issue
- 9
- Pages
- 10695–10712
- Publication date
- 27 April 2026
- Language
- EN