An Intuitive Multi-Task Manipulation Policy for Imitation Learning in Simulated and Real Robots

Imitation learning on human demonstration mechanisms and exhibits excellent skill acquisition performance. To explore the efficiency-enhancing mechanisms of human demonstration in the joint space and the robustness of multi-task imitation, we propose a novel approach called an intuitive multi-task manipulation policy for imitation learning in simulated and real robots (MIRO). First, we design a constrained joint alignment teleoperation. By incorporating kinematics, smoothness, collision, and contact constraints into joint control, the robot can intuitively imitate human motion while maintaining good data acquisition efficiency. Second, we design a multi-task context-fusion imitation policy. By integrating language-guided encoding, multi-scale context enhancement, and gated feature fusion mechanisms, we improve multi-modal trajectory learning and enhance its adaptability to diverse tasks in diverse scenarios. Finally, we validate MIRO’s effectiveness by applying it to simulated and real robotic manipulation tasks and present a set of evaluation results, demonstrating its potential in various applications.