Double Diffusion Policy for Robust Robot Learning via Human Guidance

The diffusion policy has introduced the excellent multimodal data modeling capability of the diffusion model to robotic imitation learning. However, extensive expert demonstration data is still needed during training, requiring considerable human effort to collect. Natural human action data is cheaper and easier to obtain than expert demonstration data, but its use in direct robot policy training is challenging due to the distribution gap. In this study, we propose the double diffusion policy learning paradigm, which incorporates low-cost human action data to diminish the reliance of diffusion policy on extensive expert demonstration data. Specifically, we extract human intentions from the diffusion policy modeling human actions, then integrate these intentions into training the diffusion policy for generating robot actions. This processing guides the policy model to train better on small expert demonstration datasets. Experiments show that our double diffusion policy outperforms the vanilla diffusion policy and other state-of-the-art imitation learning algorithms with limited expert demonstration data.