Toward Reliable Imitation Learning With Limited Expert Demonstrations via Search-Based Inverse Dynamic Learning

Imitation learning (IL) shows its superiority in faster strategy optimization by leveraging expert demonstrations. However, as one of the powerful IL methods, behavior cloning (BC) suffers from covariate shift problems, where the agent’s policy drifts away from the expert’s, leading to compounding errors and decreased generalization performance. To this end, we propose Search-based Inverse Dynamics Imitation Learning, namely SIDIL, to enhance the robustness of imitation learning by augmenting expert demonstrations via trajectory perturbation and stitching. Specifically, SIDIL first employs a nearest-neighbor search method to find the closest points between expert and perturbed data, generating new actions near the expert data via a stitching strategy. By exploiting this, the agent can recover from deviations and complete tasks under a wider range of conditions. Experimental results on various robotic tasks show that SIDIL outperforms baseline algorithms with higher success rates across multiple tasks. Meanwhile, SIDIL is allowed to expand the attractive region around expert demonstrations by stitching states from expert demonstrations, enjoying a higher task completion success rate even for states outside the expert distribution. These results highlight our potential to enhance humanoid locomotion and dexterous robotic hand manipulation, making it particularly suitable for industrial manufacturing automation where adaptability, robustness, and safety are essential.