Robot skill adaptation and mastery with world models

A key challenge in robotics is enabling robots to learn from their experiences and adapt their behavior effectively in unstructured environments. These settings demand robust skill acquisition to handle diverse tasks, the refinement of learned behaviors to perform better in dynamic and noisy conditions, and the generalization of capabilities to adapt to new and unfamiliar scenarios. For service robots, the challenge intensifies with tasks involving complex object interactions, extended planning horizons, and sequential multi-task manipulation, all compounded by noisy sensory perceptions. Despite decades of advancements in robot learning, existing methods still depend on large-scale labeled datasets and costly trial-and-error exploration, making deployment expensive, inefficient, and high-risk. Moreover, they often fail to equip robots with a comprehensive understanding of their surrounding world and its dynamics, limiting their ability to learn subsequent behaviors more efficiently. In this thesis, we propose multiple techniques to address the challenges of enabling robots to learn about and adapt to unstructured environments. The works focus on learning world dynamics through physical interactions without relying on labeled data and developing sample-efficient strategies for adapting and generalizing robotic skills to diverse scenarios. To achieve this, we first introduce an unsupervised method for learning structured 3D dynamics from real-world interactions, enabling robots to reason about individual object segments and predict their spatial development and 3D motions within the environment. By leveraging observational cues as learning signals and a joint forward-inverse model, it supports robust visuomotor control without requiring labeled data. We extend this approach with a self-supervised model for RGB-D video prediction, leveraging temporal reasoning to capture long-horizon dynamics and predict the evolution of complex scenes over time. Secondly, we address the challenge of adapting and refining robot skills in noisy, real-world environments. To achieve this, we introduce a method that combines dynamical systems from demonstrations with reinforcement learning, integrating the stability of dynamical systems for smooth, reactive motions with the adaptability of reinforcement learning to handle noisy sensory inputs and sparse rewards. Building on this, we propose a framework that employs Bayesian optimization to refine skill representations efficiently, enabling sample-efficient adaptation with minimal interactions in challenging environments. Thirdly, building on our framework for skill adaptation, we address the challenge of generalizing robot skills to unseen but structurally similar environments. By integrating reinforcement learning with keypoint-based scene grounding, this method enables robots to transfer and adapt skills learned in a source environment to new target environments without requiring additional ground truth data. Finally, we address the challenge of enabling robots to master their repertoire of skills for long-horizon, multi-task manipulation. Building on learned world models, we propose a framework that allows robots to practice and refine skills offline through simulated rollouts in the latent space of a world model. By combining language-conditioned goal guidance, latent planning, and intrinsic rewards that encourage matching expert trajectories in the latent space, this method enables robust skill refinement, long-horizon planning, and adaptability, facilitating seamless transfer from the learned model to real-world environments. All the approaches presented in this thesis were rigorously tested with real robots, demonstrating their effectiveness in advancing manipulation autonomy for service robots. Specifically, we demonstrate that autonomously learning 3D dynamics models allows robots to build spatiotemporal representations of their environment, enabling accurate prediction and planning. Furthermore, integrating dynamical systems with reinforcement learning provides a robust framework for skill adaptation and generalization, allowing robots to refine and transfer skills efficiently in dynamic and noisy conditions. Finally, leveraging learned world models enables robots to practice and master long-horizon, multi-task manipulation tasks offline, significantly reducing exploration costs while ensuring robust transfer to real-world environments. Together, these contributions pave the way for more autonomous, adaptable, and efficient robotic systems.