InCoM: Intent-Driven Perception and Structured Coordination for Mobile Manipulation

Read full text
Authors: Jiahao Liu, Wenbo Cui, Zhongpu Xia, Yongliang Wang, Haoran Li, Dongbin Zhao
Cited by: 0

Abstract

Mobile manipulation is a fundamental capability for general-purpose robotic agents, requiring both coordinated control of the mobile base and manipulator and robust perception under dynamically changing viewpoints. However, existing approaches face two key challenges: strong coupling between base and arm actions complicates control optimization, and perceptual attention is often poorly allocated as viewpoints shift during mobile manipulation. We propose InCoM, an intent-driven perception and structured coordination framework for mobile manipulation. InCoM infers latent motion intent to dynamically reweight multi-scale perceptual features, enabling stage-adaptive allocation of perceptual attention. To support robust cross-modal perception, InCoM further incorporates a geometric-semantic affinity refinement mechanism that enhances multimodal correspondence. On the control side, we design a decoupled coordinated flow matching action decoder that explicitly models bidirectional base-arm interactions, alleviating optimization difficulties caused by control coupling. Under matched non-privileged observations and task-specific training data, InCoM improves mean success over DSPv2 by 33.0, 26.1, and 23.6 percentage points on SetTable, TidyHouse, and PrepareGroceries, respectively. Furthermore, its effectiveness is consistently validated in real-world mobile manipulation tasks, where InCoM maintains a superior success rate over existing baselines.

Fields of Study & Subjects

Robot Manipulation and Learning Reinforcement Learning in Robotics Robotics and Sensor-Based Localization

Publication details

Journal / source
arXiv (Cornell University)
Publication date
26 February 2026
Language
EN
Document type
Preprint