STAMP: Spatial-Temporal Anchored Motion Planning for Zero-Shot Continuous Vision-and-Language Navigation

Read full text
Authors: T Liu, Xiaoyan Qi, Liuyi Wang, Jie Li, Xiao Lin, Minghao Zhu, Yulong Cui, C X Liu, Qijun Chen
Cited by: 1
References: 46

Abstract

Vision-and-Language Navigation in continuous environments (VLN-CE) requires embodied agents to ground natural language instructions into reliable long-horizon motion decisions under partial observability. Despite their strong semantic understanding and reasoning abilities, Multimodal Large Language Model (LVLM) struggle when directly applied to VLN, as they lack explicit spatial grounding, embodied memory, and awareness of geometric and reachability constraints, leading to perceptual misalignment and cascading decision errors in complex scenes. To address these limitations, we propose STAMP, a Spatial-Temporal Anchored Motion Planning framework for zero-shot VLN-CE, which systematically bridges the gap between pretrained world knowledge and embodied navigation. STAMP adopts a hierarchical design that decouples high-level semantic reasoning from low-level motion execution, enabling a frozen LVLM to operate over a structured, navigation-oriented abstraction. Its core novelty lies in a multimodal spatial-temporal anchoring mechanism that explicitly encodes instruction-relevant landmarks, action semantics, depth-aware geometry, and historical navigation context, together with an explicit Chain-of-Navigation reasoning process that constrains decision-making to navigation-critical cues. Furthermore, STAMP incrementally constructs an online, backtracking-enabled topological map, supporting robust planning under uncertainty. Extensive experiments demonstrate the effectiveness of the proposed STAMP framework, achieving performance comparable to state-of-the-art zero-shot methods on VLN-CE benchmarks and in real-world settings.

Fields of Study & Subjects

Multimodal Machine Learning Applications Robotic Path Planning Algorithms Robotics and Sensor-Based Localization

Publication details

Journal / source
Sensors
Publisher
MDPI AG
Volume
26
Issue
12
Pages
3698–3698
Publication date
10 June 2026
Language
EN