ST-Booster: An Iterative Spatiotemporal Perception Booster for Vision-and-Language Navigation in Continuous Environments

Read full text
Authors: Yue Lu, Dongliang Zhou, Liang Xie, Erwei Yin, Feitian Zhang
Cited by: 0

Abstract

Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires agents to navigate unknown, continuous spaces based on natural language instructions. Compared to discrete settings, VLN-CE poses two core perception challenges. First, the absence of predefined observation points leads to heterogeneous visual memories and weakened global spatial correlations. Second, cumulative reconstruction errors in three-dimensional scenes introduce structural noise, impairing local feature perception. To address these challenges, this paper proposes ST-Booster, an iterative spatiotemporal booster that enhances navigation performance through multi-granularity perception and instruction-aware reasoning. ST-Booster consists of three key modules -- Hierarchical SpatioTemporal Encoding (HSTE), Multi-Granularity Aligned Fusion (MGAF), and ValueGuided Waypoint Generation (VGWG). HSTE encodes long-term global memory using topological graphs and captures shortterm local details via grid maps. MGAF aligns these dualmap representations with instructions through geometry-aware knowledge fusion. The resulting representations are iteratively refined through pretraining tasks. During reasoning, VGWG generates Guided Attention Heatmaps (GAHs) to explicitly model environment-instruction relevance and optimize waypoint selection. Extensive comparative experiments and performance analyses are conducted, demonstrating that ST-Booster outperforms existing state-of-the-art methods, particularly in complex, disturbance-prone environments.

Fields of Study & Subjects

Multimodal Machine Learning Applications Advanced Image and Video Retrieval Techniques Natural Language Processing Techniques

Publication details

Journal / source
IEEE Transactions on Multimedia
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Pages
1–12
Publication date
1 January 2026
Language
EN