Learning to Localize Reference Trajectories in Image-Space for Visual Navigation

We present LoTIS, a model for visual navigation that provides robot-agnostic image-space guidance by localizing a reference RGB trajectory in the robot's current view, without requiring camera calibration, poses, or robot-specific training.Instead of predicting actions tied to specific robots, we predict the image-space coordinates of the reference trajectory as they would appear in the robot's current view.This creates robotagnostic visual guidance that easily integrates with local planning.Consequently, our model's predictions provide guidance zeroshot across diverse embodiments.By decoupling perception from action and learning to localize trajectory points rather than imitate behavioral priors, we enable a cross-trajectory training strategy that learns robust invariance to viewpoint and camera changes.We outperform state-of-the-art methods by 20-50 percentage points in success rate on forward navigation, and paired with a local planner we achieve 94-98% success rate across diverse sim and real environments.Furthermore, we achieve over 5× improvements on challenging tasks where baselines fail, such as backward traversal.The system is lightweight (∼ 22 Hz on Jetson Orin AGX) and straightforward to use: we show how even a video from a handheld phone camera directly enables different robots to navigate to any point on the trajectory.Videos, demo, and code are available at https://finnbusch.com/lotis.