Object Search in Partially-Known Environments via LLM-informed Model-based Planning and Prompt Selection

Read full text Summarise with VIRAT
Authors: Abhishek Paudel, Abhish Khanal, Raihan Islam Arnob, Shahriar Hossain, Stein, Gregory J.
Cited by: 0
References: 32

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

We present a novel LLM-informed model-based planning framework, and a novel prompt selection method, for object search in partially-known environments. Our approach uses an LLM to estimate statistics about the likelihood of finding the target object when searching various locations throughout the scene that, combined with travel costs extracted from the environment map, are used to instantiate a model, thus using the LLM to inform planning and achieve effective search performance. Moreover, the abstraction upon which our approach relies is amenable to deployment-time model selection via the recent offline replay approach, an insight we leverage to enable fast prompt and LLM selection during deployment. Simulation experiments demonstrate that our LLM-informed model-based planning approach outperforms the baseline planning strategy that fully relies on LLM and optimistic strategy with as much as 11.8% and 39.2% improvements respectively, and our bandit-like selection approach enables quick selection of best prompts and LLMs resulting in 6.5% lower average cost and 33.8% lower average cumulative regret over baseline UCB bandit selection. Real-robot experiments in an apartment demonstrate similar improvements and so further validate our approach.

Fields of Study & Subjects

AI-based Problem Solving and Planning Robotic Path Planning Algorithms Reinforcement Learning in Robotics

Publication details

Journal / source
arXiv (Cornell University)
Publication date
25 March 2026
Language
EN
Document type
Preprint