Zero-Shot Pre-Retrieval Prompt Performance Prediction for Large Language Models

Query Performance Prediction (QPP) plays a central role in information retrieval by estimating how effectively a query will perform before retrieval. However, traditional pre-retrieval QPPs rely on corpus-level statistics such as term frequency or document frequency information that is unavailable in the context of Large Language Models (LLMs), where model parameters and training data are inaccessible. In addition, post-response analysis across many LLMs is computationally costly and impractical at scale. To overcome these challenges, this work introduces a novel zero-shot framework forPrompt Performance Prediction, designed to estimate an LLM’s expertise in handling a given prompt without relying on any generated responses. The framework proposes a new family ofprompt predictorsthat capture the lexical, topical, intentional, and cognitive dimensions of the prompt text. These predictors, collectively referred to asPrompt Performance Predictors, enable pre-response estimation of model suitability based solely on the textual characteristics of the prompt. In addition, the study explores the integration of multiple predictors within a hybrid neural learning framework to assess their individual and combined contributions. Experimental evaluation on 1,130 LLMs from the TREC Million LLM Track demonstrates that the proposed prompt predictors significantly outperform traditional pre-retrieval QPPs in predicting model expertise. The results further reveal that combining diverse predictors yields the highest overall effectiveness, confirming that prompt-level linguistic, semantic, and intent features provide complementary and reliable signals for LLM expertise prediction.