A Counterfactual Explanation Framework for Retrieval Models

Explainability has become a crucial concern in modern machine learning and deep learning because it improves model transparency.Information retrieval (IR) is no exception.In the existing literature on explainable IR, the emphasis has predominantly been on explaining relevance with respect to a retrieval model.Typical questions include why a document is relevant to a query, why one document is more relevant than another, or why a particular set of documents is considered relevant for a query.However, limited attention has been given to understanding why a document is not favored by a retrieval model for a query (e.g., why it does not appear in the top-K results).To address this gap, we study the following question: which terms should be added to a document to improve its ranking?This, in turn, helps explain which missing words contribute to a document not being favored by a retrieval model for a given query.We use a counterfactual framework to address this problem.To the best of our knowledge, this is the first attempt to study this specific counterfactual setting, namely, identifying which missing words can affect a document's ranking.Our experiments demonstrate the effectiveness of the proposed approach for both statistical models (e.g., BM25) and deeplearning-based models (e.g., DRMM, DSSM, ColBERT, MonoT5).The code for our approach is available at https://github.com/ TheProParadox/Explainable-IR.