Cross-Examiner: Evaluating Consistency of Large Language Model-Generated Explanations

Large language models are often prompted to explain their outputs to enhance accuracy, trust and transparency. However, evidence suggests that these explanations may appear plausible but do not align with the models’ true reasoning processes. One of the most promising approaches to identify unfaithful explanations is consistency checking, which involves probing the explanation with targeted follow-up questions. We present the cross-examiner framework (CES), a method for assessing explanation consistency by automatically generating targeted follow-up questions from a model’s own explanations. Our framework integrates structured information extraction with LLM-driven question generation, enabling the production of higher-quality and more diverse follow-up questions than LLMs alone. Unlike many existing approaches, CES does not require access to model internals. We evaluate CES against perturbation-based self-consistency methods and demonstrate its effectiveness in detecting inconsistencies across multiple LLMs on both bias benchmark question answering and logical deduction tasks.