Do Explanations Matter? Investigating Faithfulness and Developer Trust in LLM-Based Defect Prediction

Software defect prediction (SDP) helps developers identify defect-prone code, but understanding why a model produces a prediction remains challenging. Although language models offer promising capabilities for source-code analysis, the relationship between explanation faithfulness and developer-perceived usefulness remains insufficiently understood. We investigate this relationship through an empirical study combining prediction performance, quantitative explanation evaluation, and developer perceptions. We compare four language-model-based defect predictors with traditional machine learning baselines on CodeXGLUE and PROMISE JM1. We further evaluate seven explainable AI (XAI) methods using Comprehensiveness and Sufficiency on HybridCodeBERT and conduct a study with 25 practitioners examiningexplanation clarity, trust, actionability, and preference. On CodeXGLUE, HybridCodeBERT achieves a 27.2\% relative F1-score improvement over the traditional baseline, while conventional classifiers retain higher accuracy on PROMISE JM1. Gradient-weighted Attention Rollout achieves the highest observed explanation faithfulness, whereasparticipants most frequently prefer SHAP. Despite generally positive clarity and actionability ratings, the traditional baseline receives the highest direct trust. These findings highlight a distinction between model-level explanation faithfulness and developer-perceived usefulness. Our study provides empirical insights into evaluating explainable defect prediction systems from both computational and human-centered perspectives.