XplainCVSS: Interpretable Machine Learning for Explainable CVSS Vulnerability Severity Prediction

Automating CVSS scoring can reduce analyst workload, but practical deployment requires explanations that are not only interpretable, but also faithful to model decisions. We present XplainCVSS, a framework that predicts all CVSS v3.1 base metrics from NVD vulnerability descriptions and evaluates explanation faithfulness quantitatively. Using 173,202 CVEs with strictly temporal splits, we benchmark ten models from TF-IDF+XGBoost to domainspecific and multi-task transformers. Fine-tuned SecureBERT achieves the best performance (macro-F1 = 0.774; reconstructed base-score MAE = 0.934), while multi-task variants offer competitive consistency at 7–10x lower training cost. We validate TreeSHAP and gradient-based transformer attributions across all eight CVSS components, showing that highlighted features are decision-relevant. A temporal drift decomposition further shows that label-distribution shift (concept drift) is the main driver of degradation (delta-F1 = 0.079), indicating the need for periodic retraining. Riskaware evaluation yields a low 2.79% under-prediction rate for Critical vulnerabilities, supporting analyst-in-the-loop triage. Overall, XplainCVSS highlights a practical trade-off between transformer accuracy and the direct interpretability of keyword-based models.