When LLMs Listen to Experts: Accurate Failure Diagnosis in Operating Systems

Efficient failure diagnosis is critical to maintaining the stability and reliability of operating systems (OSes) in modern industrial environments. In practice, manual failure analysis by on-call engineers (OCEs) faces increasing challenges due to the growing complexity of OS failures, while recent automated diagnosis methods often suffer from low explainability, limiting their practical adoption. Large Language Models (LLMs) hold promise for advancing automated failure diagnosis through their sophisticated reasoning and language generation capabilities. However, traditional LLM-based solutions struggle to integrate domain knowledge and lack the effective interaction mechanisms required for industrial troubleshooting. To address these practical challenges, we present OScope, an automated, explainable failure diagnosis framework powered by LLMs. OScope leverages historical failure cases to enhance the semantic understanding of anomalies, enabling precise retrieval of relevant troubleshooting guides. The framework further structures the diagnosis process using standard operating procedure (SOP) templates, supporting step-by-step verification and correction with corresponding SOP documents. Importantly, OScope facilitates human-in-the-loop collaboration, allowing OCEs to interact with the system for report refinement and practical feedback. We have evaluated OScope on a real-world OS failure dataset collected from Alibaba. Results show that OScope achieves an AC@5 of 90%, significantly outperforming baseline methods and demonstrating high diagnostic value in production settings. The diagnostic reports generated by OScope have also received positive feedback from OCEs for readability and practical usefulness. Since deployment at Alibaba, OScope has substantially improved the efficiency of engineers in resolving failures, highlighting its tangible impact as a successful case of applying automated software engineering methods in industry.