A bow tie-grounded sentence-level annotation framework for interpretable AI analysis of process safety incident narratives

Process safety reports contain significant causal and operational details, yet much of that information remains embedded in free text and is therefore difficult to use consistently for structured analysis, comparison, and learning across events. Existing process safety frameworks provide important conceptual structure, but they are not operationalized for sentence-level annotation, and prior natural language processing (NLP) work has often focused on document-level classification. To address this gap, the proposed framework defines six mutually exclusive sentence categories aligned with the bow tie causal sequence, together with explicit annotation guidelines, boundary rules, and an optional secondary failure-mode layer for barrier issue sentences. The framework was developed and evaluated using U.S. Chemical Safety Board incident summary reports. Validation included independent expert annotation and large language model (LLM) consistency analysis. The results show near-perfect human agreement (Cohen’s κ = 0.975), almost perfect agreement across three independent LLMs (Fleiss’ κ = 0.886), and substantial agreement between human and model labels (Cohen’s κ = 0.745–0.770). Confusion patterns and embedding-based analysis further indicate that the categories are meaningfully separable. These findings support the proposed framework as a stable and interpretable basis for sentence-level analysis of process safety incident narratives. This work provides a practical methodological bridge between domain-grounded incident analysis and future AI-assisted process safety applications.