Evaluation of the Reproducibility and Consistency of the RUST Radiographic Scale in Tibial Fracture Healing: Comparison Between Orthopedic Residents and Artificial Intelligence

Abstract Purpose To evaluate the reproducibility and consistency of the Radiographic Union Score for Tibial fractures (RUST) in diaphyseal fractures treated with intramedullary nailing among orthopedic residents with different levels of experience, and to compare their performance with artificial intelligence (AI) analysis. Methods Radiographs from 47 patients were assessed by four orthopedic residents (R1–R4) at two independent time points using the RUST scale. The same images were evaluated by AI. Intra- and interobserver agreement were calculated using the Intraclass Correlation Coefficient (ICC). Differences between human and AI evaluations were analyzed using Wilcoxon and Friedman tests with a 5% significance level. Results The sample consisted predominantly of men (71.7%) with a mean age of 32.9 years. AI classified 61.7% of fractures as consolidated (score ≥ 7). Absolute agreement between residents and AI was 17%, with residents overestimating scores in 45.2% and underestimating in 37.8% of cases. Observer R3 showed the best agreement with AI (ICC = 0.58), while R2 demonstrated a significant difference (p = 0.009). Interobserver reproducibility among residents was excellent (ICC = 0.93; 95% CI 0.84–0.96) but decreased to 0.61 when AI was included. Intraobserver consistency was very good (global ICC = 0.72; 95% CI 0.63–0.78), with R4 presenting the highest stability (ICC = 0.78). Conclusion The RUST scale showed high reproducibility among residents but limited agreement with AI, suggesting the need for additional training to improve consistency between human interpretation and algorithmic assessment.