Antimicrobial Stewardship in the Era of AI: A Head-to-Head Comparison of a Machine Learning HTL Algorithm and Large Language Models in Real-World Infectious Disease Cases

Background: Persistently, antimicrobial resistance is one of the major global public health challenges, associated with 4.95 million deaths in 2019, of which 1.27 million were directly attributable to resistant bacterial infections. Among all types, urinary tract infections (UTIs) are among the most common worldwide, with 4.49 billion cases in 2021, and they affect older adults the most. In this context, previous studies suggest an exacerbation in the prevalence of resistant pathogens (such as extended-spectrum β-lactamase (ESBL)-producing Enterobacteriaceae), a circumstance associated with delayed initiation of appropriate treatment, increased mortality, prolonged hospital stays, and selective pressure. These circumstances are more difficult to confront if the limited access to infectious disease specialists is considered, particularly in low- and middle-income countries. In this situation, artificial intelligence (AI) tools have emerged as a potential strategy to support antimicrobial decision-making.Methods: This observational cross-sectional concordance study evaluated the antimicrobial recommendations generated by three AI models: a human-in-the-loop machine learning (ML-HTL) for clinical decision support and two general-purpose language models (GPT-4.0 and Gemini 3.0). A total of 88 cases with positive urine cultures and complete clinical data were included. The AI-generated recommendations for each case were contrasted with the consensus response of an independent panel of three infectious disease specialists. Agreement was assessed using overall concordance, sensitivity, and Cohen’s kappa (κ), with 95% confidence intervals (CI).Results: The mean age of patients was 65.3 years; 81.8% were women, and 80.7% had uncomplicated UTIs. Escherichia coli was the most frequent pathogen (76.1%). Among isolates, 37.5% were ESBL-producing, 53.4% were fluoroquinolone-resistant, and 44.3% were multidrug-resistant. The ML-HTL system showed the highest agreement with the expert´s consensus (90.9%; 95% CI: 83.1–95.3), followed by GPT-4 (63.6%; 95% CI: 53.2–72.9) and Gemini (37.5%; 95% CI: 28.1–47.9). Also, ML-HTL showed high sensitivity and strong agreement (87.5%, κ = 0.89), whereas GPT-4 showed low agreement (κ = 0.32), and Gemini showed less than chance agreement (κ = −0.18).Conclusions: The ML-HTL system, specifically designed for antimicrobial optimization, may better align with experts' consensus than general-purpose language models, supporting their role in clinical decision-making.