Evaluation of symptom checker formats to support health literacy and trust in AI: Results from an online randomised-controlled trial

Abstract Objectives Evaluate the impact of online symptom checker formats on symptom management knowledge, symptom checker trust and acceptability, and behavioural intentions. Design Two 5-arm parallel-group online randomised controlled trials. Setting Online survey. Participants Sample of 2062 Australian adults recruited through online research panel in June 2025. Almost half identified as man/male (49%) and woman/female (51%), median age 49 years (IQR=28). Participants viewed a hypothetical health scenario (fever and vomiting) followed by a screenshot of an online symptom checker from national health service provider, healthdirect. Interventions Participants were randomised to symptom acuity level (low: self-care at home; or moderate: see a General Practitioner (GP) in 24 hours) and one of five symptom checker formats. The standard format showed the existing healthdirect symptom checker advice. The remaining formats were ‘AI-enhanced’ versions. These included an AI-enhanced version with e.g. more tailored advice, rationale for acuity level, and AI disclosure statement. The other AI-enhanced formats had additional features: numbered steps, multimedia, and more detailed information about the use of AI. Main outcome measures Primary outcomes were intentions to follow the symptom checker’s self-care advice and intentions to see a GP in 24 hours. Secondary outcomes were trust in advice, knowledge of symptom management, and acceptability of the tool. All outcomes were assessed immediately post-intervention; knowledge was also assessed after 2 weeks. Results When advised to self-care at home, the AI-enhanced groups reported lower intentions to see a GP in 24 hours (median 3.00 out of 5), compared to the standard (original) tool (median 4.00; adjusted p = 0.003). There were no other significant effects on intentions. Immediately following the intervention, participants who viewed an AI-enhanced format reported greater knowledge about how to manage current and changing symptoms, across both acuity levels (adjusted p’s