When AI Gives Advice: Evaluating AI and Human Responses to Online Advice-Seeking for Well-Being
Abstract
Seeking advice is a core human behavior that the internet has reinvented twice: first through forums and Q&A communities that crowdsource public guidance, and now through large language models (LLMs). Yet the quality of this LLM advice for everyday well-being scenarios remains unclear. How does it compare, not only against human comments, but against the wisdom of the online crowd? We ran two studies (N=210) in which experts compared top-voted Reddit advice with LLM-generated advice. LLMs ranked significantly higher overall and on effectiveness, warmth, and willingness to seek advice again. GPT-4o beat GPT-5 on all metrics except sycophancy, suggesting that benchmark gains need not improve advice-giving. In Study-2, we examined how human and algorithmic advice could be combined, and found that human advice can be unobtrusively polished to compete with AI-generated comments. We conclude with design implications for advice-giving agents and ecosystems blending AI, crowd input, and expert oversight.
Fields of Study & Subjects
AI in Service Interactions
Artificial Intelligence in Healthcare and Education
Ethics and Social Impacts of AI
Publication details
- Journal / source
- Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Pages
- 1–20
- Publication date
- 13 April 2026