When AI Gives Advice: Evaluating AI and Human Responses to Online Advice-Seeking for Well-Being

Read full text
Authors: Harsh Kumar, Jasmine Chahal, Yinuo Zhao, Zeling Zhang, Annika Z Wei, Louis Tay, Ashton Anderson
Cited by: 1
References: 73

Abstract

Seeking advice is a core human behavior that the internet has reinvented twice: first through forums and Q&A communities that crowdsource public guidance, and now through large language models (LLMs). Yet the quality of this LLM advice for everyday well-being scenarios remains unclear. How does it compare, not only against human comments, but against the wisdom of the online crowd? We ran two studies (N=210) in which experts compared top-voted Reddit advice with LLM-generated advice. LLMs ranked significantly higher overall and on effectiveness, warmth, and willingness to seek advice again. GPT-4o beat GPT-5 on all metrics except sycophancy, suggesting that benchmark gains need not improve advice-giving. In Study-2, we examined how human and algorithmic advice could be combined, and found that human advice can be unobtrusively polished to compete with AI-generated comments. We conclude with design implications for advice-giving agents and ecosystems blending AI, crowd input, and expert oversight.

Fields of Study & Subjects

AI in Service Interactions Artificial Intelligence in Healthcare and Education Ethics and Social Impacts of AI

Publication details

Journal / source
Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Pages
1–20
Publication date
13 April 2026