Evaluating an LLM-Powered Chatbot for Cognitive Restructuring: Insights from Users and Mental Health Professionals

Recent advances in large language models (LLMs) create opportunities to expand access to mental healthcare. An emerging body of work focuses on LLM-based cognitive restructuring (CR), yet existing systems are often limited to generating alternative beliefs or engaging in short, scripted exchanges, offering little of the sustained interactivity required for effective restructuring. In this work, we present CRBot , an LLM-powered chatbot designed with mental health professionals to deliver CR. We conducted a two-week user study with 19 participants, complemented by expert review of conversation transcripts. Results show that CRBot can scaffold the core steps of CR, support self-reflection through Socratic questioning, and provide effective psychoeducation. At the same time, challenges arose, including rigidity, oversimplified questioning, pathologization of appropriate emotions, and misinterpretation of nuanced context. We conclude with design implications for building more interactive, flexible, and ethically robust LLM-enabled CR systems.