A Hybrid NLP Framework for Measuring Therapist Empathy in Online Counseling Responses

Abstract Empathy is central to effective therapeutic communication, but measuring empathy in large-scale online counseling interactions remains challenging. This paper presents a hybrid natural language processing framework for estimating therapist empathy in online counseling responses. We define empathy using linguistic and semantic markers such as emotional acknowledgment, validation, supportiveness, reflective listening, reassurance, and non-directive language. We then fine-tune a BERT-based classifier to identify empathetic expressions and combine its output with sentiment scores and normalized response length to construct an interpretable empathy score. The framework is applied to therapist responses from the CounselChat dataset, which contains real online counseling questions and therapist answers. We evaluate the model using classifier performance, compare the generated empathy scores with ChatGPT-based scores as a consistency check, and examine whether estimated empathy is associated with user engagement measures such as views and upvotes. The results suggest that transformer-based models can capture meaningful empathy-related language patterns in therapist responses, while the weak relationship between empathy scores and engagement indicates that platform metrics capture only part of perceived therapeutic quality. The study highlights the potential of NLP methods for large-scale analysis of therapeutic communication while emphasizing the need for expert clinical annotations and stronger validation in future work.