Evaluating Supportive LLM Behavior Over Multiple Turns across Demographics

Large Language Models (LLMs) are increasingly deployed as first-line mental health support, yet evaluations often rely on single-turn prompts that miss how support unfolds over time. In real help-seeking, users disclose context incrementally, requiring the assistant to sustain coherent support across turns rather than responding to a fully specified one-shot prompt [12, 16]. We introduce a multi-turn simulation framework to audit supportive LLM behavior across demographic contexts. By simulating user agents that gradually reveal pre-existing Reddit narratives to a support agent, we compare response patterns across gender and parenting roles. We find a “Dad Deficit” in sentiment category agreement: mothers’ emotional tone is matched 80.6% of the time versus 45.7% for fathers (p