The TME Framework: Multimodal Learner Modeling for Active Listening Skills in Collaborative Problem Solving

Active listening is an important social–emotional skill for productive and inclusive collaboration, yet it remains difficult to assess in small-group collaborative problem-solving (CPS) activities. Teachers cannot continuously observe multiple groups simultaneously, while existing assessment approaches rely either on self-report—which is prone to bias—or on manual coding, which is labor-intensive and difficult to scale. We present the Theory-Measurement-Evaluation (TME) framework, a theory-based and explainable approach for modeling active listening in CPS grounded in the HURIER active listening framework. TME comprises three interlocking models. First, a Theory Model operationalizes HURIER into eight constructs, each translated into observable behavioral indicators. Second, a Measurement Model automatically extracts construct-aligned multimodal behavioral traces from audio and video and situates them within layered collaboration context derived from idea-evolution analysis across multiple time scales (micro-moments, collaboration phases, and whole-conversation dynamics). Building on evidence generated by the Measurement Model, we introduce an Evaluation Model that applies Bayesian Skill Tracing to estimate time-varying construct expression while preserving uncertainty. This produces an interpretable Multimodal Learner Model that combines quantitative metrics, exemplary behaviors, and descriptive narratives, and can be used by both human stakeholders and intelligent agents for downstream applications such as providing feedback or personalized learning. We evaluate the faithfulness and usefulness of the learner model outputs through expert ratings. Results suggest that TME can assess active listening skills from multimodal CPS data in a scalable and interpretable way.