Measuring Teaching Efficacy Through Comprehension Delta: Large Language Models as Standardized Learner Proxies
Abstract
Measuring teaching quality remains one of the most persistent challenges in educational research. Existing approaches depend on classroom observation protocols that require trained human raters, or on student evaluations that correlate poorly with actual learning outcomes. Both suffer from subjectivity, scalability constraints, and well-documented biases. This paper proposes a novel measurement construct, the dual-run comprehension delta, which reframes teaching efficacy as the measurable change in a standardized evaluator's comprehension before and after receiving a teacher's explanation. We argue that large language models can serve as controlled learner proxies for this purpose, and we develop a formal framework in which a domain-specific assessment is administered to an LLM twice: once without access to the teacher's explanation, and once with it. The score difference constitutes the delta. We ground this construct in established validity theory from the Standards for Educational and Psychological Testing, analyze it through the lens of pedagogical content knowledge and explanation quality research, and present a reference architecture that instantiates the methodology. We argue that the dual-run delta offers a reproducible and scalable complement to existing teaching quality measures, particularly suited for asynchronous instruction, teacher certification, and professional development contexts. We outline a validity argument organized around Kane's interpretation/use framework, discuss the construct boundaries of the approach, and identify the empirical studies required to evaluate its claims.
Fields of Study & Subjects
Evaluation of Teaching Practices
Teacher Education and Assessments
Intelligent Tutoring Systems and Adaptive Learning
Publication details
- Publication date
- 6 March 2026
- Language
- EN
- Document type
- Preprint