ADAPTING THE DYNAMIC MODEL OF EDUCATIONAL EFFECTIVENESS FOR TEXT-BASED ANALYSIS – CODEBOOK DEVELOPMENT AND CONTENT VALIDATION FRAMEWORK

Read full text Summarise with VIRAT
Authors: Jogaila Vaitekaitis, Leonidas E. Kyriakides, Mantas Lukoševičius
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

As lesson transcripts become increasingly accessible, there is a growing need for tools that can capture teaching quality from text alone, laying the foundation for scalable, AI-driven analysis. This study adapts the classroom-level components of the Dynamic Model of Educational Effectiveness (DMEE) into a low-inference codebook specifically optimized for textual data. The goal is to preserve the theoretical integrity of the original in-class observation instruments while ensuring that each indicator is operationalized through explicit verbal evidence, creating a robust framework suitable for both human annotation and future machine learning applications. We started by analyzing all existing elements of low- and high-inference observation instruments, adding new sub-items to capture characteristics specific to classroom transcripts. Each element was reviewed to determine its dependence on nonverbal cues, precise timing, or holistic assessment. The elements were then classified according to a systematic discernibility scheme: textually discernible, conditionally discernible, impossible to discern from the text, or omitted due to redundancy. Where possible, complex indicators were reformulated into smaller verbal events so that the main construct could still be presented in a concise manner. This article describes in detail the methodological framework used to develop the final instrument. To establish content validity, we employed a structured expert panel review in which experts in the DMEE independently assessed the relevance and clarity of each indicator. The analysis showed high content validity, and only minor improvements were needed based on expert feedback. In the validated codebook, more than a hundred indicators are grouped into conceptual levels that distinguish simple turn-level codes from context-dependent or derived metrics. The developed framework offers researchers and practitioners a DMEE theory-grounded, content-validated approach for analysing lesson transcripts at scale. It clearly shows which aspects of effective teaching can be captured from text alone and which remain poorly represented, thus defining a realistic scope for text-based classroom observation and laying the groundwork for subsequent reliability testing and semi-automated analysis. Published version: INTED2026 Proceedings, Article 1441, IATED. DOI: https://doi.org/10.21125/inted.2026.1441. Publisher citation: https://library.iated.org/view/VAITEKAITIS2026ADA. The original published PDF is deposited with IATED's written permission dated 15 July 2026. This permission permits self-archiving and does not grant a Creative Commons licence or additional reuse rights. Supplementary item-level adaptation ledger (Appendix A): https://doi.org/10.5281/zenodo.21196611. Funding: Research Council of Lithuania (LMT), agreement No. S-PD-24-109; application registration No. P-PD-23-133-PATIKSLINTA; KTU internal contract No. SV3240042.

Fields of Study & Subjects

Evaluation of Teaching Practices Educational Research and Methods Computational and Text Analysis Methods

Publication details

Journal / source
INTED proceedings
Volume
1
Publication date
1 March 2026
Language
EN