A Multi-Criteria Software Quality Evaluation of AI Meeting Assistants for English-Medium University Lectures Using ISO/IEC 25010 and TOPSIS

This study aims to systematically evaluate the effectiveness of ten Artificial Intelligence Meeting Assistants (AIMAs) in supporting English-medium university lectures. The research was carried out at The Chinese University of Hong Kong and involved both teachers and students as stakeholders. Using a within-subjects design, twenty participants (twelve students and eight teaching staff, recruited through snowball sampling across multiple faculties) tested each AIMA in simulated lecture contexts. Data were collected through structured questionnaires based on the ISO/IEC 25010 software quality framework, covering nine criteria including functional suitability, performance efficiency, compatibility, usability, reliability, security, satisfaction, sustainability, and scalability. The Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) was used to combine participants’ importance ratings and performance scores, resulting in a final ranking of the AIMAs. TOPSIS analysis of participant evaluations ranked Tl;dv most favorably, followed by Grain and Microsoft Teams. Notably, teachers rated security (p < 0.001) and performance efficiency (p = 0.009) significantly higher than students, highlighting differing user priorities. This study provides empirical benchmarks and a replicable framework for selecting educational technologies. The findings may help institutions make evidence-based decisions about using AIMAs to improve student understanding and participation in linguistically diverse classrooms.