Expert judgment vs. psychometric analysis: Enhancing high school exam development in Albania

In Albania, high school graduates undergo State Matura exams that determine university entrance through a merit-based system. Since 2006, these pencil and paper exams have included three mandatory exams: Albanian language and literature, Mathematics, and a foreign language, plus one elective exam from a list of eight. Each exam totals 60 points: 20 points for multiple-choice items and 40 points for open-response items (structured and essay-type) covering Albanian literacy and a foreign language. Historically, item difficulty has been set primarily by expert judgment, with limited psychometric validation, and student proficiency has been computed via Classical Test Theory (CTT) on a 4-10 decimal scale. This study highlights the lack of psychometric analysis in Matura exams, emphasizing the need for improved assessment methods. By focusing on the limitations of relying solely on expert judgment in an era of technological innovations, we address the challenges posed by insufficient historical item parameters. To support expert judgment, we present a ShinyApp that integrates exam data to provide fast, transparent, and replicable psychometric insights. The tool demonstrates how technology can support evidence-based decision-making and contribute to modernizing Albania’s e assessment framework.