Test Quality Assessment and Adaptive Algorithm Based on IRT Models

Abstract

In this paper the algorithm for adaptive testing of students’ knowledge in distance learning and an assessment of its effectiveness in the educational process has been proposed. The results of the study are based on the achievements of modern testing theory IRT. The objective of the study was to build test items that allow adequately assessing the student’s achievements and automating the assessment process using an adaptive algorithm. To achieve this goal, mathematical models of modern testing theory IRT were used, namely, the Rasch model, 2-PL and 3-PL Birnbaum models. The important outcomes of this work are a thorough analysis of the developed test tasks, identification of their shortcomings and automation of the process of assessing students’ knowledge using an adaptive algorithm based on the methods of modern testing theory IRT. The paper provides an overview of the results of the application of modern test theory, a description and block diagram of the proposed algorithm and the results of its application in the real educational process. The effectiveness of using this algorithm for the objective assessment of students’ knowledge has been experimentally shown. The test quality has been assessed using the IRT models.