Differences in the educational software evaluation process for experts and novice students

Abstract

This comparative case study investigated the educational software evaluation processes of both experts and novices in conjunction with a software evaluation checklist. Twenty novice elementary education students, divided into groups of five, and three experts participated. Each novice group and the three experts evaluated educational software selected by the novice groups in accordance with the checklist. Data were collected through focus group interviews, classroom observations and document analysis. Evaluation processes were analysed through thematic comparisons. The results showed that the expert-novice agreement rate was as low as 48%, with novice students tending to grade the software higher. While the experts used a systematic approach, including understanding and assessing each criterion, supporting the process with literature and evaluating the software as a whole, novice students lacked such methods, indicating a need for additional training and development.