Interactive computer based assessment tasks: How problem-solving process data can inform instruction

Abstract

This article presents key steps in the design and analysis of a computer based problem-solving assessment featuring interactive tasks. The purpose of the assessment is to support targeted instruction for students by diagnosing strengths and weaknesses at different stages of problem-solving. The first focus of this article is the task piloting methodology, which demonstrates the relationship between process data and a priori documented problem-solving behaviours. This work culminated in the design of a Microsoft Excel template for data transcription named a Temporal Evidence Map. The second focus of this article is to illustrate how evidence from process data can be accumulated to produce and report instructionally useful information not available through traditional assessment approaches. This is demonstrated through the production of reports profiling individual student outcomes against important aspects of problem solving.