Beyond test accuracy: Benefits of measuring response time in computerised testing
AbstractThis paper investigates how monitoring the time spent on a question in a test of basic mathematics skills can provide insights into learning processes, the quality of test takers' knowledge, and cognitive demands and performance of test items that otherwise would remain undiscovered if the usual test outcome of accuracy only format ("correct/incorrect") was used. Data from three tests of basic mathematics skills taken by tertiary students in 2004-2006 were analysed. Means and distributions of individual response times on parallel test questions were examined and differences were further investigated.
Analysis of response time data revealed a number of surprising findings in regard to the impact of variables on preferences for written and mental calculation methods and regarding additional cognitive demands of a question. The study examined how simple statistical analysis of response time distribution can be used to investigate abnormalities of the item functioning. These findings may be of value to educators and to test producers by informing them about the potential of utilising response time measurements as a diagnostic facility in computerised tests, for the purposes of improving teaching and learning.
Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC-ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY-NC-ND 4.0. Users have the right to read, download, copy, distribute, print, search, or link to the full texts of articles in this journal, and to use them for any other lawful purpose.
Articles published in AJET can be copied, communicated and shared in their published form for non-commercial purposes provided full attribution is given to the author and the journal. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal’s published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
This copyright notice applies to articles published in AJET volumes 36 onwards. Please read about the copyright notices for previous volumes under Journal History.