Online course quality evaluation instruments: A scoping review

Abstract

How do we make judgements about the quality of online courses? Checklists and rubrics are commonplace in higher education for establishing and measuring design features of online courses. They are created and used by institutions, academics and educational designers to standardise measures for quality online course design. Despite an intensifying spotlight on quality learning and teaching in higher education, no large-scale review of course quality instruments has occurred. This scoping review aimed to ascertain the conceptions of quality being promoted by course quality evaluation instruments and the capability-building resources that underpin these instruments. Seventy-five instruments used to measure quality in online course design in higher education were identified via a systematic search. These instruments were charted and coded. This paper reports on findings that summarise the key attributes of the course quality evaluation instruments, conceptualises a shared definition of course quality and proposes specific core criteria for measuring course quality under the domains of learning design, assessment and evaluation, usability and accessibility, social interaction and technology. The findings indicate that course quality instruments and their support resources have capability-building potential; yet, mostly this potential is not fully realised. We recommend shifting the capability-building potential of these tools from checking compliance to enabling skills development.

Implications for practice:

• Teaching academics can ensure course quality by using evaluation instruments featuring the five overarching conceptualisations of course quality.
• Teaching academics should customise quality evaluation instruments around agreed standards and definitions of the elements of course quality.
• In collaboration with teaching academics, third-space professionals should leverage the affordances of course quality evaluation instruments to build staff capability and agency.