A pedagogical evaluation of an institution's digital assessment platform (DAP): Integrating pedagogical, technical and contextual factors

Authors

DOI:

https://doi.org/10.14742/ajet.9448

Keywords:

digital assessment, digital assessment platform, e-assessment, pedagogical evaluation, pedagogical and contextual indicators, institutional context

Abstract

An assessment for learning approach is foundational for student learning. The necessity to shift teaching and learning online as a response to COVID-19 has propelled digital assessment into the mainstream within higher education institutions. User experience is a common indicator of effectiveness of technologically enhanced initiatives; however, measuring the pedagogical impact of digital assessment initiatives has not been straightforward. This paper presents a pedagogical evaluation, incorporating staff and student perspectives, of a digital assessment platform (DAP) at a large Australian metropolitan university. Data were collected using surveys, interviews, focus groups and case studies and incorporated findings from student evaluations of courses and a student-staff partnership project. Results highlight strong pedagogical reasons for continuing the digital assessment initiative, while also identifying constraints and opportunities for improvement. The insights generated through this evaluation emphasise the importance of considering technical, pedagogical and contextual factors and the interplay between the factors. In a quality assurance climate, where the demand for this type of institution-wide evaluation is increasing, this study contributes an approach that addresses some of the complexities and challenges of evaluating digital assessment initiatives in higher education. Future research is needed to understand more fully the relationships between pedagogical and contextual factors when undertaking an evaluation.

 

 Implications for practice or policy:

  • The DAP enabled the (re)design of authentic, inclusive, engaging and secure assessment tasks.
  • Educator support is critical in learning the system and  creating assessment with high pedagogical merit.
  • Student familiarisation opportunities with using the DAP are beneficial.
  • The DAP’s longevity within the institution is crucial considering the significant effort required of educators and students.
  • Contextual influences blur understanding of pedagogical factors.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

Author Biographies

Associate Professor Christine Slade, Associate Professor in Higher Education, Institute for Teaching and Learning Innovation, The University of Queensland

ssociate Professor Christine Slade PFHEA is an academic at The University of Queensland’s Institute for Teaching and Learning Innovation (ITaLI). Her primary areas of expertise are assessment, academic integrity, and more recently artificial intelligence in education; she contributes to UQ’s strategic priorities in these areas. Christine has been recognised for her outstanding contributions to student learning and academic integrity. In 2022 she was awarded a UQ Citation for Outstanding Contributions to Student Learning for her assessment work. She is also a Principal Fellow of the Higher Education Academy UK, (the highest level of fellowship), awarded for sustained strategic impact and thought leadership in teaching and learning at international and institutional levels.  Christine is involved in several initiatives and projects aimed at promoting academic integrity and supporting schools in Queensland and nationally with their artificial intelligence priorities. She recently led a HERDSA-funded grant exploring why students use ‘buy-sell-trade’ filing sharing sites with the view to inform institutional decision making. Additionally, she was a finalist in the 2021 augural Tracey Bretag Prize with the UQ Academic Integrity Team, which she led. Christine has been part of numerous academic integrity development initiatives, including the TEQSA-funded national academic integrity workshops and toolkit of resources. She has also represented UQ on the international Epigeum Academic Integrity Development Collaborative, which designed contemporary academic integrity modules for students and staff.

Dr Kathleen Mahon, Senior Lecturer in Higher Education Innovation , Institute for Teaching and Learning Innovation, The University of Queensland

Kathleen has been a university educator and educational researcher based in Australia, and more recently Sweden, for over 14 years. Her teaching and research focus on higher education pedagogy, professional learning of university educators, conditions for educational practice, and research approaches that simultaneously investigate and nurture educational practice and praxis. Since completing her PhD on critical pedagogical praxis in higher education in 2014, Kathleen has become known internationally for her work on educational praxis and practice theory, with her co-authored publications on the theory of practice architectures being particularly widely cited.

Kathleen's role is to support UQ staff in their efforts to enhance their own teaching practice, their courses and programs, the experiences of their students, and their scholarly engagement with teaching and learning challenges and opportunities, and also to support the UQ community more broadly to enhance educational practices and conditions for teaching and learning across the University.

Kathleen holds the title of ‘Docent in Pedagogical Work’ in Sweden and is a co-founding editor of the Journal of Praxis in Higher Education. Her professional background includes twenty years of teaching and leading in schools and outdoor education settings in Queensland and the Northern Territory.

Jacqui Lynagh, Senior Project Manager, Teaching and Learning, Institute for Teaching and Learning Innovation, The University of Queensland

Jacqui has been working as a Learning Designer since 2012 and is currently a key member of the eAssessment project team, leading assessment redesign and the implementation of Inspera Assessment in Schools. Before moving across to ITaLI, Jacqui worked on curriculum re-development at UQ’s TC Beirne School of Law. Jacqui has a combined BA/LLB; P.G. Dip. Ed., and a Master of Education majoring in online learning and teaching. Her PhD (Education) investigates how young professionals exercise reflexivity to work through uncertainty, with the aim of using the findings to design curriculum that equips graduates to navigate the complexity that now characterises the systems and contexts of professional work.

Jacqui specialises in working collaboratively with subject matter experts to prepare relevant assessment and resources to effectively facilitate learning. Throughout her education and management careers, Jacqui has worked as a learning designer, curriculum writer, work-integrated learning researcher, e-moderator, educational consultant, law firm practice manager, and teacher.

Dom McGrath, Senior Manager, Teaching and Learning, Institute for Teaching and Learning Innovation, The University of Queensland

Dom has held learning design and management roles at UQ since 2008 particularly focusing on professional learning, assessment and digital learning initiatives.

Between being a UQ student and staff member, Dom worked as a teacher, events coordinator and software developer. More recently, he completed a Master’s of Education Studies, focusing on academic professional learning, and became a Senior Fellow of the HEA.

Dom has been part of teams that have received UQ and national awards for teaching and service.

Dr Karen Sheppard, Manager, Learning Design, Institute for Teaching and Learning Innovation, The University of Queensland

Karen is an experienced teacher who has worked in higher education since 2010. She holds a Master of Education (Professional Learning) and has completed a PhD examining the influence of data in education systems at The University of Queensland. Karen has experience in education evaluation, project management, policy development, curriculum and pedagogical transformation, and staff development.

Karen’s primary responsibilities at ITaLI include work on professional learning opportunities and staff development, evaluation design and implementation, curriculum and assessment design and peer observation. 

Professor Karen Benson, Director, Institute for Teaching and Learning Innovation, The University of Queensland

Karen has been employed by The University of Queensland (UQ) since 2001, initially as a finance academic and more recently in the Office of the Academic Board and the Institute for Teaching and Learning Innovation (ITaLI).

Over the last 12 years, she has held significant leadership roles, both within the Business School and at the broader University level. Her experience has resulted in a deep understanding of UQ’s teaching and learning goals, processes and policies which enable leadership and delivery of ITaLI's agenda.

With a background in finance education and research, Karen has strengths in curriculum design for large undergraduate and postgraduate programs, teaching diverse student groups, delivery of executive education, assessment design, and qualitative and quantitative research methods.

Downloads

Published

2024-10-23

How to Cite

Slade, C., Mahon, K., Lynagh, J., McGrath, D., Sheppard, K., Ahsan, S. Q. M., & Benson, K. (2024). A pedagogical evaluation of an institution’s digital assessment platform (DAP): Integrating pedagogical, technical and contextual factors. Australasian Journal of Educational Technology. https://doi.org/10.14742/ajet.9448

Issue

Early Release Articles

Section

Special Issue 2024 - Advancements in Technology-Enhanced Assessment in Higher Education

License

Copyright (c) 2024 Associate Professor Christine Slade, Dr Kathleen Mahon, Jacqui Lynagh, Dom McGrath, Dr Karen Sheppard, Qasim Ahsan, Professor Karen Benson

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC-ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY-NC-ND 4.0.

This copyright notice applies to articles published in AJET volumes 36 onwards. Please read about the copyright notices for previous volumes under Journal History.