Where's the harm? Screening student evaluations of teaching for offensive, threatening or distressing comments

Authors

DOI:

https://doi.org/10.14742/ajet.6133

Keywords:

offensive comments, student experience, survey screening, student evaluation, teaching

Abstract

Student evaluation surveys provide educational institutions with important feedback regarding the student experience of teaching and courses; however, qualitative comments can contain offensive, insulting or threatening content. Large educational institutions generate thousands of comments per academic term; therefore, manual screening processes to find potentially harmful comments are not generally feasible. We developed a methodology for semi-automated screening of student comments that incorporates a machine learning decision support system and a detailed psychological assessment protocol. In a case study at a large public Australian university, our system identified 4,258 out of 62,049 (6.9%) comments as potentially harmful and requiring further review. Feedback from stakeholders demonstrates that this methodology is useful in reducing staff workload and could be broadly applied to different settings.

Implications for practice or policy:

  • Educational institutions can adopt this methodology to dramatically decrease the number of working hours required to screen harmful free-text comments.
  • Researchers can use the proposed psychology-based assessment as an example of how to develop a protocol to categorise comments.
  • Educators and researchers can use this case study to follow best practices to develop their own decision support system that implements free-text comment classifiers.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

Downloads

Published

2022-04-19

How to Cite

Gibson, M. J., Luong, J., Cho, H. ., Moh, B., Zanin, S., Djatmiko, M. ., & Aandahl, R. Z. (2022). Where’s the harm? Screening student evaluations of teaching for offensive, threatening or distressing comments. Australasian Journal of Educational Technology, 1–14. https://doi.org/10.14742/ajet.6133

Issue

Section

Articles