Where's the harm? Screening student evaluations of teaching for offensive, threatening or distressing comments
Keywords:offensive comments, student experience, survey screening, student evaluation, teaching
Student evaluation surveys provide educational institutions with important feedback regarding the student experience of teaching and courses; however, qualitative comments can contain offensive, insulting or threatening content. Large educational institutions generate thousands of comments per academic term; therefore, manual screening processes to find potentially harmful comments are not generally feasible. We developed a methodology for semi-automated screening of student comments that incorporates a machine learning decision support system and a detailed psychological assessment protocol. In a case study at a large public Australian university, our system identified 4,258 out of 62,049 (6.9%) comments as potentially harmful and requiring further review. Feedback from stakeholders demonstrates that this methodology is useful in reducing staff workload and could be broadly applied to different settings.
Implications for practice or policy:
- Educational institutions can adopt this methodology to dramatically decrease the number of working hours required to screen harmful free-text comments.
- Researchers can use the proposed psychology-based assessment as an example of how to develop a protocol to categorise comments.
- Educators and researchers can use this case study to follow best practices to develop their own decision support system that implements free-text comment classifiers.
How to Cite
Copyright (c) 2022 Matthew J. Gibson, Justin Luong, Hanbit Cho, Bryan Moh, Simone Zanin, Mentari Djatmiko, R. Zach Aandahl
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC-ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY-NC-ND 4.0.
This copyright notice applies to articles published in AJET volumes 36 onwards. Please read about the copyright notices for previous volumes under Journal History.