Integrating generative AI chatbot beyond classrooms: Exploring students’ interactions with chatbot and competencies in systems thinking

Abstract

Novice learners often struggle to engage with the complexity inherent in systems thinking. This study explores how a customised generative artificial intelligence (GenAI) chatbot can scaffold students’ systems thinking processes in a critical thinking and problem-solving skills module at an institute of higher education. Adopting a convergent mixed methods design, the study involved 52 students who interacted with the chatbot and completed an assignment addressing complex sustainability issues. Data from chatbot logs, reflection journals and assignments were analysed using rubrics that scored interaction depth and learning quality. Linear regression revealed a significant relationship between the depth of chatbot interaction and assignment quality. Building on these quantitative scores, content analysis of the chat logs was conducted to identify distinctive behaviours associated with each interaction level. For example, students with deep-level interactions often engaged in iterative questioning and elaboration, while those with moderate-level interactions sought feedback but did not always pursue further dialogue. Finally, interviews with six students provided insight into the factors shaping their interaction patterns, including how they perceived the role of the chatbot and their sense of agency.

Implications for practice or policy:

• Educators could model effective GenAI chatbot use by emphasising the intended pedagogical purpose of GenAI tools.
• Instructional designers could design complementary classroom activities to support diverse learners and extend GenAI-mediated learning.