Development and validation of the Generative AI Literacy for Learning Scale (GenAI-LLs)

Abstract

Research on artificial intelligence (AI) in education has mainly focused on the measurement instruments relevant to learning about AI and framing AI literacy as learning about AI. The present study's focus, however, was on developing and validating a scale pertinent to learning with generative artificial intelligence (GenAI) in higher education. Specifically, this study aimed to develop and validate the Generative AI Literacy for Learning Scale (GenAI-LLs). For this purpose, data were collected from three groups of university students at three time points. Exploratory factor analysis was conducted to discover the psychometric structure of the scale. Confirmatory factor analysis was conducted to confirm the explored structure with the data collected from a separate group of participants. Further item and internal consistency analyses were conducted to ensure the validity and reliability of the scale. Test-retest analysis was applied to the data from a different group to ensure the stability of the scale. The findings showed that the GenAI-LLs is a valid and reliable instrument that can be used to measure and evaluate the literacy university students need to learn with GenAI in diverse learning environments, including face-to-face, open, distance and flexible learning. The findings are discussed based on the literature on GenAI, and recommendations for future studies are offered.

Implications for practice or policy:

• Instructors can use GenAI-LLs as a valid and reliable instrument to measure and evaluate university students’ GenAI literacy for learning.
• Instructors and curriculum developers may use GenAI-LLs as a framework – covering needs analysis, prompt and language skills, autonomous learning and critical thinking – during curriculum development and instructional design for integrating GenAI into learning environments.
• Higher education stakeholders should develop policies and actionable strategies for integrating GenAI into learning design, based on data from the GenAI-LLs.