Indonesian Journal of Instruction
Vol. 6 No. 3 (2025): September

Evaluating Students’ Artificial Intelligence Literacy: Development of an Assessment Instrument Based on Experiential Learning Elements

Ahmad Ahmad (Department of Non-Formal Education, Faculty of Education, State University of Malang, Malang, Indonesia)
Lasi Purwito (Universitas Negeri Malang, East Java, Indonesia)
R. Anggia Listyaningrum (Universitas Negeri Malang, East Java, Indonesia)
Riyan Hidayat (Universiti Putra Malaysia, Selangor, Malaysia)



Article Info

Publish Date
25 Sep 2025

Abstract

Artificial Intelligence (AI) literacy has become an essential competency for students in responding to rapid digital transformation across various fields. However, the limited availability of valid and reliable assessment instruments constrains the comprehensive evaluation of students’ AI literacy levels. This study aims to analyze, develop, and validate an AI literacy assessment instrument based on experiential learning elements and a deep learning approach. The study employed a Research and Development (R&D) design consisting of needs analysis, instrument design, development, validation testing, and data analysis stages. The research involved 500 students selected through multistage sampling techniques. Data were collected using an AI literacy questionnaire designed to measure three main components: knowledge, affective attitudes, and critical thinking skills, which are integrated with global competencies required in the 21st century. Construct validity was examined using factor analysis, while reliability was assessed through internal consistency analysis. The findings indicate that the developed instrument demonstrates strong validity and reliability and effectively represents the multidimensional structure of AI literacy. These results suggest that the instrument can function as a reliable assessment tool for evaluating students’ readiness to understand, utilize, and critically reflect on AI technologies in an ethical and responsible manner. The study provides important implications for higher education institutions in integrating AI literacy assessment into teaching practices and curriculum development.

Copyrights © 2025






Journal Info

Abbrev

IJI

Publisher

Subject

Social Sciences

Description

Indonesian Journal of Instruction (IJI) is an internationally recognized journal in the field of education and is published four times a year (in January, mei september). The aim of this journal is to publish high quality studies in the areas of instruction, learning, teaching, curriculum ...