Evaluating chatGPT’s accuracy in understanding statistical concepts: Performance and implications for learning

Rahmawati, Fitriana
Noviyana, Hesti
Mauliana, Yunita
Pradana, Kenny Candra
Pratama, Ryan
Zakia, Atiqo Haya

Al-Jabar : Jurnal Pendidikan Matematika
Vol 16 No 2 (2025): Al-Jabar: Jurnal Pendidikan Matematika

Purpose: This study examines students’ ability to comprehend statistical concepts explained by ChatGPT and explores both students’ and lecturers’ perceptions regarding the clarity and instructional appropriateness of ChatGPT’s explanations within the framework of Intelligent Tutoring Systems (ITS), particularly in learning contexts that require high cognitive engagement. Method: A qualitative approach with a quasi-experimental design was employed. Participants were divided into an experimental group that utilized ChatGPT during learning activities and a control group that received conventional lecturer-led instruction. Data were gathered through quizzes, classroom observations, and Focus Group Discussions (FGDs) involving students and lecturers. The collected data were analyzed descriptively using the Miles and Huberman model to identify recurring themes related to conceptual understanding, learning behavior, and instructional effectiveness. Findings: The results reveal that most students experienced substantial difficulty in understanding statistical explanations generated by ChatGPT. Students frequently relied on copying answers without critically engaging with the reasoning process. Major obstacles included fragmented explanations, unfamiliar symbolic representations, and the use of technical language that exceeded students’ conceptual readiness, leading to weak conceptual understanding. Conversely, students who possessed a solid foundation in statistical concepts demonstrated better learning outcomes, even when using ChatGPT. Lecturers consistently emphasized that ChatGPT cannot substitute the pedagogical role of lecturers but may provide limited support when its use is guided and supervised. Significance: This study underscores that ChatGPT should be positioned as a supplementary instructional tool rather than a primary source of explanation in statistics education. Effective integration requires structured guidance, conceptual reinforcement, and the cultivation of students’ critical thinking skills to avoid misconceptions. These findings contribute to the discourse on responsible human AI collaboration in higher education and offer practical implications for implementing generative AI within ITS based learning environments.