Ida Bagus Indrabudhi Kusuma
School of Computing, Telkom University, Indonesia

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

Comparative Evaluation of Retrieval Augmented Generation Based Large Language Model with Formative Feedback and Traditional Instruction in Learning Fraction Ida Bagus Indrabudhi Kusuma; Dade Nurjanah
Jurnal Teknik Informatika (Jutif) Vol. 7 No. 5 (2026): JUTIF Volume 7, Number 5, October 2026
Publisher : Informatika, Universitas Jenderal Soedirman

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.52436/1.jutif.2026.7.5.5847

Abstract

Learning fractions poses challenges for students, and while traditional one-on-one tutoring is effective, it faces human resource limitations. While generative artificial intelligence (AI) offers scalable personalized support, large language models (LLMs) risk generating hallucinations, leading students to misconceptions. To address these challenges, this study evaluates the effect of a Retrieval-Augmented Generation (RAG) based Intelligent Tutoring System (ITS) integrated with formative feedback on students' learning performance in fraction arithmetic, comparing it to traditional instruction. A quasi-experimental study was conducted with 56 7th grader students, divided into an experimental group (N=26) utilizing the ITS and a control group (N=30) receiving regular classroom instruction. The ITS employed GPT-4o and RAG grounded with fraction educational content to deliver personalized, step-by-step formative feedback based on the Mason and Bruning framework. Students’ learning performance was measured through pre and post-test and were analyzed using ANCOVA. The experimental group outperformed the control group with higher learning gain of 47.02% compared to the control group’s 31.54%. ANCOVA results confirmed the significant main effect (p = 0.002) suggesting that the RAG-based tutor improved the mathematical comprehension and reduced knowledge variance among students. These results underscore the grounded LLM integrated with formative feedback demonstrates potential to improve students learning performance. This study provides a comparative evaluation of RAG-based ITS integrated with formative feedback and traditional classroom instruction in improving student learning performance in fraction. Furthermore, this study advances AI-supported tutoring by introducing an LLM-based ITS that combines RAG-grounded generation, formative feedback, and LLM-based judgment for dynamic and adaptive student modelling.