IAES International Journal of Artificial Intelligence (IJ-AI)
Vol 14, No 3: June 2025

Generative artificial intelligence as an evaluator and feedback tool in distance learning: a case study on law implementation

Nurdiana, Dian (Unknown)
Maulana, Muhamad Riyan (Unknown)
Hasanah, Siti Hadijah (Unknown)
Chairunnisa, Madiha Dzakiyyah (Unknown)
Komuna, Avelyn Pingkan (Unknown)
Rif'an, Muhammad (Unknown)



Article Info

Publish Date
01 Jun 2025

Abstract

The development of generative artificial intelligence (GAI) has impacted various fields, including higher education. This research examines the use of GAI as an evaluator and feedback provider in distance legal education. This study tested five GAI models: ChatGPT, Perplexity, Gemini, Bing, and You, using a sample of 20 students and evaluations from legal experts. Descriptive statistical analysis and non-parametric tests, including Wilcoxon, intraclass correlation coefficient (ICC), Kappa, and Kendall's W, were used to assess accuracy, feedback quality, and usability. The results showed that ChatGPT was the most effective GAI, with the highest mean scores of 4.22 from experts and 4.12 from students, followed by Gemini with scores of 4.15 and 4.07. In terms of binary judgement accuracy, Gemini scored 80%, ChatGPT 60%, while Perplexity, Bing, and You had lower scores. Statistical analysis showed moderate agreement (ICC=0.439) and low alignment (Kappa=-0.058) between the GAIs and expert evaluations, with a Kendall's W value of 0.576 indicating moderate consistency in judgements. These findings emphasize the importance of selecting effective GAIs such as ChatGPT and Gemini to improve academic evaluation and learning in legal education, and pave the way for further innovations in the use of AI.

Copyrights © 2025






Journal Info

Abbrev

IJAI

Publisher

Subject

Computer Science & IT Engineering

Description

IAES International Journal of Artificial Intelligence (IJ-AI) publishes articles in the field of artificial intelligence (AI). The scope covers all artificial intelligence area and its application in the following topics: neural networks; fuzzy logic; simulated biological evolution algorithms (like ...