Information Technology Education Journal
Vol. 4, No. 2, May (2025)

Indonesian Automated Essay Scoring: A Comparative Study of Pretrained Transformer Models

Pulung Hendro Prastyo (Unknown)
Eddy Tungadi (Unknown)
Shaifudin Zuhdi (Unknown)



Article Info

Publish Date
02 Jun 2025

Abstract

Manual essay scoring is often characterized by inefficiency and inconsistency. This process is notably time-consuming, leading to delayed feedback and increased susceptibility to evaluator fatigue and subjective bias, thereby posing significant challenges. Automated Essay Scoring (AES) offers a scalable, robust, and consistent solution to these issues. However, the performance of AES models can vary considerably depending on the specific application. Therefore, this study evaluated ten Indonesian pretrained transformer models from Hugging Face for AES tasks, using 300 essay responses from a Research Methodology quiz at Politeknik Negeri Ujung Pandang. Performance was assessed using Root Mean Square Error (RMSE) and Quadratic Weighted Kappa (QWK). Among the evaluated models, Indobenchmark/indobert-base-p2 (BERT-02) demonstrated superior performance. It achieved the lowest RMSE of 5.664 and the highest QWK score of 0.6745. The findings suggest that BERT-02 is the most effective model for Indonesian AES tasks. Future research could explore larger datasets and different models to further enhance the performance and understanding of Indonesian AES systems.

Copyrights © 2025






Journal Info

Abbrev

INTEC

Publisher

Subject

Computer Science & IT Education

Description

INTEC Journal is published by the Informatics and Computer Engineering Education Study Program at Makassar State University. INTEC Journal is published periodically three times a year, containing articles on research results and / or critical studies in the field of Informatics and Computer ...