Indonesian Automated Essay Scoring: A Comparative Study of Pretrained Transformer Models - Pulung Hendro Prastyo; Eddy Tungadi; Shaifudin Zuhdi (Jurusan Teknik Informatika dan Komputer, 2025)
Manual essay scoring is often characterized by inefficiency and inconsistency. This process is notably time-consuming, leading to delayed feedback and increased susceptibility to evaluator fatigue and subjective bias, thereby posing significant challenges. Automated Essay Scoring (AES) offers a scalable, robust, and consistent solution to these issues. However, the performance of AES models can vary considerably depending on the specific application. Therefore, this study evaluated ten Indonesian pretrained transformer models from Hugging Face for AES tasks, using 300 essay responses from a Research Methodology quiz at Politeknik Negeri Ujung Pandang. Performance was assessed using Root Mean Square Error (RMSE) and Quadratic Weighted Kappa (QWK). Among the evaluated models, Indobenchmark/indobert-base-p2 (BERT-02) demonstrated superior performance. It achieved the lowest RMSE of 5.664 and the highest QWK score of 0.6745. The findings suggest that BERT-02 is the most effective model for Indonesian AES tasks. Future research could explore larger datasets and different models to further enhance the performance and understanding of Indonesian AES systems.