Article Per Year (5 Year)
p-Index From 2021 - 2026
0.23
P-Index
This Author published in this journals
All Journal Jurnal Nasional Pendidikan Teknik Informatika (JANAPATI)
Agung Wicaksono
Directorat of Information System, Universitas Terbuka, Tangerang Selatan, Indonesia
Computer Science & IT Education Engineering

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

 1 
Artificial Intelligence in Essay Scoring: A Comparative Study of Transformer-Based Models Unggul Utan Sufandi; Dwi Astuti Aprijani; Bachriah Fatwa Dhini; Agung Wicaksono
Jurnal Nasional Pendidikan Teknik Informatika: JANAPATI Vol. 15 No. 1 (2026)
Publisher : Prodi Pendidikan Teknik Informatika Universitas Pendidikan Ganesha

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.23887/janapati.v15i1.103823

Abstract

We compare transformer-based approaches for automated essay scoring (AES) on 3,077 student essays using two reference answers (RA1, RA2) and two models (all-MiniLM-L6-v2, all-mpnet-base-v2). Accuracy is defined as within ±10 points of human scores. all-MiniLM-L6-v2 yields the best accuracy—57.16% (RA1) and 61.13% (RA2) with stop-words kept—while all-mpnet-base-v2 attains 55.37% (RA1) and 59.59% (RA2). Removing stop-words generally reduces accuracy by ~1–2.5 points (e.g., MiniLM RA2: 61.13% → 58.70%). On efficiency, mpnet is ≈2.5× faster (avg 0.041–0.036 s/essay) than MiniLM (0.102–0.094 s/essay) across settings, indicating a clear accuracy–throughput trade-off. Complementary error metrics show MAE as low as 9.75, RMSE 12.95–14.26, and correlation up to 0.556, corroborating moderate but usable agreement. Practically, MiniLM is preferable for high-stakes precision, whereas mpnet favors large-scale, time-critical deployments. Findings also suggest retaining common words helps preserve syntactic/contextual cues, slightly boosting similarity-based scoring. We conclude with deployment guidance: report within-±10 accuracy alongside MAE/RMSE and correlation, choose the model per operational goal (precision vs. speed), and consider hybrid enhancements (rubric features + embeddings) to raise reliability for inclusive, at-scale AES in higher education.
Co-Authors Bachriah Fatwa Dhini Dwi Astuti Aprijani Unggul Utan Sufandi