JOURNAL OF INFORMATION SYSTEM RESEARCH (JOSH)
Vol 8 No 1 (2026): October 2026

Comparing Machine Learning and BERT for Classifying Grammatical Error Types in Indonesian EFL Essays

Meily Ivane Esther Neman (Universitas Klabat, Minahasa Utara)
George Morris William Tangka (Universitas Klabat, Minahasa Utara)
Regi Fernando Najoan (Universitas Klabat, Minahasa Utara)
Enrico Samuel Djimesha (Universitas Klabat, Minahasa Utara)
Daniel Semuel Warouw (Universitas Klabat, Minahasa Utara)



Article Info

Publish Date
31 Oct 2026

Abstract

Automated writing feedback tools are usually trained on learner corpora from outside Indonesia, so their accuracy on Indonesian students' grammatical errors is unknown. This study compares Multinomial Naive Bayes, a linear Support Vector Machine (SVM), Random Forest, and a fine-tuned BERT model for classifying nine grammatical error types. The models were trained on 48,011 error edits from the BEA-2019 W&I+LOCNESS and FCE corpora, re-annotated with ERRANT 3, and tested in-domain on 5,129 edits and cross-corpus on 1,900 edits from 66 essays by 33 Indonesian university students in the ICNALE Edited Essays. BERT achieved the highest macro-F1 on both test sets, 0.917 in-domain and 0.849 on the Indonesian essays, and McNemar tests confirmed its advantage over each classical model. All models lost 0.035 to 0.068 macro-F1 under the corpus shift. Noun number errors formed 24% of the Indonesian errors, three times their training share, while word form and pronoun were the hardest classes. This study contributes a cross-corpus benchmark of classical and Transformer-based error-type classifiers on Indonesian students' essays, a reproducible pipeline that labels both corpora with the same ERRANT version, and a class-level reliability profile for classroom feedback. For English teachers in Indonesia, the results support explicit, contrastive practice on articles and plural marking and the use of BERT for immediate feedback on reliably recognised classes, while its word form and pronoun suggestions are confirmed by the teacher.

Copyrights © 2026






Journal Info

Abbrev

josh

Publisher

Subject

Computer Science & IT Decision Sciences, Operations Research & Management

Description

Artikel yang dimuat melalui proses Blind Review oleh Jurnal JOSH, dengan mempertimbangkan antara lain: terpenuhinya persyaratan baku publikasi jurnal, metodologi riset yang digunakan, dan signifikansi kontribusi hasil riset terhadap pengembangan keilmuan bidang teknologi dan informasi. Fokus Journal ...