<![CDATA[Purpose: This study develops and evaluates a fine-tuned IndoBERT model for detecting ChatGPT-generated Indonesian text, addressing the gap left by AI-generated text detection systems built primarily for English. Design/methodology/approach – The M4 Indonesian dataset (5,774 samples after preprocessing, group-split by article source identifier into 4,619 training, 577 validation, and 578 test samples to keep matched human-written and ChatGPT-generated pairs from leaking across partitions) was used as the basis of this study. A fixed 1,000-sample subset of the training partition was used to fine-tune IndoBERT for binary text classification using PyTorch and Hugging Face Transformers (learning rate 2e-5, three epochs, AdamW) and to train the support vector machine and Naïve Bayes benchmarks under identical conditions. Performance was evaluated on the full 578-sample test set using accuracy, precision, recall and F1-score. Findings: On 578 held-out test samples, IndoBERT reached 95.85% accuracy, 92.57% precision, 100% recall, and an F1-score of 0.9614 (macro F1 = 0.9582), against 97.06% for SVM and 93.08% for Naive Bayes. An exact McNemar's test found no statistically significant difference between IndoBERT and SVM (p = .324), indicating that IndoBERT's advantage under an earlier, non-grouped split largely reflected data leakage rather than a genuine architectural advantage. Research implications/limitations – The dataset is limited to the news domain and to text generated by a single model (ChatGPT); generalisation to academic text, such as student essays, and to outputs from other LLMs, such as Gemini or LLaMA, requires further validation. Originality/value – This study demonstrates that language-specific fine-tuning of IndoBERT can support academic integrity efforts in Indonesia, offering a practical and reproducible detection pipeline that can be adapted and further validated to identify ChatGPT-generated text in educational settings.]]>
Copyrights © 2026
