DEVELOPMENT AND OPTIMIZATION OF SIMPLE-O: AN AUTOMATED ESSAY SCORING SYSTEM FOR JAPANESE LANGUAGE BASED ON BERT, BILSTM, AND BIGRU - Junaidi, Muhammad Aidan Daffa; Ratna, Anak Agung Putri (International Journal of Electrical, Computer, and Biomedical Engineering, 2025)
This study aims to develop an Automatic Essay Scoring System (SIMPLE-O) for Japanese essays, consisting of five short essay questions. SIMPLE-O is designed to enhance scoring accuracy by leveraging deep learning models such as BERT, BiLSTM, and BiGRU. The research evaluates deep-level score predictions for each question, rather than only considering the total score across the five questions, to provide more reliable and accurate assessments. SIMPLE-O compares student responses with three predefined answer keys using two similarity measurement methods: Cosine Similarity and Manhattan Distance. The study employs datasets developed through data augmentation techniques applied to lecturer and student responses. The system is implemented using Python, and its performance is evaluated through analyses of various architectures based on specified hyperparameters. The best results were achieved using a BERT-BiLSTM architecture with the Cosine Similarity method, configured with a batch size of 8, 256 hidden state units, a learning rate of 0.00001, and 100 epochs. The evaluation demonstrated that this approach achieved a Mean Absolute Percentage Error (MAPE) of 7.230% and an average score difference of 5.689. This research highlights the potential of SIMPLE-O for automated scoring of Japanese essays, offering improved accuracy, reliability, and deeper analytical insights.