Implementation of BERT in Sentiment Analysis of National Digital Samsat (SIGNAL) User Reviews Based on Machine Learning - Savitri, Ratna; Rizki, Fido; Sobri, Ahmad (Department of Informatics Engineering, 2025)
The SIGNAL application facilitates online vehicle tax payments for the public. The application's quality is frequently evaluated through user reviews on platforms like the Google Play Store. This study aims to analyze the sentiment of SIGNAL user reviews using a Machine Learning-based approach, specifically the BERT (Bidirectional Encoder Representations from Transformers) model. The dataset consists of 20,000 user reviews. After preprocessing, the remaining data comprises 17,287 reviews, categorized into 12,758 positive reviews, 2,160 neutral reviews, and 2,369 negative reviews. To address data imbalance, the Random Over Sampling (ROS) technique was applied. The evaluation was performed using metrics such as accuracy, precision, recall, and F1-score. The results of the study indicate that the IndoBERT model can classify sentiments with an accuracy of 99% and a validation accuracy of 98% after five epochs of training. Confusion matrix analysis shows that the model achieved an overall accuracy of 99.72% on training data and 98.68% on testing data. This study demonstrates that the IndoBERT model is highly effective in classifying sentiment and makes a significant contribution to understanding the user experience of SIGNAL, which can serve as a foundation for future improvements to the application.