Evaluating the Limitations of English Lexicon-Based Sentiment Analysis for Indonesian E-Wallet Reviews: A Comparison of VADER and Indonesian RoBERTa - Imam asrowardi; Septafiansyah Dwi Putra; Nadia Nabiha Dziqra (Bulletin of Network Engineer and Informatics, 2026)
Sentiment analysis of mobile application reviews supports service evaluation in the fast-growing Indonesian digital finance sector. This study examined whether an English lexicon-based method remains reliable for Indonesian-language reviews by comparing VADER with a fine-tuned Indonesian RoBERTa model. A total of 1,000 user reviews of the DANA e-wallet application were collected from Google Play and preprocessed through case folding, removal of numbers and punctuation, tokenization, and Indonesian stopword removal. Both methods classified each review as positive, neutral, or negative. VADER labelled 879 reviews as neutral, 102 as positive, and 19 as negative, whereas the Indonesian RoBERTa model produced a more balanced distribution of 362 negative, 327 positive, and 311 neutral reviews. The inter-method agreement, measured by Cohen's kappa, was only 0.027, indicating almost no agreement beyond chance. The results showed that VADER systematically assigned neutral labels because most Indonesian words were absent from its English lexicon, while the transformer model captured sentiment far more effectively. The findings demonstrated that language-specific transformer models are essential for sentiment analysis of Indonesian application reviews and that English lexicon-based tools are unsuitable for this task.