Sistemasi: Jurnal Sistem Informasi
Vol 15, No 10 (2026): Sistemasi: Jurnal Sistem Informasi

Benchmarking IndoBERT and IndoBERTweet on Emotion and Sentiment of Indonesian App Reviews

Natalie Grace Widjaja Kuswanto (Universitas Ciputra)
Rinabi Tanamal (Universitas Ciputra)



Article Info

Publish Date
31 Oct 2026

Abstract

Indonesian mobile application reviews are informal free-text feedback. Their sentiment and emotion labels are costly to obtain and only moderately reliable. This study determines which pre-trained encoder, IndoBERT or IndoBERTweet, classifies emotion and sentiment more reliably in Indonesian app reviews, and tests whether combining or routing between the two encoders improves on the reference single model. IndoBERT was pre-trained on general Indonesian text and IndoBERTweet on Indonesian tweets. Both were fine-tuned for six-class emotion and three-class sentiment classification on 21,065 de-duplicated reviews. Five random seeds, McNemar tests, and paired bootstrap confidence intervals evaluated confidence-based and task-based routing, probability averaging, sentiment derived from emotion, and a lexical register analysis. IndoBERTweet outperformed IndoBERT on emotion, reaching a macro-F1 of 0.627 against 0.596, with the advantage concentrated on reviews containing slang. Because IndoBERTweet performed at least as well on both formal and informal reviews, confidence-based routing gave no additional gain. Averaging the probabilities of ten models raised emotion macro-F1 significantly to 0.652, an improvement of 0.024 over the reference single model. Sentiment derived from the predicted emotion was never worse than a dedicated sentiment head. It performed significantly better for a single IndoBERTweet model, reaching 0.807 against 0.796, eliminated 10.2 to 14.6% of reviews with contradictory separate heads, and halved inference cost. Single-model modifications that did not degrade performance converged near 0.62 to 0.63, consistent with annotation reliability limiting attainable performance. These findings give practitioners an evidence-based default configuration: IndoBERTweet as the encoder, with sentiment read directly from the predicted emotion label.

Copyrights © 2026






Journal Info

Abbrev

stmsi

Publisher

Subject

Computer Science & IT Electrical & Electronics Engineering

Description

Sistemasi adalah nama terbitan jurnal ilmiah dalam bidang ilmu sains komputer program studi Sistem Informasi Universitas Islam Indragiri, Tembilahan Riau. Jurnal Sistemasi Terbit 3x setahun yaitu bulan Januari, Mei dan September,Focus dan Scope Umum dari Sistemasi yaitu Bidang Sistem Informasi, ...