Jahda Rusti Putri
Sriwijaya University

Published : 2 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 2 Documents
Search

SENTIMENT ANALYSIS USING MACHINE LEARNING FOR DIGITAL SERVICE DEVELOPMENT Rugaiyah Balqis; Jahda Rusti Putri; Mira Afrina; Ali Ibrahim; Fathoni Fathoni
JURTEKSI (jurnal Teknologi dan Sistem Informasi) Vol. 12 No. 2 (2026): Maret 2026
Publisher : Lembaga Penelitian dan Pengabdian Kepada Masyarakat (LPPM) STMIK Royal Kisaran

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.33330/jurteksi.v12i2.4476

Abstract

Abstract: The rapid growth of e-commerce mobile applications has generated large volumes of user reviews, making manual sentiment analysis increasingly impractical. This study aims to compare the effectiveness of three machine learning algorithms Support Vector Machine (SVM), Random Forest, and Naive Bayes for automated sentiment classification of Indonesian-language mobile application reviews. A dataset of 3,000 user reviews from the RupaRupa application on the Google Play Store was collected and preprocessed through normalization, tokenization, stopword removal, and stemming. TF-IDF vectorization was applied for feature extraction, while the Synthetic Minority Over-sampling Technique (SMOTE) was used to address class imbalance across three sentiment categories: positive, negative, and neutral. The results show that SVM achieved the highest accuracy of 90.02%, while Random Forest obtained the best F1-score of 88.08% when sufficient training data were available. Naive Bayes demonstrated relatively stable performance across varying training data sizes. Furthermore, TF-IDF keyword analysis revealed that negative reviews were primarily associated with delivery issues, technical problems, and pricing concerns. These findings demonstrate the effectiveness of machine learning approaches for sentiment classification and provide practical insights for improving mobile application services. Keywords: sentiment analysis; machine learning; SMOTE; TF-IDF; text classification Abstrak: Pertumbuhan pesat aplikasi mobile e-commerce telah menghasilkan volume ulasan pengguna yang sangat besar, sehingga analisis sentimen secara manual menjadi semakin tidak praktis. Penelitian ini bertujuan untuk membandingkan efektivitas tiga algoritma machine learning Support Vector Machine (SVM), Random Forest, dan Naive Bayes dalam melakukan klasifikasi sentimen otomatis terhadap ulasan aplikasi mobile berbahasa Indonesia. Dataset yang digunakan terdiri dari 3.000 ulasan pengguna aplikasi RupaRupa yang dikumpulkan dari Google Play Store. Data kemudian diproses melalui tahapan preprocessing yang meliputi normalisasi, tokenisasi, penghapusan stopword, dan stemming. Ekstraksi fitur dilakukan menggunakan metode Term Frequency–Inverse Document Frequency (TF-IDF), sedangkan ketidakseimbangan kelas ditangani menggunakan Synthetic Minority Over-sampling Technique (SMOTE) pada tiga kategori sentimen, yaitu positif, negatif, dan netral. Hasil penelitian menunjukkan bahwa SVM mencapai tingkat akurasi tertinggi sebesar 90,02%, sementara Random Forest memperoleh nilai F1-score terbaik sebesar 88,08% ketika tersedia data pelatihan yang memadai. Naive Bayes menunjukkan performa yang relatif stabil pada berbagai ukuran data pelatihan. Selain itu, analisis kata kunci berbasis TF-IDF mengungkapkan bahwa ulasan negatif terutama berkaitan dengan masalah pengiriman, kendala teknis aplikasi, dan isu harga. Temuan ini menunjukkan bahwa pendekatan machine learning efektif untuk klasifikasi sentimen serta memberikan wawasan yang bermanfaat dalam meningkatkan kualitas layanan aplikasi mobile. Kata Kunci: analisis sentimen; pembelajaran mesin; SMOTE; TF-IDF; klasifikasi teks.
Class-Level Behavior Analysis under Metric Disagreement in Imbalanced Multi-Label Indonesian Emotion Classification Jahda Rusti Putri; Ermatita; Abdiansah
Journal of Information System and Informatics Vol 8 No 3 (2026): June
Publisher : Asosiasi Doktor Sistem Informasi Indonesia

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.63158/journalisi.v8i3.1664

Abstract

This study aims to analyze class-level model behavior under metric disagreement in imbalanced multi-label Indonesian emotion classification, using the divergence between Macro F1 and Micro F1 as a diagnostic signal rather than a mere performance indicator. A machine-translated Indonesian version of the GoEmotions dataset, comprising approximately 58,000 samples across 28 fine-grained emotion categories, is used as the experimental setting. The translated dataset was not manually revalidated, and findings are scoped to this translated GoEmotions setting. Two transformer-based models are evaluated: IndoBERT, a monolingual Indonesian model, and DistilBERT, a multilingual model, both fine-tuned with class-specific threshold optimization. The results reveal opposing divergence patterns: IndoBERT achieves higher Micro F1 than Macro F1, indicating performance concentrated on high-frequency classes, while DistilBERT exhibits the reverse pattern, suggesting broader but less precise label activation. Per-class analysis further shows that most minority classes consistently fall into unstable or non-functional performance regimes across both models. This study concludes that aggregate metrics alone are insufficient for evaluating model behavior in imbalanced multi-label settings. A behavior-oriented interpretation framework for Macro–Micro F1 divergence and a regime-based class reliability categorization are proposed to support more structured and informative evaluation practices.