Viviana Purba
Universitas Pembangunan Nasional “Veteran” Jawa Timur

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

Multilabel Aspect-Based Emotion Analysis Pada Ulasan Aplikasi IKD: Pengaruh Focal Loss dan Threshold Tuning Menggunakan Indobert Viviana Purba; Eka Dyar Wahyuni; Tri Luhur Indayanti Sugata
Jurnal Ilmu Komputer dan Sistem Informasi Vol. 5 No. 2 (2025): Mei 2026
Publisher : LKP Unity Academy

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.70340/jirsi.v5i2.448

Abstract

User reviews of the Identitas Kependudukan Digital (IKD) application contain various emotions toward different service aspects. These reviews not only reflect the level of service satisfaction but also encompass user experiences, complaints, expectations, and public perceptions regarding the quality of the system. This study aims to develop a multi-label Aspect-Based Emotion Analysis (ABEA) model using an end-to-end IndoBERT architecture to identify user emotions across each service aspect of the IKD application. Additionally, it analyzes the impact of implementing Focal Loss and threshold tuning on classification performance under highly imbalanced label distributions. Data were collected from 13,197 user reviews on the Google Play Store spanning from June 2024 to November 2025 using web scraping methods, which were subsequently cleaned and filtered to yield 6,891 data entries. Service aspects were empirically identified using BERTopics. Labeling was conducted by three human annotators and two AI annotators, with the final labels determined through majority voting. The model was developed across 6 experimental scenarios varying in preprocessing, Focal Loss, threshold tuning, and data split ratios. Evaluation was performed using F1 Score Macro, F1 Score Micro, Precision, Recall, and Hamming Loss metrics. BERTopic achieved a Coherence Score of 0.6196 and a Topic Diversity of 0.92 with 5 representative aspects. The most optimal model was obtained using a configuration of Focal Loss, a threshold of 0.4, and a 60:20:20 split ratio, achieving an F1 Score Macro of 0.3916, a 24.1% increase from the baseline, alongside an F1 Score Micro of 0.9134 and a Recall of 0.9423. The selected model was successfully integrated into a web-based system using the Flask framework to visualize the classification results. Anger dominated the reviews concerning the Login & Akses Akun and Scan Barcode ke Dukcapil aspects, whereas the Dokumen & Layanan Digital aspect recorded the highest joy emotion. The combination of Focal Loss and threshold tuning proved effective in handling imbalanced label distributions in Indonesian multi-label ABEA classification.