Indonesian Journal on Computing (Indo-JC)
Vol. 10 No. 1 (2025): August, 2025

Emotion Recognition from Text and Gesture Generation for an Early Marriage Counseling Chatbot in Lombok Using BERT

Adam Zahran Ramadhdan (Telkom University)
Rifki Wijaya (Telkom University)
Shaufiah (Telkom University)



Article Info

Publish Date
03 Oct 2025

Abstract

Early marriage remains a pressing issue among adolescents in Lombok, Indonesia, influenced by cultural norms, educational barriers, and economic challenges. This study develops an emotion classification and reason identification framework for a virtual counseling chatbot to support prevention efforts. Five functional emotion categories ‘Enthusiastic’, ‘Gentle’, ‘Analytical’, ‘Inspirational’, and ‘Cautionary’ were defined to capture counseling tones. The system leverages IndoBERT with a two-phase fine-tuning strategy. Phase 1 used a balanced dataset of 2,000 samples and achieved a macro F1-score of 0.95, while Phase 2 refined the model using 10,000 imbalanced pseudo-labeled samples, yielding a macro F1-score of 0.88 and improved sensitivity to minority classes. In addition, a semantic similarity-based reason identification module was implemented to classify user inputs into Education, Economy, Religion, or Culture categories, enhancing context awareness beyond simple keyword matching. Performance evaluation employed accuracy, precision, recall, and F1-score, supported by confusion matrices and training plots for generalization analysis. A descriptive emotion-to-gesture mapping was also designed to link each emotion category with static body pose visualizations, providing a conceptual basis for future multimodal applications.

Copyrights © 2025






Journal Info

Abbrev

indojc

Publisher

Subject

Computer Science & IT Control & Systems Engineering Education Engineering

Description

Indonesian Journal on Computing (Indo-JC) is an open access scientific journal intended to bring together researchers and practitioners dealing with the general field of computing. Indo-JC is published by School of Computing, Telkom University (Indonesia). The journal coverage includes, but is not ...