Claim Missing Document
Check
Articles

Found 2 Documents
Search
Journal : Journal of Applied Data Sciences

Enhancing the Performance of Machine Learning Algorithm for Intent Sentiment Analysis on Village Fund Topic Anam, M. Khairul; Putra, Pandu Pratama; Malik, Rio Andika; Karfindo, Karfindo; Putra, Teri Ade; Elva, Yesri; Mahessya, Raja Ayu; Firdaus, Muhammad Bambang; Ikhsan, Ikhsan; Gunawan, Chichi Rizka
Journal of Applied Data Sciences Vol 6, No 2: MAY 2025
Publisher : Bright Publisher

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47738/jads.v6i2.637

Abstract

This study explores the implementation of Intent Sentiment Analysis on Twitter data related to the Village Fund program, leveraging Multinomial Naïve Bayes (MNB) and enhancing it with Synthetic Minority Over-sampling Technique (SMOTE) and XGBoost (XGB). The analysis categorizes tweets into six labels: Optimistic, Pessimistic, Advice, Satire, Appreciation, and No Intent. Initially, the MNB model achieved an accuracy of 67% on a 90:10 data split. By applying SMOTE, accuracy improved by 12%, reaching 89%. However, adding Chi-Square feature selection did not increase accuracy further. Incorporating XGB into the MNB+SMOTE model led to a 6% improvement, achieving a final accuracy of 95%. Comprehensive model evaluation revealed that the MNB+SMOTE+XGB model achieved 96% accuracy, 96% precision, 96% recall, and a 96% F1-score, with an AUC of 99%, categorizing it as excellent. These findings demonstrate that the combination of SMOTE for addressing class imbalance and XGBoost for boosting performance significantly enhances the MNB model's classification capabilities. The novelty lies in the integration of these techniques to improve intent sentiment classification for public opinion analysis on the Village Fund program. The results indicate that the majority of tweets labeled as "No Intent" reflect a lack of specific sentiment or actionable intent, providing valuable insights into public perception of the program.
IndoBERT-SupCon: A Supervised Contrastive Learning Model for Analyzing Public Perception on Halal Tourism Octafia, Sri Mona; Malik, Rio Andika; Weriframayeni, Annisa; Delpa, Delpa
Journal of Applied Data Sciences Vol 7, No 1: January 2026
Publisher : Bright Publisher

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47738/jads.v7i1.1045

Abstract

The primary objective of this research is to develop and evaluate a robust deep learning model for accurately analyzing stakeholder perceptions of halal tourism development in Pariaman, West Sumatra, based on qualitative textual data. The main contribution is the introduction of IndoBERT-SupCon, a novel architecture that enhances the Indonesian BERT model with a Supervised Contrastive Learning (SupCon) mechanism. A novel method for producing more discriminative feature representations for complex viewpoints is presented in this paper, which is one of the first to use this sophisticated fine-tuning technique to Indonesian socio-political sentiment analysis. Conceptually, the model is trained to simultaneously minimize classification error while optimizing the feature space, pulling representations of similar sentiments closer together and pushing dissimilar ones further apart. To achieve this, we collected 1,022 primary textual responses through online surveys with tourists and in-depth interviews with key stakeholders, including SME owners and government officials. The SMOTE oversampling technique was employed on the training data to mitigate class imbalance. Experimental results on the test data demonstrate that the IndoBERT-SupCon model achieved outstanding performance, with a final accuracy of 96.59% and a macro F1-score of 0.97. These results significantly surpass the performance of a standard fine-tuned IndoBERT baseline, confirming the effectiveness of the SupCon approach. The findings provide the Pariaman local government with a highly valid, data-driven tool for more responsive and effective policy formulation. This research offers a robust framework that can be applied to other public policy domains, showcasing the value of advanced deep learning in transforming qualitative stakeholder feedback into actionable insights.