Claim Missing Document
Check
Articles

Found 2 Documents
Search
Journal : Journal of Intelligent Systems Technology and Informatics

Automatic Sentiment Annotation Using Grok AI for Opinion Mining in a University Learning Management System Julianto, Indri Tri; Sidqi, Muhammad Affan Al
Journal of Intelligent Systems Technology and Informatics Vol 1 No 3 (2025): JISTICS, November 2025
Publisher : Aliansi Peneliti Informatika

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.64878/jistics.v1i3.42

Abstract

Sentiment analysis has become an essential tool in evaluating user feedback on digital learning platforms. Understanding student sentiments toward Learning Management Systems (LMS) in higher education can offer critical insights for system development and service improvement. This study aims to evaluate the effectiveness of AI-assisted sentiment labeling using Grok AI and ChatGPT compared to manual labeling for sentiment classification of student opinions on LMS at Institut Teknologi Garut. The research involved distributing an online questionnaire to 96 students across four academic levels, collecting open-ended responses regarding their LMS usage experiences. These responses were preprocessed through case folding, cleaning, tokenization, stopword removal, and stemming. The sentiment labels were assigned using Grok AI, ChatGPT, and manual annotation, and the resulting datasets were used to build classification models using the Naïve Bayes algorithm in Altair RapidMiner with 10-Fold Cross Validation. The performance evaluation shows that manual labeling yielded the highest accuracy (52.22%) and Cohen's Kappa (0.137), followed by ChatGPT (50.11%, 0.119) and Grok AI (48.00%, 0.087). Word cloud visualizations further revealed the dominant themes within each sentiment class, indicating that positive opinions emphasized helpfulness and ease of use, while negative ones focused on access issues and system lags. This research suggests that AI-assisted labeling methods can be viable alternatives, although manual labeling still offers slightly better accuracy.
User Sentiment Analysis X Towards Makan Bergizi Gratis Program Using Automatic Labeling Technique with Deepseek AI Julianto, Indri Tri; Nurpajar, Dini Siti
Journal of Intelligent Systems Technology and Informatics Vol 1 No 2 (2025): JISTICS, July 2025
Publisher : Aliansi Peneliti Informatika

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.64878/jistics.v1i2.43

Abstract

Public perception of national nutrition initiatives is instrumental in shaping inclusive and data-driven policy development. In Indonesia, the "Makan Bergizi Gratis" (MBG) program introduced by President Prabowo has drawn significant attention, particularly on the X platform (formerly Twitter). This research topic was selected due to its national urgency and political significance, as the MBG program emerged as a key agenda during the 2024–2025 political transition. Therefore, examining public sentiment is essential to assess policy acceptance and identify areas for improvement. This study analyzes user sentiment toward the MBG policy using an automatic labeling approach supported by DeepSeek AI and the VADER Lexicon, followed by sentiment classification through the K-Nearest Neighbor (KNN) algorithm. The research involved five main stages: collecting 1,704 tweets from X between January 2024 and March 2025, preprocessing the text, conducting automatic sentiment labeling, applying TF-IDF for vectorization, handling class imbalance using the Synthetic Minority Over-sampling Technique (SMOTE), and classifying sentiments using KNN. The results indicate that without SMOTE, the VADER model achieved higher accuracy (93.49%) but lower Cohen's Kappa (0.16), while DeepSeek AI yielded lower accuracy (73.67%) but slightly higher Kappa (0.17). After SMOTE was applied, accuracy declined (VADER to 77.25%, DeepSeek AI to 64.72%), but Kappa scores improved significantly (VADER to 0.65, DeepSeek AI to 0.47), indicating more balanced and consistent sentiment predictions across classes. In conclusion, integrating automatic labeling, SMOTE, and KNN provides a reliable and scalable framework for analyzing large-scale sentiment on social media platforms, particularly in contexts with imbalanced opinion distributions.
Co-Authors Abdullah, Angga Abdulrohman, Muhammad Haviz Ade Sutedi Ade Sutedi, Ade Aditriyana, Muhammad Rizky Agisni Nurlela, Agni Agreindra Helmiawan, Muhammad Akhdan Hidayat, Fairuz Alamsyah, Restu Alisha Fauzia, Fathia Apriliani, Insani Ardana, Alwan Arif Rahman, Rifal Arif Syamsudin, Muhammad Asri Mulyani B. Balilo Jr , Benedicto B. Balilo Jr, Benedicto Balilo Jr, Benedicto B. Baswardono, Wiyoga Burhanudin, Asep Chaerunisa, Adinda Citra Indahsari, Ajeng Dede Kurniadi Deni Heryanto, Deni Dewi Tresnawati Dikdik, Dikdik Dinata, Messy Suryani Jaya Dwi Anggara, Krisna Dzulkhomzah, Moh Rival Fajar, Sigit Sihab Fauzi Pratama, Andhika Fauziah, Fathia Alisha Fikri Fahru Roji Fiqriansyah, Agung Firdaus, Ardy Reza Ginanjar, Ahmad Gotama, Dwi Haris, Gendhi Hartono, Ali Hidayat, Ramdan Rahmat Hidayat, Rangga Huwaidah, Alya Ilham Maulana Ilyasin, Yasa Tiyas Kurnia, Ahmad Hopan Leni Fitriani, Leni Lindawati Lindawati Mahesa, Restu Gusti Malik Ibrahim, Maulana Meta Regita Muhammad Ajif, Arvin Muhammad Rikza Nashrulloh Muhammad Sambas, Phadil Mulyani, Neng Cici Munparik, Riyan Hakim Mutiara, Sani Muzaky, Rifky Khoerul N, Firza Much Asrizal Nawawi, Irfan Ahmad Nurandhini, Rosa Eliza Nurdiansyah, Farhan Nurdin, Kaila Fashla Nurfauziah, Hanifah Nurhalimah, Seli Nurhaqiqi, Lisda Nurpajar, Dini Siti Nursalapiah, Sopa Nurul Muttaqin, Epwan Octaviansyah, Rizqi Moch Pardiansyah, Irgi Pratama, Rizky Muhammad Rahayu, Raden Erwin Gunadhi Rahman, Jaohari Rahmawati, Deby Ricky Rohmanto Ricky Rohmanto Ridwan Ridwan Ibrahim, Maulana Ridwan Setiawan Rinda Cahyana Rinda Cahyana Rohman, Fauza Rohmanto, Ricky Rusdiawan, Mohamad Mihradi Rustandy, Sandy Sadikin, M. Fitroh Saepul Jamil, Alwis Sanusi, Aini Fauziah Putu Septian Rheno Widianto Sermana, Elsa Maharani Setiawan Putra, Achmad Dhani Sidqi, Muhammad Affan Al Sirojudin, Naufal Suryadi, Khaila Thsabita Suryani, Isma Taupik Hidayat, Taupik Tizani, Sofyan Tria Afini Ujang Sarifudin Yoga Handoko Agustin Yosep Septiana