Claim Missing Document
Check
Articles

Found 4 Documents
Search
Journal : Journal of Intelligent Systems Technology and Informatics

Automatic Sentiment Annotation Using Grok AI for Opinion Mining in a University Learning Management System Julianto, Indri Tri; Sidqi, Muhammad Affan Al
Journal of Intelligent Systems Technology and Informatics Vol 1 No 3 (2025): JISTICS, November 2025
Publisher : Aliansi Peneliti Informatika

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.64878/jistics.v1i3.42

Abstract

Sentiment analysis has become an essential tool in evaluating user feedback on digital learning platforms. Understanding student sentiments toward Learning Management Systems (LMS) in higher education can offer critical insights for system development and service improvement. This study aims to evaluate the effectiveness of AI-assisted sentiment labeling using Grok AI and ChatGPT compared to manual labeling for sentiment classification of student opinions on LMS at Institut Teknologi Garut. The research involved distributing an online questionnaire to 96 students across four academic levels, collecting open-ended responses regarding their LMS usage experiences. These responses were preprocessed through case folding, cleaning, tokenization, stopword removal, and stemming. The sentiment labels were assigned using Grok AI, ChatGPT, and manual annotation, and the resulting datasets were used to build classification models using the Naïve Bayes algorithm in Altair RapidMiner with 10-Fold Cross Validation. The performance evaluation shows that manual labeling yielded the highest accuracy (52.22%) and Cohen's Kappa (0.137), followed by ChatGPT (50.11%, 0.119) and Grok AI (48.00%, 0.087). Word cloud visualizations further revealed the dominant themes within each sentiment class, indicating that positive opinions emphasized helpfulness and ease of use, while negative ones focused on access issues and system lags. This research suggests that AI-assisted labeling methods can be viable alternatives, although manual labeling still offers slightly better accuracy.
User Sentiment Analysis X Towards Makan Bergizi Gratis Program Using Automatic Labeling Technique with Deepseek AI Julianto, Indri Tri; Nurpajar, Dini Siti
Journal of Intelligent Systems Technology and Informatics Vol 1 No 2 (2025): JISTICS, July 2025
Publisher : Aliansi Peneliti Informatika

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.64878/jistics.v1i2.43

Abstract

Public perception of national nutrition initiatives is instrumental in shaping inclusive and data-driven policy development. In Indonesia, the "Makan Bergizi Gratis" (MBG) program introduced by President Prabowo has drawn significant attention, particularly on the X platform (formerly Twitter). This research topic was selected due to its national urgency and political significance, as the MBG program emerged as a key agenda during the 2024–2025 political transition. Therefore, examining public sentiment is essential to assess policy acceptance and identify areas for improvement. This study analyzes user sentiment toward the MBG policy using an automatic labeling approach supported by DeepSeek AI and the VADER Lexicon, followed by sentiment classification through the K-Nearest Neighbor (KNN) algorithm. The research involved five main stages: collecting 1,704 tweets from X between January 2024 and March 2025, preprocessing the text, conducting automatic sentiment labeling, applying TF-IDF for vectorization, handling class imbalance using the Synthetic Minority Over-sampling Technique (SMOTE), and classifying sentiments using KNN. The results indicate that without SMOTE, the VADER model achieved higher accuracy (93.49%) but lower Cohen's Kappa (0.16), while DeepSeek AI yielded lower accuracy (73.67%) but slightly higher Kappa (0.17). After SMOTE was applied, accuracy declined (VADER to 77.25%, DeepSeek AI to 64.72%), but Kappa scores improved significantly (VADER to 0.65, DeepSeek AI to 0.47), indicating more balanced and consistent sentiment predictions across classes. In conclusion, integrating automatic labeling, SMOTE, and KNN provides a reliable and scalable framework for analyzing large-scale sentiment on social media platforms, particularly in contexts with imbalanced opinion distributions.
Sentiment Analysis of the Residency Policy Launch in the New Student Admission System Using Automatic Labeling with Meta AI Julianto, Indri Tri; Lindawati
Journal of Intelligent Systems Technology and Informatics Vol 2 No 1 (2026): JISTICS, March 2026
Publisher : Aliansi Peneliti Informatika

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.64878/jistics.v2i1.44

Abstract

The launch of a domicile-based policy in Indonesia's New Student Admission System (SPMB) has triggered various public responses, especially on social media platforms. Understanding these sentiments is essential for evaluating policy acceptance and guiding future improvements in educational governance. This study aims to analyze public sentiment toward the policy using automatic labeling techniques and machine learning classification, with a focus on identifying dominant public perceptions. The research applies the CRISP-DM methodology, consisting of six stages: business understanding, data understanding, data preparation, modeling, evaluation, and deployment. A total of 1,105 comments were collected from Instagram and YouTube via web scraping and then preprocessed using text cleaning, stemming, and tokenization. Sentiment labels were generated using three automatic methods: Meta AI, RoBERTa, and TextBlob. Classification was performed using the Support Vector Machine (SVM) algorithm with four kernel variations. The results indicate that the combination of TextBlob labeling and an SVM with the Sigmoid kernel achieved the highest accuracy (0.99), along with strong precision, recall, and F1 Scores. Word cloud visualizations revealed that positive sentiment was related to educational access and teacher appreciation, while negative sentiment focused on dissatisfaction with fairness and system transparency. In conclusion, this study demonstrates that automated sentiment analysis, when supported by proper preprocessing and class balancing, is a powerful approach to extracting meaningful insights from public discourse. The findings are expected to support policymakers in developing data-driven strategies for improving future education policies.
Load Testing-Based Performance Evaluation of the SiUKT API System Rachman, Andi Nur; Shofa, Rahmi Nur; Sjamsuddin, Irfan Nafis; Tarempa, Genta Najwar; Julianto, Indri Tri; Athoillah , Bifahmi Ahmad
Journal of Intelligent Systems Technology and Informatics Vol 2 No 1 (2026): JISTICS, March 2026
Publisher : Aliansi Peneliti Informatika

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.64878/jistics.v2i1.89

Abstract

Software testing is a crucial stage in the system development lifecycle. Previous studies on SiUKT have used only the black-box method, focusing on functionality without providing insights into performance optimization. This study aims to analyze and improve the performance of the SiUKT API using an intelligent load-testing approach with Apache JMeter. The testing measures three key indicators—response time, throughput, and error rate—across 10 API endpoints with concurrent user simulations of 10, 100, 250, and 500 users. The results show that the SiUKT website performs effectively under moderate load conditions, with an average response time of 338 ms and a throughput of 8.2 requests per second for 10 users. Under high load (500 users), performance declines, with response times ranging from 6 to 8 seconds, while throughput remains stable and the error rate stays at 0.00%. Only the register endpoint experienced a 100% error rate due to validation conflicts. These findings demonstrate the system's ability to maintain stability under varying loads and highlight performance degradation patterns as user traffic increases. The research contributes to the optimization of intelligent system performance by establishing quantitative benchmarks for API scalability and providing recommendations for adaptive infrastructure improvements to support automated intelligent load management.
Co-Authors Abdullah, Angga Abdulrohman, Muhammad Haviz Ade Sutedi Ade Sutedi, Ade Aditriyana, Muhammad Rizky Agisni Nurlela, Agni Akhdan Hidayat, Fairuz Alamsyah, Restu Alisha Fauzia, Fathia Apriliani, Insani Ardana, Alwan Arif Rahman, Rifal Arif Syamsudin, Muhammad Asri Mulyani Athoillah , Bifahmi Ahmad B. Balilo Jr , Benedicto B. Balilo Jr, Benedicto Balilo Jr, Benedicto B. Baswardono, Wiyoga Burhanudin, Asep Chaerunisa, Adinda Citra Indahsari, Ajeng Dede Kurniadi Deni Heryanto, Deni Dewi Tresnawati Dikdik, Dikdik Dinata, Messy Suryani Jaya Dwi Anggara, Krisna Dzulkhomzah, Moh Rival Fajar, Sigit Sihab Fauzi Pratama, Andhika Fauziah, Fathia Alisha Fikri Fahru Roji Fiqriansyah, Agung Firdaus, Ardy Reza Ginanjar, Ahmad Gotama, Dwi Haris, Gendhi Hartono, Ali Hidayat, Ramdan Rahmat Hidayat, Rangga Huwaidah, Alya Ilham Maulana Ilyasin, Yasa Tiyas Kurnia, Ahmad Hopan Leni Fitriani, Leni LINDAWATI Lindawati Lindawati Mahesa, Restu Gusti Malik Ibrahim, Maulana Meta Regita Muhammad Agreindra Helmiawan Muhammad Ajif, Arvin Muhammad Rikza Nashrulloh Muhammad Sambas, Phadil Mulyani, Neng Cici Munparik, Riyan Hakim Mutiara, Sani Muzaky, Rifky Khoerul N, Firza Much Asrizal Nafis Sjamsuddin, Irfan Nawawi, Irfan Ahmad Nurandhini, Rosa Eliza Nurdiansyah, Farhan Nurdin, Kaila Fashla Nurfauziah, Hanifah Nurhalimah, Seli Nurhaqiqi, Lisda Nurpajar, Dini Siti Nursalapiah, Sopa Nurul Muttaqin, Epwan Octaviansyah, Rizqi Moch Pardiansyah, Irgi Pratama, Rizky Muhammad Rachman, Andi Nur Rahayu, Raden Erwin Gunadhi Rahman, Jaohari Rahmawati, Deby Ricky Rohmanto Ricky Rohmanto Ridwan Ridwan Ibrahim, Maulana Ridwan Setiawan Rinda Cahyana Rinda Cahyana Rohman, Fauza Rohmanto, Ricky Rusdiawan, Mohamad Mihradi Rustandy, Sandy Sadikin, M. Fitroh Saepul Jamil, Alwis Sanusi, Aini Fauziah Putu Septian Rheno Widianto Sermana, Elsa Maharani Setiawan Putra, Achmad Dhani Shofa, Rahmi Nur Sidqi, Muhammad Affan Al Sirojudin, Naufal Suryadi, Khaila Thsabita Suryani, Isma Tarempa, Genta Najwar Taupik Hidayat, Taupik Tizani, Sofyan Tria Afini Ujang Sarifudin Yoga Handoko Agustin Yosep Septiana