Claim Missing Document
Check
Articles

Systematic Review of Hyperparameter Adjustment and Evaluation Metrics in Bert-Based Sentiment Analysis Bahari, Aris Rifki Setiya; Utomo, Fandy Setyo; Berlilana, Berlilana
Journal La Multiapp Vol. 7 No. 2 (2026): Journal La Multiapp
Publisher : Newinera Publisher

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.37899/journallamultiapp.v7i2.3046

Abstract

The development of sentiment analysis towards Aspect-Based Sentiment Analysis (ABSA) has made significant progress thanks to deep learning technology, especially through the Bidirectional Encoder Representations from Transformers (BERT) architecture. Despite its increasing popularity, a comprehensive synthesis of global research patterns and optimal model configurations is still urgently needed. This study presents a Systematic Literature Review (SLR) combined with bibliometric analysis to examine BERT-based ABSA research indexed in Scopus. Using the PRISMA and VOSviewer frameworks for visualization, a total of 62 eligible articles up to mid-2025 were analyzed. The results of the study show a strong upward trend of publications with a peak in 2024, where China, India, and Indonesia emerged as the major contributors in this domain. Further, the review identified a critical technical standard for effective model training: the Adam optimizer was the most dominant choice, typically paired with a learning rate between 1e-5 to 2e-5 and a batch size of 16. Regarding performance evaluation, Accuracy and F1-Score are set as de facto standard metrics. These findings provide strategic guidance for researchers to optimize BERT implementation and identify future directions in more in-depth sentiment analysis tasks.
IMPROVING HANDWRITTEN DIGIT RECOGNITION USING CYCLEGAN-AUGMENTED DATA WITH CNN–BILSTM HYBRID MODEL Utomo, Fandy Setyo; Barkah, Azhari Shouni; Muhtyas Yugi
JITK (Jurnal Ilmu Pengetahuan dan Teknologi Komputer) Vol. 11 No. 2 (2025): JITK Issue November 2025
Publisher : LPPM Nusa Mandiri

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.33480/jitk.v11i2.6982

Abstract

Handwritten digit recognition presents persistent challenges in computer vision due to the high variability in human handwriting styles, which necessitates robust generalization in classification models. This study proposes an advanced data augmentation strategy using Cycle-Consistent Generative Adversarial Networks (CycleGAN) to improve recognition accuracy on the MNIST dataset. Two architectures are evaluated: a standard Convolutional Neural Network (CNN) and a hybrid model combining CNN for spatial feature extraction and Bidirectional Long Short-Term Memory (BiLSTM) for sequential pattern modeling. The CycleGAN-based augmentation generates realistic synthetic images that enrich the training data distribution. Experimental results demonstrate that both models benefit from the augmentation, with the CNN-BiLSTM model achieving the highest accuracy of 99.22%, outperforming the CNN model’s 99.01%. The study’s novelty lies in the integration of CycleGAN-generated data with a CNN–BiLSTM architecture, which has been rarely explored in previous works. These findings contribute to the development of more generalized and accurate deep learning models for handwritten digit classification and similar pattern recognition tasks.
Adaptive Test Model Enhancement Based on Salmon Salar Optimization and Partially Observable Markov Decision Process Saputro, Rujianto Eko; Utomo, Fandy Setyo; Wanti, Linda Perdana
Journal of Applied Data Sciences Vol 7, No 1: January 2026
Publisher : Bright Publisher

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47738/jads.v7i1.1065

Abstract

Cognitive Diagnosis Models (CDMs) in Computerized Adaptive Testing (CAT) are widely used to assess students’ cognitive abilities; however, existing approaches face significant limitations. The Latent Trait Model often suffers from specification errors due to its complexity, the Diagnostic Classification Model encounters difficulties in integrating hierarchical structures, and Deep Learning Models demand substantial computational resources. To address these challenges, this study introduces Salmon Salar Optimization (SSO) to enhance CDM performance and integrates the Partially Observable Markov Decision Process (POMDP) to improve dynamic question selection. The proposed adaptive testing framework comprises three components: preprocessing, CDM, and a selection algorithm. Experimental results on the ASSISTments 2009-2010 dataset demonstrate that SSO outperforms representative baselines from both deep learning: Neural CD and Latent Trait Model: MIRT approaches. Using 5-fold cross-validation, the proposed model achieved superior predictive performance with 75.51% accuracy and an AUC of 0.8191, highlighting its robustness compared to existing state-of-the-art methods. Furthermore, adaptive test simulations reveal that the SSO- and POMDP-based model delivers superior outcomes, attaining 80.3% accuracy with a reward of 8.03 for 10-question exams and 79.8% accuracy with a reward of 11.97 for 15-question exams. These findings confirm the effectiveness of the proposed model in enhancing cognitive diagnosis and adaptive testing performance.
Performance Comparison Of Xgboost Lightgbm And Lstm For E-Commerce Repeat Buyer Prediction Nugroho, Lustiyono Prasetyo; Saputro, Rujianto Eko; Utomo, Fandy Setyo
Jurnal Teknik Informatika (Jutif) Vol. 7 No. 1 (2026): JUTIF Volume 7, Number 1, February 2026
Publisher : Informatika, Universitas Jenderal Soedirman

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.52436/1.jutif.2026.7.1.5746

Abstract

Repeat buyer behavior is a critical indicator of customer retention success in e-commerce platforms. However, accurately predicting repeat buyers remains a challenging problem due to the complexity of user behavior patterns and the temporal characteristics embedded in interaction data. Existing studies often focus on single modeling approaches or limited sequence exploration, resulting in insufficient comparative insight between ensemble-based machine learning and sequence-based deep learning models. Therefore, this study aims to systematically compare the performance of tree-based ensemble models (XGBoost and LightGBM) and a sequence-based deep learning model (LSTM) in predicting repeat buyers using user behavior data. To ensure fair evaluation, data preprocessing and feature engineering were carefully designed to prevent data leakage by utilizing user behavior prior to the first purchase. Model performance was evaluated using Accuracy, F1-score, and ROC–AUC metrics. Experimental results show that XGBoost and LightGBM achieve stable classification performance with accuracy values of 86.11% and 85.84%, respectively, while the LSTM model attains the highest ROC–AUC value of 0.937, indicating superior capability in capturing temporal behavioral patterns. This study provides valuable insights for e-commerce platforms seeking to optimize predictive models for repeat buyers, contributing to more effective customer retention strategies.
WhatsApp Hybrid Chatbot Architecture Rasa-DeepSeek: Design and Performance Evaluation Had, Iqbaluddin Syam; Utomo, Fandy Setyo; Karyono, Giat; Kinding, Dwi Putriana Nuramanah
Sistemasi: Jurnal Sistem Informasi Vol 15, No 2 (2026): Sistemasi: Jurnal Sistem Informasi
Publisher : Program Studi Sistem Informasi Fakultas Teknik dan Ilmu Komputer

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.32520/stmsi.v15i2.5791

Abstract

This study designed and evaluated a hybrid chatbot for a domain-specific application by addressing two main issues: limited NLU coverage and the variability of latency and cost when all queries are routed directly to an LLM. The proposed solution integrates a deterministic Rasa-based pipeline with a DeepSeek fallback mechanism. In this architecture, Rasa handles NLU processing, rules, stories, and context storage for mk and jk, while the LLM is only invoked when the NLU confidence score falls below a defined threshold. The methodology includes end-to-end implementation through a Node.js bridge connected to Rasa, functional testing to validate the intent–entity–action flow, and performance testing using load (stress) testing across two access paths: the Rasa REST endpoint and the Node-to-Rasa bridge. Meanwhile, the LLM pipeline was profiled separately through instrumented action calls. The results indicate that domain-specific conversations were successfully answered using curated knowledge, and both deterministic access paths met the service level objective (SLO), achieving a median latency of approximately 32 milliseconds with no observed errors. This study contributes by demonstrating that a hybrid chatbot architecture separating deterministic and generative pipelines can maintain SLO compliance in domain-specific settings. In addition, it highlights limitations of LLMs in understanding domain ontologies, reinforcing the need for semantic guardrails.
Optimization of Phishing Detection Performance with Variable Correlation Analysis and Imbalance Learning Arifin, Samsul; Setyo Utomo, Fandy
Sistemasi: Jurnal Sistem Informasi Vol 15, No 2 (2026): Sistemasi: Jurnal Sistem Informasi
Publisher : Program Studi Sistem Informasi Fakultas Teknik dan Ilmu Komputer

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.32520/stmsi.v15i2.4671

Abstract

Phishing is a common cyber security threat in which attackers attempt to deceive users into disclosing personal information such as passwords, credit card numbers, and other sensitive data. With the rapid advancement of technology, phishing techniques have become increasingly sophisticated and harder to detect using traditional methods. Therefore, it is essential to develop detection techniques capable of identifying phishing websites with high accuracy. This study aims to optimize phishing detection performance by integrating variable correlation analysis for feature selection and applying imbalanced learning techniques to address data imbalance. The research stages include Data Collection, Data Preprocessing, and Data Exploration, which involve correlation analysis, removal of low-correlation features, and data visualization. In the Model Building and Training phase, the dataset is split into features and labels, followed by training and the application of data balancing techniques, ending with Model Evaluation. The evaluated algorithms include Logistic Regression, Naive Bayes, K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Multi-Layer Perceptron, Decision Tree, Random Forest, Gradient Boosting, and CatBoost. The results show that the KNN algorithm delivers the best performance, achieving an accuracy of 91.25% and optimal scores in Precision (0.906943), Recall (0.927858), and F1-Score (0.922141), along with the lowest Hamming Loss at 0.0875. In contrast, the SVM algorithm recorded the lowest performance among the tested models. The implementation of this method is expected to contribute to the development of more reliable and accurate phishing detection systems in the future.
Evaluation of User Satisfaction in Web-based Library Information Systems: A Systematic Literature Review Faradina, Faradina; Hariguna, Taqwa; Utomo, Fandy Setyo
Sistemasi: Jurnal Sistem Informasi Vol 15, No 3 (2026): Sistemasi: Jurnal Sistem Informasi
Publisher : Program Studi Sistem Informasi Fakultas Teknik dan Ilmu Komputer

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.32520/stmsi.v15i3.6204

Abstract

The transformation of library management today is highly influenced by the acceleration of information and communication technology (ICT), particularly through the adoption of web-based information systems. While these systems can optimize productivity and service accessibility, their effectiveness ultimately depends on the level of user satisfaction. This study evaluates various user satisfaction assessment methodologies through a Systematic Literature Review (SLR) using the PRISMA protocol on 25 selected articles published between 2020 and 2024. The findings indicate a shift in the dominance of evaluation tools toward the Human-Organization-Technology Fit (HOT-Fit) model and the Net Promoter Score (NPS). Key determinants of satisfaction were identified in terms of information quality, system reliability, and responsiveness of technical support.
Usability Evaluation of a School Library OPAC Using Heuristic Evaluation and User Testing Faradina Faradina; Taqwa Hariguna; Fandy Setyo Utomo
Journal of Information System and Informatics Vol 8 No 1 (2026): February
Publisher : Asosiasi Doktor Sistem Informasi Indonesia

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.63158/journalisi.v8i1.1528

Abstract

This study evaluates the usability of the Online Public Access Catalog (OPAC) at SMK Negeri 1 Purwokerto to address the persistent gap between traditional library information architectures and the modern search behaviors of vocational students within the Kurikulum Merdeka ecosystem. The research aims to solve the problem of "mental model dissonance" that hinders independent information literacy among digital native learners. A hybrid evaluation approach was employed, integrating a Heuristic Evaluation by three experts with empirical User Testing involving students. The study utilized the Think-Aloud protocol and the System Usability Scale (SUS) to capture both performance and perception data. Result: The expert inspection identified 18 significant usability violations, primarily in library technical jargon (H2) and error prevention (H5). Empirical testing revealed a low average Task Success Rate (TSR) of 49.3% and a mean SUS score of 55.0, placing the system in the "Unacceptable" category. These figures confirm that current cataloging logic significantly obstructs retrieval efficiency. The originality of this research lies in the identification of specific dissonance points between vocational students' mental models and bibliographic metadata. It provides a strategic framework for interface restructuring through semantic simplification and department-based navigation, offering a practical model for developing user-centric "smart" library services in vocational education.
ENHANCING HANDWRITTEN DIGIT RECOGNITION ACCURACY ON THE MNIST DATASET USING A HYBRID CNN-BILSTM MODEL WITH DATA AUGMENTATION Yugi, Muhtyas; Latif, Ahmad; Utomo, Fandy Setyo; Barkah, Azhari Shouni
JIPI (Jurnal Ilmiah Penelitian dan Pembelajaran Informatika) Vol 11, No 1 (2026)
Publisher : STKIP PGRI Tulungagung

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.29100/jipi.v11i1.7758

Abstract

Handwritten digit recognition is a classic challenge in the field of computer vision and machine learning, and continues to be developed to achieve higher accuracy. This study proposes a hybrid method that combines Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) to enhance performance in handwritten digit classification using the MNIST dataset. CNNs are em-ployed to extract spatial features from digit images, while BiLSTMs are used to capture the temporal patterns and sequential context from the extracted features. To address limitations in data variation and improve the model’s generalization capabilities, the study also applies data augmentation techniques based on image transformations such as rota-tion, translation, scaling, and flipping. Experimental results demonstrate that the hybrid CNN-BiLSTM model with data augmentation signifi-cantly improves classification accuracy compared to baseline ap-proaches without augmentation or without BiLSTM. The model achieved the following accuracy on the MNIST test data: CNN Model Accuracy: Before Augmentation: 98.0%. After Augmentation: 98.5%; CNN-BiLSTM Model Accuracy: Before Augmentation: 98.0%. After Augmentation: 98.7%. These results highlight the effectiveness of the hybrid approach in enhancing handwritten digit recognition perfor-mance. This research contributes to the development of more accurate and robust deep learning models for handwritten image processing
COMPARISON OF THE PERFORMANCE OF SVM, RANDOM FOREST, AND NEURAL NETWORK ALGORITHMS IN SENTIMENT ANALYSIS OF OPENAI APPLICATION REVIEWS ON THE GOOGLE PLAY STORE Latif, Ahmad; Yugi, Muhtyas; Utomo, Fandy Setyo; Hariguna, Taqwa
JIPI (Jurnal Ilmiah Penelitian dan Pembelajaran Informatika) Vol 11, No 1 (2026)
Publisher : STKIP PGRI Tulungagung

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.29100/jipi.v11i1.7793

Abstract

This study compares the performance of three machine learning algo-rithms—Support Vector Machine (SVM), Random Forest (RF), and Neural Network (NN)—in sentiment analysis of user reviews for the OpenAI application on the Google Play Store. The primary objective of this study is to evaluate the effectiveness of each algorithm in clas-sifying user reviews into three sentiment categories: positive, negative, and neutral. The dataset used consists of user reviews of the OpenAI application, collected directly from the Google Play Store. Model per-formance was evaluated using accuracy, precision, recall, and F1-score metrics. The results indicate that the Neural Network algorithm achieved the best overall performance in terms of accuracy and F1-score. SVM demonstrated competitive performance, particularly in classifying positive and neutral sentiments, while Random Forest showed an advantage in terms of precision but performed lower over-all, especially in classifying negative sentiments. Therefore, the Neural Network is considered the most effective algorithm for sentiment analysis tasks in this study
Co-Authors Adiya, Az Zahra Dwi Nur Afit Ajis Solihin Aisha Hukama Setyowati Aji Saeful Aji Septa, Adrian Ajis Solihin, Afit Amar Al Farizi Anas Nur Khafid Anggini, Melisa Anggraeni, Mutia Dwi Anggraini, Nova Anggriani, Epri Anies Indah Hariyanti Azhari Shouni Barkah Azmi, Mohd Sanusi Bagus Adhi Kusuma Bahari, Aris Ridky Setiya Bahari, Aris Rifki Setiya Baihaqi, Wiga Maulana Balit, Muhamad Naufal Burhanuddin Berlilana Berlilana Berlilana Burhanuddin Balit, Muhamad Naufal Churil Aeni, Agustina Chyntia Raras Ajeng Widiawati Chyntia Raras Ajeng Widiawati Darmono Dedi Purwanto, Dedi Didi Prasetyo Dwi Krisbiantoro, Dwi Dwi Putriana Nuramanah Kinding Dzaky Candy Fahrezy Fadhilah, Siti Nur Faradina Faradina Faradina, Faradina Febriansyah Husni Adiatma Giat Karyono Giat Karyono Had, Iqbaluddin Syam Hanif Hidayatulloh Hendra Marcos, Hendra hidayatulloh, hanif Ilham, Rifqi Arifin Imam Tahyudin Indriyani, Ria Jamie Mayliana Alyza Kafilla, Princess Iqlima Kusuma, Bagus Adhi Kusuma, Velizha Sandy Lasmedi Afuan Latif, Ahmad Lubna, Zuhriyatul Lukita, Dita Maulana Baihaqi, Wiga Mohd Fairuz Iskandar Othman Mohd Nazrin Muhammad Mohd Sanusi Azmi Muaziz, Imam Muhamad Naufal Burhanuddin Balit Muhtyas Yugi Murtiyoso Murtiyoso Nandang Hermanto Nanna Suryana Nikmah Trinarsih Nugroho, Khabib Adi Nugroho, Lustiyono Prasetyo Nur Cholis Romadhon Octavia, Annisa Suci Prayoga, Fandhi Dhuga Pungkas Subarkah Purbo, Yevi Septiray Purwidiantoro, Moch. Hari Pyawai, Hero Galuh R. Vitto Mahendra Putranto Ramadhan, Aziz Ramadhan, Rio Fadly Rifqi Arifin Ilham RR. Ella Evrita Hestiandari Rujianto Eko Saputro Sagita, Selvi Samsul Arifin Sarmini - Sarmini Sarmini Sarmini Sekhudin, Sekhudin Setiabudi, Rizki Setiawan, Ito Shafira, Lulu Shendy Filanzi Slamet Widodo Slamet Widodo Sofa, Nur Sri Hartini Sugianto, Dwi Suryana, Nanna Taqwa Hariguna Taqwa Hariguna Titi Safitri Maharani Trinarsih, Nikmah Turino, Turino Utomo, Dadang Wahyu Wahid, Arif Mu'amar Wanti, Linda Perdana Wibisono, Arif Cahyo Wiga Maulana Baihaqi Yugi, Muhtyas Yuli Purwat Yuli Purwati Yuli Purwati Yulianto, Koko Edy